System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving an image of an interior of a mouth captured by an image capture device; means for preprocessing the received image of the interior of the mouth and inputting the preprocessed image to a AI model; means for analyzing the image of the interior of the mouth using the AI model and determining a health condition; means for automatically generating a health condition report and an improvement proposal for a user based on the determination result; and means for transmitting the generated report and improvement proposal to a user device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Regular dental checkups are important for maintaining dental health, but many people tend to put off dental checkups due to busy schedules, anxiety about visiting the dentist, time constraints, and other factors. As a result, as people get older, untreated cavities and periodontal disease progress, making it increasingly difficult for them to eat using their own teeth. This issue stems from the reality that regular dental checkups are not easily accessible, and there is a need for a system that allows people to easily monitor their dental health at home. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides the following means. The system includes: means for receiving image data of the oral cavity captured by an image capture device; means for preprocessing the received image data of the oral cavity and inputting it into an AI model; means for analyzing the image data of the oral cavity using the AI ​​model to assess health status; means for automatically generating a health status report and improvement suggestions for the user based on the assessment results; and means for transmitting the generated report and improvement suggestions to a user terminal. Furthermore, if a detailed diagnosis by a specialist is desired, the system includes means for transmitting the captured image data of the oral cavity to a specialist via a separate communication means and receiving the diagnosis results from the specialist. The system also includes database means for accumulating health data for each user and using it to improve the accuracy of future analyses and improvement suggestions.

[0006] An "imaging device" is a device that captures images of the inside of the oral cavity and acquires them as digital data.

[0007] "Image data" refers to image information expressed in digital format, and is data that indicates the state of the inside of the oral cavity.

[0008] "Preprocessing" refers to the process of converting image data into an appropriate format before inputting it into an AI model, and includes processes such as resizing and normalization.

[0009] An "AI model" is an analytical algorithm that uses artificial intelligence and is used to determine health status from image data.

[0010] "Assessing health status" means that the AI ​​model analyzes image data to identify conditions such as cavities, periodontal disease, stains, discoloration, and tooth wear.

[0011] "Reports and Improvement Suggestions" refers to information for users that is generated based on the results of health status analysis, and includes explanations of health status and advice.

[0012] "User Device" means a device, such as a smartphone or tablet, used by a User to receive notifications and suggestions.

[0013] "Expert diagnosis results" refers to diagnostic information provided by an expert such as a dentist after analyzing image data.

[0014] "Another means of communication" refers to the means that users use to communicate with experts, such as messaging services like LINE or email.

[0015] The "database" is a system that stores users' health data and uses it to improve the accuracy of future analysis and improvement suggestions. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] As one embodiment of the present invention, the AI ​​Dental Guard system is a system that allows users to easily and accurately monitor their dental health at home and receive suggestions for improvement. The processing of the system program is explained below in natural language.

[0038] User takes and uploads a photo

[0039] Users launch the AI ​​Dental guard app on their smartphone and use the camera to take a photo of their teeth. After taking the photo, they press the "Start Analysis" button in the app to upload the captured data to the server. For example, users can take photos of the front and back teeth with their smartphone and send the images through the app.

[0040] Upload the image to the server

[0041] The device (user's smartphone) takes a photo and sends the image data to the server. This requires an internet connection. For example, the user can use Wi-Fi or mobile data to upload the image data to the server over a secure network.

[0042] The image analysis server performs the analysis

[0043] The server receives and stores image data sent from the device. The received images are preprocessed and input into the AI ​​model. The AI ​​model analyzes the image data to determine health conditions such as cavities, risk of periodontal disease, stains, discoloration, and tooth wear. For example, the server analyzes an image uploaded by a user and detects signs of early cavities in the upper right molar.

[0044] Save the analysis results in a database

[0045] The server associates the analysis results with the user ID and saves them in a database. This allows each user's health history to be accumulated, which will be useful for future analysis and advice. For example, the server saves the results in the database as "User A's dental check results for October 2023."

[0046] Notify users of results and send suggestions for improvement

[0047] Based on the analysis, the server automatically generates a health report and recommendations for improvement, which are sent to the user's app. For example, the report might include a warning about a high risk of cavities in the upper right molar, along with recommendations for specific brushing and flossing techniques.

[0048] The user checks the results

[0049] Users will receive a notification within the app and can view the generated report and improvement suggestions. If necessary, they can send images via messaging services such as LINE for expert diagnosis. For example, a user may view a warning in the app, request a more detailed diagnosis, and send images to the LINE official account.

[0050] Each step can be easily performed at home, allowing for daily monitoring of dental health and early diagnosis and prevention without visiting a dental clinic. This system is extremely beneficial for people who lead busy lives or who are anxious about visiting the dentist.

[0051] The processing flow will be explained below.

[0052] Step 1:

[0053] The user launches the app.

[0054] The user taps and launches the AI ​​Dental guard app on their smartphone.

[0055] Step 2:

[0056] The user takes a photo of their teeth.

[0057] Users use the app's camera function to take a photo of their teeth with their smartphone camera, and are encouraged to clearly capture both the front and back teeth.

[0058] Step 3:

[0059] The user checks the shooting data.

[0060] Users can review the photos they've taken, take new ones if necessary, and once they're satisfied, press the "Start Analysis" button within the app.

[0061] Step 4:

[0062] The device sends the captured image data to the server.

[0063] The image data captured by the device is uploaded to a dedicated server via an internet connection, using Wi-Fi or mobile data.

[0064] Step 5:

[0065] The server receives the image data.

[0066] The server receives the image data sent from the terminal and stores it in a secure storage.

[0067] Step 6:

[0068] The server preprocesses the received image data.

[0069] The server performs preprocessing on the received image data, such as formatting, resizing, and normalization. This preprocessing converts the image data into a format suitable for the AI ​​model.

[0070] Step 7:

[0071] The server inputs preprocessed image data into the AI ​​model.

[0072] The server inputs the preprocessed image data into the AI ​​model and prepares it for analysis.

[0073] Step 8:

[0074] The AI ​​model analyzes the image data.

[0075] The AI ​​model analyzes the image data to determine health conditions such as risk of cavities, periodontal disease, stains, discoloration, and tooth wear.

[0076] Step 9:

[0077] The server generates the analysis results.

[0078] The server generates a health report based on the analysis results obtained from the AI ​​model, which includes a description of the health condition and individualized suggestions for improvement.

[0079] Step 10:

[0080] The server stores the analysis results in a database.

[0081] The analysis results generated by the server are stored in a database along with the user ID, allowing each user's health history to be accumulated.

[0082] Step 11:

[0083] The server sends the analysis results and improvement suggestions.

[0084] The server generates a health report and sends it to the user's app, which displays the user's health status and appropriate actions.

[0085] Step 12:

[0086] The user checks the results in the app.

[0087] Users receive in-app notifications and receive health status reports and suggestions for improvement.

[0088] Step 13:

[0089] If the user wishes to have a specialist diagnose the image, they can submit it.

[0090] If the user wishes to receive a more detailed diagnosis, they can send the image to a specialist from within the app using another means of communication, such as the official LINE account, and wait for a reply from the specialist.

[0091] Step 14:

[0092] The expert will send the diagnosis results to the user.

[0093] The expert will analyze the images sent and return the diagnosis and specific advice to the user, allowing the user to obtain more detailed health information.

[0094] Example 1

[0095] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0096] In modern society, many people lead busy lives and often neglect regular dental checkups. As a result, dental health is more likely to deteriorate and early detection becomes more difficult. Furthermore, for people who have anxiety about visiting the dentist, regular visits can be a psychological burden. Given this background, there is a need for a system that can easily and accurately monitor dental health at home and provide guidelines for when to seek professional diagnosis.

[0097] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0098] In this invention, the server includes means for receiving image data of the inside of the oral cavity captured by an image capturing device, means for preprocessing the received image data of the inside of the oral cavity and inputting it into a generative AI model, means for sending prompts to the generative AI model, analyzing the image data of the inside of the oral cavity, and determining the health condition, means for automatically generating a health condition report and improvement suggestions for the user based on the determination results, and means for sending the generated report and improvement suggestions to a user terminal. This enables users to easily and accurately monitor their dental health condition at home, detect problems early, and take appropriate measures.

[0099] An "imaging device" is a device for taking images of the inside of the oral cavity, and includes smartphones, digital cameras, etc.

[0100] "Image data" refers to a digital representation of visual information generated by an image capture device.

[0101] "Means for receiving" refers to a device or software capable of receiving image data into a server or other system component.

[0102] "Preprocessing" refers to a series of processes performed on received image data to facilitate analysis, and specific examples include adjusting resolution and removing noise.

[0103] A "generative AI model" refers to a model that uses artificial intelligence techniques to analyze data and is trained to perform a specific task.

[0104] A "prompt" is a textual instruction that instructs a generative AI model to perform a specific analytical task.

[0105] "Means for analysis" refers to a device or software that has the function of inputting image data into a generative AI model, analyzing the data, and determining health status.

[0106] "Determination results" refer to information about the health status of the oral cavity obtained based on analysis by the generative AI model.

[0107] "Means for automatically generating reports and improvement suggestions" refers to a device or software that has the function of explaining the user's health condition and suggesting ways to improve it based on the assessment results.

[0108] "User terminal" refers to a device used by a user to receive reports and improvement suggestions, such as a computer or smartphone.

[0109] As one embodiment of the present invention, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home and receive improvement suggestions. This system is composed of an image capture device, a generative AI model, and a user terminal.

[0110] First, a user launches the AI ​​Dental guard app on their smartphone (for example, an iPhone or Android device) and takes a photo of their teeth using the app's camera function. The captured image data is then uploaded from the smartphone to a server via the internet. Uploading is done using a Wi-Fi or mobile data connection. For example, a user can take a photo of the front and back teeth with their smartphone and send the image through the app.

[0111] The server receives the image data sent from the device and temporarily stores it in a database. The received image data is preprocessed (for example, by adjusting the resolution or removing noise) and then input into the generative AI model. Image processing software (for example, OpenCV) is used for preprocessing.

[0112] The generative AI model uses deep learning frameworks such as TensorFlow and PyTorch to analyze image data by receiving prompts such as the following:

[0113] "Analyze this image to determine the user's dental health (cavities, risk of periodontal disease, stains, discoloration, tooth wear)."

[0114] This allows the AI ​​model to analyze image data and determine health conditions such as risk of cavities and periodontal disease, dirt, staining, and tooth wear.

[0115] The server receives the analysis results obtained from the generative AI model, associates them with the user ID, and stores them in a database. This database accumulates health data for each user and is used to improve the accuracy of future analyses and improvement suggestions.

[0116] Based on the analysis results, the server automatically generates a health status report and improvement suggestions. These reports and suggestions are sent to the user's device, and the user receives notifications within the app and can review the generated report and improvement suggestions. For example, a warning indicating a high risk of cavities in the upper right molar and suggestions for specific tooth brushing and flossing techniques may be displayed.

[0117] If necessary, users can use the app's functions to receive further professional advice. For example, users can use messaging services such as LINE to send images they have taken to a specialist for further diagnosis.

[0118] In this way, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home, detect problems early, and take appropriate measures. It also helps prevent dental visits and supports continuous health management.

[0119] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0120] Step 1:

[0121] The user launches the AI ​​Dental guard app on their smartphone and takes a photo of their teeth using the app's camera function. The captured image data is the input and is saved on the user's device. Next, when the user taps the "Start Analysis" button, the captured data is uploaded to the server. The output is the image data sent to the server. Specifically, when the user takes a photo of the front teeth with their iPhone and presses the "Start Analysis" button, the captured image file (e.g., JPEG) is sent to the server.

[0122] Step 2:

[0123] The device compresses the captured image data (e.g., using the JPEG compression algorithm) and sends it to a server via an Internet connection. The input is the uncompressed image data, and the output is the compressed image data. Specifically, the device uses a Wi-Fi connection to send the image data to the server via an SSL / TLS encrypted channel.

[0124] Step 3:

[0125] The server receives image data sent from the device and temporarily stores it in a database. The input is image data from the device, and the output is image data stored in the database. For example, the server stores image data in a specified format in a MySQL database.

[0126] Step 4:

[0127] The server preprocesses the received image data. Preprocessing includes adjusting the resolution and removing noise. The input is image data stored in the database, and the output is the preprocessed image data. Specifically, the server uses the OpenCV library to adjust the image resolution and remove noise.

[0128] Step 5:

[0129] The server inputs the preprocessed image data into the generative AI model and sends a prompt. The input is the preprocessed image data and the prompt, and the output is the analysis result. For example, the server sends the following prompt to the TensorFlow model: "Analyze this image to determine the user's dental health status (cavities, periodontal disease risk, stains, discoloration, and tooth wear)."

[0130] Step 6:

[0131] The generative AI model analyzes the received image data and determines health conditions such as risk of cavities and periodontal disease, stains, discoloration, and tooth wear. The input is preprocessed image data and prompt text, and the output is the analysis results. Specifically, the AI ​​model analyzes each pixel in the image, identifies specific patterns, and generates a judgment result.

[0132] Step 7:

[0133] The server receives the generated analysis results, associates them with the user ID, and stores them in a database. The input is the analysis results from the AI ​​model, and the output is the analysis results stored in the database. Specifically, "User A's dental check results for October 2023" is stored in the server's database.

[0134] Step 8:

[0135] Based on the analysis results, the server automatically generates a health status report and improvement suggestions. These reports and suggestions are sent to the user's app. The input is the analysis results, and the output is the report and suggestions sent to the user's device. Specifically, the server generates a warning such as "You have a high risk of cavities in your upper right molars" and "suggestions for specific tooth brushing methods and flossing."

[0136] Step 9:

[0137] Users receive notifications within the app and review the generated report and improvement suggestions. The input is the report and suggestions sent to the device, and the output is the user's acknowledgment. If necessary, users can use the app's functions to send images to experts. Specific actions include tapping the "Check analysis results" button in the app, viewing the detailed analysis, and then sharing the images and analysis results using the "Send to expert via LINE" button.

[0138] (Application example 1)

[0139] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0140] Traditionally, dental clinics have spent a lot of time and effort diagnosing patients' oral conditions. Furthermore, there were limited ways for patients to monitor their dental health at home, making it difficult to detect problems early. Furthermore, there was a need for efficient methods for managing patient data and proposing treatment plans.

[0141] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0142] In this invention, the server includes a means for receiving image data of the oral cavity captured by an imaging device, a means for preprocessing the received image data of the oral cavity and inputting it into a generated AI model, a means for analyzing the image data of the oral cavity using the AI ​​model to assess the user's health status, a means for automatically generating a health status report and improvement suggestions for the user based on the assessment results, a means for transmitting the generated report and improvement suggestions to a user terminal, a means for collecting patient data at the reception desk and saving it together with the image data, and a means for proposing a treatment plan based on the analysis results. This enables efficient patient care at dental clinics and rapid diagnosis and treatment plan proposals by dentists. Furthermore, patients can easily monitor their oral health status at home and obtain appropriate improvement measures.

[0143] An "imaging device" is a device used to take images of the inside of the oral cavity, and typically refers to a smartphone with a camera function or a dedicated digital camera.

[0144] The "receiving means" is a mechanism for receiving data sent from outside, and is a device that includes a server or software for receiving image data via a network connection.

[0145] "Preprocessing" refers to processing performed to make received image data easier to analyze, and typically refers to image processing such as noise removal, brightness adjustment, and cropping.

[0146] An "AI model" is a model of artificial intelligence that has been trained using machine learning algorithms to perform a specific task.

[0147] The "means of judgment" is a function that derives results based on the data being analyzed; specifically, the AI ​​model analyzes image data to evaluate health status.

[0148] "Means for automatically generating reports and improvement suggestions" refers to a function that automatically generates reports on the user's health status and methods for improving it based on the assessment results obtained by the AI ​​model.

[0149] "Means for sending to user devices" refers to a function for sending generated reports and improvement suggestions to devices used by users, and is a mechanism for sending data to smartphones and other devices via the Internet.

[0150] "Means for collecting patient data at the reception desk" refers to a function for collecting basic patient information and intraoral images at the dental clinic reception desk, and involves obtaining this information using smart glasses or a tablet.

[0151] "Means for proposing a treatment plan based on the analysis results" refers to a function that proposes specific treatment methods and procedures appropriate for the patient based on the analysis results of the AI ​​model.

[0152] One embodiment of this invention is a system in which an application called "Dental Reception Assistant" that supports reception work at dental clinics is installed on smart glasses or a head-mounted display. This system takes intraoral images of patients at the reception desk, analyzes the images using the AI ​​Dental guard system, and immediately provides diagnosis results and treatment plans.

[0153] Key Hardware and Software

[0154] The hardware used includes, for example, smart glasses (Vuzix M400) and a head-mounted display (Microsoft HoloLens) for capturing intraoral images and displaying the data, while the server hosts the high-performance image processing unit and AI models.

[0155] The software uses OpenCV as a library for image processing and AI analysis, and trained generative AI models. In addition, the HTTP protocol is used for data communication, sending and receiving image data and analysis results.

[0156] Program Overview

[0157] The system works as follows:

[0158] 1. Patient data collection and photography

[0159] Using a device (smart glasses or a head-mounted display), an image of the patient's oral cavity is taken. At the same time, basic information about the patient is also entered. The device then sends this image data to a server.

[0160] 2. Image data preprocessing

[0161] The server preprocesses the received image data, specifically performing noise removal, brightness adjustment, cropping, etc.

[0162] 3. Analysis using AI models

[0163] The pre-processed image data is then fed into a generative AI model, which analyzes the images and determines health conditions such as risk of cavities, risk of periodontal disease, staining level, and wear status.

[0164] 4. Diagnosis results and treatment plan proposal

[0165] Based on the results, the server automatically generates a health status report and recommendations for improvement, as well as specific treatment plans, which are then sent to the user's device and displayed visually.

[0166] 5. Checking the results and taking action

[0167] The user (dentist or staff) receives a notification on their device, reviews the generated report and improvement suggestions, and, if necessary, proposes an appropriate treatment plan for the patient.

[0168] Specific examples

[0169] When a patient visits the dental clinic, the dental clinic's reception staff uses smart glasses to take a photo of the patient's mouth. The captured image is uploaded to a server in real time and analyzed by an AI model. Based on the analysis results, the risk of tooth decay and the necessary treatment plan are displayed on the spot, and appropriate treatment can be immediately proposed to the patient.

[0170] Prompt Sentence Examples

[0171] A patient comes to your clinic. Use smart glasses to take intraoral photos of the patient and analyze them with the AI ​​Dental Guard system. Based on the analysis results, propose the best treatment plan for the patient.

[0172] This invention is expected to improve the efficiency of reception work and diagnostic accuracy at dental clinics, and will be a particularly useful solution for busy dentists and clinics with many patients.

[0173] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0174] Step 1: Collect and photograph patient data

[0175] The user (dental clinic staff) uses smart glasses or a head-mounted display to take intraoral images of the patient. At this time, the patient's basic information is also entered. The entered data (patient's basic information and intraoral images) is saved on the terminal. The terminal then sends this image data to the server. The input is the intraoral images and patient information, and the output is data transmission to the server. Specifically, the user takes several images using the terminal's camera, and then manually selects and uploads them.

[0176] Step 2: Preprocessing the image data

[0177] The server preprocesses the received intraoral image data. Specifically, it uses the OpenCV library to perform image processing such as noise removal, brightness adjustment, and cropping. The input is the received raw image data, and the output is preprocessed image data. Specifically, the server automatically performs image filtering and correction, and then converts the data into a format suitable for the AI ​​model.

[0178] Step 3: Analysis by AI model

[0179] The server inputs the preprocessed image data into the generated AI model. The generated AI model analyzes the image data and determines health conditions such as caries risk, periodontal disease risk, staining level, and wear status. The input is the preprocessed image data, and the output is the health condition assessment result. Specifically, the AI ​​model analyzes the characteristics of each pixel and detects patterns of risk factors.

[0180] Step 4: Generate a diagnosis and treatment plan

[0181] The server automatically generates a health status report and improvement suggestions based on the analysis results of the AI ​​model. It also generates a specific treatment plan. The input is the analysis results, and the output is an automatically generated report and treatment plan. Specifically, it uses templates to create a report and adds a treatment plan tailored to each patient's situation.

[0182] Step 5: Notification and display of results

[0183] The server sends the generated report and treatment plan to the user's device. The user (dentist or staff member) receives a notification on their device and reviews the generated report and recommendations. The input is the generated report and treatment plan, and the output is the information displayed on the user's device. Specifically, the application on the device displays a pop-up notification and takes the user to a screen where detailed results can be viewed.

[0184] Step 6: Handling the results

[0185] The user (dentist or staff) checks the notification on the user terminal and proposes the optimal treatment method for the patient based on the generated report and treatment plan. The input is the diagnosis results and treatment plan displayed on the terminal, and the output is a treatment proposal to the patient. In concrete terms, the dentist explains the treatment contents to the patient and sets a treatment schedule if necessary.

[0186] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0187] As one embodiment of the present invention, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home, receive professional diagnosis as needed, and recognizes the user's emotions to provide appropriate suggestions and advice for improvement. The system program processing is explained below in natural language.

[0188] User takes and uploads a photo

[0189] Users launch the AI ​​Dental guard app on their smartphone and use the camera to take a photo of their teeth. After taking the photo, they press the "Start Analysis" button in the app to upload the captured data to the server. For example, users can take photos of the front and back teeth with their smartphone and send the images through the app.

[0190] Upload the image to the server

[0191] The device (user's smartphone) sends the captured image data to the server. An internet connection is required for transmission. For example, the user can use Wi-Fi or mobile data to upload the captured image data to the server via a secure network.

[0192] The image analysis server performs the analysis

[0193] The server receives and stores the image data sent from the device. The received image is preprocessed and input into the AI ​​model. The AI ​​model analyzes the image data and determines health conditions such as tooth decay, risk of periodontal disease, stains, discoloration, and tooth wear. For example, the AI ​​model may detect early signs of tooth decay in the user's upper right molar.

[0194] Save the analysis results in a database

[0195] The server associates the analysis results with the user ID and saves them in a database. This allows each user's health history to be accumulated and used to improve the accuracy of future analyses and advice. For example, the server may save the results in the database as "User A's dental check results for October 2023."

[0196] Notify users of results and send suggestions for improvement

[0197] Based on the analysis, the server automatically generates a health report and recommendations for improvement, which are sent to the user's app. For example, the report might include a warning about a high risk of cavities in the upper right molar, along with recommendations for specific brushing and flossing techniques.

[0198] Emotion engine recognizes user emotions

[0199] When a user checks the results, the emotion engine uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to recognize their current emotional state. For example, the emotion engine can detect if the user is surprised or anxious about the results.

[0200] Adjusting improvement suggestions with emotion engine

[0201] The emotion engine recognizes the user's emotions and tailors the suggestions to them accordingly. For example, if the user is feeling anxious, the engine generates a suggestion that includes a gentle, encouraging message or supporting information to recommend a visit to the dentist.

[0202] If you would like a professional diagnosis, please send us your images.

[0203] If a user requests a more detailed diagnosis, they can use another means of communication, such as the LINE official account, to send the image to a specialist and wait for a reply from the specialist. For example, a user can use the LINE app to send a photo they have taken to a dentist and request a more detailed diagnosis.

[0204] The expert will send the diagnosis to the user.

[0205] The expert will analyze the images and return a diagnosis and specific advice to the user, allowing the user to obtain detailed health information. For example, a dentist may diagnose the image and recommend appropriate treatment or preventative measures.

[0206] The above system supports users in dental care by conveniently checking their dental health at home and providing appropriate responses based on their emotions. It also allows users to easily receive a diagnosis from a specialist, enabling timely treatment and countermeasures.

[0207] The processing flow will be explained below.

[0208] Step 1:

[0209] The user launches the app.

[0210] The user taps and launches the AI ​​Dental guard app on their smartphone.

[0211] Step 2:

[0212] The user takes a photo of their teeth.

[0213] Users use the app's camera function to take a photo of their teeth with their smartphone camera, and are encouraged to clearly capture both the front and back teeth.

[0214] Step 3:

[0215] The user checks the shooting data.

[0216] Users can review the photos they've taken, take new ones if necessary, and once they're satisfied, press the "Start Analysis" button within the app.

[0217] Step 4:

[0218] The device sends the captured image data to the server.

[0219] The image data captured by the device is uploaded to a dedicated server via an internet connection, using Wi-Fi or mobile data.

[0220] Step 5:

[0221] The server receives the image data.

[0222] The server receives the image data sent from the terminal and stores it in a secure storage.

[0223] Step 6:

[0224] The server preprocesses the received image data.

[0225] The server performs preprocessing on the received image data, such as formatting, resizing, and normalization. This preprocessing converts the image data into a format suitable for the AI ​​model.

[0226] Step 7:

[0227] The server inputs preprocessed image data into the AI ​​model.

[0228] The server inputs the preprocessed image data into the AI ​​model and prepares it for analysis.

[0229] Step 8:

[0230] The AI ​​model analyzes the image data.

[0231] The AI ​​model analyzes the image data to determine health conditions such as risk of cavities, periodontal disease, stains, discoloration, and tooth wear.

[0232] Step 9:

[0233] The server generates the analysis results.

[0234] The server generates a health report based on the analysis results obtained from the AI ​​model, which includes a description of the health condition and individualized suggestions for improvement.

[0235] Step 10:

[0236] The server stores the analysis results in a database.

[0237] The analysis results generated by the server are stored in a database along with the user ID, allowing each user's health history to be accumulated.

[0238] Step 11:

[0239] The server sends the analysis results and improvement suggestions.

[0240] The server generates a health report and sends it to the user's app, which displays the user's health status and appropriate actions.

[0241] Step 12:

[0242] The user checks the results in the app.

[0243] Users receive in-app notifications and receive health status reports and suggestions for improvement.

[0244] Step 13:

[0245] The emotion engine recognizes the user's emotions.

[0246] When the user checks the results, the emotion engine uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to recognize their current emotional state. For example, the emotion engine can detect if the user is surprised or anxious about the results.

[0247] Step 14:

[0248] Adjustment of improvement suggestions by emotion engine.

[0249] The emotion engine recognizes the user's emotions and tailors the suggestions to them accordingly. For example, if the user is feeling anxious, the engine generates a suggestion that includes a gentle, encouraging message or supporting information to recommend a visit to the dentist.

[0250] Step 15:

[0251] If the user wishes to have a specialist diagnose the image, they can submit it.

[0252] If a user requests a more detailed diagnosis, they can use another means of communication, such as the LINE official account, to send the image to a specialist and wait for a reply from the specialist. For example, a user can use the LINE app to send a photo they have taken to a dentist and request a more detailed diagnosis.

[0253] Step 16:

[0254] The expert will send the diagnosis results to the user.

[0255] The expert will analyze the images and return a diagnosis and specific advice to the user, allowing the user to obtain detailed health information. For example, a dentist may diagnose the image and recommend appropriate treatment or preventative measures.

[0256] Example 2

[0257] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0258] Until now, users have had limited means to monitor their dental health at home, making it difficult to easily receive detailed analyses of their health or receive professional diagnoses. Furthermore, there were no systems that took into account the user's emotional state when reporting their health status or providing recommendations for improvement. This meant users were unable to take care of their teeth with peace of mind and found it difficult to receive professional diagnoses at the appropriate time.

[0259] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data of the oral cavity captured by an image capturing device, a means for preprocessing the received image data of the oral cavity and inputting it into a generative AI model, a means for analyzing the image data of the oral cavity using the generative AI model and determining the health status, a means for automatically generating a health status report and improvement suggestions for the user based on the determination results, a means for transmitting the generated report and improvement suggestions to a user terminal, and a means for recognizing the user's emotional state and adjusting the content of the improvement suggestions according to the emotion. This allows the user to easily check the health status of their teeth at home and receive appropriate improvement suggestions according to their emotion. Furthermore, if a detailed diagnosis by a specialist is desired, a prompt response is also possible.

[0260] An "imaging device" is a device that a user uses to take images of the inside of the oral cavity.

[0261] "Image data" refers to digital data of an image of the inside of the oral cavity captured by an image capturing device.

[0262] The "receiving means" is a communication means for transferring image data from the user terminal to the server.

[0263] "Preprocessing" refers to data processing, such as removing noise from image data and correcting resolution, to enable accurate analysis by the AI ​​model.

[0264] A "generative AI model" is an artificial intelligence (AI) model used to analyze image data from inside the oral cavity and determine health status.

[0265] "Analysis" refers to the generative AI model analyzing image data of the inside of the mouth to determine the health of the teeth.

[0266] A "health report" is a report containing information describing a user's dental health based on the analysis results.

[0267] "Improvement suggestions" are recommendations on how to brush and care for your teeth based on the user's health condition.

[0268] A "user device" is an electronic device used by a user, such as a smartphone or tablet.

[0269] "Emotional state" refers to the user's current emotional state, as detected from facial expressions, tone of voice, etc.

[0270] An "emotion engine" is an artificial intelligence system used to analyze a user's emotional state.

[0271] "Means of communication" refers to methods of data transfer, including internet connection and messaging services such as LINE.

[0272] "Professionals" are medical professionals with professional qualifications, such as dentists and dental hygienists.

[0273] A "database" is an information system for storing health data and analysis results for each user.

[0274] As one embodiment of this invention, we provide the "AI Dental Guard System," which allows users to easily monitor their dental health at home and receive a diagnosis from a specialist if necessary. This system recognizes the user's emotional state and provides appropriate improvement suggestions and advice according to their emotions.

[0275] First, the user launches the dedicated app on their smartphone and uses the app's camera function to take an image of the inside of their mouth. Next, the captured image data is uploaded to the server by pressing the "Start Analysis" button within the app. The server saves the received image data and performs preprocessing such as noise removal.

[0276] The preprocessed image data is input into a generative AI model, which analyzes the input image data and determines the health status of the user based on factors such as the risk of cavities and periodontal disease, stains, discoloration, and tooth wear. The results of the assessment are associated with the user's ID and stored in a database.

[0277] The server automatically generates a report of the user's dental health based on the analysis results, along with suggestions for improvement. This report and suggestions are sent to the user's smartphone app. For example, it could warn the user that they are at high risk for cavities in their upper right molar, and include suggestions for specific brushing and flossing techniques.

[0278] When the user checks the results, the smartphone's camera and microphone are activated, and the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotional state. Based on the recognized emotional state, the server adjusts the content of the improvement suggestions. For example, if the user is feeling anxious, the server will provide suggestions that include a gentle, encouraging message or support information recommending a visit to the dentist.

[0279] Additionally, if a user desires a more detailed diagnosis, they can send the captured image to a specialist using other communication methods within the dedicated app (e.g., the official LINE account). The specialist will then respond to the user with a detailed diagnosis based on the image. For example, a dentist may analyze the captured image and advise, "There are signs of early tooth decay in the upper right molar, so immediate treatment is required."

[0280] This system allows users to easily check the health of their teeth at home and receive appropriate suggestions for improvement. Furthermore, by receiving a detailed diagnosis from a specialist promptly, timely treatment and measures can be taken.

[0281] An example prompt might look like this:

[0282] Please provide a detailed description of the system where users upload photos of their teeth taken with the AI ​​Dental guard app to the server.

[0283] In this way, the invention is designed to make users' lives more comfortable and support their health management.

[0284] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0285] Step 1:

[0286] The user launches the "AI Dental guard" app. The user uses the camera function of their smartphone to take images of the inside of their mouth. For example, the user takes photos of their front and back teeth. The input is the image data taken by the user, and the output is the captured image data.

[0287] Step 2:

[0288] The user presses the "Start Analysis" button to upload the captured image data to the server. Specifically, the app uses an internet connection to send the image data to the server. The input is the captured image data, and the output is the image data sent to the server.

[0289] Step 3:

[0290] The server receives image data sent from the terminal and stores it in cloud storage. The input is the sent image data, and the output is the stored image data. Specifically, the server stores the image data in secure storage.

[0291] Step 4:

[0292] The server performs preprocessing such as noise removal and resolution correction on the image data. The input is the stored image data, and the output is the preprocessed image data. Specifically, the server applies noise filtering and resolution correction algorithms.

[0293] Step 5:

[0294] The server inputs the preprocessed data into the generative AI model. The input is the preprocessed image data, and the output is the analysis results. Specifically, the generative AI model analyzes the image data and determines health conditions such as the risk of cavities and periodontal disease, stains, discoloration, and tooth wear.

[0295] Step 6:

[0296] The server associates the analysis results with the user ID and stores them in a database. The input is the analysis results and the user ID, and the output is the associated data. Specifically, the server organizes the analysis results by user and stores them in a database for future reference.

[0297] Step 7:

[0298] The server automatically generates a report of the user's dental health status based on the analysis results. It also automatically generates improvement suggestions. The input is the analysis results, and the output is the generated report and improvement suggestions. Specifically, the server uses an automatic generation algorithm to create the report and improvement suggestions.

[0299] Step 8:

[0300] The server sends the generated report and improvement suggestions to the user's app. The input is the generated report and improvement suggestions, and the output is the data sent to the user's device. Specifically, the server sends the data using an Internet connection.

[0301] Step 9:

[0302] When the user wants to check the results, the smartphone's camera and microphone are activated. The emotion engine analyzes the user's facial expressions and tone of voice to recognize their current emotional state. The input is data from the camera and microphone, and the output is the recognized emotional state. Specifically, the emotion engine uses video and audio analysis algorithms.

[0303] Step 10:

[0304] Based on the analysis results of the emotion engine, the server adjusts the content of the improvement proposal. The input is the emotional state and the initial improvement proposal, and the output is the adjusted improvement proposal. Specifically, the server adds or modifies messages and proposals according to the user's emotions.

[0305] Step 11:

[0306] If the user wishes to receive a more detailed diagnosis, they can select another communication method, such as the LINE official account, from within the app and send the captured image to an expert. The input is the captured image data and a request to send it to the expert, and the output is the image data sent to the expert. Specifically, the user sends the data to the expert using an app such as LINE.

[0307] Step 12:

[0308] The expert analyzes the sent images and returns the diagnosis and specific advice to the user. The input is the sent image data, and the output is the expert's diagnosis and advice. Specifically, the expert diagnoses based on the images and notifies the user of appropriate treatment methods and preventive measures.

[0309] In this way, the AI ​​Dental Guard system allows users to easily monitor their dental health at home and receive professional diagnosis, and supports users' dental care by providing appropriate improvement suggestions based on their emotions.

[0310] (Application example 2)

[0311] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0312] Conventional oral examination systems have the drawback of making it difficult for users to easily monitor their dental health at home, and they are unable to receive appropriate advice or suggestions for improvement based on their emotions. Furthermore, they lack the means to alleviate users' anxiety, making it difficult to receive a detailed diagnosis from a specialist promptly. This makes it difficult to detect dental problems early and take appropriate measures, potentially hindering users' dental health maintenance.

[0313] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0314] In this invention, the server includes means for receiving image data of the oral cavity captured by an imaging device, means for preprocessing the received image data of the oral cavity and inputting it into an AI model, means for analyzing the image data of the oral cavity using the AI ​​model to assess the health status, means for automatically generating a health status report and improvement suggestions for the user based on the assessment results, means for transmitting the generated report and improvement suggestions to a user terminal, and means for recognizing the user's emotions and adjusting the improvement suggestions according to the emotions. This allows users to easily monitor their dental health at home and receive appropriate improvement suggestions according to their emotions. It also enables prompt and detailed diagnosis by experts, reducing user anxiety and enabling early detection of dental problems and appropriate measures.

[0315] (Definitions of important words)

[0316] "Imaging device" refers to an electronic device used to take images of the inside of the oral cavity.

[0317] "Intraoral image data" refers to digital data that digitizes visual information about the oral cavity obtained by an image capturing device.

[0318] "Means for receiving" refers to a mechanism that allows a server to acquire data sent from a user terminal via a network.

[0319] "Preprocessing" refers to the process of converting and processing image data into an analyzable format before inputting it into an AI model.

[0320] "AI model" refers to a model that includes neural networks and machine learning algorithms built on the foundation of artificial intelligence technology.

[0321] "Means for determining health status" refers to the system that evaluates the health status of teeth and the oral cavity from the results of analysis by the AI ​​model and derives the results.

[0322] "Determination result" refers to the conclusion regarding health status output after the AI ​​model analyzes image data.

[0323] "Means for automatically generating improvement proposals" refers to a system that has the function of automatically generating improvement measures and advice appropriate for users based on the assessment results.

[0324] "User device" refers to electronic devices such as smartphones and tablets used by users.

[0325] "Means of recognizing emotions" refers to technology that analyzes the user's emotions and determines the appropriate response based on that.

[0326] "Communication means" refers to the methods and technologies used to send and receive data, including the Internet, mobile networks, and dedicated applications.

[0327] "Expert" refers to a medical professional with specialized knowledge and skills to diagnose oral health conditions and abnormalities.

[0328] MODE FOR CARRYING OUT THE INVENTION

[0329] The present invention is a system that allows users to easily monitor their dental health at home. The system recognizes the user's emotions and provides corresponding suggestions for improvement. It also allows users to receive detailed diagnostics from a specialist if necessary.

[0330] System configuration

[0331] The main components of the system are:

[0332] Image capture device (smartphone, etc.)

[0333] server

[0334] AI model

[0335] Emotion Recognition Engine

[0336] User devices (smartphones, tablets)

[0337] Program processing

[0338] 1. Image capture and data reception

[0339] Users take pictures of the inside of their mouths using the camera function on their smartphones, and the image data they take is uploaded to a server via the Internet, using Wi-Fi or mobile data communication.

[0340] 2. Preprocessing and input to the AI ​​model

[0341] The server preprocesses the received image data and converts it into a format suitable for the AI ​​model, including image resizing and normalization.

[0342] 3. Health status analysis

[0343] The server inputs the preprocessed image data into an AI model to analyze the health condition, which detects abnormalities in the teeth and oral cavity and assesses health risks such as cavities, periodontal disease, and stains.

[0344] 4. Emotional Recognition

[0345] The emotion recognition engine uses the smartphone's camera and microphone to analyze the user's emotions when they check the results, analyzing their facial expressions and tone of voice to recognize their current emotional state.

[0346] 5. Generate reports and improvement suggestions

[0347] Based on the analysis results and emotion recognition results, the server automatically generates a health status report for the user and improvement suggestions according to their emotions, including specific tooth brushing methods and lifestyle improvements.

[0348] 6. Notifications to User Devices

[0349] The generated reports and improvement suggestions are sent to the user's smartphone or tablet, where they can view the information through the app.

[0350] 7. Detailed diagnosis by an expert

[0351] If the user wishes to receive a more detailed diagnosis, the captured images of the oral cavity can be sent to a specialist via another communication method. The specialist will analyze the images and provide the user with a diagnosis and specific advice.

[0352] Specific examples

[0353] For example, if a user notices something wrong with their teeth, they can take a picture of their teeth using their smartphone and upload it to the server through the app. The AI ​​model then analyzes the image and detects signs of tooth decay. The emotion recognition engine recognizes that the user is feeling anxious and generates a gentle message saying, "You are at risk of tooth decay. Don't worry, we'll support you. We recommend that you visit the dentist as soon as possible." This suggestion is immediately sent to the user's smartphone.

[0354] Prompt Sentence Examples

[0355] Below are some examples of specific prompts that can be fed into a generative AI model:

[0356] Implement an API that allows users to upload photos taken with their smartphones to check their dental health at home. The API will analyze the dental health status using an AI model and return a warning message if there is an abnormality. It will also analyze the user's emotions using an emotion recognition engine and customize improvement suggestions based on the results. The output should be in JSON format, containing the health status and suggestions.

[0357] In this way, by utilizing specific prompt sentences, generative AI models can be used efficiently and system development can proceed smoothly.

[0358] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0359] Program processing steps

[0360] Step 1:

[0361] Image capture and data upload

[0362] The user takes a photograph of the inside of the mouth using a smartphone. This image data is uploaded from the smartphone to the server. The user presses the "Start Analysis" button in the app, and the device sends the image data to the server over a secure network. The input is the image data of the inside of the mouth that was photographed, and the output is the image data sent to the server.

[0363] Step 2:

[0364] Image data preprocessing

[0365] The server receives the received image data and performs preprocessing. Specifically, it resizes, normalizes, and removes noise if necessary. The preprocessed image data is converted into a format suitable for the AI ​​model. The input is the raw image data uploaded to the server, and the output is image data converted into a format that can be input into the AI ​​model.

[0366] Step 3:

[0367] Health status analysis using AI models

[0368] The server inputs the preprocessed image data into the AI ​​model and performs the analysis. The AI ​​model evaluates health risks such as cavities, periodontal disease, stains, discoloration, and tooth wear. The input is the preprocessed image data, and the output is a judgment result regarding the health status. For example, the AI ​​model may detect signs of early cavities from the image.

[0369] Step 4:

[0370] Emotion recognition

[0371] When a user checks the results of the AI ​​analysis, the emotion recognition engine uses the device's camera and microphone to analyze the user's facial expressions and tone of voice. This allows the system to recognize the user's current emotional state. The input is facial expression and voice data acquired from the smartphone, and the output is the user's emotional state. For example, the emotion engine can detect that the user is surprised by the results.

[0372] Step 5:

[0373] Generate reports and improvement suggestions

[0374] The server automatically generates a report and improvement suggestions based on the AI's health assessment results and the emotional state obtained from the emotion recognition engine. This includes advice on proper tooth brushing methods and lifestyle habits. The input is the AI ​​assessment results and the user's emotional state, and the output is a customized improvement suggestion message. For example, it might say, "Incipient tooth decay has been detected. Don't worry. We recommend that you visit a dentist as soon as possible."

[0375] Step 6:

[0376] Notifications on user devices

[0377] The reports and improvement suggestions generated by the server are sent to the user's smartphone, where the user can check the information through the app. The input is the generated report and suggestion data, and the output is the report and suggestion message displayed on the user's device.

[0378] Step 7:

[0379] Detailed diagnosis by an expert

[0380] If the user requests a detailed diagnosis, the image data captured within the app is sent to a specialist via another means of communication (e.g., a messaging app). The specialist analyzes the data and provides the user with a diagnosis and specific advice. The input is the image data of the inside of the oral cavity, and the output is a detailed diagnosis and advice from the specialist.

[0381] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0382] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0383] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0384] [Second embodiment]

[0385] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0386] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0387] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0388] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0389] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0390] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0391] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0392] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0393] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0394] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0395] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0396] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0397] As one embodiment of the present invention, the AI ​​Dental Guard system is a system that allows users to easily and accurately monitor their dental health at home and receive suggestions for improvement. The processing of the system program is explained below in natural language.

[0398] User takes and uploads a photo

[0399] Users launch the AI ​​Dental guard app on their smartphone and use the camera to take a photo of their teeth. After taking the photo, they press the "Start Analysis" button in the app to upload the captured data to the server. For example, users can take photos of the front and back teeth with their smartphone and send the images through the app.

[0400] Upload the image to the server

[0401] The device (user's smartphone) takes a photo and sends the image data to the server. This requires an internet connection. For example, the user can use Wi-Fi or mobile data to upload the image data to the server over a secure network.

[0402] The image analysis server performs the analysis

[0403] The server receives and stores image data sent from the device. The received images are preprocessed and input into the AI ​​model. The AI ​​model analyzes the image data to determine health conditions such as cavities, risk of periodontal disease, stains, discoloration, and tooth wear. For example, the server analyzes an image uploaded by a user and detects signs of early cavities in the upper right molar.

[0404] Save the analysis results in a database

[0405] The server associates the analysis results with the user ID and saves them in a database. This allows each user's health history to be accumulated, which will be useful for future analysis and advice. For example, the server saves the results in the database as "User A's dental check results for October 2023."

[0406] Notify users of results and send suggestions for improvement

[0407] Based on the analysis, the server automatically generates a health report and recommendations for improvement, which are sent to the user's app. For example, the report might include a warning about a high risk of cavities in the upper right molar, along with recommendations for specific brushing and flossing techniques.

[0408] The user checks the results

[0409] Users will receive a notification within the app and can view the generated report and improvement suggestions. If necessary, they can send images via messaging services such as LINE for expert diagnosis. For example, a user may view a warning in the app, request a more detailed diagnosis, and send images to the LINE official account.

[0410] Each step can be easily performed at home, allowing for daily monitoring of dental health and early diagnosis and prevention without visiting a dental clinic. This system is extremely beneficial for people who lead busy lives or who are anxious about visiting the dentist.

[0411] The processing flow will be explained below.

[0412] Step 1:

[0413] The user launches the app.

[0414] The user taps and launches the AI ​​Dental guard app on their smartphone.

[0415] Step 2:

[0416] The user takes a photo of their teeth.

[0417] Users use the app's camera function to take a photo of their teeth with their smartphone camera, and are encouraged to clearly capture both the front and back teeth.

[0418] Step 3:

[0419] The user checks the shooting data.

[0420] Users can review the photos they've taken, take new ones if necessary, and once they're satisfied, press the "Start Analysis" button within the app.

[0421] Step 4:

[0422] The device sends the captured image data to the server.

[0423] The image data captured by the device is uploaded to a dedicated server via an internet connection, using Wi-Fi or mobile data.

[0424] Step 5:

[0425] The server receives the image data.

[0426] The server receives the image data sent from the terminal and stores it in a secure storage.

[0427] Step 6:

[0428] The server preprocesses the received image data.

[0429] The server performs preprocessing on the received image data, such as formatting, resizing, and normalization. This preprocessing converts the image data into a format suitable for the AI ​​model.

[0430] Step 7:

[0431] The server inputs preprocessed image data into the AI ​​model.

[0432] The server inputs the preprocessed image data into the AI ​​model and prepares it for analysis.

[0433] Step 8:

[0434] The AI ​​model analyzes the image data.

[0435] The AI ​​model analyzes the image data to determine health conditions such as risk of cavities, periodontal disease, stains, discoloration, and tooth wear.

[0436] Step 9:

[0437] The server generates the analysis results.

[0438] The server generates a health report based on the analysis results obtained from the AI ​​model, which includes a description of the health condition and individualized suggestions for improvement.

[0439] Step 10:

[0440] The server stores the analysis results in a database.

[0441] The analysis results generated by the server are stored in a database along with the user ID, allowing each user's health history to be accumulated.

[0442] Step 11:

[0443] The server sends the analysis results and improvement suggestions.

[0444] The server generates a health report and sends it to the user's app, which displays the user's health status and appropriate actions.

[0445] Step 12:

[0446] The user checks the results in the app.

[0447] Users receive in-app notifications and receive health status reports and suggestions for improvement.

[0448] Step 13:

[0449] If the user wishes to have a specialist diagnose the image, they can submit it.

[0450] If the user wishes to receive a more detailed diagnosis, they can send the image to a specialist from within the app using another means of communication, such as the official LINE account, and wait for a reply from the specialist.

[0451] Step 14:

[0452] The expert will send the diagnosis results to the user.

[0453] The expert will analyze the images sent and return the diagnosis and specific advice to the user, allowing the user to obtain more detailed health information.

[0454] Example 1

[0455] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0456] In modern society, many people lead busy lives and often neglect regular dental checkups. As a result, dental health is more likely to deteriorate and early detection becomes more difficult. Furthermore, for people who have anxiety about visiting the dentist, regular visits can be a psychological burden. Given this background, there is a need for a system that can easily and accurately monitor dental health at home and provide guidelines for when to seek professional diagnosis.

[0457] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0458] In this invention, the server includes means for receiving image data of the inside of the oral cavity captured by an image capturing device, means for preprocessing the received image data of the inside of the oral cavity and inputting it into a generative AI model, means for sending prompts to the generative AI model, analyzing the image data of the inside of the oral cavity, and determining the health condition, means for automatically generating a health condition report and improvement suggestions for the user based on the determination results, and means for sending the generated report and improvement suggestions to a user terminal. This enables users to easily and accurately monitor their dental health condition at home, detect problems early, and take appropriate measures.

[0459] An "imaging device" is a device for taking images of the inside of the oral cavity, and includes smartphones, digital cameras, etc.

[0460] "Image data" refers to a digital representation of visual information generated by an image capture device.

[0461] "Means for receiving" refers to a device or software capable of receiving image data into a server or other system component.

[0462] "Preprocessing" refers to a series of processes performed on received image data to facilitate analysis, and specific examples include adjusting resolution and removing noise.

[0463] A "generative AI model" refers to a model that uses artificial intelligence techniques to analyze data and is trained to perform a specific task.

[0464] A "prompt" is a textual instruction that instructs a generative AI model to perform a specific analytical task.

[0465] "Means for analysis" refers to a device or software that has the function of inputting image data into a generative AI model, analyzing the data, and determining health status.

[0466] "Determination results" refer to information about the health status of the oral cavity obtained based on analysis by the generative AI model.

[0467] "Means for automatically generating reports and improvement suggestions" refers to a device or software that has the function of explaining the user's health condition and suggesting ways to improve it based on the assessment results.

[0468] "User terminal" refers to a device used by a user to receive reports and improvement suggestions, such as a computer or smartphone.

[0469] As one embodiment of the present invention, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home and receive improvement suggestions. This system is composed of an image capture device, a generative AI model, and a user terminal.

[0470] First, a user launches the AI ​​Dental guard app on their smartphone (for example, an iPhone or Android device) and takes a photo of their teeth using the app's camera function. The captured image data is then uploaded from the smartphone to a server via the internet. Uploading is done using a Wi-Fi or mobile data connection. For example, a user can take a photo of the front and back teeth with their smartphone and send the image through the app.

[0471] The server receives the image data sent from the device and temporarily stores it in a database. The received image data is preprocessed (for example, by adjusting the resolution or removing noise) and then input into the generative AI model. Image processing software (for example, OpenCV) is used for preprocessing.

[0472] The generative AI model uses deep learning frameworks such as TensorFlow and PyTorch to analyze image data by receiving prompts such as the following:

[0473] "Analyze this image to determine the user's dental health (cavities, risk of periodontal disease, stains, discoloration, tooth wear)."

[0474] This allows the AI ​​model to analyze image data and determine health conditions such as risk of cavities and periodontal disease, dirt, staining, and tooth wear.

[0475] The server receives the analysis results obtained from the generative AI model, associates them with the user ID, and stores them in a database. This database accumulates health data for each user and is used to improve the accuracy of future analyses and improvement suggestions.

[0476] Based on the analysis results, the server automatically generates a health status report and improvement suggestions. These reports and suggestions are sent to the user's device, and the user receives notifications within the app and can review the generated report and improvement suggestions. For example, a warning indicating a high risk of cavities in the upper right molar and suggestions for specific tooth brushing and flossing techniques may be displayed.

[0477] If necessary, users can use the app's functions to receive further professional advice. For example, users can use messaging services such as LINE to send images they have taken to a specialist for further diagnosis.

[0478] In this way, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home, detect problems early, and take appropriate measures. It also helps prevent dental visits and supports continuous health management.

[0479] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0480] Step 1:

[0481] The user launches the AI ​​Dental guard app on their smartphone and takes a photo of their teeth using the app's camera function. The captured image data is the input and is saved on the user's device. Next, when the user taps the "Start Analysis" button, the captured data is uploaded to the server. The output is the image data sent to the server. Specifically, when the user takes a photo of the front teeth with their iPhone and presses the "Start Analysis" button, the captured image file (e.g., JPEG) is sent to the server.

[0482] Step 2:

[0483] The device compresses the captured image data (e.g., using the JPEG compression algorithm) and sends it to a server via an Internet connection. The input is the uncompressed image data, and the output is the compressed image data. Specifically, the device uses a Wi-Fi connection to send the image data to the server via an SSL / TLS encrypted channel.

[0484] Step 3:

[0485] The server receives image data sent from the device and temporarily stores it in a database. The input is image data from the device, and the output is image data stored in the database. For example, the server stores image data in a specified format in a MySQL database.

[0486] Step 4:

[0487] The server preprocesses the received image data. Preprocessing includes adjusting the resolution and removing noise. The input is image data stored in the database, and the output is the preprocessed image data. Specifically, the server uses the OpenCV library to adjust the image resolution and remove noise.

[0488] Step 5:

[0489] The server inputs the preprocessed image data into the generative AI model and sends a prompt. The input is the preprocessed image data and the prompt, and the output is the analysis result. For example, the server sends the following prompt to the TensorFlow model: "Analyze this image to determine the user's dental health status (cavities, periodontal disease risk, stains, discoloration, and tooth wear)."

[0490] Step 6:

[0491] The generative AI model analyzes the received image data and determines health conditions such as risk of cavities and periodontal disease, stains, discoloration, and tooth wear. The input is preprocessed image data and prompt text, and the output is the analysis results. Specifically, the AI ​​model analyzes each pixel in the image, identifies specific patterns, and generates a judgment result.

[0492] Step 7:

[0493] The server receives the generated analysis results, associates them with the user ID, and stores them in a database. The input is the analysis results from the AI ​​model, and the output is the analysis results stored in the database. Specifically, "User A's dental check results for October 2023" is stored in the server's database.

[0494] Step 8:

[0495] Based on the analysis results, the server automatically generates a health status report and improvement suggestions. These reports and suggestions are sent to the user's app. The input is the analysis results, and the output is the report and suggestions sent to the user's device. Specifically, the server generates a warning such as "You have a high risk of cavities in your upper right molars" and "suggestions for specific tooth brushing methods and flossing."

[0496] Step 9:

[0497] Users receive notifications within the app and review the generated report and improvement suggestions. The input is the report and suggestions sent to the device, and the output is the user's acknowledgment. If necessary, users can use the app's functions to send images to experts. Specific actions include tapping the "Check analysis results" button in the app, viewing the detailed analysis, and then sharing the images and analysis results using the "Send to expert via LINE" button.

[0498] (Application example 1)

[0499] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0500] Traditionally, dental clinics have spent a lot of time and effort diagnosing patients' oral conditions. Furthermore, there were limited ways for patients to monitor their dental health at home, making it difficult to detect problems early. Furthermore, there was a need for efficient methods for managing patient data and proposing treatment plans.

[0501] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0502] In this invention, the server includes a means for receiving image data of the oral cavity captured by an imaging device, a means for preprocessing the received image data of the oral cavity and inputting it into a generated AI model, a means for analyzing the image data of the oral cavity using the AI ​​model to assess the user's health status, a means for automatically generating a health status report and improvement suggestions for the user based on the assessment results, a means for transmitting the generated report and improvement suggestions to a user terminal, a means for collecting patient data at the reception desk and saving it together with the image data, and a means for proposing a treatment plan based on the analysis results. This enables efficient patient care at dental clinics and rapid diagnosis and treatment plan proposals by dentists. Furthermore, patients can easily monitor their oral health status at home and obtain appropriate improvement measures.

[0503] An "imaging device" is a device used to take images of the inside of the oral cavity, and typically refers to a smartphone with a camera function or a dedicated digital camera.

[0504] The "receiving means" is a mechanism for receiving data sent from outside, and is a device that includes a server or software for receiving image data via a network connection.

[0505] "Preprocessing" refers to processing performed to make received image data easier to analyze, and typically refers to image processing such as noise removal, brightness adjustment, and cropping.

[0506] An "AI model" is a model of artificial intelligence that has been trained using machine learning algorithms to perform a specific task.

[0507] The "means of judgment" is a function that derives results based on the data being analyzed; specifically, the AI ​​model analyzes image data to evaluate health status.

[0508] "Means for automatically generating reports and improvement suggestions" refers to a function that automatically generates reports on the user's health status and methods for improving it based on the assessment results obtained by the AI ​​model.

[0509] "Means for sending to user devices" refers to a function for sending generated reports and improvement suggestions to devices used by users, and is a mechanism for sending data to smartphones and other devices via the Internet.

[0510] "Means for collecting patient data at the reception desk" refers to a function for collecting basic patient information and intraoral images at the dental clinic reception desk, and involves obtaining this information using smart glasses or a tablet.

[0511] "Means for proposing a treatment plan based on the analysis results" refers to a function that proposes specific treatment methods and procedures appropriate for the patient based on the analysis results of the AI ​​model.

[0512] One embodiment of this invention is a system in which an application called "Dental Reception Assistant" that supports reception work at dental clinics is installed on smart glasses or a head-mounted display. This system takes intraoral images of patients at the reception desk, analyzes the images using the AI ​​Dental guard system, and immediately provides diagnosis results and treatment plans.

[0513] Key Hardware and Software

[0514] The hardware used includes, for example, smart glasses (Vuzix M400) and a head-mounted display (Microsoft HoloLens) for capturing intraoral images and displaying the data, while the server hosts the high-performance image processing unit and AI models.

[0515] The software uses OpenCV as a library for image processing and AI analysis, and trained generative AI models. In addition, the HTTP protocol is used for data communication, sending and receiving image data and analysis results.

[0516] Program Overview

[0517] The system works as follows:

[0518] 1. Patient data collection and photography

[0519] Using a device (smart glasses or a head-mounted display), an image of the patient's oral cavity is taken. At the same time, basic information about the patient is also entered. The device then sends this image data to a server.

[0520] 2. Image data preprocessing

[0521] The server preprocesses the received image data, specifically performing noise removal, brightness adjustment, cropping, etc.

[0522] 3. Analysis using AI models

[0523] The pre-processed image data is then fed into a generative AI model, which analyzes the images and determines health conditions such as risk of cavities, risk of periodontal disease, staining level, and wear status.

[0524] 4. Diagnosis results and treatment plan proposal

[0525] Based on the results, the server automatically generates a health status report and recommendations for improvement, as well as specific treatment plans, which are then sent to the user's device and displayed visually.

[0526] 5. Checking the results and taking action

[0527] The user (dentist or staff) receives a notification on their device, reviews the generated report and improvement suggestions, and, if necessary, proposes an appropriate treatment plan for the patient.

[0528] Specific examples

[0529] When a patient visits the dental clinic, the dental clinic's reception staff uses smart glasses to take a photo of the patient's mouth. The captured image is uploaded to a server in real time and analyzed by an AI model. Based on the analysis results, the risk of tooth decay and the necessary treatment plan are displayed on the spot, and appropriate treatment can be immediately proposed to the patient.

[0530] Prompt Sentence Examples

[0531] A patient comes to your clinic. Use smart glasses to take intraoral photos of the patient and analyze them with the AI ​​Dental Guard system. Based on the analysis results, propose the best treatment plan for the patient.

[0532] This invention is expected to improve the efficiency of reception work and diagnostic accuracy at dental clinics, and will be a particularly useful solution for busy dentists and clinics with many patients.

[0533] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0534] Step 1: Collect and photograph patient data

[0535] The user (dental clinic staff) uses smart glasses or a head-mounted display to take intraoral images of the patient. At this time, the patient's basic information is also entered. The entered data (patient's basic information and intraoral images) is saved on the terminal. The terminal then sends this image data to the server. The input is the intraoral images and patient information, and the output is data transmission to the server. Specifically, the user takes several images using the terminal's camera, and then manually selects and uploads them.

[0536] Step 2: Preprocessing the image data

[0537] The server preprocesses the received intraoral image data. Specifically, it uses the OpenCV library to perform image processing such as noise removal, brightness adjustment, and cropping. The input is the received raw image data, and the output is preprocessed image data. Specifically, the server automatically performs image filtering and correction, and then converts the data into a format suitable for the AI ​​model.

[0538] Step 3: Analysis by AI model

[0539] The server inputs the preprocessed image data into the generated AI model. The generated AI model analyzes the image data and determines health conditions such as caries risk, periodontal disease risk, staining level, and wear status. The input is the preprocessed image data, and the output is the health condition assessment result. Specifically, the AI ​​model analyzes the characteristics of each pixel and detects patterns of risk factors.

[0540] Step 4: Generate a diagnosis and treatment plan

[0541] The server automatically generates a health status report and improvement suggestions based on the analysis results of the AI ​​model. It also generates a specific treatment plan. The input is the analysis results, and the output is an automatically generated report and treatment plan. Specifically, it uses templates to create a report and adds a treatment plan tailored to each patient's situation.

[0542] Step 5: Notification and display of results

[0543] The server sends the generated report and treatment plan to the user's device. The user (dentist or staff member) receives a notification on their device and reviews the generated report and recommendations. The input is the generated report and treatment plan, and the output is the information displayed on the user's device. Specifically, the application on the device displays a pop-up notification and takes the user to a screen where detailed results can be viewed.

[0544] Step 6: Handling the results

[0545] The user (dentist or staff) checks the notification on the user terminal and proposes the optimal treatment method for the patient based on the generated report and treatment plan. The input is the diagnosis results and treatment plan displayed on the terminal, and the output is a treatment proposal to the patient. In concrete terms, the dentist explains the treatment contents to the patient and sets a treatment schedule if necessary.

[0546] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0547] As one embodiment of the present invention, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home, receive professional diagnosis as needed, and recognizes the user's emotions to provide appropriate suggestions and advice for improvement. The system program processing is explained below in natural language.

[0548] User takes and uploads a photo

[0549] Users launch the AI ​​Dental guard app on their smartphone and use the camera to take a photo of their teeth. After taking the photo, they press the "Start Analysis" button in the app to upload the captured data to the server. For example, users can take photos of the front and back teeth with their smartphone and send the images through the app.

[0550] Upload the image to the server

[0551] The device (user's smartphone) sends the captured image data to the server. An internet connection is required for transmission. For example, the user can use Wi-Fi or mobile data to upload the captured image data to the server via a secure network.

[0552] The image analysis server performs the analysis

[0553] The server receives and stores the image data sent from the device. The received image is preprocessed and input into the AI ​​model. The AI ​​model analyzes the image data and determines health conditions such as tooth decay, risk of periodontal disease, stains, discoloration, and tooth wear. For example, the AI ​​model may detect early signs of tooth decay in the user's upper right molar.

[0554] Save the analysis results in a database

[0555] The server associates the analysis results with the user ID and saves them in a database. This allows each user's health history to be accumulated and used to improve the accuracy of future analyses and advice. For example, the server may save the results in the database as "User A's dental check results for October 2023."

[0556] Notify users of results and send suggestions for improvement

[0557] Based on the analysis, the server automatically generates a health report and recommendations for improvement, which are sent to the user's app. For example, the report might include a warning about a high risk of cavities in the upper right molar, along with recommendations for specific brushing and flossing techniques.

[0558] Emotion engine recognizes user emotions

[0559] When a user checks the results, the emotion engine uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to recognize their current emotional state. For example, the emotion engine can detect if the user is surprised or anxious about the results.

[0560] Adjusting improvement suggestions with emotion engine

[0561] The emotion engine recognizes the user's emotions and tailors the suggestions to them accordingly. For example, if the user is feeling anxious, the engine generates a suggestion that includes a gentle, encouraging message or supporting information to recommend a visit to the dentist.

[0562] If you would like a professional diagnosis, please send us your images.

[0563] If a user requests a more detailed diagnosis, they can use another means of communication, such as the LINE official account, to send the image to a specialist and wait for a reply from the specialist. For example, a user can use the LINE app to send a photo they have taken to a dentist and request a more detailed diagnosis.

[0564] The expert will send the diagnosis to the user.

[0565] The expert will analyze the images and return a diagnosis and specific advice to the user, allowing the user to obtain detailed health information. For example, a dentist may diagnose the image and recommend appropriate treatment or preventative measures.

[0566] The above system supports users in dental care by conveniently checking their dental health at home and providing appropriate responses based on their emotions. It also allows users to easily receive a diagnosis from a specialist, enabling timely treatment and countermeasures.

[0567] The processing flow will be explained below.

[0568] Step 1:

[0569] The user launches the app.

[0570] The user taps and launches the AI ​​Dental guard app on their smartphone.

[0571] Step 2:

[0572] The user takes a photo of their teeth.

[0573] Users use the app's camera function to take a photo of their teeth with their smartphone camera, and are encouraged to clearly capture both the front and back teeth.

[0574] Step 3:

[0575] The user checks the shooting data.

[0576] Users can review the photos they've taken, take new ones if necessary, and once they're satisfied, press the "Start Analysis" button within the app.

[0577] Step 4:

[0578] The device sends the captured image data to the server.

[0579] The image data captured by the device is uploaded to a dedicated server via an internet connection, using Wi-Fi or mobile data.

[0580] Step 5:

[0581] The server receives the image data.

[0582] The server receives the image data sent from the terminal and stores it in a secure storage.

[0583] Step 6:

[0584] The server preprocesses the received image data.

[0585] The server performs preprocessing on the received image data, such as formatting, resizing, and normalization. This preprocessing converts the image data into a format suitable for the AI ​​model.

[0586] Step 7:

[0587] The server inputs preprocessed image data into the AI ​​model.

[0588] The server inputs the preprocessed image data into the AI ​​model and prepares it for analysis.

[0589] Step 8:

[0590] The AI ​​model analyzes the image data.

[0591] The AI ​​model analyzes the image data to determine health conditions such as risk of cavities, periodontal disease, stains, discoloration, and tooth wear.

[0592] Step 9:

[0593] The server generates the analysis results.

[0594] The server generates a health report based on the analysis results obtained from the AI ​​model, which includes a description of the health condition and individualized suggestions for improvement.

[0595] Step 10:

[0596] The server stores the analysis results in a database.

[0597] The analysis results generated by the server are stored in a database along with the user ID, allowing each user's health history to be accumulated.

[0598] Step 11:

[0599] The server sends the analysis results and improvement suggestions.

[0600] The server generates a health report and sends it to the user's app, which displays the user's health status and appropriate actions.

[0601] Step 12:

[0602] The user checks the results in the app.

[0603] Users receive in-app notifications and receive health status reports and suggestions for improvement.

[0604] Step 13:

[0605] The emotion engine recognizes the user's emotions.

[0606] When the user checks the results, the emotion engine uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to recognize their current emotional state. For example, the emotion engine can detect if the user is surprised or anxious about the results.

[0607] Step 14:

[0608] Adjustment of improvement suggestions by emotion engine.

[0609] The emotion engine recognizes the user's emotions and tailors the suggestions to them accordingly. For example, if the user is feeling anxious, the engine generates a suggestion that includes a gentle, encouraging message or supporting information to recommend a visit to the dentist.

[0610] Step 15:

[0611] If the user wishes to have a specialist diagnose the image, they can submit it.

[0612] If a user requests a more detailed diagnosis, they can use another means of communication, such as the LINE official account, to send the image to a specialist and wait for a reply from the specialist. For example, a user can use the LINE app to send a photo they have taken to a dentist and request a more detailed diagnosis.

[0613] Step 16:

[0614] The expert will send the diagnosis results to the user.

[0615] The expert will analyze the images and return a diagnosis and specific advice to the user, allowing the user to obtain detailed health information. For example, a dentist may diagnose the image and recommend appropriate treatment or preventative measures.

[0616] Example 2

[0617] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0618] Until now, users have had limited means to monitor their dental health at home, making it difficult to easily receive detailed analyses of their health or receive professional diagnoses. Furthermore, there were no systems that took into account the user's emotional state when reporting their health status or providing recommendations for improvement. This meant users were unable to take care of their teeth with peace of mind and found it difficult to receive professional diagnoses at the appropriate time.

[0619] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data of the oral cavity captured by an image capturing device, a means for preprocessing the received image data of the oral cavity and inputting it into a generative AI model, a means for analyzing the image data of the oral cavity using the generative AI model and determining the health status, a means for automatically generating a health status report and improvement suggestions for the user based on the determination results, a means for transmitting the generated report and improvement suggestions to a user terminal, and a means for recognizing the user's emotional state and adjusting the content of the improvement suggestions according to the emotion. This allows the user to easily check the health status of their teeth at home and receive appropriate improvement suggestions according to their emotion. Furthermore, if a detailed diagnosis by a specialist is desired, a prompt response is also possible.

[0620] An "imaging device" is a device that a user uses to take images of the inside of the oral cavity.

[0621] "Image data" refers to digital data of an image of the inside of the oral cavity captured by an image capturing device.

[0622] The "receiving means" is a communication means for transferring image data from the user terminal to the server.

[0623] "Preprocessing" refers to data processing, such as removing noise from image data and correcting resolution, to enable accurate analysis by the AI ​​model.

[0624] A "generative AI model" is an artificial intelligence (AI) model used to analyze image data from inside the oral cavity and determine health status.

[0625] "Analysis" refers to the generative AI model analyzing image data of the inside of the mouth to determine the health of the teeth.

[0626] A "health report" is a report containing information describing a user's dental health based on the analysis results.

[0627] "Improvement suggestions" are recommendations on how to brush and care for your teeth based on the user's health condition.

[0628] A "user device" is an electronic device used by a user, such as a smartphone or tablet.

[0629] "Emotional state" refers to the user's current emotional state, as detected from facial expressions, tone of voice, etc.

[0630] An "emotion engine" is an artificial intelligence system used to analyze a user's emotional state.

[0631] "Means of communication" refers to methods of data transfer, including internet connection and messaging services such as LINE.

[0632] "Professionals" are medical professionals with professional qualifications, such as dentists and dental hygienists.

[0633] A "database" is an information system for storing health data and analysis results for each user.

[0634] As one embodiment of this invention, we provide the "AI Dental Guard System," which allows users to easily monitor their dental health at home and receive a diagnosis from a specialist if necessary. This system recognizes the user's emotional state and provides appropriate improvement suggestions and advice according to their emotions.

[0635] First, the user launches the dedicated app on their smartphone and uses the app's camera function to take an image of the inside of their mouth. Next, the captured image data is uploaded to the server by pressing the "Start Analysis" button within the app. The server saves the received image data and performs preprocessing such as noise removal.

[0636] The preprocessed image data is input into a generative AI model, which analyzes the input image data and determines the health status of the user based on factors such as the risk of cavities and periodontal disease, stains, discoloration, and tooth wear. The results of the assessment are associated with the user's ID and stored in a database.

[0637] The server automatically generates a report of the user's dental health based on the analysis results, along with suggestions for improvement. This report and suggestions are sent to the user's smartphone app. For example, it could warn the user that they are at high risk for cavities in their upper right molar, and include suggestions for specific brushing and flossing techniques.

[0638] When the user checks the results, the smartphone's camera and microphone are activated, and the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotional state. Based on the recognized emotional state, the server adjusts the content of the improvement suggestions. For example, if the user is feeling anxious, the server will provide suggestions that include a gentle, encouraging message or support information recommending a visit to the dentist.

[0639] Additionally, if a user desires a more detailed diagnosis, they can send the captured image to a specialist using other communication methods within the dedicated app (e.g., the official LINE account). The specialist will then respond to the user with a detailed diagnosis based on the image. For example, a dentist may analyze the captured image and advise, "There are signs of early tooth decay in the upper right molar, so immediate treatment is required."

[0640] This system allows users to easily check the health of their teeth at home and receive appropriate suggestions for improvement. Furthermore, by receiving a detailed diagnosis from a specialist promptly, timely treatment and measures can be taken.

[0641] An example prompt might look like this:

[0642] Please provide a detailed description of the system where users upload photos of their teeth taken with the AI ​​Dental guard app to the server.

[0643] In this way, the invention is designed to make users' lives more comfortable and support their health management.

[0644] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0645] Step 1:

[0646] The user launches the "AI Dental guard" app. The user uses the camera function of their smartphone to take images of the inside of their mouth. For example, the user takes photos of their front and back teeth. The input is the image data taken by the user, and the output is the captured image data.

[0647] Step 2:

[0648] The user presses the "Start Analysis" button to upload the captured image data to the server. Specifically, the app uses an internet connection to send the image data to the server. The input is the captured image data, and the output is the image data sent to the server.

[0649] Step 3:

[0650] The server receives image data sent from the terminal and stores it in cloud storage. The input is the sent image data, and the output is the stored image data. Specifically, the server stores the image data in secure storage.

[0651] Step 4:

[0652] The server performs preprocessing such as noise removal and resolution correction on the image data. The input is the stored image data, and the output is the preprocessed image data. Specifically, the server applies noise filtering and resolution correction algorithms.

[0653] Step 5:

[0654] The server inputs the preprocessed data into the generative AI model. The input is the preprocessed image data, and the output is the analysis results. Specifically, the generative AI model analyzes the image data and determines health conditions such as the risk of cavities and periodontal disease, stains, discoloration, and tooth wear.

[0655] Step 6:

[0656] The server associates the analysis results with the user ID and stores them in a database. The input is the analysis results and the user ID, and the output is the associated data. Specifically, the server organizes the analysis results by user and stores them in a database for future reference.

[0657] Step 7:

[0658] The server automatically generates a report of the user's dental health status based on the analysis results. It also automatically generates improvement suggestions. The input is the analysis results, and the output is the generated report and improvement suggestions. Specifically, the server uses an automatic generation algorithm to create the report and improvement suggestions.

[0659] Step 8:

[0660] The server sends the generated report and improvement suggestions to the user's app. The input is the generated report and improvement suggestions, and the output is the data sent to the user's device. Specifically, the server sends the data using an Internet connection.

[0661] Step 9:

[0662] When the user wants to check the results, the smartphone's camera and microphone are activated. The emotion engine analyzes the user's facial expressions and tone of voice to recognize their current emotional state. The input is data from the camera and microphone, and the output is the recognized emotional state. Specifically, the emotion engine uses video and audio analysis algorithms.

[0663] Step 10:

[0664] Based on the analysis results of the emotion engine, the server adjusts the content of the improvement proposal. The input is the emotional state and the initial improvement proposal, and the output is the adjusted improvement proposal. Specifically, the server adds or modifies messages and proposals according to the user's emotions.

[0665] Step 11:

[0666] If the user wishes to receive a more detailed diagnosis, they can select another communication method, such as the LINE official account, from within the app and send the captured image to an expert. The input is the captured image data and a request to send it to the expert, and the output is the image data sent to the expert. Specifically, the user sends the data to the expert using an app such as LINE.

[0667] Step 12:

[0668] The expert analyzes the sent images and returns the diagnosis and specific advice to the user. The input is the sent image data, and the output is the expert's diagnosis and advice. Specifically, the expert diagnoses based on the images and notifies the user of appropriate treatment methods and preventive measures.

[0669] In this way, the AI ​​Dental Guard system allows users to easily monitor their dental health at home and receive professional diagnosis, and supports users' dental care by providing appropriate improvement suggestions based on their emotions.

[0670] (Application example 2)

[0671] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0672] Conventional oral examination systems have the drawback of making it difficult for users to easily monitor their dental health at home, and they are unable to receive appropriate advice or suggestions for improvement based on their emotions. Furthermore, they lack the means to alleviate users' anxiety, making it difficult to receive a detailed diagnosis from a specialist promptly. This makes it difficult to detect dental problems early and take appropriate measures, potentially hindering users' dental health maintenance.

[0673] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0674] In this invention, the server includes means for receiving image data of the oral cavity captured by an imaging device, means for preprocessing the received image data of the oral cavity and inputting it into an AI model, means for analyzing the image data of the oral cavity using the AI ​​model to assess the health status, means for automatically generating a health status report and improvement suggestions for the user based on the assessment results, means for transmitting the generated report and improvement suggestions to a user terminal, and means for recognizing the user's emotions and adjusting the improvement suggestions according to the emotions. This allows users to easily monitor their dental health at home and receive appropriate improvement suggestions according to their emotions. It also enables prompt and detailed diagnosis by experts, reducing user anxiety and enabling early detection of dental problems and appropriate measures.

[0675] (Definitions of important words)

[0676] "Imaging device" refers to an electronic device used to take images of the inside of the oral cavity.

[0677] "Intraoral image data" refers to digital data that digitizes visual information about the oral cavity obtained by an image capturing device.

[0678] "Means for receiving" refers to a mechanism that allows a server to acquire data sent from a user terminal via a network.

[0679] "Preprocessing" refers to the process of converting and processing image data into an analyzable format before inputting it into an AI model.

[0680] "AI model" refers to a model that includes neural networks and machine learning algorithms built on the foundation of artificial intelligence technology.

[0681] "Means for determining health status" refers to the system that evaluates the health status of teeth and the oral cavity from the results of analysis by the AI ​​model and derives the results.

[0682] "Determination result" refers to the conclusion regarding health status output after the AI ​​model analyzes image data.

[0683] "Means for automatically generating improvement proposals" refers to a system that has the function of automatically generating improvement measures and advice appropriate for users based on the assessment results.

[0684] "User device" refers to electronic devices such as smartphones and tablets used by users.

[0685] "Means of recognizing emotions" refers to technology that analyzes the user's emotions and determines the appropriate response based on that.

[0686] "Communication means" refers to the methods and technologies used to send and receive data, including the Internet, mobile networks, and dedicated applications.

[0687] "Expert" refers to a medical professional with specialized knowledge and skills to diagnose oral health conditions and abnormalities.

[0688] MODE FOR CARRYING OUT THE INVENTION

[0689] The present invention is a system that allows users to easily monitor their dental health at home. The system recognizes the user's emotions and provides corresponding suggestions for improvement. It also allows users to receive detailed diagnostics from a specialist if necessary.

[0690] System configuration

[0691] The main components of the system are:

[0692] Image capture device (smartphone, etc.)

[0693] server

[0694] AI model

[0695] Emotion Recognition Engine

[0696] User devices (smartphones, tablets)

[0697] Program processing

[0698] 1. Image capture and data reception

[0699] Users take pictures of the inside of their mouths using the camera function on their smartphones, and the image data they take is uploaded to a server via the Internet, using Wi-Fi or mobile data communication.

[0700] 2. Preprocessing and input to the AI ​​model

[0701] The server preprocesses the received image data and converts it into a format suitable for the AI ​​model, including image resizing and normalization.

[0702] 3. Health status analysis

[0703] The server inputs the preprocessed image data into an AI model to analyze the health condition, which detects abnormalities in the teeth and oral cavity and assesses health risks such as cavities, periodontal disease, and stains.

[0704] 4. Emotional Recognition

[0705] The emotion recognition engine uses the smartphone's camera and microphone to analyze the user's emotions when they check the results, analyzing their facial expressions and tone of voice to recognize their current emotional state.

[0706] 5. Generate reports and improvement suggestions

[0707] Based on the analysis results and emotion recognition results, the server automatically generates a health status report for the user and improvement suggestions according to their emotions, including specific tooth brushing methods and lifestyle improvements.

[0708] 6. Notifications to User Devices

[0709] The generated reports and improvement suggestions are sent to the user's smartphone or tablet, where they can view the information through the app.

[0710] 7. Detailed diagnosis by an expert

[0711] If the user wishes to receive a more detailed diagnosis, the captured images of the oral cavity can be sent to a specialist via another communication method. The specialist will analyze the images and provide the user with a diagnosis and specific advice.

[0712] Specific examples

[0713] For example, if a user notices something wrong with their teeth, they can take a picture of their teeth using their smartphone and upload it to the server through the app. The AI ​​model then analyzes the image and detects signs of tooth decay. The emotion recognition engine recognizes that the user is feeling anxious and generates a gentle message saying, "You are at risk of tooth decay. Don't worry, we'll support you. We recommend that you visit the dentist as soon as possible." This suggestion is immediately sent to the user's smartphone.

[0714] Prompt Sentence Examples

[0715] Below are some examples of specific prompts that can be fed into a generative AI model:

[0716] Implement an API that allows users to upload photos taken with their smartphones to check their dental health at home. The API will analyze the dental health status using an AI model and return a warning message if there is an abnormality. It will also analyze the user's emotions using an emotion recognition engine and customize improvement suggestions based on the results. The output should be in JSON format, containing the health status and suggestions.

[0717] In this way, by utilizing specific prompt sentences, generative AI models can be used efficiently and system development can proceed smoothly.

[0718] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0719] Program processing steps

[0720] Step 1:

[0721] Image capture and data upload

[0722] The user takes a photograph of the inside of the mouth using a smartphone. This image data is uploaded from the smartphone to the server. The user presses the "Start Analysis" button in the app, and the device sends the image data to the server over a secure network. The input is the image data of the inside of the mouth that was photographed, and the output is the image data sent to the server.

[0723] Step 2:

[0724] Image data preprocessing

[0725] The server receives the received image data and performs preprocessing. Specifically, it resizes, normalizes, and removes noise if necessary. The preprocessed image data is converted into a format suitable for the AI ​​model. The input is the raw image data uploaded to the server, and the output is image data converted into a format that can be input into the AI ​​model.

[0726] Step 3:

[0727] Health status analysis using AI models

[0728] The server inputs the preprocessed image data into the AI ​​model and performs the analysis. The AI ​​model evaluates health risks such as cavities, periodontal disease, stains, discoloration, and tooth wear. The input is the preprocessed image data, and the output is a judgment result regarding the health status. For example, the AI ​​model may detect signs of early cavities from the image.

[0729] Step 4:

[0730] Emotion recognition

[0731] When a user checks the results of the AI ​​analysis, the emotion recognition engine uses the device's camera and microphone to analyze the user's facial expressions and tone of voice. This allows the system to recognize the user's current emotional state. The input is facial expression and voice data acquired from the smartphone, and the output is the user's emotional state. For example, the emotion engine can detect that the user is surprised by the results.

[0732] Step 5:

[0733] Generate reports and improvement suggestions

[0734] The server automatically generates a report and improvement suggestions based on the AI's health assessment results and the emotional state obtained from the emotion recognition engine. This includes advice on proper tooth brushing methods and lifestyle habits. The input is the AI ​​assessment results and the user's emotional state, and the output is a customized improvement suggestion message. For example, it might say, "Incipient tooth decay has been detected. Don't worry. We recommend that you visit a dentist as soon as possible."

[0735] Step 6:

[0736] Notifications on user devices

[0737] The reports and improvement suggestions generated by the server are sent to the user's smartphone, where the user can check the information through the app. The input is the generated report and suggestion data, and the output is the report and suggestion message displayed on the user's device.

[0738] Step 7:

[0739] Detailed diagnosis by an expert

[0740] If the user requests a detailed diagnosis, the image data captured within the app is sent to a specialist via another means of communication (e.g., a messaging app). The specialist analyzes the data and provides the user with a diagnosis and specific advice. The input is the image data of the inside of the oral cavity, and the output is a detailed diagnosis and advice from the specialist.

[0741] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0742] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0743] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0744] [Third embodiment]

[0745] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0746] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0747] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0748] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0749] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0750] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0751] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0752] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0753] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0754] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0755] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0756] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0757] As one embodiment of the present invention, the AI ​​Dental Guard system is a system that allows users to easily and accurately monitor their dental health at home and receive suggestions for improvement. The processing of the system program is explained below in natural language.

[0758] User takes and uploads a photo

[0759] Users launch the AI ​​Dental guard app on their smartphone and use the camera to take a photo of their teeth. After taking the photo, they press the "Start Analysis" button in the app to upload the captured data to the server. For example, users can take photos of the front and back teeth with their smartphone and send the images through the app.

[0760] Upload the image to the server

[0761] The device (user's smartphone) sends the captured image data to the server. An internet connection is required for transmission. For example, the user can use Wi-Fi or mobile data to upload the captured image data to the server over a secure network.

[0762] The image analysis server performs the analysis

[0763] The server receives and stores image data sent from the device. The received images are preprocessed and input into the AI ​​model. The AI ​​model analyzes the image data to determine health conditions such as cavities, risk of periodontal disease, stains, discoloration, and tooth wear. For example, the server analyzes an image uploaded by a user and detects signs of early cavities in the upper right molar.

[0764] Save the analysis results in a database

[0765] The server associates the analysis results with the user ID and saves them in a database. This allows each user's health history to be accumulated, which will be useful for future analysis and advice. For example, the server saves the results in the database as "User A's dental check results for October 2023."

[0766] Notify users of results and send suggestions for improvement

[0767] Based on the analysis, the server automatically generates a health report and recommendations for improvement, which are sent to the user's app. For example, the report might include a warning about a high risk of cavities in the upper right molar, along with recommendations for specific brushing and flossing techniques.

[0768] The user checks the results

[0769] Users will receive a notification within the app and can view the generated report and improvement suggestions. If necessary, they can send images via messaging services such as LINE for expert diagnosis. For example, a user may view a warning in the app and send images to the LINE official account for further diagnosis.

[0770] Each step can be easily performed at home, allowing for daily monitoring of dental health and early diagnosis and prevention without visiting a dental clinic. This system is extremely beneficial for people who lead busy lives or who are anxious about visiting the dentist.

[0771] The processing flow will be explained below.

[0772] Step 1:

[0773] The user launches the app.

[0774] The user taps and launches the AI ​​Dental guard app on their smartphone.

[0775] Step 2:

[0776] The user takes a photo of their teeth.

[0777] Users use the app's in-app camera to take a photo of their teeth with their smartphone camera, and are encouraged to clearly capture both the front and back teeth.

[0778] Step 3:

[0779] The user checks the shooting data.

[0780] Users can review the photos they've taken, take new ones if necessary, and once they're satisfied, press the "Start Analysis" button within the app.

[0781] Step 4:

[0782] The device sends the captured image data to the server.

[0783] The image data captured by the device is uploaded to a dedicated server via an internet connection, using Wi-Fi or mobile data.

[0784] Step 5:

[0785] The server receives the image data.

[0786] The server receives the image data sent from the terminal and stores it in a secure storage.

[0787] Step 6:

[0788] The server preprocesses the received image data.

[0789] The server performs preprocessing on the received image data, such as formatting, resizing, and normalization. This preprocessing converts the image data into a format suitable for the AI ​​model.

[0790] Step 7:

[0791] The server inputs preprocessed image data into the AI ​​model.

[0792] The server inputs the preprocessed image data into the AI ​​model and prepares it for analysis.

[0793] Step 8:

[0794] The AI ​​model analyzes the image data.

[0795] The AI ​​model analyzes the image data to determine health conditions such as risk of cavities, periodontal disease, stains, discoloration, and tooth wear.

[0796] Step 9:

[0797] The server generates the analysis results.

[0798] The server generates a health report based on the analysis results obtained from the AI ​​model, which includes a description of the health condition and individualized suggestions for improvement.

[0799] Step 10:

[0800] The server stores the analysis results in a database.

[0801] The analysis results generated by the server are stored in a database along with the user ID, allowing each user's health history to be accumulated.

[0802] Step 11:

[0803] The server sends the analysis results and improvement suggestions.

[0804] The server generates a health report and sends it to the user's app, which displays the user's health status and appropriate actions.

[0805] Step 12:

[0806] The user checks the results in the app.

[0807] Users receive in-app notifications and receive health status reports and suggestions for improvement.

[0808] Step 13:

[0809] If the user wishes to have a specialist diagnose the image, they can submit it.

[0810] If the user wishes to receive a more detailed diagnosis, they can send the image to a specialist from within the app using another means of communication, such as the official LINE account, and wait for a reply from the specialist.

[0811] Step 14:

[0812] The expert will send the diagnosis results to the user.

[0813] The expert will analyze the images sent and return the diagnosis and specific advice to the user, allowing the user to obtain more detailed health information.

[0814] Example 1

[0815] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0816] In modern society, many people lead busy lives and often neglect regular dental checkups. As a result, dental health is more likely to deteriorate and early detection becomes more difficult. Furthermore, for people who have anxiety about visiting the dentist, regular visits can be a psychological burden. Given this background, there is a need for a system that can easily and accurately monitor dental health at home and provide guidelines for when to seek professional diagnosis.

[0817] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0818] In this invention, the server includes means for receiving image data of the inside of the oral cavity captured by an image capturing device, means for preprocessing the received image data of the inside of the oral cavity and inputting it into a generative AI model, means for sending prompts to the generative AI model, analyzing the image data of the inside of the oral cavity, and determining the health condition, means for automatically generating a health condition report and improvement suggestions for the user based on the determination results, and means for sending the generated report and improvement suggestions to a user terminal. This enables users to easily and accurately monitor their dental health condition at home, detect problems early, and take appropriate measures.

[0819] An "imaging device" is a device for taking images of the inside of the oral cavity, and includes smartphones, digital cameras, etc.

[0820] "Image data" refers to a digital representation of visual information generated by an image capture device.

[0821] "Means for receiving" refers to a device or software capable of receiving image data into a server or other system component.

[0822] "Preprocessing" refers to a series of processes performed on received image data to facilitate analysis, and specific examples include adjusting resolution and removing noise.

[0823] A "generative AI model" refers to a model that uses artificial intelligence techniques to analyze data and is trained to perform a specific task.

[0824] A "prompt" is a textual instruction that instructs a generative AI model to perform a specific analytical task.

[0825] "Means for analysis" refers to a device or software that has the function of inputting image data into a generative AI model, analyzing the data, and determining health status.

[0826] "Determination results" refer to information about the health status of the oral cavity obtained based on analysis by the generative AI model.

[0827] "Means for automatically generating reports and improvement suggestions" refers to a device or software that has the function of explaining the user's health condition and suggesting ways to improve it based on the assessment results.

[0828] "User terminal" refers to a device used by a user to receive reports and improvement suggestions, such as a computer or smartphone.

[0829] As one embodiment of the present invention, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home and receive improvement suggestions. This system is composed of an image capture device, a generative AI model, and a user terminal.

[0830] First, a user launches the AI ​​Dental guard app on their smartphone (for example, an iPhone or Android device) and takes a photo of their teeth using the app's camera function. The captured image data is then uploaded from the smartphone to a server via the internet. Uploading is done using a Wi-Fi or mobile data connection. For example, a user can take a photo of the front and back teeth with their smartphone and send the image through the app.

[0831] The server receives the image data sent from the device and temporarily stores it in a database. The received image data is preprocessed (for example, by adjusting the resolution or removing noise) and then input into the generative AI model. Image processing software (for example, OpenCV) is used for preprocessing.

[0832] The generative AI model uses deep learning frameworks such as TensorFlow and PyTorch to analyze image data by receiving prompts such as the following:

[0833] "Analyze this image to determine the user's dental health (cavities, risk of periodontal disease, stains, discoloration, tooth wear)."

[0834] This allows the AI ​​model to analyze image data and determine health conditions such as risk of cavities and periodontal disease, dirt, staining, and tooth wear.

[0835] The server receives the analysis results obtained from the generative AI model, associates them with the user ID, and stores them in a database. This database accumulates health data for each user and is used to improve the accuracy of future analyses and improvement suggestions.

[0836] Based on the analysis results, the server automatically generates a health status report and improvement suggestions. These reports and suggestions are sent to the user's device, and the user receives notifications within the app and can review the generated report and improvement suggestions. For example, a warning indicating a high risk of cavities in the upper right molar and suggestions for specific tooth brushing and flossing techniques may be displayed.

[0837] If necessary, users can use the app's functions to receive further professional advice. For example, users can use messaging services such as LINE to send images they have taken to a specialist for further diagnosis.

[0838] In this way, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home, detect problems early, and take appropriate measures. It also helps prevent dental visits and supports continuous health management.

[0839] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0840] Step 1:

[0841] The user launches the AI ​​Dental guard app on their smartphone and takes a photo of their teeth using the app's camera function. The captured image data is the input and is saved on the user's device. Next, when the user taps the "Start Analysis" button, the captured data is uploaded to the server. The output is the image data sent to the server. Specifically, when the user takes a photo of the front teeth with their iPhone and presses the "Start Analysis" button, the captured image file (e.g., JPEG) is sent to the server.

[0842] Step 2:

[0843] The device compresses the captured image data (e.g., using the JPEG compression algorithm) and sends it to a server via an Internet connection. The input is the uncompressed image data, and the output is the compressed image data. Specifically, the device uses a Wi-Fi connection to send the image data to the server via an SSL / TLS encrypted channel.

[0844] Step 3:

[0845] The server receives image data sent from the device and temporarily stores it in a database. The input is image data from the device, and the output is image data stored in the database. For example, the server stores image data in a specified format in a MySQL database.

[0846] Step 4:

[0847] The server preprocesses the received image data. Preprocessing includes adjusting the resolution and removing noise. The input is image data stored in the database, and the output is the preprocessed image data. Specifically, the server uses the OpenCV library to adjust the image resolution and remove noise.

[0848] Step 5:

[0849] The server inputs the preprocessed image data into the generative AI model and sends a prompt. The input is the preprocessed image data and the prompt, and the output is the analysis result. For example, the server sends the following prompt to the TensorFlow model: "Analyze this image to determine the user's dental health status (cavities, periodontal disease risk, stains, discoloration, and tooth wear)."

[0850] Step 6:

[0851] The generative AI model analyzes the received image data and determines health conditions such as risk of cavities and periodontal disease, stains, discoloration, and tooth wear. The input is preprocessed image data and prompt text, and the output is the analysis results. Specifically, the AI ​​model analyzes each pixel in the image, identifies specific patterns, and generates a judgment result.

[0852] Step 7:

[0853] The server receives the generated analysis results, associates them with the user ID, and stores them in a database. The input is the analysis results from the AI ​​model, and the output is the analysis results stored in the database. Specifically, "User A's dental check results for October 2023" is stored in the server's database.

[0854] Step 8:

[0855] Based on the analysis results, the server automatically generates a health status report and improvement suggestions. These reports and suggestions are sent to the user's app. The input is the analysis results, and the output is the report and suggestions sent to the user's device. Specifically, the server generates a warning such as "You have a high risk of cavities in your upper right molars" and "suggestions for specific tooth brushing methods and flossing."

[0856] Step 9:

[0857] Users receive notifications within the app and review the generated report and improvement suggestions. The input is the report and suggestions sent to the device, and the output is the user's acknowledgment. If necessary, users can use the app's functions to send images to experts. Specific actions include tapping the "Check analysis results" button in the app, viewing the detailed analysis, and then sharing the images and analysis results using the "Send to expert via LINE" button.

[0858] (Application example 1)

[0859] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0860] Traditionally, dental clinics have spent a lot of time and effort diagnosing patients' oral conditions. Furthermore, there were limited ways for patients to monitor their dental health at home, making it difficult to detect problems early. Furthermore, there was a need for efficient methods for managing patient data and proposing treatment plans.

[0861] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0862] In this invention, the server includes a means for receiving image data of the oral cavity captured by an imaging device, a means for preprocessing the received image data of the oral cavity and inputting it into a generated AI model, a means for analyzing the image data of the oral cavity using the AI ​​model to assess the user's health status, a means for automatically generating a health status report and improvement suggestions for the user based on the assessment results, a means for transmitting the generated report and improvement suggestions to a user terminal, a means for collecting patient data at the reception desk and saving it together with the image data, and a means for proposing a treatment plan based on the analysis results. This enables efficient patient care at dental clinics and rapid diagnosis and treatment plan proposals by dentists. Furthermore, patients can easily monitor their oral health status at home and obtain appropriate improvement measures.

[0863] An "imaging device" is a device used to take images of the inside of the oral cavity, and typically refers to a smartphone with a camera function or a dedicated digital camera.

[0864] The "receiving means" is a mechanism for receiving data sent from outside, and is a device that includes a server or software for receiving image data via a network connection.

[0865] "Preprocessing" refers to processing performed to make received image data easier to analyze, and typically refers to image processing such as noise removal, brightness adjustment, and cropping.

[0866] An "AI model" is a model of artificial intelligence that has been trained using machine learning algorithms to perform a specific task.

[0867] The "means of judgment" is a function that derives results based on the data being analyzed; specifically, the AI ​​model analyzes image data to evaluate health status.

[0868] "Means for automatically generating reports and improvement suggestions" refers to a function that automatically generates reports on the user's health status and methods for improving it based on the assessment results obtained by the AI ​​model.

[0869] "Means for sending to user devices" refers to a function for sending generated reports and improvement suggestions to devices used by users, and is a mechanism for sending data to smartphones and other devices via the Internet.

[0870] "Means for collecting patient data at the reception desk" refers to a function for collecting basic patient information and intraoral images at the dental clinic reception desk, and involves obtaining this information using smart glasses or a tablet.

[0871] "Means for proposing a treatment plan based on the analysis results" refers to a function that proposes specific treatment methods and procedures appropriate for the patient based on the analysis results of the AI ​​model.

[0872] One embodiment of this invention is a system in which an application called "Dental Reception Assistant" that supports reception work at dental clinics is installed on smart glasses or a head-mounted display. This system takes intraoral images of patients at the reception desk, analyzes the images using the AI ​​Dental guard system, and immediately provides diagnosis results and treatment plans.

[0873] Key Hardware and Software

[0874] The hardware used includes, for example, smart glasses (Vuzix M400) and a head-mounted display (Microsoft HoloLens) for capturing intraoral images and displaying the data, while the server hosts the high-performance image processing unit and AI models.

[0875] The software uses OpenCV as a library for image processing and AI analysis, and trained generative AI models. In addition, the HTTP protocol is used for data communication, sending and receiving image data and analysis results.

[0876] Program Overview

[0877] The system works as follows:

[0878] 1. Patient data collection and photography

[0879] Using a device (smart glasses or a head-mounted display), an image of the patient's oral cavity is taken. At the same time, basic information about the patient is also entered. The device then sends this image data to a server.

[0880] 2. Image data preprocessing

[0881] The server preprocesses the received image data, specifically performing noise removal, brightness adjustment, cropping, etc.

[0882] 3. Analysis using AI models

[0883] The pre-processed image data is then fed into a generative AI model, which analyzes the images and determines health conditions such as risk of cavities, risk of periodontal disease, staining level, and wear status.

[0884] 4. Diagnosis results and treatment plan proposal

[0885] Based on the results, the server automatically generates a health status report and recommendations for improvement, as well as specific treatment plans, which are then sent to the user's device and displayed visually.

[0886] 5. Checking the results and taking action

[0887] The user (dentist or staff) receives a notification on their device, reviews the generated report and improvement suggestions, and, if necessary, proposes an appropriate treatment plan for the patient.

[0888] Specific examples

[0889] When a patient visits the dental clinic, the dental clinic's reception staff uses smart glasses to take a photo of the patient's mouth. The captured image is uploaded to a server in real time and analyzed by an AI model. Based on the analysis results, the risk of tooth decay and the necessary treatment plan are displayed on the spot, and appropriate treatment can be immediately proposed to the patient.

[0890] Prompt Sentence Examples

[0891] A patient comes to your clinic. Use smart glasses to take intraoral photos of the patient and analyze them with the AI ​​Dental Guard system. Based on the analysis results, propose the best treatment plan for the patient.

[0892] This invention is expected to improve the efficiency of reception work and diagnostic accuracy at dental clinics, and will be a particularly useful solution for busy dentists and clinics with many patients.

[0893] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0894] Step 1: Collect and photograph patient data

[0895] The user (dental clinic staff) uses smart glasses or a head-mounted display to take intraoral images of the patient. At this time, the patient's basic information is also entered. The entered data (patient's basic information and intraoral images) is saved on the terminal. The terminal then sends this image data to the server. The input is the intraoral images and patient information, and the output is data transmission to the server. Specifically, the user takes several images using the terminal's camera, and then manually selects and uploads them.

[0896] Step 2: Preprocessing the image data

[0897] The server preprocesses the received intraoral image data. Specifically, it uses the OpenCV library to perform image processing such as noise removal, brightness adjustment, and cropping. The input is the received raw image data, and the output is preprocessed image data. Specifically, the server automatically performs image filtering and correction, and then converts the data into a format suitable for the AI ​​model.

[0898] Step 3: Analysis by AI model

[0899] The server inputs the preprocessed image data into the generated AI model. The generated AI model analyzes the image data and determines health conditions such as caries risk, periodontal disease risk, staining level, and wear status. The input is the preprocessed image data, and the output is the health condition assessment result. Specifically, the AI ​​model analyzes the characteristics of each pixel and detects patterns of risk factors.

[0900] Step 4: Generate a diagnosis and treatment plan

[0901] The server automatically generates a health status report and improvement suggestions based on the analysis results of the AI ​​model. It also generates a specific treatment plan. The input is the analysis results, and the output is an automatically generated report and treatment plan. Specifically, it uses templates to create a report and adds a treatment plan tailored to each patient's situation.

[0902] Step 5: Notification and display of results

[0903] The server sends the generated report and treatment plan to the user's device. The user (dentist or staff member) receives a notification on their device and reviews the generated report and recommendations. The input is the generated report and treatment plan, and the output is the information displayed on the user's device. Specifically, the application on the device displays a pop-up notification and takes the user to a screen where detailed results can be viewed.

[0904] Step 6: Handling the results

[0905] The user (dentist or staff) checks the notification on the user terminal and proposes the optimal treatment method for the patient based on the generated report and treatment plan. The input is the diagnosis results and treatment plan displayed on the terminal, and the output is a treatment proposal to the patient. In concrete terms, the dentist explains the treatment contents to the patient and sets a treatment schedule if necessary.

[0906] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0907] As one embodiment of the present invention, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home, receive professional diagnosis as needed, and recognizes the user's emotions to provide appropriate suggestions and advice for improvement. The system program processing is explained below in natural language.

[0908] User takes and uploads a photo

[0909] Users launch the AI ​​Dental guard app on their smartphone and use the camera to take a photo of their teeth. After taking the photo, they press the "Start Analysis" button in the app to upload the captured data to the server. For example, users can take photos of the front and back teeth with their smartphone and send the images through the app.

[0910] Upload the image to the server

[0911] The device (user's smartphone) sends the captured image data to the server. An internet connection is required for transmission. For example, the user can use Wi-Fi or mobile data to upload the captured image data to the server via a secure network.

[0912] The image analysis server performs the analysis

[0913] The server receives and stores the image data sent from the device. The received image is preprocessed and input into the AI ​​model. The AI ​​model analyzes the image data and determines health conditions such as tooth decay, risk of periodontal disease, stains, discoloration, and tooth wear. For example, the AI ​​model may detect early signs of tooth decay in the user's upper right molar.

[0914] Save the analysis results in a database

[0915] The server associates the analysis results with the user ID and saves them in a database. This allows each user's health history to be accumulated and used to improve the accuracy of future analyses and advice. For example, the server may save the results in the database as "User A's dental check results for October 2023."

[0916] Notify users of results and send suggestions for improvement

[0917] Based on the analysis, the server automatically generates a health report and recommendations for improvement, which are sent to the user's app. For example, the report might include a warning about a high risk of cavities in the upper right molar, along with recommendations for specific brushing and flossing techniques.

[0918] Emotion engine recognizes user emotions

[0919] When a user checks the results, the emotion engine uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to recognize their current emotional state. For example, the emotion engine can detect if the user is surprised or anxious about the results.

[0920] Adjusting improvement suggestions with emotion engine

[0921] The emotion engine recognizes the user's emotions and tailors the suggestions to them accordingly. For example, if the user is feeling anxious, the engine generates a suggestion that includes a gentle, encouraging message or supporting information to recommend a visit to the dentist.

[0922] If you would like a professional diagnosis, please send us your images.

[0923] If a user requests a more detailed diagnosis, they can use another means of communication, such as the LINE official account, to send the image to a specialist and wait for a reply from the specialist. For example, a user can use the LINE app to send a photo they have taken to a dentist and request a more detailed diagnosis.

[0924] The expert will send the diagnosis to the user.

[0925] The expert will analyze the images and return a diagnosis and specific advice to the user, allowing the user to obtain detailed health information. For example, a dentist may diagnose the image and recommend appropriate treatment or preventative measures.

[0926] The above system supports users in dental care by conveniently checking their dental health at home and providing appropriate responses based on their emotions. It also allows users to easily receive a diagnosis from a specialist, enabling timely treatment and countermeasures.

[0927] The processing flow will be explained below.

[0928] Step 1:

[0929] The user launches the app.

[0930] The user taps and launches the AI ​​Dental guard app on their smartphone.

[0931] Step 2:

[0932] The user takes a photo of their teeth.

[0933] Users use the app's camera function to take a photo of their teeth with their smartphone camera, and are encouraged to clearly capture both the front and back teeth.

[0934] Step 3:

[0935] The user checks the shooting data.

[0936] Users can review the photos they've taken, take new ones if necessary, and once they're satisfied, press the "Start Analysis" button within the app.

[0937] Step 4:

[0938] The device sends the captured image data to the server.

[0939] The image data captured by the device is uploaded to a dedicated server via an internet connection, using Wi-Fi or mobile data.

[0940] Step 5:

[0941] The server receives the image data.

[0942] The server receives the image data sent from the terminal and stores it in a secure storage.

[0943] Step 6:

[0944] The server preprocesses the received image data.

[0945] The server performs preprocessing on the received image data, such as formatting, resizing, and normalization. This preprocessing converts the image data into a format suitable for the AI ​​model.

[0946] Step 7:

[0947] The server inputs preprocessed image data into the AI ​​model.

[0948] The server inputs the preprocessed image data into the AI ​​model and prepares it for analysis.

[0949] Step 8:

[0950] The AI ​​model analyzes the image data.

[0951] The AI ​​model analyzes the image data to determine health conditions such as risk of cavities, periodontal disease, stains, discoloration, and tooth wear.

[0952] Step 9:

[0953] The server generates the analysis results.

[0954] The server generates a health report based on the analysis results obtained from the AI ​​model, which includes a description of the health condition and individualized suggestions for improvement.

[0955] Step 10:

[0956] The server stores the analysis results in a database.

[0957] The analysis results generated by the server are stored in a database along with the user ID, allowing each user's health history to be accumulated.

[0958] Step 11:

[0959] The server sends the analysis results and improvement suggestions.

[0960] The server generates a health report and sends it to the user's app, which displays the user's health status and appropriate actions.

[0961] Step 12:

[0962] The user checks the results in the app.

[0963] Users receive in-app notifications and receive health status reports and suggestions for improvement.

[0964] Step 13:

[0965] The emotion engine recognizes the user's emotions.

[0966] When the user checks the results, the emotion engine uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to recognize their current emotional state. For example, the emotion engine can detect if the user is surprised or anxious about the results.

[0967] Step 14:

[0968] Adjustment of improvement suggestions by emotion engine.

[0969] The emotion engine recognizes the user's emotions and tailors the suggestions to them accordingly. For example, if the user is feeling anxious, the engine generates a suggestion that includes a gentle, encouraging message or supporting information to recommend a visit to the dentist.

[0970] Step 15:

[0971] If the user wishes to have a specialist diagnose the image, they can submit it.

[0972] If a user requests a more detailed diagnosis, they can use another means of communication, such as the LINE official account, to send the image to a specialist and wait for a reply from the specialist. For example, a user can use the LINE app to send a photo they have taken to a dentist and request a more detailed diagnosis.

[0973] Step 16:

[0974] The expert will send the diagnosis results to the user.

[0975] The expert will analyze the images and return a diagnosis and specific advice to the user, allowing the user to obtain detailed health information. For example, a dentist may diagnose the image and recommend appropriate treatment or preventative measures.

[0976] Example 2

[0977] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0978] Until now, users have had limited means to monitor their dental health at home, making it difficult to easily receive detailed analyses of their health or receive professional diagnoses. Furthermore, there were no systems that took into account the user's emotional state when reporting their health status or providing recommendations for improvement. This meant users were unable to take care of their teeth with peace of mind and found it difficult to receive professional diagnoses at the appropriate time.

[0979] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data of the oral cavity captured by an image capturing device, a means for preprocessing the received image data of the oral cavity and inputting it into a generative AI model, a means for analyzing the image data of the oral cavity using the generative AI model and determining the health status, a means for automatically generating a health status report and improvement suggestions for the user based on the determination results, a means for transmitting the generated report and improvement suggestions to a user terminal, and a means for recognizing the user's emotional state and adjusting the content of the improvement suggestions according to the emotion. This allows the user to easily check the health status of their teeth at home and receive appropriate improvement suggestions according to their emotion. Furthermore, if a detailed diagnosis by a specialist is desired, a prompt response is also possible.

[0980] An "imaging device" is a device that a user uses to take images of the inside of the oral cavity.

[0981] "Image data" refers to digital data of an image of the inside of the oral cavity captured by an image capturing device.

[0982] The "receiving means" is a communication means for transferring image data from the user terminal to the server.

[0983] "Preprocessing" refers to data processing, such as removing noise from image data and correcting resolution, to enable accurate analysis by the AI ​​model.

[0984] A "generative AI model" is an artificial intelligence (AI) model used to analyze image data from inside the oral cavity and determine health status.

[0985] "Analysis" refers to the generative AI model analyzing image data of the inside of the mouth to determine the health of the teeth.

[0986] A "health report" is a report containing information describing a user's dental health based on the analysis results.

[0987] "Improvement suggestions" are recommendations on how to brush and care for your teeth based on the user's health condition.

[0988] A "user device" is an electronic device used by a user, such as a smartphone or tablet.

[0989] "Emotional state" refers to the user's current emotional state, as detected from facial expressions, tone of voice, etc.

[0990] An "emotion engine" is an artificial intelligence system used to analyze a user's emotional state.

[0991] "Means of communication" refers to methods of data transfer, including internet connection and messaging services such as LINE.

[0992] "Professionals" are medical professionals with professional qualifications, such as dentists and dental hygienists.

[0993] A "database" is an information system for storing health data and analysis results for each user.

[0994] As one embodiment of this invention, we provide the "AI Dental Guard System," which allows users to easily monitor their dental health at home and receive a diagnosis from a specialist if necessary. This system recognizes the user's emotional state and provides appropriate improvement suggestions and advice according to their emotions.

[0995] First, the user launches the dedicated app on their smartphone and uses the app's camera function to take an image of the inside of their mouth. Next, the captured image data is uploaded to the server by pressing the "Start Analysis" button within the app. The server saves the received image data and performs preprocessing such as noise removal.

[0996] The preprocessed image data is input into a generative AI model, which analyzes the input image data and determines the health status of the user based on factors such as the risk of cavities and periodontal disease, stains, discoloration, and tooth wear. The results of the assessment are associated with the user's ID and stored in a database.

[0997] The server automatically generates a report of the user's dental health based on the analysis results, along with suggestions for improvement. This report and suggestions are sent to the user's smartphone app. For example, it could warn the user that they are at high risk for cavities in their upper right molar, and include suggestions for specific brushing and flossing techniques.

[0998] When the user checks the results, the smartphone's camera and microphone are activated, and the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotional state. Based on the recognized emotional state, the server adjusts the content of the improvement suggestions. For example, if the user is feeling anxious, the server will provide suggestions that include a gentle, encouraging message or support information recommending a visit to the dentist.

[0999] Additionally, if a user desires a more detailed diagnosis, they can send the captured image to a specialist using other communication methods within the dedicated app (e.g., the official LINE account). The specialist will then respond to the user with a detailed diagnosis based on the image. For example, a dentist may analyze the captured image and advise, "There are signs of early tooth decay in the upper right molar, so immediate treatment is required."

[1000] This system allows users to easily check the health of their teeth at home and receive appropriate suggestions for improvement. Furthermore, by receiving a detailed diagnosis from a specialist promptly, timely treatment and measures can be taken.

[1001] An example prompt might look like this:

[1002] Please provide a detailed description of the system where users upload photos of their teeth taken with the AI ​​Dental guard app to the server.

[1003] In this way, the invention is designed to make users' lives more comfortable and support their health management.

[1004] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1005] Step 1:

[1006] The user launches the "AI Dental guard" app. The user uses the camera function of their smartphone to take images of the inside of their mouth. For example, the user takes photos of their front and back teeth. The input is the image data taken by the user, and the output is the captured image data.

[1007] Step 2:

[1008] The user presses the "Start Analysis" button to upload the captured image data to the server. Specifically, the app uses an internet connection to send the image data to the server. The input is the captured image data, and the output is the image data sent to the server.

[1009] Step 3:

[1010] The server receives image data sent from the terminal and stores it in cloud storage. The input is the sent image data, and the output is the stored image data. Specifically, the server stores the image data in secure storage.

[1011] Step 4:

[1012] The server performs preprocessing such as noise removal and resolution correction on the image data. The input is the stored image data, and the output is the preprocessed image data. Specifically, the server applies noise filtering and resolution correction algorithms.

[1013] Step 5:

[1014] The server inputs the preprocessed data into the generative AI model. The input is the preprocessed image data, and the output is the analysis results. Specifically, the generative AI model analyzes the image data and determines health conditions such as the risk of cavities and periodontal disease, stains, discoloration, and tooth wear.

[1015] Step 6:

[1016] The server associates the analysis results with the user ID and stores them in a database. The input is the analysis results and the user ID, and the output is the associated data. Specifically, the server organizes the analysis results by user and stores them in a database for future reference.

[1017] Step 7:

[1018] The server automatically generates a report of the user's dental health status based on the analysis results. It also automatically generates improvement suggestions. The input is the analysis results, and the output is the generated report and improvement suggestions. Specifically, the server uses an automatic generation algorithm to create the report and improvement suggestions.

[1019] Step 8:

[1020] The server sends the generated report and improvement suggestions to the user's app. The input is the generated report and improvement suggestions, and the output is the data sent to the user's device. Specifically, the server sends the data using an Internet connection.

[1021] Step 9:

[1022] When the user wants to check the results, the smartphone's camera and microphone are activated. The emotion engine analyzes the user's facial expressions and tone of voice to recognize their current emotional state. The input is data from the camera and microphone, and the output is the recognized emotional state. Specifically, the emotion engine uses video and audio analysis algorithms.

[1023] Step 10:

[1024] Based on the analysis results of the emotion engine, the server adjusts the content of the improvement proposal. The input is the emotional state and the initial improvement proposal, and the output is the adjusted improvement proposal. Specifically, the server adds or modifies messages and proposals according to the user's emotions.

[1025] Step 11:

[1026] If the user wishes to receive a more detailed diagnosis, they can select another communication method, such as the LINE official account, from within the app and send the captured image to an expert. The input is the captured image data and a request to send it to the expert, and the output is the image data sent to the expert. Specifically, the user sends the data to the expert using an app such as LINE.

[1027] Step 12:

[1028] The expert analyzes the sent images and returns the diagnosis and specific advice to the user. The input is the sent image data, and the output is the expert's diagnosis and advice. Specifically, the expert diagnoses based on the images and notifies the user of appropriate treatment methods and preventive measures.

[1029] In this way, the AI ​​Dental Guard system allows users to easily monitor their dental health at home and receive professional diagnosis, and supports users' dental care by providing appropriate improvement suggestions based on their emotions.

[1030] (Application example 2)

[1031] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1032] Conventional oral examination systems have the drawback of making it difficult for users to easily monitor their dental health at home, and they are unable to receive appropriate advice or suggestions for improvement based on their emotions. Furthermore, they lack the means to alleviate users' anxiety, making it difficult to receive a detailed diagnosis from a specialist promptly. This makes it difficult to detect dental problems early and take appropriate measures, potentially hindering users' dental health maintenance.

[1033] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1034] In this invention, the server includes means for receiving image data of the oral cavity captured by an imaging device, means for preprocessing the received image data of the oral cavity and inputting it into an AI model, means for analyzing the image data of the oral cavity using the AI ​​model to assess the health status, means for automatically generating a health status report and improvement suggestions for the user based on the assessment results, means for transmitting the generated report and improvement suggestions to a user terminal, and means for recognizing the user's emotions and adjusting the improvement suggestions according to the emotions. This allows users to easily monitor their dental health at home and receive appropriate improvement suggestions according to their emotions. It also enables prompt and detailed diagnosis by experts, reducing user anxiety and enabling early detection of dental problems and appropriate measures.

[1035] (Definitions of important words)

[1036] "Imaging device" refers to an electronic device used to take images of the inside of the oral cavity.

[1037] "Intraoral image data" refers to digital data that digitizes visual information about the oral cavity obtained by an image capturing device.

[1038] "Means for receiving" refers to a mechanism that allows a server to acquire data sent from a user terminal via a network.

[1039] "Preprocessing" refers to the process of converting and processing image data into an analyzable format before inputting it into an AI model.

[1040] "AI model" refers to a model that includes neural networks and machine learning algorithms built on the foundation of artificial intelligence technology.

[1041] "Means for determining health status" refers to the system that evaluates the health status of teeth and the oral cavity from the results of analysis by the AI ​​model and derives the results.

[1042] "Determination result" refers to the conclusion regarding health status output after the AI ​​model analyzes image data.

[1043] "Means for automatically generating improvement proposals" refers to a system that has the function of automatically generating improvement measures and advice appropriate for users based on the assessment results.

[1044] "User device" refers to electronic devices such as smartphones and tablets used by users.

[1045] "Means of recognizing emotions" refers to technology that analyzes the user's emotions and determines the appropriate response based on that.

[1046] "Communication means" refers to the methods and technologies used to send and receive data, including the Internet, mobile networks, and dedicated applications.

[1047] "Expert" refers to a medical professional with specialized knowledge and skills to diagnose oral health conditions and abnormalities.

[1048] MODE FOR CARRYING OUT THE INVENTION

[1049] The present invention is a system that allows users to easily monitor their dental health at home. The system recognizes the user's emotions and provides corresponding suggestions for improvement. It also allows users to receive detailed diagnostics from a specialist if necessary.

[1050] System configuration

[1051] The main components of the system are:

[1052] Image capture device (smartphone, etc.)

[1053] server

[1054] AI model

[1055] Emotion Recognition Engine

[1056] User devices (smartphones, tablets)

[1057] Program processing

[1058] 1. Image capture and data reception

[1059] Users take pictures of the inside of their mouths using the camera function on their smartphones, and the image data they take is uploaded to a server via the Internet, using Wi-Fi or mobile data communication.

[1060] 2. Preprocessing and input to the AI ​​model

[1061] The server preprocesses the received image data and converts it into a format suitable for the AI ​​model, including image resizing and normalization.

[1062] 3. Health status analysis

[1063] The server inputs the preprocessed image data into an AI model to analyze the health condition, which detects abnormalities in the teeth and oral cavity and assesses health risks such as cavities, periodontal disease, and stains.

[1064] 4. Emotional Recognition

[1065] The emotion recognition engine uses the smartphone's camera and microphone to analyze the user's emotions when they check the results, analyzing their facial expressions and tone of voice to recognize their current emotional state.

[1066] 5. Generate reports and improvement suggestions

[1067] Based on the analysis results and emotion recognition results, the server automatically generates a health status report for the user and improvement suggestions according to their emotions, including specific tooth brushing methods and lifestyle improvements.

[1068] 6. Notifications to User Devices

[1069] The generated reports and improvement suggestions are sent to the user's smartphone or tablet, where they can view the information through the app.

[1070] 7. Detailed diagnosis by an expert

[1071] If the user wishes to receive a more detailed diagnosis, the captured images of the oral cavity can be sent to a specialist via another communication method. The specialist will analyze the images and provide the user with a diagnosis and specific advice.

[1072] Specific examples

[1073] For example, if a user notices something wrong with their teeth, they can take a picture of their teeth using their smartphone and upload it to the server through the app. The AI ​​model then analyzes the image and detects signs of tooth decay. The emotion recognition engine recognizes that the user is feeling anxious and generates a gentle message saying, "You are at risk of tooth decay. Don't worry, we'll support you. We recommend that you visit the dentist as soon as possible." This suggestion is immediately sent to the user's smartphone.

[1074] Prompt Sentence Examples

[1075] Below are some examples of specific prompts that can be fed into a generative AI model:

[1076] Implement an API that allows users to upload photos taken with their smartphones to check their dental health at home. The API will analyze the dental health status using an AI model and return a warning message if there is an abnormality. It will also analyze the user's emotions using an emotion recognition engine and customize improvement suggestions based on the results. The output should be in JSON format, containing the health status and suggestions.

[1077] In this way, by utilizing specific prompt sentences, generative AI models can be used efficiently and system development can proceed smoothly.

[1078] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1079] Program processing steps

[1080] Step 1:

[1081] Image capture and data upload

[1082] The user takes a photograph of the inside of the mouth using a smartphone. This image data is uploaded from the smartphone to the server. The user presses the "Start Analysis" button in the app, and the device sends the image data to the server over a secure network. The input is the image data of the inside of the mouth that was photographed, and the output is the image data sent to the server.

[1083] Step 2:

[1084] Image data preprocessing

[1085] The server receives the received image data and performs preprocessing. Specifically, it resizes, normalizes, and removes noise if necessary. The preprocessed image data is converted into a format suitable for the AI ​​model. The input is the raw image data uploaded to the server, and the output is image data converted into a format that can be input into the AI ​​model.

[1086] Step 3:

[1087] Health status analysis using AI models

[1088] The server inputs the preprocessed image data into the AI ​​model and performs the analysis. The AI ​​model evaluates health risks such as cavities, periodontal disease, stains, discoloration, and tooth wear. The input is the preprocessed image data, and the output is a judgment result regarding the health status. For example, the AI ​​model may detect signs of early cavities from the image.

[1089] Step 4:

[1090] Emotion recognition

[1091] When a user checks the results of the AI ​​analysis, the emotion recognition engine uses the device's camera and microphone to analyze the user's facial expressions and tone of voice. This allows the system to recognize the user's current emotional state. The input is facial expression and voice data acquired from the smartphone, and the output is the user's emotional state. For example, the emotion engine can detect that the user is surprised by the results.

[1092] Step 5:

[1093] Generate reports and improvement suggestions

[1094] The server automatically generates a report and improvement suggestions based on the AI's health assessment results and the emotional state obtained from the emotion recognition engine. This includes advice on proper tooth brushing methods and lifestyle habits. The input is the AI ​​assessment results and the user's emotional state, and the output is a customized improvement suggestion message. For example, it might say, "Incipient tooth decay has been detected. Don't worry. We recommend that you visit a dentist as soon as possible."

[1095] Step 6:

[1096] Notifications on user devices

[1097] The reports and improvement suggestions generated by the server are sent to the user's smartphone, where the user can check the information through the app. The input is the generated report and suggestion data, and the output is the report and suggestion message displayed on the user's device.

[1098] Step 7:

[1099] Detailed diagnosis by an expert

[1100] If the user requests a detailed diagnosis, the image data captured within the app is sent to a specialist via another means of communication (e.g., a messaging app). The specialist analyzes the data and provides the user with a diagnosis and specific advice. The input is the image data of the inside of the oral cavity, and the output is a detailed diagnosis and advice from the specialist.

[1101] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1102] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1103] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1104] [Fourth embodiment]

[1105] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1106] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1107] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1108] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1109] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1110] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1111] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1112] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1113] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1114] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1115] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1116] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1117] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1118] As one embodiment of the present invention, the AI ​​Dental Guard system is a system that allows users to easily and accurately monitor their dental health at home and receive improvement suggestions. The processing of the system program is explained below in natural language.

[1119] User takes and uploads a photo

[1120] Users launch the AI ​​Dental guard app on their smartphone and use the camera to take a photo of their teeth. After taking the photo, they press the "Start Analysis" button in the app to upload the captured data to the server. For example, users can take photos of the front and back teeth with their smartphone and send the images through the app.

[1121] Upload the image to the server

[1122] The device (user's smartphone) sends the captured image data to the server. An internet connection is required for transmission. For example, the user can use Wi-Fi or mobile data to upload the captured image data to the server over a secure network.

[1123] The image analysis server performs the analysis

[1124] The server receives and stores image data sent from the device. The received images are preprocessed and input into the AI ​​model. The AI ​​model analyzes the image data to determine health conditions such as cavities, risk of periodontal disease, stains, discoloration, and tooth wear. For example, the server analyzes an image uploaded by a user and detects signs of early cavities in the upper right molar.

[1125] Save the analysis results in a database

[1126] The server associates the analysis results with the user ID and saves them in a database. This allows each user's health history to be accumulated, which will be useful for future analysis and advice. For example, the server saves the results in the database as "User A's dental check results for October 2023."

[1127] Notify users of results and send suggestions for improvement

[1128] Based on the analysis, the server automatically generates a health report and recommendations for improvement, which are sent to the user's app. For example, the report might include a warning about a high risk of cavities in the upper right molar, along with recommendations for specific brushing and flossing techniques.

[1129] The user checks the results

[1130] Users will receive a notification within the app and can view the generated report and improvement suggestions. If necessary, they can send images via messaging services such as LINE for expert diagnosis. For example, a user may view a warning in the app, request a more detailed diagnosis, and send images to the LINE official account.

[1131] Each step can be easily performed at home, allowing for daily monitoring of dental health and early diagnosis and prevention without visiting a dental clinic. This system is extremely beneficial for people who lead busy lives or who are anxious about visiting the dentist.

[1132] The processing flow will be explained below.

[1133] Step 1:

[1134] The user launches the app.

[1135] The user taps and launches the AI ​​Dental guard app on their smartphone.

[1136] Step 2:

[1137] The user takes a photo of their teeth.

[1138] Users use the app's in-app camera to take a photo of their teeth with their smartphone camera, and are encouraged to clearly capture both the front and back teeth.

[1139] Step 3:

[1140] The user checks the shooting data.

[1141] Users can review the photos they've taken, take new ones if necessary, and once they're satisfied, press the "Start Analysis" button within the app.

[1142] Step 4:

[1143] The device sends the captured image data to the server.

[1144] The image data captured by the device is uploaded to a dedicated server via an internet connection, using Wi-Fi or mobile data.

[1145] Step 5:

[1146] The server receives the image data.

[1147] The server receives the image data sent from the terminal and stores it in a secure storage.

[1148] Step 6:

[1149] The server preprocesses the received image data.

[1150] The server performs preprocessing on the received image data, such as formatting, resizing, and normalization. This preprocessing converts the image data into a format suitable for the AI ​​model.

[1151] Step 7:

[1152] The server inputs preprocessed image data into the AI ​​model.

[1153] The server inputs the preprocessed image data into the AI ​​model and prepares it for analysis.

[1154] Step 8:

[1155] The AI ​​model analyzes the image data.

[1156] The AI ​​model analyzes the image data to determine health conditions such as risk of cavities, periodontal disease, stains, discoloration, and tooth wear.

[1157] Step 9:

[1158] The server generates the analysis results.

[1159] The server generates a health report based on the analysis results obtained from the AI ​​model, which includes a description of the health condition and individualized suggestions for improvement.

[1160] Step 10:

[1161] The server stores the analysis results in a database.

[1162] The analysis results generated by the server are stored in a database along with the user ID, allowing each user's health history to be accumulated.

[1163] Step 11:

[1164] The server sends the analysis results and improvement suggestions.

[1165] The server generates a health report and sends it to the user's app, which displays the user's health status and appropriate actions.

[1166] Step 12:

[1167] The user checks the results in the app.

[1168] Users receive in-app notifications and receive health status reports and suggestions for improvement.

[1169] Step 13:

[1170] If the user wishes to have a specialist diagnose the image, they can submit it.

[1171] If the user wishes to receive a more detailed diagnosis, they can send the image to a specialist from within the app using another means of communication, such as the official LINE account, and wait for a reply from the specialist.

[1172] Step 14:

[1173] The expert will send the diagnosis results to the user.

[1174] The expert will analyze the images sent and return the diagnosis and specific advice to the user, allowing the user to obtain more detailed health information.

[1175] Example 1

[1176] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1177] In modern society, many people lead busy lives and often neglect regular dental checkups. As a result, dental health is more likely to deteriorate and early detection becomes more difficult. Furthermore, for people who have anxiety about visiting the dentist, regular visits can be a psychological burden. Given this background, there is a need for a system that can easily and accurately monitor dental health at home and provide guidelines for when to seek professional diagnosis.

[1178] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1179] In this invention, the server includes means for receiving image data of the inside of the oral cavity captured by an image capturing device, means for preprocessing the received image data of the inside of the oral cavity and inputting it into a generative AI model, means for sending prompts to the generative AI model, analyzing the image data of the inside of the oral cavity, and determining the health condition, means for automatically generating a health condition report and improvement suggestions for the user based on the determination results, and means for sending the generated report and improvement suggestions to a user terminal. This enables users to easily and accurately monitor their dental health condition at home, detect problems early, and take appropriate measures.

[1180] An "imaging device" is a device for taking images of the inside of the oral cavity, and includes smartphones, digital cameras, etc.

[1181] "Image data" refers to a digital representation of visual information generated by an image capture device.

[1182] "Means for receiving" refers to a device or software capable of receiving image data into a server or other system component.

[1183] "Preprocessing" refers to a series of processes performed on received image data to facilitate analysis, and specific examples include adjusting resolution and removing noise.

[1184] A "generative AI model" refers to a model that uses artificial intelligence techniques to analyze data and is trained to perform a specific task.

[1185] A "prompt" is a textual instruction that instructs a generative AI model to perform a specific analytical task.

[1186] "Means for analysis" refers to a device or software that has the function of inputting image data into a generative AI model, analyzing the data, and determining health status.

[1187] "Determination results" refer to information about the health status of the oral cavity obtained based on analysis by the generative AI model.

[1188] "Means for automatically generating reports and improvement suggestions" refers to a device or software that has the function of explaining the user's health condition and suggesting ways to improve it based on the assessment results.

[1189] "User terminal" refers to a device used by a user to receive reports and improvement suggestions, such as a computer or smartphone.

[1190] As one embodiment of the present invention, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home and receive improvement suggestions. This system is composed of an image capture device, a generative AI model, and a user terminal.

[1191] First, a user launches the AI ​​Dental guard app on their smartphone (for example, an iPhone or Android device) and takes a photo of their teeth using the app's camera function. The captured image data is then uploaded from the smartphone to a server via the internet. Uploading is done using a Wi-Fi or mobile data connection. For example, a user can take a photo of the front and back teeth with their smartphone and send the image through the app.

[1192] The server receives the image data sent from the device and temporarily stores it in a database. The received image data is preprocessed (for example, by adjusting the resolution or removing noise) and then input into the generative AI model. Image processing software (for example, OpenCV) is used for preprocessing.

[1193] The generative AI model uses deep learning frameworks such as TensorFlow and PyTorch to analyze image data by receiving prompts such as the following:

[1194] "Analyze this image to determine the user's dental health (cavities, risk of periodontal disease, stains, discoloration, tooth wear)."

[1195] This allows the AI ​​model to analyze image data and determine health conditions such as risk of cavities and periodontal disease, dirt, staining, and tooth wear.

[1196] The server receives the analysis results obtained from the generative AI model, associates them with the user ID, and stores them in a database. This database accumulates health data for each user and is used to improve the accuracy of future analyses and improvement suggestions.

[1197] Based on the analysis results, the server automatically generates a health status report and improvement suggestions. These reports and suggestions are sent to the user's device, and the user receives notifications within the app and can review the generated report and improvement suggestions. For example, a warning indicating a high risk of cavities in the upper right molar and suggestions for specific tooth brushing and flossing techniques may be displayed.

[1198] If necessary, users can use the app's functions to receive further professional advice. For example, users can use messaging services such as LINE to send images they have taken to a specialist for further diagnosis.

[1199] In this way, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home, detect problems early, and take appropriate measures. It also helps prevent dental visits and supports continuous health management.

[1200] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1201] Step 1:

[1202] The user launches the AI ​​Dental guard app on their smartphone and takes a photo of their teeth using the app's camera function. The captured image data is the input and is saved on the user's device. Next, when the user taps the "Start Analysis" button, the captured data is uploaded to the server. The output is the image data sent to the server. Specifically, when the user takes a photo of the front teeth with their iPhone and presses the "Start Analysis" button, the captured image file (e.g., JPEG) is sent to the server.

[1203] Step 2:

[1204] The device compresses the captured image data (e.g., using the JPEG compression algorithm) and sends it to a server via an Internet connection. The input is the uncompressed image data, and the output is the compressed image data. Specifically, the device uses a Wi-Fi connection to send the image data to the server via an SSL / TLS encrypted channel.

[1205] Step 3:

[1206] The server receives image data sent from the device and temporarily stores it in a database. The input is image data from the device, and the output is image data stored in the database. For example, the server stores image data in a specified format in a MySQL database.

[1207] Step 4:

[1208] The server preprocesses the received image data. Preprocessing includes adjusting the resolution and removing noise. The input is image data stored in the database, and the output is the preprocessed image data. Specifically, the server uses the OpenCV library to adjust the image resolution and remove noise.

[1209] Step 5:

[1210] The server inputs the preprocessed image data into the generative AI model and sends a prompt. The input is the preprocessed image data and the prompt, and the output is the analysis result. For example, the server sends the following prompt to the TensorFlow model: "Analyze this image to determine the user's dental health status (cavities, periodontal disease risk, stains, discoloration, and tooth wear)."

[1211] Step 6:

[1212] The generative AI model analyzes the received image data and determines health conditions such as risk of cavities and periodontal disease, stains, discoloration, and tooth wear. The input is preprocessed image data and prompt text, and the output is the analysis results. Specifically, the AI ​​model analyzes each pixel in the image, identifies specific patterns, and generates a judgment result.

[1213] Step 7:

[1214] The server receives the generated analysis results, associates them with the user ID, and stores them in a database. The input is the analysis results from the AI ​​model, and the output is the analysis results stored in the database. Specifically, "User A's dental check results for October 2023" is stored in the server's database.

[1215] Step 8:

[1216] Based on the analysis results, the server automatically generates a health status report and improvement suggestions. These reports and suggestions are sent to the user's app. The input is the analysis results, and the output is the report and suggestions sent to the user's device. Specifically, the server generates a warning such as "You have a high risk of cavities in your upper right molars" and "suggestions for specific tooth brushing methods and flossing."

[1217] Step 9:

[1218] Users receive notifications within the app and review the generated report and improvement suggestions. The input is the report and suggestions sent to the device, and the output is the user's acknowledgment. If necessary, users can use the app's functions to send images to experts. Specific actions include tapping the "Check analysis results" button in the app, viewing the detailed analysis, and then sharing the images and analysis results using the "Send to expert via LINE" button.

[1219] (Application example 1)

[1220] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1221] Traditionally, dental clinics have spent a lot of time and effort diagnosing patients' oral conditions. Furthermore, there were limited ways for patients to monitor their dental health at home, making it difficult to detect problems early. Furthermore, there was a need for efficient methods for managing patient data and proposing treatment plans.

[1222] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1223] In this invention, the server includes a means for receiving image data of the oral cavity captured by an imaging device, a means for preprocessing the received image data of the oral cavity and inputting it into a generated AI model, a means for analyzing the image data of the oral cavity using the AI ​​model to assess the user's health status, a means for automatically generating a health status report and improvement suggestions for the user based on the assessment results, a means for transmitting the generated report and improvement suggestions to a user terminal, a means for collecting patient data at the reception desk and saving it together with the image data, and a means for proposing a treatment plan based on the analysis results. This enables efficient patient care at dental clinics and rapid diagnosis and treatment plan proposals by dentists. Furthermore, patients can easily monitor their oral health status at home and obtain appropriate improvement measures.

[1224] An "imaging device" is a device used to take images of the inside of the oral cavity, and typically refers to a smartphone with a camera function or a dedicated digital camera.

[1225] The "receiving means" is a mechanism for receiving data sent from outside, and is a device that includes a server or software for receiving image data via a network connection.

[1226] "Preprocessing" refers to processing performed to make received image data easier to analyze, and typically refers to image processing such as noise removal, brightness adjustment, and cropping.

[1227] An "AI model" is a model of artificial intelligence that has been trained using machine learning algorithms to perform a specific task.

[1228] The "means of judgment" is a function that derives results based on the data being analyzed; specifically, the AI ​​model analyzes image data to evaluate health status.

[1229] "Means for automatically generating reports and improvement suggestions" refers to a function that automatically generates reports on the user's health status and methods for improving it based on the assessment results obtained by the AI ​​model.

[1230] "Means for sending to user devices" refers to a function for sending generated reports and improvement suggestions to devices used by users, and is a mechanism for sending data to smartphones and other devices via the Internet.

[1231] "Means for collecting patient data at the reception desk" refers to a function for collecting basic patient information and intraoral images at the dental clinic reception desk, and involves obtaining this information using smart glasses or a tablet.

[1232] "Means for proposing a treatment plan based on the analysis results" refers to a function that proposes specific treatment methods and procedures appropriate for the patient based on the analysis results of the AI ​​model.

[1233] One embodiment of this invention is a system in which an application called "Dental Reception Assistant" that supports reception work at dental clinics is installed on smart glasses or a head-mounted display. This system takes intraoral images of patients at the reception desk, analyzes the images using the AI ​​Dental guard system, and immediately provides diagnosis results and treatment plans.

[1234] Key Hardware and Software

[1235] The hardware used includes, for example, smart glasses (Vuzix M400) and a head-mounted display (Microsoft HoloLens) for capturing intraoral images and displaying the data, while the server hosts the high-performance image processing unit and AI models.

[1236] The software uses OpenCV as a library for image processing and AI analysis, and trained generative AI models. In addition, the HTTP protocol is used for data communication, sending and receiving image data and analysis results.

[1237] Program Overview

[1238] The system works as follows:

[1239] 1. Patient data collection and photography

[1240] Using a device (smart glasses or a head-mounted display), an image of the patient's oral cavity is taken. At the same time, basic information about the patient is also entered. The device then sends this image data to a server.

[1241] 2. Image data preprocessing

[1242] The server preprocesses the received image data, specifically performing noise removal, brightness adjustment, cropping, etc.

[1243] 3. Analysis using AI models

[1244] The pre-processed image data is then fed into a generative AI model, which analyzes the images and determines health conditions such as risk of cavities, risk of periodontal disease, staining level, and wear status.

[1245] 4. Diagnosis results and treatment plan proposal

[1246] Based on the results, the server automatically generates a health status report and recommendations for improvement, as well as specific treatment plans, which are then sent to the user's device and displayed visually.

[1247] 5. Checking the results and taking action

[1248] The user (dentist or staff) receives a notification on their device, reviews the generated report and improvement suggestions, and, if necessary, proposes an appropriate treatment plan for the patient.

[1249] Specific examples

[1250] When a patient visits the dental clinic, the dental clinic's reception staff uses smart glasses to take a photo of the patient's mouth. The captured image is uploaded to a server in real time and analyzed by an AI model. Based on the analysis results, the risk of tooth decay and the necessary treatment plan are displayed on the spot, and appropriate treatment can be immediately proposed to the patient.

[1251] Prompt Sentence Examples

[1252] A patient comes to your clinic. Use smart glasses to take intraoral photos of the patient and analyze them with the AI ​​Dental Guard system. Based on the analysis results, propose the best treatment plan for the patient.

[1253] This invention is expected to improve the efficiency of reception work and diagnostic accuracy at dental clinics, and will be a particularly useful solution for busy dentists and clinics with many patients.

[1254] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1255] Step 1: Collect and photograph patient data

[1256] The user (dental clinic staff) uses smart glasses or a head-mounted display to take intraoral images of the patient. At this time, the patient's basic information is also entered. The entered data (patient's basic information and intraoral images) is saved on the terminal. The terminal then sends this image data to the server. The input is the intraoral images and patient information, and the output is data transmission to the server. Specifically, the user takes several images using the terminal's camera, and then manually selects and uploads them.

[1257] Step 2: Preprocessing the image data

[1258] The server preprocesses the received intraoral image data. Specifically, it uses the OpenCV library to perform image processing such as noise removal, brightness adjustment, and cropping. The input is the received raw image data, and the output is preprocessed image data. Specifically, the server automatically performs image filtering and correction, and then converts the data into a format suitable for the AI ​​model.

[1259] Step 3: Analysis by AI model

[1260] The server inputs the preprocessed image data into the generated AI model. The generated AI model analyzes the image data and determines health conditions such as caries risk, periodontal disease risk, staining level, and wear status. The input is the preprocessed image data, and the output is the health condition assessment result. Specifically, the AI ​​model analyzes the characteristics of each pixel and detects patterns of risk factors.

[1261] Step 4: Generate a diagnosis and treatment plan

[1262] The server automatically generates a health status report and improvement suggestions based on the analysis results of the AI ​​model. It also generates a specific treatment plan. The input is the analysis results, and the output is an automatically generated report and treatment plan. Specifically, it uses templates to create a report and adds a treatment plan tailored to each patient's situation.

[1263] Step 5: Notification and display of results

[1264] The server sends the generated report and treatment plan to the user's device. The user (dentist or staff member) receives a notification on their device and reviews the generated report and recommendations. The input is the generated report and treatment plan, and the output is the information displayed on the user's device. Specifically, the application on the device displays a pop-up notification and takes the user to a screen where detailed results can be viewed.

[1265] Step 6: Handling the results

[1266] The user (dentist or staff) checks the notification on the user terminal and proposes the optimal treatment method for the patient based on the generated report and treatment plan. The input is the diagnosis results and treatment plan displayed on the terminal, and the output is a treatment proposal to the patient. In concrete terms, the dentist explains the treatment contents to the patient and sets a treatment schedule if necessary.

[1267] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1268] As one embodiment of the present invention, the AI ​​Dental Guard system allows users to easily and accurately monitor their dental health at home, receive professional diagnosis as needed, and recognizes the user's emotions to provide appropriate suggestions and advice for improvement. The system program processing is explained below in natural language.

[1269] User takes and uploads a photo

[1270] Users launch the AI ​​Dental guard app on their smartphone and use the camera to take a photo of their teeth. After taking the photo, they press the "Start Analysis" button in the app to upload the captured data to the server. For example, users can take photos of the front and back teeth with their smartphone and send the images through the app.

[1271] Upload the image to the server

[1272] The device (user's smartphone) sends the captured image data to the server. An internet connection is required for transmission. For example, the user can use Wi-Fi or mobile data to upload the captured image data to the server via a secure network.

[1273] The image analysis server performs the analysis

[1274] The server receives and stores the image data sent from the device. The received image is preprocessed and input into the AI ​​model. The AI ​​model analyzes the image data and determines health conditions such as tooth decay, risk of periodontal disease, stains, discoloration, and tooth wear. For example, the AI ​​model may detect early signs of tooth decay in the user's upper right molar.

[1275] Save the analysis results in a database

[1276] The server associates the analysis results with the user ID and saves them in a database. This allows each user's health history to be accumulated and used to improve the accuracy of future analyses and advice. For example, the server may save the results in the database as "User A's dental check results for October 2023."

[1277] Notify users of results and send suggestions for improvement

[1278] Based on the analysis, the server automatically generates a health report and recommendations for improvement, which are sent to the user's app. For example, the report might include a warning about a high risk of cavities in the upper right molar, along with recommendations for specific brushing and flossing techniques.

[1279] Emotion engine recognizes user emotions

[1280] When a user checks the results, the emotion engine uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to recognize their current emotional state. For example, the emotion engine can detect if the user is surprised or anxious about the results.

[1281] Adjusting improvement suggestions with an emotion engine

[1282] The emotion engine recognizes the user's emotions and tailors the suggestions to them accordingly. For example, if the user is feeling anxious, the engine generates a suggestion that includes a gentle, encouraging message or supporting information to recommend a visit to the dentist.

[1283] If you would like a professional diagnosis, please send us your images.

[1284] If a user requests a more detailed diagnosis, they can use another means of communication, such as the LINE official account, to send the image to a specialist and wait for a reply from the specialist. For example, a user can use the LINE app to send a photo they have taken to a dentist and request a more detailed diagnosis.

[1285] The expert will send the diagnosis to the user.

[1286] The expert will analyze the images and return a diagnosis and specific advice to the user, allowing the user to obtain detailed health information. For example, a dentist may diagnose the image and recommend appropriate treatment or preventative measures.

[1287] The above system supports users in dental care by conveniently checking their dental health at home and providing appropriate responses based on their emotions. It also allows users to easily receive a diagnosis from a specialist, enabling timely treatment and countermeasures.

[1288] The processing flow will be explained below.

[1289] Step 1:

[1290] The user launches the app.

[1291] The user taps and launches the AI ​​Dental guard app on their smartphone.

[1292] Step 2:

[1293] The user takes a photo of their teeth.

[1294] Users use the app's in-app camera to take a photo of their teeth with their smartphone camera, and are encouraged to clearly capture both the front and back teeth.

[1295] Step 3:

[1296] The user checks the shooting data.

[1297] Users can review the photos they've taken, take new ones if necessary, and once they're satisfied, press the "Start Analysis" button within the app.

[1298] Step 4:

[1299] The device sends the captured image data to the server.

[1300] The image data captured by the device is uploaded to a dedicated server via an internet connection, using Wi-Fi or mobile data.

[1301] Step 5:

[1302] The server receives the image data.

[1303] The server receives the image data sent from the terminal and stores it in a secure storage.

[1304] Step 6:

[1305] The server preprocesses the received image data.

[1306] The server performs preprocessing on the received image data, such as formatting, resizing, and normalization. This preprocessing converts the image data into a format suitable for the AI ​​model.

[1307] Step 7:

[1308] The server inputs preprocessed image data into the AI ​​model.

[1309] The server inputs the preprocessed image data into the AI ​​model and prepares it for analysis.

[1310] Step 8:

[1311] The AI ​​model analyzes the image data.

[1312] The AI ​​model analyzes the image data to determine health conditions such as risk of cavities, periodontal disease, stains, discoloration, and tooth wear.

[1313] Step 9:

[1314] The server generates the analysis results.

[1315] The server generates a health report based on the analysis results obtained from the AI ​​model, which includes a description of the health condition and individualized suggestions for improvement.

[1316] Step 10:

[1317] The server stores the analysis results in a database.

[1318] The analysis results generated by the server are stored in a database along with the user ID, allowing each user's health history to be accumulated.

[1319] Step 11:

[1320] The server sends the analysis results and improvement suggestions.

[1321] The server generates a health report and sends it to the user's app, which displays the user's health status and appropriate actions.

[1322] Step 12:

[1323] The user checks the results in the app.

[1324] Users receive in-app notifications and receive health status reports and suggestions for improvement.

[1325] Step 13:

[1326] The emotion engine recognizes the user's emotions.

[1327] When the user checks the results, the emotion engine uses the smartphone's camera and microphone to analyze the user's facial expressions and tone of voice to recognize their current emotional state. For example, the emotion engine can detect if the user is surprised or anxious about the results.

[1328] Step 14:

[1329] Adjustment of improvement suggestions by emotion engine.

[1330] The emotion engine recognizes the user's emotions and tailors the suggestions to them accordingly. For example, if the user is feeling anxious, the engine generates a suggestion that includes a gentle, encouraging message or supporting information to recommend a visit to the dentist.

[1331] Step 15:

[1332] If the user wishes to have a specialist diagnose the image, they can submit it.

[1333] If a user requests a more detailed diagnosis, they can use another means of communication, such as the LINE official account, to send the image to a specialist and wait for a reply from the specialist. For example, a user can use the LINE app to send a photo they have taken to a dentist and request a more detailed diagnosis.

[1334] Step 16:

[1335] The expert will send the diagnosis results to the user.

[1336] The expert will analyze the images and return a diagnosis and specific advice to the user, allowing the user to obtain detailed health information. For example, a dentist may diagnose the image and recommend appropriate treatment or preventative measures.

[1337] Example 2

[1338] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1339] Until now, users have had limited means to monitor their dental health at home, making it difficult to easily receive detailed analyses of their health or receive professional diagnoses. Furthermore, there were no systems that took into account the user's emotional state when reporting their health status or providing recommendations for improvement. This meant users were unable to take care of their teeth with peace of mind and found it difficult to receive professional diagnoses at the appropriate time.

[1340] The identification process by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes a means for receiving image data of the oral cavity captured by an image capturing device, a means for preprocessing the received image data of the oral cavity and inputting it into a generative AI model, a means for analyzing the image data of the oral cavity using the generative AI model and determining the health status, a means for automatically generating a health status report and improvement suggestions for the user based on the determination results, a means for transmitting the generated report and improvement suggestions to a user terminal, and a means for recognizing the user's emotional state and adjusting the content of the improvement suggestions according to the emotion. This allows the user to easily check the health status of their teeth at home and receive appropriate improvement suggestions according to their emotion. Furthermore, if a detailed diagnosis by a specialist is desired, a prompt response is also possible.

[1341] An "imaging device" is a device that a user uses to take images of the inside of the oral cavity.

[1342] "Image data" refers to digital data of an image of the inside of the oral cavity captured by an image capturing device.

[1343] The "receiving means" is a communication means for transferring image data from the user terminal to the server.

[1344] "Preprocessing" refers to data processing, such as removing noise from image data and correcting resolution, to enable accurate analysis by the AI ​​model.

[1345] A "generative AI model" is an artificial intelligence (AI) model used to analyze image data from inside the oral cavity and determine health status.

[1346] "Analysis" refers to the generative AI model analyzing image data of the inside of the mouth to determine the health of the teeth.

[1347] A "health report" is a report containing information describing a user's dental health based on the analysis results.

[1348] "Improvement suggestions" are recommendations on how to brush and care for your teeth based on the user's health condition.

[1349] A "user device" is an electronic device used by a user, such as a smartphone or tablet.

[1350] "Emotional state" refers to the user's current emotional state, as detected from facial expressions, tone of voice, etc.

[1351] An "emotion engine" is an artificial intelligence system used to analyze a user's emotional state.

[1352] "Means of communication" refers to methods of data transfer, including internet connection and messaging services such as LINE.

[1353] "Professionals" are medical professionals with professional qualifications, such as dentists and dental hygienists.

[1354] A "database" is an information system for storing health data and analysis results for each user.

[1355] As one embodiment of this invention, we provide the "AI Dental Guard System," which allows users to easily monitor their dental health at home and receive a diagnosis from a specialist if necessary. This system recognizes the user's emotional state and provides appropriate improvement suggestions and advice according to their emotions.

[1356] First, the user launches the dedicated app on their smartphone and uses the app's camera function to take an image of the inside of their mouth. Next, the captured image data is uploaded to the server by pressing the "Start Analysis" button within the app. The server saves the received image data and performs preprocessing such as noise removal.

[1357] The preprocessed image data is input into a generative AI model, which analyzes the input image data and determines the health status of the user based on factors such as the risk of cavities and periodontal disease, stains, discoloration, and tooth wear. The results of the assessment are associated with the user's ID and stored in a database.

[1358] The server automatically generates a report of the user's dental health based on the analysis results, along with suggestions for improvement. This report and suggestions are sent to the user's smartphone app. For example, it could warn the user that they are at high risk for cavities in their upper right molar, and include suggestions for specific brushing and flossing techniques.

[1359] When the user checks the results, the smartphone's camera and microphone are activated, and the emotion engine analyzes the user's facial expressions and tone of voice to recognize their emotional state. Based on the recognized emotional state, the server adjusts the content of the improvement suggestions. For example, if the user is feeling anxious, the server will provide suggestions that include a gentle, encouraging message or support information recommending a visit to the dentist.

[1360] Additionally, if a user desires a more detailed diagnosis, they can send the captured image to a specialist using other communication methods within the dedicated app (e.g., the official LINE account). The specialist will then respond to the user with a detailed diagnosis based on the image. For example, a dentist may analyze the captured image and advise, "There are signs of early tooth decay in the upper right molar, so immediate treatment is required."

[1361] This system allows users to easily check the health of their teeth at home and receive appropriate suggestions for improvement. Furthermore, by receiving a detailed diagnosis from a specialist promptly, timely treatment and measures can be taken.

[1362] An example prompt might look like this:

[1363] Please provide a detailed description of the system where users upload photos of their teeth taken with the AI ​​Dental guard app to the server.

[1364] In this way, the invention is designed to make users' lives more comfortable and support their health management.

[1365] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1366] Step 1:

[1367] The user launches the "AI Dental guard" app. The user uses the camera function of their smartphone to take images of the inside of their mouth. For example, the user takes photos of their front and back teeth. The input is the image data taken by the user, and the output is the captured image data.

[1368] Step 2:

[1369] The user presses the "Start Analysis" button to upload the captured image data to the server. Specifically, the app uses an internet connection to send the image data to the server. The input is the captured image data, and the output is the image data sent to the server.

[1370] Step 3:

[1371] The server receives image data sent from the terminal and stores it in cloud storage. The input is the sent image data, and the output is the stored image data. Specifically, the server stores the image data in secure storage.

[1372] Step 4:

[1373] The server performs preprocessing such as noise removal and resolution correction on the image data. The input is the stored image data, and the output is the preprocessed image data. Specifically, the server applies noise filtering and resolution correction algorithms.

[1374] Step 5:

[1375] The server inputs the preprocessed data into the generative AI model. The input is the preprocessed image data, and the output is the analysis results. Specifically, the generative AI model analyzes the image data and determines health conditions such as the risk of cavities and periodontal disease, stains, discoloration, and tooth wear.

[1376] Step 6:

[1377] The server associates the analysis results with the user ID and stores them in a database. The input is the analysis results and the user ID, and the output is the associated data. Specifically, the server organizes the analysis results by user and stores them in a database for future reference.

[1378] Step 7:

[1379] The server automatically generates a report of the user's dental health status based on the analysis results. It also automatically generates improvement suggestions. The input is the analysis results, and the output is the generated report and improvement suggestions. Specifically, the server uses an automatic generation algorithm to create the report and improvement suggestions.

[1380] Step 8:

[1381] The server sends the generated report and improvement suggestions to the user's app. The input is the generated report and improvement suggestions, and the output is the data sent to the user's device. Specifically, the server sends the data using an Internet connection.

[1382] Step 9:

[1383] When the user wants to check the results, the smartphone's camera and microphone are activated. The emotion engine analyzes the user's facial expressions and tone of voice to recognize their current emotional state. The input is data from the camera and microphone, and the output is the recognized emotional state. Specifically, the emotion engine uses video and audio analysis algorithms.

[1384] Step 10:

[1385] Based on the analysis results of the emotion engine, the server adjusts the content of the improvement proposal. The input is the emotional state and the initial improvement proposal, and the output is the adjusted improvement proposal. Specifically, the server adds or modifies messages and proposals according to the user's emotions.

[1386] Step 11:

[1387] If the user wishes to receive a more detailed diagnosis, they can select another communication method, such as the LINE official account, from within the app and send the captured image to an expert. The input is the captured image data and a request to send it to the expert, and the output is the image data sent to the expert. Specifically, the user sends the data to the expert using an app such as LINE.

[1388] Step 12:

[1389] The expert analyzes the sent images and returns the diagnosis and specific advice to the user. The input is the sent image data, and the output is the expert's diagnosis and advice. Specifically, the expert diagnoses based on the images and notifies the user of appropriate treatment methods and preventive measures.

[1390] In this way, the AI ​​Dental Guard system allows users to easily monitor their dental health at home and receive professional diagnosis, and supports users' dental care by providing appropriate improvement suggestions based on their emotions.

[1391] (Application example 2)

[1392] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1393] Conventional oral examination systems have the drawback of making it difficult for users to easily monitor their dental health at home, and they are unable to receive appropriate advice or suggestions for improvement based on their emotions. Furthermore, they lack the means to alleviate users' anxiety, making it difficult to receive a detailed diagnosis from a specialist promptly. This makes it difficult to detect dental problems early and take appropriate measures, potentially hindering users' dental health maintenance.

[1394] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1395] In this invention, the server includes means for receiving image data of the oral cavity captured by an imaging device, means for preprocessing the received image data of the oral cavity and inputting it into an AI model, means for analyzing the image data of the oral cavity using the AI ​​model to assess the health status, means for automatically generating a health status report and improvement suggestions for the user based on the assessment results, means for transmitting the generated report and improvement suggestions to a user terminal, and means for recognizing the user's emotions and adjusting the improvement suggestions according to the emotions. This allows users to easily monitor their dental health at home and receive appropriate improvement suggestions according to their emotions. It also enables prompt and detailed diagnosis by experts, reducing user anxiety and enabling early detection of dental problems and appropriate measures.

[1396] (Definitions of important words)

[1397] "Imaging device" refers to an electronic device used to take images of the inside of the oral cavity.

[1398] "Intraoral image data" refers to digital data that digitizes visual information about the oral cavity obtained by an image capturing device.

[1399] "Means for receiving" refers to a mechanism that allows a server to acquire data sent from a user terminal via a network.

[1400] "Preprocessing" refers to the process of converting and processing image data into an analyzable format before inputting it into an AI model.

[1401] "AI model" refers to a model that includes neural networks and machine learning algorithms built on the foundation of artificial intelligence technology.

[1402] "Means for determining health status" refers to the system that evaluates the health status of teeth and the oral cavity from the results of analysis by the AI ​​model and derives the results.

[1403] "Determination result" refers to the conclusion regarding health status output after the AI ​​model analyzes image data.

[1404] "Means for automatically generating improvement proposals" refers to a system that has the function of automatically generating improvement measures and advice appropriate for users based on the assessment results.

[1405] "User device" refers to electronic devices such as smartphones and tablets used by users.

[1406] "Means of recognizing emotions" refers to technology that analyzes the user's emotions and determines the appropriate response based on that.

[1407] "Communication means" refers to the methods and technologies used to send and receive data, including the Internet, mobile networks, and dedicated applications.

[1408] "Expert" refers to a medical professional with specialized knowledge and skills to diagnose oral health conditions and abnormalities.

[1409] MODE FOR CARRYING OUT THE INVENTION

[1410] The present invention is a system that allows users to easily monitor their dental health at home. The system recognizes the user's emotions and provides corresponding suggestions for improvement. It also allows users to receive detailed diagnostics from a specialist if necessary.

[1411] System configuration

[1412] The main components of the system are:

[1413] Image capture device (smartphone, etc.)

[1414] server

[1415] AI model

[1416] Emotion Recognition Engine

[1417] User devices (smartphones, tablets)

[1418] Program processing

[1419] 1. Image capture and data reception

[1420] Users take pictures of the inside of their mouths using the camera function on their smartphones, and the image data they take is uploaded to a server via the Internet, using Wi-Fi or mobile data communication.

[1421] 2. Preprocessing and input to the AI ​​model

[1422] The server preprocesses the received image data and converts it into a format suitable for the AI ​​model, including image resizing and normalization.

[1423] 3. Health status analysis

[1424] The server inputs the preprocessed image data into an AI model to analyze the health condition, which detects abnormalities in the teeth and oral cavity and assesses health risks such as cavities, periodontal disease, and stains.

[1425] 4. Emotional Recognition

[1426] The emotion recognition engine uses the smartphone's camera and microphone to analyze the user's emotions when they check the results, analyzing their facial expressions and tone of voice to recognize their current emotional state.

[1427] 5. Generate reports and improvement suggestions

[1428] Based on the analysis results and emotion recognition results, the server automatically generates a health status report for the user and improvement suggestions according to their emotions, including specific tooth brushing methods and lifestyle improvements.

[1429] 6. Notifications to User Devices

[1430] The generated reports and improvement suggestions are sent to the user's smartphone or tablet, where they can view the information through the app.

[1431] 7. Detailed diagnosis by an expert

[1432] If the user wishes to receive a more detailed diagnosis, the captured images of the oral cavity can be sent to a specialist via another means of communication. The specialist will analyze the images and provide the user with a diagnosis and specific advice.

[1433] Specific examples

[1434] For example, if a user notices something wrong with their teeth, they can take a picture of their teeth using their smartphone and upload it to the server through the app. The AI ​​model then analyzes the image and detects signs of tooth decay. The emotion recognition engine recognizes that the user is feeling anxious and generates a gentle message saying, "You are at risk of tooth decay. Don't worry, we'll support you. We recommend that you visit the dentist as soon as possible." This suggestion is immediately sent to the user's smartphone.

[1435] Prompt Sentence Examples

[1436] Below are some examples of specific prompts that can be fed into a generative AI model:

[1437] Implement an API that allows users to upload photos taken with their smartphones to check their dental health at home. The API will analyze the dental health status using an AI model and return a warning message if there is an abnormality. It will also analyze the user's emotions using an emotion recognition engine and customize improvement suggestions based on the results. The output should be in JSON format, containing the health status and suggestions.

[1438] In this way, by utilizing specific prompt sentences, generative AI models can be used efficiently and system development can proceed smoothly.

[1439] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1440] Program processing steps

[1441] Step 1:

[1442] Image capture and data upload

[1443] The user takes a photograph of the inside of the mouth using a smartphone. This image data is uploaded from the smartphone to the server. The user presses the "Start Analysis" button in the app, and the device sends the image data to the server over a secure network. The input is the image data of the inside of the mouth that was photographed, and the output is the image data sent to the server.

[1444] Step 2:

[1445] Image data preprocessing

[1446] The server receives the received image data and performs preprocessing. Specifically, it resizes, normalizes, and removes noise if necessary. The preprocessed image data is converted into a format suitable for the AI ​​model. The input is the raw image data uploaded to the server, and the output is image data converted into a format that can be input into the AI ​​model.

[1447] Step 3:

[1448] Health status analysis using AI models

[1449] The server inputs the preprocessed image data into the AI ​​model and performs the analysis. The AI ​​model evaluates health risks such as cavities, periodontal disease, stains, discoloration, and tooth wear. The input is the preprocessed image data, and the output is a judgment result regarding the health status. For example, the AI ​​model may detect signs of early cavities from the image.

[1450] Step 4:

[1451] Emotion recognition

[1452] When a user checks the results of the AI ​​analysis, the emotion recognition engine uses the device's camera and microphone to analyze the user's facial expressions and tone of voice. This allows the system to recognize the user's current emotional state. The input is facial expression and voice data acquired from the smartphone, and the output is the user's emotional state. For example, the emotion engine can detect that the user is surprised by the results.

[1453] Step 5:

[1454] Generate reports and improvement suggestions

[1455] The server automatically generates a report and improvement suggestions based on the AI's health assessment results and the emotional state obtained from the emotion recognition engine. This includes advice on proper tooth brushing methods and lifestyle habits. The input is the AI ​​assessment results and the user's emotional state, and the output is a customized improvement suggestion message. For example, it might say, "Incipient tooth decay has been detected. Don't worry. We recommend that you visit a dentist as soon as possible."

[1456] Step 6:

[1457] Notifications on user devices

[1458] The reports and improvement suggestions generated by the server are sent to the user's smartphone, where the user can check the information through the app. The input is the generated report and suggestion data, and the output is the report and suggestion message displayed on the user's device.

[1459] Step 7:

[1460] Detailed diagnosis by an expert

[1461] If the user requests a detailed diagnosis, the image data captured within the app is sent to a specialist via another means of communication (e.g., a messaging app). The specialist analyzes the data and provides the user with a diagnosis and specific advice. The input is the image data of the inside of the oral cavity, and the output is a detailed diagnosis and advice from the specialist.

[1462] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1463] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1464] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1465] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1466] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1467] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1468] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1469] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1470] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1471] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1472] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1473] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1474] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1475] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1476] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1477] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1478] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1479] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1480] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1481] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1482] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1483] The following is further disclosed regarding the above embodiment.

[1484] (Claim 1)

[1485] a means for receiving image data of the inside of the oral cavity captured by the image capturing device;

[1486] A means for preprocessing the received intraoral image data and inputting it into an AI model;

[1487] A means of analyzing image data from inside the oral cavity using an AI model to determine health status,

[1488] A means for automatically generating a report of the user's health status and suggestions for improvement based on the assessment results;

[1489] means for transmitting the generated report and improvement suggestions to a user terminal;

[1490] A system including:

[1491] (Claim 2)

[1492] The system according to claim 1, further comprising a means for transmitting the captured image data of the oral cavity to an expert via another communication means and receiving the expert's diagnosis results, if a detailed diagnosis by an expert is desired.

[1493] (Claim 3)

[1494] 2. The system according to claim 1, further comprising a database means for storing health data for each user and using the data to improve the accuracy of future analyses and improvement suggestions.

[1495] "Example 1"

[1496] (Claim 1)

[1497] a means for receiving image data of the inside of the oral cavity captured by the image capturing device;

[1498] A means for preprocessing the received intraoral image data and inputting it into a generative AI model;

[1499] A means for sending prompts to the generative AI model, analyzing image data of the oral cavity, and determining the health status;

[1500] A means for automatically generating a report of the user's health status and suggestions for improvement based on the assessment results;

[1501] means for transmitting the generated report and improvement suggestions to a user terminal;

[1502] A system including:

[1503] (Claim 2)

[1504] The system according to claim 1, further comprising a means for transmitting the captured image data of the oral cavity to an expert via another communication means and receiving the expert's diagnosis results, if a detailed diagnosis by an expert is desired.

[1505] (Claim 3)

[1506] 2. The system according to claim 1, further comprising a database means for storing health data for each user and using the data to improve the accuracy of future analyses and improvement suggestions.

[1507] "Application Example 1"

[1508] (Claim 1)

[1509] a means for receiving image data of the inside of the oral cavity captured by the image capturing device;

[1510] A means for preprocessing the received intraoral image data and inputting it into the generated AI model;

[1511] A means of analyzing image data from inside the oral cavity using an AI model to determine health status,

[1512] A means for automatically generating a report of the user's health status and suggestions for improvement based on the assessment results;

[1513] means for transmitting the generated report and improvement suggestions to a user terminal;

[1514] A means for collecting patient data at the reception desk and storing it together with the image data;

[1515] A means for proposing a treatment plan based on the analysis results;

[1516] A system including:

[1517] (Claim 2)

[1518] The system according to claim 1, further comprising a means for transmitting the captured image data of the oral cavity to an expert via another communication means and receiving the expert's diagnosis results, if a detailed diagnosis by an expert is desired.

[1519] (Claim 3)

[1520] 2. The system according to claim 1, further comprising a database means for storing health data for each user and using the data to improve the accuracy of future analyses and improvement suggestions.

[1521] "Example 2: Combining Emotion Engines"

[1522] (Claim 1)

[1523] a means for receiving image data of the inside of the oral cavity captured by the image capturing device;

[1524] A means for preprocessing the received intraoral image data and inputting it into a generative AI model;

[1525] A means to analyze image data of the oral cavity using a generative AI model and determine the health status,

[1526] A means for automatically generating a report of the user's health status and suggestions for improvement based on the assessment results;

[1527] means for transmitting the generated report and improvement suggestions to a user terminal;

[1528] A means of recognizing the user's emotional state and adjusting the content of improvement suggestions according to the emotion;

[1529] A system including:

[1530] (Claim 2)

[1531] The system according to claim 1, further comprising a means for transmitting the captured image data of the oral cavity to an expert via another communication means and receiving the expert's diagnosis results, if a detailed diagnosis by an expert is desired.

[1532] (Claim 3)

[1533] 2. The system according to claim 1, further comprising a database means for storing health data for each user and using the data to improve the accuracy of future analyses and improvement suggestions.

[1534] "Application example 2 when combining emotion engines"

[1535] (Claim 1)

[1536] a means for receiving image data of the inside of the oral cavity captured by the image capturing device;

[1537] A means for preprocessing the received intraoral image data and inputting it into an AI model;

[1538] A means of analyzing image data from inside the oral cavity using an AI model to determine health status,

[1539] A means for automatically generating a report of the user's health status and suggestions for improvement based on the assessment results;

[1540] means for transmitting the generated report and improvement suggestions to a user terminal;

[1541] A means of recognizing user emotions and tailoring improvement suggestions accordingly;

[1542] A system including:

[1543] (Claim 2)

[1544] The system according to claim 1, further comprising a means for transmitting the captured image data of the oral cavity to an expert via another communication means and receiving the expert's diagnosis results, if a detailed diagnosis by an expert is desired.

[1545] (Claim 3)

[1546] 2. The system according to claim 1, further comprising a database means for storing health data for each user and using the data to improve the accuracy of future analyses and improvement suggestions. [Explanation of symbols]

[1547] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for receiving image data of the inside of the oral cavity captured by the image capturing device; A means for preprocessing the received intraoral image data and inputting it into an AI model; A means of analyzing image data from inside the oral cavity using an AI model to determine health status, A means for automatically generating a report of the user's health status and suggestions for improvement based on the assessment results; means for transmitting the generated report and improvement suggestions to a user terminal; A system including:

2. The system of claim 1 further comprises a means for transmitting the captured image data of the oral cavity to an expert via another communication means and receiving the expert's diagnosis results if a detailed diagnosis by an expert is desired.

3. 2. The system according to claim 1, further comprising database means for storing health data for each user and using the data to improve the accuracy of future analyses and improvement suggestions.

Images