system

Abstract

We provide the system. [Solution] A means for acquiring images of multiple surfaces inside the oral cavity using an image acquisition device, A means for preprocessing acquired images to generate image data, An analytical means using a generative model that analyzes image data to determine the presence and progression of dental caries, Based on the analysis results, a means to predict the number of hospital visits and treatment costs required for treatment, A means of outputting the prediction results to the terminal, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance that responds to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Many modern people are busy and have difficulty undergoing regular dental check-ups. As a result, cases of leaving cavities untreated without realizing it and allowing them to worsen are increasing. Worsened cavities require long-term and costly treatment, and early detection is important to avoid this. However, in conventional dental check-ups, patients need to visit a dental clinic directly, which takes time and effort. This results in the problem of missing the opportunity for early treatment.

Means for Solving the Problems

[0005] This invention provides a system that enables users to easily understand their dental health at home and create an early treatment plan. To achieve this, the user acquires images of multiple surfaces in the oral cavity using an imaging device and uploads them to a server via a dedicated application. On the server, the acquired images are preprocessed, and a generative model performs image analysis. This analysis determines the presence and progression of cavities and predicts the number of visits and treatment costs required. This result is output to the user's terminal, allowing for appropriate treatment planning before visiting a dental clinic.

[0006] An "image acquisition device" is a device used to acquire images of the inside of the oral cavity, and may include the camera function of a smart device.

[0007] "Preprocessing" refers to the process of converting acquired images into a format that can be analyzed, and includes operations such as format conversion and resolution adjustment of image data.

[0008] A "generative model" is an AI technology used to create new information based on input data, and in this system, it is a model used to determine the presence and progression of tooth decay.

[0009] "Analysis means" refers to a processing method that uses a generative model to analyze image data and perform the detection and evaluation of tooth decay.

[0010] "Prediction results" refer to the number of hospital visits and treatment costs predicted based on the image analysis results, and are provided as reference information for users when creating a treatment plan. [Brief explanation of the drawing]

[0011] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3]This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0012] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0013] First, let's explain the terminology used in the following explanation.

[0014] In the following embodiments, the labeled processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0015] In the following embodiments, the labeled RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0016] In the following embodiments, the labeled storage is one or more non-volatile storage devices that store various programs, various parameters, and the like. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0017] In the following embodiments, the labeled communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0018] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0019] [First Embodiment]

[0020] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0021] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0022] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0023] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0024] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0027] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0028] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0030] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0031] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0032] The present invention aims to provide a dental diagnostic support system that allows users to easily and quickly understand the condition of their oral cavity at home. Specific embodiments of the present invention are described below.

[0033] This system begins with the user taking images of their upper, lower, left, and right teeth using an image-capturing device such as a smartphone or digital camera. Users must take the images under adequate lighting and ensure that all teeth are clearly visible. Since the captured images may not be suitable for analysis on the server as is, the system performs image format conversion and compression as needed, depending on the device.

[0034] Next, the device uploads the captured images to a server using a dedicated application or web portal. During this process, a secure communication protocol is used to safely transmit the data.

[0035] The server inputs the received images into an AI analysis model. The AI ​​model has been pre-trained with a large amount of dental image data and uses deep learning algorithms to identify areas in the images that are likely to have cavities. Through AI analysis, the locations and progression of cavities that are most likely to occur are extracted as numerical data.

[0036] Based on the analysis results, the server then determines the need for treatment. Using a machine learning model based on past treatment data, it predicts the number of hospital visits and treatment costs for a specific stage of progression. At this stage, comparisons with similar cases and statistical analysis are performed to improve prediction accuracy.

[0037] These prediction results are sent back to the device in real time, allowing the user to develop an appropriate treatment plan based on them. Ultimately, the user can use this information to make rational appointments at the dental clinic.

[0038] For example, suppose an image taken by a user is analyzed by an AI model, and an early-stage cavity is detected in the upper right molar. In this case, one visit to the dentist is recommended for treatment, with an estimated cost of approximately 5,000 yen, allowing the user to schedule their appointment appropriately. In this way, this system is expected to contribute to maintaining oral health while reducing the burden on the user.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] Users will use a smartphone or digital camera to take images of their upper, lower, left, and right teeth. Each image must be clear and sharp, and sufficient lighting should be ensured during shooting.

[0042] Step 2:

[0043] The device uploads the captured images to a dedicated application or web portal. The images are formatted and compressed as needed, and then prepared for transmission to the server.

[0044] Step 3:

[0045] The server receives the image sent from the terminal and verifies its appropriate format and quality. At this point, pre-processing such as adjusting the image resolution and reducing unnecessary noise is performed.

[0046] Step 4:

[0047] The server passes the pre-processed images to an AI image analysis model. The model uses a multi-layer neural network to analyze the images and identify areas that may have tooth decay.

[0048] Step 5:

[0049] The server analyzes data obtained from the AI ​​model to determine the presence and progression of tooth decay. The diagnosis integrates information about the location and size of the decay.

[0050] Step 6:

[0051] Based on the diagnostic results, the server uses a machine learning model based on historical data to predict the number of hospital visits and costs required for treatment. Statistical estimations are also performed based on similar cases.

[0052] Step 7:

[0053] The server sends the results to the terminal. This allows the user to receive information that enables them to properly develop a treatment plan.

[0054] Step 8:

[0055] Users can review the prediction results and use them to make an appointment at a dental clinic. They can also take and submit images again if necessary.

[0056] (Example 1)

[0057] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0058] This diagnostic system aims to solve the challenge of enabling users to easily and quickly understand their oral health status and create appropriate treatment plans without requiring them to expend much effort. Specifically, it is required to accurately identify areas potentially affected by tooth decay and provide the diagnostic results to the user in an easily usable format.

[0059] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0060] In this invention, the server includes means for acquiring images of multiple surfaces inside the oral cavity using an image capture function, means for preprocessing the acquired images to generate image information, and means for transmitting the image information using a dedicated application or web portal. This enables the user to quickly understand the health status of their oral cavity and to develop an appropriate treatment plan.

[0061] "Image capture function" refers to the function of imaging devices or equipment used to capture multiple surfaces within the oral cavity.

[0062] "Multiple surfaces within the oral cavity" refers to different parts and angles within the oral cavity, including individual teeth and gums.

[0063] "Image information" refers to data obtained by pre-processing captured images and converting them into a format suitable for analysis.

[0064] "Means of transmission" refers to technical methods for securely transmitting image information to a remote server via a dedicated application or web portal.

[0065] "Received image information" refers to the data in the state after the server has received the data sent from the user's terminal.

[0066] A "generative model" is a model that includes an algorithm that learns from a large amount of training data beforehand and performs data analysis and prediction.

[0067] A "deep learning algorithm" is a method that uses a multi-layered neural network structure to perform feature extraction and pattern recognition based on large amounts of data.

[0068] "Diagnostic data" refers to the results of analyzing image information to generate data regarding the presence and progression of tooth decay.

[0069] This invention provides a diagnostic support system that allows users to easily and quickly understand the condition of their oral cavity at home. First, the user uses a device with image capture capabilities, such as a smartphone or digital camera, to capture images of multiple surfaces in the oral cavity, i.e., the surfaces of the teeth and gums. The captured images are pre-processed on the device and generated as image information. Pre-processing includes image format conversion and compression.

[0070] Next, the terminal uses a dedicated application or web portal to send image information to the server via a secure communication protocol. On the server side, the received image information is input into a generative model. This generative AI model uses a deep learning algorithm and has been pre-trained on a large amount of dental images. As a result, the server can identify areas that may have cavities with high accuracy.

[0071] Next, the server predicts the progression of tooth decay, the number of visits required for treatment, and the treatment cost based on the analysis results. The server sends this information back to the terminal in real time, allowing the user to use the diagnostic results for self-management. For example, if the AI ​​detects an early-stage cavity in the upper right molar through an image taken by the user, one treatment is recommended, and the cost is estimated at 5,000 yen. This information is then used in the user's treatment plan.

[0072] An example of a prompt for a generated AI model is, "Please describe a system that uploads images of the inside of the mouth taken with a smartphone to an AI and automatically detects cavities." In this way, the system is expected to reduce the burden on users and support the maintenance of oral health.

[0073] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0074] Step 1:

[0075] The user takes images of their oral cavity. The user uses the image capture function of a smartphone or digital camera to take detailed images of their teeth and gums. During this process, sufficient lighting is ensured, and multiple surfaces of the oral cavity are captured from various angles. The input is the captured images. The output is visual image data.

[0076] Step 2:

[0077] The device performs image preprocessing. The device takes in the acquired image and converts or compresses it into a format suitable for analysis. For example, it may reduce the image size or remove noise. The input is the captured image data. The output is the preprocessed image information.

[0078] Step 3:

[0079] The terminal sends image information to the server. Data is uploaded to the server via a dedicated application or web portal using a secure communication protocol. The input is pre-processed image information. The output is the image information sent to the server.

[0080] Step 4:

[0081] The server inputs image information into an AI analysis model. The server takes in the image information and passes it to the generating AI model to perform the analysis. This analysis uses a deep learning algorithm to automatically identify areas with a high probability of tooth decay. The input is the image information received by the server. The output is the location information of the analyzed tooth decay.

[0082] Step 5:

[0083] The server generates diagnostic data. The server compiles the AI ​​analysis results and derives numerical data regarding the progression of the disease and the need for treatment. It also makes treatment predictions based on past data. The input is the output data of the AI ​​analysis model. The output consists of diagnostic data and treatment prediction information.

[0084] Step 6:

[0085] The server sends the diagnostic results to the terminal. The generated diagnostic data is sent back to the terminal in real time, allowing the user to quickly check the results. The input is the diagnostic data on the server. The output is the diagnostic results transferred to the terminal.

[0086] Step 7:

[0087] The user develops a treatment plan based on the diagnosis results. The user adjusts the treatment content and appointment schedule based on the received information, and makes appointments if necessary. The input is the diagnosis results displayed on the terminal. The output is the specific treatment plan.

[0088] (Application Example 1)

[0089] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0090] Conventional oral cavity diagnostic support systems often presented users with difficulties in image capture and understanding diagnostic results. Furthermore, the inconsistency in image quality due to user-initiated capture made accurate diagnosis challenging. A solution to these problems is needed.

[0091] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0092] In this invention, the server includes means for acquiring images of multiple parts of the oral cavity using an image acquisition device, means for preprocessing the acquired visual information to generate visual data, and analysis means using a generative model to analyze the visual data and determine the presence and progression level of tooth decay. As a result, with the support of a home-use device, users can easily photograph the inside of their mouths, and the analyzed results are provided in an easy-to-understand manner, enabling prompt and accurate dental treatment.

[0093] An "image acquisition device" is a device used to visually record the condition of the oral cavity, and its role is to allow the user to take images of the inside of their mouth.

[0094] "Visual information" refers to digital visual data captured by an image acquisition device, and is information that represents the condition of the oral cavity.

[0095] "Visual data" refers to digital data generated based on visual information, which has been converted into a format suitable for analysis and processing.

[0096] A "generative model" is an algorithm that analyzes visual data to determine the condition of teeth and the progression level of tooth decay; it is a computational model based on machine learning technology.

[0097] "Analysis means" refers to a device or program that uses a generative model to process visual data and perform a process to make judgments about the health status of teeth.

[0098] A "home-use device" is a device designed to support the user's daily tasks within their home, and in this invention, it is a device that assists with intraoral photography and presents the results of the analysis.

[0099] The system of the present invention is designed to allow users to easily monitor their oral health at home. First, the user takes images of their oral cavity using an image acquisition device via a home-use device. The home-use device facilitates the shooting process by guiding the user to the optimal shooting angle, for example, using dedicated voice assistance. The images are acquired as visual information and subsequently converted into visual data.

[0100] The server analyzes visual data through a generative model to determine the presence and progression of tooth decay. The generative model utilizes a multi-layer neural network, enabling highly accurate diagnoses based on large amounts of data. The analysis results are fed back to the user through a home device, allowing the user to quickly understand the results and plan the necessary treatment.

[0101] As a concrete example, suppose a user instructs a home-use device to "take a picture of the inside of the mouth and start the diagnosis." The home-use device assists with the photography and sends the captured image to a server. The server analyzes the received image, sends the analysis results back to the home-use device in real time, and notifies the user of the results. This entire process allows the user to receive an initial diagnosis at home before going to a specialist, enabling a quick and appropriate response.

[0102] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0103] Step 1:

[0104] The user instructs the home device to take intraoral photographs. The user initiates the process by using the prompt "Take intraoral photographs and begin diagnosis" and operating the home device. The input is the user's voice command, and the output is the home device starting camera preparation.

[0105] Step 2:

[0106] A home-use device activates the image acquisition system and takes images of the user's oral cavity. The terminal adjusts the optimal angle and captures images at appropriate brightness. The input is the lighting environment in the home and the user's eye position, and the output is a high-resolution intraoral image.

[0107] Step 3:

[0108] The acquired images are preprocessed as visual data. The image data captured by the device is converted to the optimal format (e.g., JPEG) and compressed as needed. The input is raw image data of the oral cavity, and the output is visual data for transmission to the server.

[0109] Step 4:

[0110] The terminal sends pre-processed visual data to the server. A secure communication protocol is used to upload the visual data to the server. The input is compressed visual data, and the output is the server that received the data.

[0111] Step 5:

[0112] The server inputs visual data into an AI model for analysis. Based on a deep learning algorithm, it evaluates the presence and progression level of tooth decay. The input is visual data uploaded to the server, and the output is numerical data of the analysis results.

[0113] Step 6:

[0114] The server sends feedback to the user based on the analysis results. The analyzed results are sent back to the home device in real time, allowing the user to check them immediately. The input is analyzed numerical data, and the output is diagnostic information displayed on the home device.

[0115] Step 7:

[0116] The user receives feedback and decides on the next course of action as needed. After reviewing the analysis results, the user then develops a treatment plan. The input is the diagnostic information displayed on the home device, and the output is the user's decision to schedule a treatment appointment.

[0117] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0118] This invention provides a more personalized diagnostic experience by combining an emotion recognition function with a dental diagnostic support system that allows users to understand the condition of their oral cavity. Specific embodiments of each element are described below.

[0119] This system begins with the user taking images of various parts of their oral cavity using a smartphone or digital camera. The user uploads the images to the system via a terminal. The terminal converts these images to the appropriate format and sends them to the server.

[0120] The server preprocesses the received images and inputs this data into an AI image analysis model. The model used here includes a multi-layer neural network and can detect areas potentially affected by tooth decay with high accuracy. The model analyzes the acquired data to determine the presence and progression of tooth decay. Based on the diagnosis, the server predicts the number of visits and costs required for treatment, based on past cases.

[0121] Furthermore, this system incorporates an emotion engine. This engine analyzes the user's voice or facial expression input to determine the user's emotional state. Based on this, the server adjusts how diagnostic and predictive results are presented according to the user's emotions. For example, if the user is feeling anxious, the results will be explained more carefully, ensuring optimal communication for the user.

[0122] For example, suppose a user uses the system to check the condition of their teeth. The analysis reveals a minor cavity, and it is predicted that treatment can be completed in a single visit. In this case, the emotion engine analyzes the user's voice and, if it detects that the user is slightly nervous, the server reads the results aloud in a dedicated, calming tone and provides additional reassuring messages as needed. In this way, the system helps users gain a sense of security through a diagnosis at home while planning an appropriate treatment.

[0123] This invention is expected to improve conventional dental diagnostic processes and enhance the user experience.

[0124] The following describes the processing flow.

[0125] Step 1:

[0126] The user takes images of various parts of their oral cavity using a smartphone or digital camera. Ensure sufficient lighting and capture all teeth clearly.

[0127] Step 2:

[0128] The device receives the captured images via a dedicated application and converts them to the appropriate format. This conversion process includes adjusting the image resolution and compressing the image.

[0129] Step 3:

[0130] The device uploads the converted image to the server. The image is transmitted using a secure communication protocol, thus maintaining data confidentiality.

[0131] Step 4:

[0132] The server preprocesses the received images. This process involves denoising and enhancing edges to facilitate recognition by the AI ​​analysis model.

[0133] Step 5:

[0134] The server inputs pre-processed images into an AI image analysis model. The model, based on a multi-layer neural network, determines the presence and progression of tooth decay.

[0135] Step 6:

[0136] After the analysis is complete, the server uses a machine learning model based on historical data to calculate the predicted number of treatments and costs. This result is statistically estimated based on treatment patterns.

[0137] Step 7:

[0138] The server simultaneously analyzes the audio and facial images received from the terminal via the emotion engine. The engine uses this data to determine the user's emotional state.

[0139] Step 8:

[0140] Depending on the user's emotional state, the server sends the diagnostic and predictive results to the user's device in a format appropriate to their needs. If the user appears anxious, the results are explained carefully in a gentle tone.

[0141] Step 9:

[0142] Users can review the information presented on their device and confidently make appointments and create treatment plans at dental clinics. This process allows users to have a diagnostic experience that takes their feelings into consideration.

[0143] (Example 2)

[0144] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0145] The objective of this invention is not only to enable users to easily diagnose the condition of their oral cavity at home, but also to consider the user's emotional state and provide individually optimized diagnostic information. This aims to alleviate user anxiety and doubts, and provide a more satisfying treatment experience.

[0146] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0147] In this invention, the server includes an analysis means that uses a generative model to analyze image data and determine the presence and progression of dental caries, a means that predicts the number of hospital visits and treatment costs required for treatment based on the analysis results, and a means that analyzes emotional information and adjusts the method of presenting the results. This makes it possible not only to provide highly accurate oral diagnosis results but also to provide information that takes into account the user's emotions.

[0148] An "image capture device" is a device used by a user to photograph the condition of their oral cavity, and is a device that includes an imaging function built into a general information device.

[0149] "Preprocessing" refers to the process of preparing acquired image data to make it easier to analyze, and includes steps such as noise reduction and contrast adjustment.

[0150] A "generative model" is an analytical model used to determine the presence or absence of dental caries and its progression, and it enables highly accurate diagnosis, particularly through the use of multi-layer algorithms.

[0151] "Analysis means" refers to a process within a system that uses a generative model to analyze image data and determine the patient's condition.

[0152] A "predictive method" is a procedure for predicting the number of hospital visits and treatment costs required for treatment, based on the analysis results.

[0153] "Emotional information" refers to data obtained from the user's voice or facial expressions, which is used to determine the user's emotional state through analysis.

[0154] "Means for analyzing emotional information" refers to a server-based process that processes a user's emotional information and determines their emotional state.

[0155] "Means for adjusting the method of presenting results" refers to functions that optimize how diagnostic results and predictions are communicated according to the user's emotional state.

[0156] This invention is a diagnostic support system that allows users to easily diagnose the condition of their oral cavity. The system includes an image capture device for users to take images of their oral cavity using a smartphone or digital camera. The user imports the captured images into their device and uploads them to a server via a dedicated application.

[0157] First, the terminal converts the received image into an appropriate format. This allows the server to handle the image efficiently. The converted image is then sent to the server using a secure communication protocol.

[0158] Next, the server preprocesses the received images. This preprocessing removes noise and adjusts contrast to generate image data suitable for analysis. The preprocessed image data is then input into a generative AI model using a multi-layer algorithm. This model is capable of accurately determining the presence and progression of dental caries.

[0159] Based on the analysis results, the server predicts the number of hospital visits and treatment costs required, referencing past cases. This prediction is optimized based on the user's emotional information. The server uses an emotion recognition engine to analyze emotional information from the user's voice and facial expressions, and reflects the results in how the diagnostic information is presented. Therefore, if the user is feeling anxious, the diagnostic results will be presented in a more careful and reassuring manner.

[0160] For example, if a user uses the system for the first time and a minor cavity is detected, the system predicts that the treatment can be completed in a single visit. If the emotion engine detects that the user is feeling anxious, the server will output a reassuring message such as, "The treatment is simple, so please don't worry." In this way, the system helps users gain a sense of security through home-based diagnosis and supports them in planning an appropriate treatment.

[0161] Examples of prompts for the generating AI model include: "Use intraoral images to check for the presence and progression of cavities, and adjust the diagnosis results by taking into account emotional information from the user's facial expressions and voice."

[0162] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0163] Step 1:

[0164] The user takes images of the inside of their mouth using a smartphone or digital camera. These images serve as input. It is desirable to take these images from multiple angles to record the condition of the oral cavity in detail.

[0165] Step 2:

[0166] The user imports the captured image into their device and uploads it to the system via a dedicated application. The input to this process is the captured image, and the output is the image file imported into the device. The device then prepares this image to facilitate conversion to an appropriate format (e.g., JPEG or PNG).

[0167] Step 3:

[0168] The terminal converts the captured image to an appropriate format. The input for this step is the captured image file, and the output is the converted image file. Converting the image format facilitates analysis on the server. The terminal then sends this converted image to the server.

[0169] Step 4:

[0170] The server preprocesses the received image data. The input for this step is the converted image file, and the output is the preprocessed image data. Specifically, the server performs denoising and contrast adjustment on the image to prepare it for use in the analysis model.

[0171] Step 5:

[0172] The server inputs pre-processed image data into the AI ​​model. The input for each step is the pre-processed image data, and the output is the analysis result. The AI ​​model uses this data to diagnose dental caries and determine the presence and progression of cavities.

[0173] Step 6:

[0174] The server makes predictions for treatment based on the analysis results. The input for this step is the analysis results, and the output is a prediction of the number of hospital visits required and the treatment costs. The server refers to past case data and derives predictions from similar cases.

[0175] Step 7:

[0176] After capturing an image, the user inputs voice and facial expression data into the system. The server analyzes this data using an emotion recognition engine. The input for this step is the user's voice and facial expression data, and the output is a judgment about the user's emotional state.

[0177] Step 8:

[0178] The server optimizes how the diagnostic results are presented based on the user's emotional state. The input for this step is the predicted results and the user's emotional information, while the output is a personalized presentation of the diagnostic results. For example, if the user is feeling anxious, the server provides reassurance by adding detailed explanations in a gentle tone.

[0179] (Application Example 2)

[0180] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0181] In modern dental diagnosis, patients often need to visit a medical facility in person, which can be time-consuming, laborious, and emotionally burdensome. Furthermore, traditional diagnostic processes often lack individualized care that takes into account the patient's emotional state, leading to anxiety and tension. There is a growing need for improved diagnostic accuracy and, consequently, a better user experience.

[0182] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0183] In this invention, the server includes means for acquiring images of multiple surfaces inside the oral cavity using an image acquisition device; means for preprocessing the acquired images to generate image information; means for analyzing the image information and determining the presence and progression of dental caries using a generative model; means for predicting the number of hospital visits and treatment costs required for treatment based on the analysis results; means for adjusting the presentation method of the analysis results using an emotion recognition engine that determines the emotional state from the user's voice or facial expressions; and means for outputting the prediction results to the device. As a result, the user can receive a highly accurate dental diagnosis from the comfort of their home and obtain personalized feedback tailored to their emotional state.

[0184] An "image acquisition device" is a device used to capture images of multiple surfaces inside the oral cavity.

[0185] "Image information" refers to data generated by preprocessing acquired images.

[0186] A "generative model" is an analytical algorithm used to determine the presence and progression of dental caries, and it includes multilayer neural networks.

[0187] "Analysis means" refers to a function that performs a process of determining the presence and progression of dental caries using image information.

[0188] An "emotion recognition engine" is a system that identifies the user's emotional state from their voice or facial expressions and incorporates that information into the way the diagnostic results are presented.

[0189] "Prediction results" refer to the estimated number of hospital visits and treatment costs required based on the analysis.

[0190] "Device" refers to equipment including a terminal or display for outputting prediction results to the user.

[0191] This invention describes a specific example of a system that efficiently diagnoses the oral condition of a user and enables personalized responses based on their emotions.

[0192] First, the user uses the image acquisition function of the computing device, which is an image acquisition device, to capture images of multiple surfaces inside the oral cavity. The acquired images are pre-processed by the terminal and sent to the server as image information. The server uses this image information to perform analysis using a generation AI model and a multi-layer neural network to determine the presence and progression of dental caries. To improve the accuracy of this analysis, the model employs pattern recognition technology that takes past data into consideration.

[0193] Based on the analysis results, the server uses a generated AI model to predict the number of hospital visits and treatment costs required. In this process, an emotion recognition engine is used to detect the user's emotional state through voice or facial expressions, and the presentation of the diagnostic results is adjusted accordingly. For example, if the user is feeling anxious, the server will add calming information to the presentation.

[0194] This allows users to obtain highly accurate diagnostic results at home and develop treatment plans with peace of mind. For example, if a user detects a minor cavity, the server may provide feedback such as, "Please rest assured that the treatment will be completed in a short time."

[0195] An example of a prompt to a generative AI model is, "Generate a calming phrase to use when the user is feeling anxious." This allows the system to respond in a way that is sensitive to the user's emotions.

[0196] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0197] Step 1:

[0198] The user captures images of multiple surfaces within the oral cavity using the image acquisition function of the computing device. The input is the raw intraoral images obtained from the camera, and the output is this image data. At this stage, the terminal performs basic contrast adjustment and noise reduction preprocessing.

[0199] Step 2:

[0200] The terminal sends pre-processed image data to the server. The input is the pre-processed image data, and the output is the image data that has been successfully received by the server. The server then confirms receipt of the data and performs a format check.

[0201] Step 3:

[0202] The server uses a generative AI model as an analysis tool to determine the presence and progression of dental caries from image data. Here, a multilayer neural network is applied to analyze the input image data. The output is a judgment regarding the presence and characteristics of tooth decay.

[0203] Step 4:

[0204] The server predicts the number of hospital visits and treatment costs required based on the analysis results. The input is the analysis results, and the output is the predicted number of hospital visits and costs based on a probabilistic model. The server records this information and prepares to communicate it appropriately to the user.

[0205] Step 5:

[0206] Similarly, the server uses an emotion recognition engine to analyze the user's voice input and identify their emotional state. The input is voice data from the user, and an emotion is determined using a voice analysis algorithm. The output is information about the user's emotional state.

[0207] Step 6:

[0208] The server uses prompts to optimally present diagnostic and predictive results based on the user's emotional state. Input consists of analysis results and emotional state information, and, if necessary, retrieves phrases to alleviate tension from an AI model. Output is personalized information presented to the user.

[0209] Step 7:

[0210] The user develops a treatment plan based on the information received from the device. Input is personalized diagnostic results and predictive information, while output is a reasonable treatment schedule based on their own health condition. The user can then consider this and confidently proceed to the next step.

[0211] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0212] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0213] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0214] [Second Embodiment]

[0215] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0216] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0217] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0218] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0219] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0220] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0221] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0222] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0223] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0224] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0225] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0226] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0227] The present invention aims to provide a dental diagnostic support system that allows users to easily and quickly understand the condition of their oral cavity at home. Specific embodiments of the present invention are described below.

[0228] This system begins with the user taking images of their upper, lower, left, and right teeth using an image-capturing device such as a smartphone or digital camera. Users must take the images under adequate lighting and ensure that all teeth are clearly visible. Since the captured images may not be suitable for analysis on the server as is, the system performs image format conversion and compression as needed, depending on the device.

[0229] Next, the device uploads the captured images to a server using a dedicated application or web portal. During this process, a secure communication protocol is used to safely transmit the data.

[0230] The server inputs the received images into an AI analysis model. The AI ​​model has been pre-trained with a large amount of dental image data and uses deep learning algorithms to identify areas in the images that are likely to have cavities. Through AI analysis, the locations and progression of cavities that are most likely to occur are extracted as numerical data.

[0231] Based on the analysis results, the server then determines the need for treatment. Using a machine learning model based on past treatment data, it predicts the number of hospital visits and treatment costs for a specific stage of progression. At this stage, comparisons with similar cases and statistical analysis are performed to improve prediction accuracy.

[0232] These prediction results are sent back to the device in real time, allowing the user to develop an appropriate treatment plan based on them. Ultimately, the user can use this information to make rational appointments at the dental clinic.

[0233] For example, suppose an image taken by a user is analyzed by an AI model, and an early-stage cavity is detected in the upper right molar. In this case, one visit to the dentist is recommended for treatment, with an estimated cost of approximately 5,000 yen, allowing the user to schedule their appointment appropriately. In this way, this system is expected to contribute to maintaining oral health while reducing the burden on the user.

[0234] The following describes the processing flow.

[0235] Step 1:

[0236] Users will use a smartphone or digital camera to take images of their upper, lower, left, and right teeth. Each image must be clear and sharp, and sufficient lighting should be ensured during shooting.

[0237] Step 2:

[0238] The device uploads the captured images to a dedicated application or web portal. The images are formatted and compressed as needed, and then prepared for transmission to the server.

[0239] Step 3:

[0240] The server receives the image sent from the terminal and verifies its appropriate format and quality. At this point, pre-processing such as adjusting the image resolution and reducing unnecessary noise is performed.

[0241] Step 4:

[0242] The server passes the pre-processed images to an AI image analysis model. The model uses a multi-layer neural network to analyze the images and identify areas that may have tooth decay.

[0243] Step 5:

[0244] The server analyzes data obtained from the AI ​​model to determine the presence and progression of tooth decay. The diagnosis integrates information about the location and size of the decay.

[0245] Step 6:

[0246] Based on the diagnostic results, the server uses a machine learning model based on historical data to predict the number of hospital visits and costs required for treatment. Statistical estimations are also performed based on similar cases.

[0247] Step 7:

[0248] The server sends the results to the terminal. This allows the user to receive information that enables them to properly develop a treatment plan.

[0249] Step 8:

[0250] Users can review the prediction results and use them to make an appointment at a dental clinic. They can also take and submit images again if necessary.

[0251] (Example 1)

[0252] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0253] This diagnostic system aims to solve the challenge of enabling users to easily and quickly understand their oral health status and create appropriate treatment plans without requiring them to expend much effort. Specifically, it is required to accurately identify areas potentially affected by tooth decay and provide the diagnostic results to the user in an easily usable format.

[0254] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0255] In this invention, the server includes means for acquiring images of multiple surfaces inside the oral cavity using an image capture function, means for preprocessing the acquired images to generate image information, and means for transmitting the image information using a dedicated application or web portal. This enables the user to quickly understand the health status of their oral cavity and to develop an appropriate treatment plan.

[0256] "Image capture function" refers to the function of imaging devices or equipment used to capture multiple surfaces within the oral cavity.

[0257] "Multiple surfaces within the oral cavity" refers to different parts and angles within the oral cavity, including individual teeth and gums.

[0258] "Image information" refers to data obtained by pre-processing captured images and converting them into a format suitable for analysis.

[0259] "Means of transmission" refers to technical methods for securely transmitting image information to a remote server via a dedicated application or web portal.

[0260] "Received image information" refers to the data in the state after the server has received the data sent from the user's terminal.

[0261] A "generative model" is a model that includes an algorithm that learns from a large amount of training data beforehand and performs data analysis and prediction.

[0262] A "deep learning algorithm" is a method that uses a multi-layered neural network structure to perform feature extraction and pattern recognition based on large amounts of data.

[0263] "Diagnostic data" refers to the results of analyzing image information to generate data regarding the presence and progression of tooth decay.

[0264] This invention provides a diagnostic support system that allows users to easily and quickly understand the condition of their oral cavity at home. First, the user uses a device with image capture capabilities, such as a smartphone or digital camera, to capture images of multiple surfaces in the oral cavity, i.e., the surfaces of the teeth and gums. The captured images are pre-processed on the device and generated as image information. Pre-processing includes image format conversion and compression.

[0265] Next, the terminal uses a dedicated application or web portal to send image information to the server via a secure communication protocol. On the server side, the received image information is input into a generative model. This generative AI model uses a deep learning algorithm and has been pre-trained on a large amount of dental images. As a result, the server can identify areas that may have cavities with high accuracy.

[0266] Next, the server predicts the progression of tooth decay, the number of visits required for treatment, and the treatment cost based on the analysis results. The server sends this information back to the terminal in real time, allowing the user to use the diagnostic results for self-management. For example, if the AI ​​detects an early-stage cavity in the upper right molar through an image taken by the user, one treatment is recommended, and the cost is estimated at 5,000 yen. This information is then used in the user's treatment plan.

[0267] An example of a prompt for a generated AI model is, "Please describe a system that uploads images of the inside of the mouth taken with a smartphone to an AI and automatically detects cavities." In this way, the system is expected to reduce the burden on users and support the maintenance of oral health.

[0268] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0269] Step 1:

[0270] The user takes images of their oral cavity. The user uses the image capture function of a smartphone or digital camera to take detailed images of their teeth and gums. During this process, sufficient lighting is ensured, and multiple surfaces of the oral cavity are captured from various angles. The input is the captured images. The output is visual image data.

[0271] Step 2:

[0272] The device performs image preprocessing. The device takes in the acquired image and converts or compresses it into a format suitable for analysis. For example, it may reduce the image size or remove noise. The input is the captured image data. The output is the preprocessed image information.

[0273] Step 3:

[0274] The terminal sends image information to the server. Data is uploaded to the server via a dedicated application or web portal using a secure communication protocol. The input is pre-processed image information. The output is the image information sent to the server.

[0275] Step 4:

[0276] The server inputs image information into an AI analysis model. The server takes in the image information and passes it to the generating AI model to perform the analysis. This analysis uses a deep learning algorithm to automatically identify areas with a high probability of tooth decay. The input is the image information received by the server. The output is the location information of the analyzed tooth decay.

[0277] Step 5:

[0278] The server generates diagnostic data. The server compiles the AI ​​analysis results and derives numerical data regarding the progression of the disease and the need for treatment. It also makes treatment predictions based on past data. The input is the output data of the AI ​​analysis model. The output consists of diagnostic data and treatment prediction information.

[0279] Step 6:

[0280] The server sends the diagnostic results to the terminal. The generated diagnostic data is sent back to the terminal in real time, allowing the user to quickly check the results. The input is the diagnostic data on the server. The output is the diagnostic results transferred to the terminal.

[0281] Step 7:

[0282] The user develops a treatment plan based on the diagnosis results. The user adjusts the treatment content and appointment schedule based on the received information, and makes appointments if necessary. The input is the diagnosis results displayed on the terminal. The output is the specific treatment plan.

[0283] (Application Example 1)

[0284] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart glasses 214 are referred to as a "terminal".

[0285] In a conventional intraoral diagnosis support system, users often had difficulty in taking images and understanding diagnosis results. In addition, there was a problem that the quality of images varied due to the user taking the images themselves, making accurate diagnosis difficult. A method for solving this problem is required.

[0286] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0287] In this invention, the server includes means for acquiring images of a plurality of parts inside the oral cavity using an image acquisition device, means for preprocessing the acquired visual information to generate visual data, and analysis means using a generation model for analyzing the visual data and determining the presence or absence and progression level of dental caries. As a result, with the assistance of household devices, users can easily take pictures inside the oral cavity, and the analyzed results are provided in an easy-to-understand manner, enabling rapid and accurate dental treatment.

[0288] The "image acquisition device" is a device for visually recording the state inside the oral cavity and plays a role of allowing the user to take pictures of the inside of the oral cavity.

[0289] The "visual information" refers to digital visual data captured by the image acquisition device and is information representing the state inside the oral cavity.

[0290] The "visual data" is digital data generated based on visual information and is information converted into a format suitable for analysis and processing.

[0291] The "generation model" is an algorithm for analyzing visual data and determining the state of teeth and the progression level of dental caries, and is a computational model based on machine learning technology.

[0292] "Analysis means" refers to a device or program that uses a generative model to process visual data and perform a process to make judgments about the health status of teeth.

[0293] A "home-use device" is a device designed to support the user's daily tasks within their home, and in this invention, it is a device that assists with intraoral photography and presents the results of the analysis.

[0294] The system of the present invention is designed to allow users to easily monitor their oral health at home. First, the user takes images of their oral cavity using an image acquisition device via a home-use device. The home-use device facilitates the shooting process by guiding the user to the optimal shooting angle, for example, using dedicated voice assistance. The images are acquired as visual information and subsequently converted into visual data.

[0295] The server analyzes visual data through a generative model to determine the presence and progression of tooth decay. The generative model utilizes a multi-layer neural network, enabling highly accurate diagnoses based on large amounts of data. The analysis results are fed back to the user through a home device, allowing the user to quickly understand the results and plan the necessary treatment.

[0296] As a concrete example, suppose a user instructs a home-use device to "take a picture of the inside of the mouth and start the diagnosis." The home-use device assists with the photography and sends the captured image to a server. The server analyzes the received image, sends the analysis results back to the home-use device in real time, and notifies the user of the results. This entire process allows the user to receive an initial diagnosis at home before going to a specialist, enabling a quick and appropriate response.

[0297] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0298] Step 1:

[0299] The user gives an instruction for intraoral imaging to the household device. By using the prompt sentence "Start the diagnosis by imaging the inside of the mouth" and operating the household device to indicate that the user wants to image the inside of the mouth, the imaging process is started. The input is the user's voice instruction, and the output is the start of camera preparation by the household device.

[0300] Step 2:

[0301] The household device activates the image acquisition device to image the user's oral cavity. The terminal adjusts to the optimal angle and captures an image with appropriate brightness. The input is the light environment in the home and the position of the user's eyes, and the output is a high-resolution intraoral image.

[0302] Step 3:

[0303] Preprocess the acquired image as visual data. The terminal converts the captured image data into an optimal format (e.g., JPEG) and performs compression processing if necessary. The input is the raw image data of the oral cavity, and the output is the visual data for server transmission.

[0304] Step 4:

[0305] The terminal sends the preprocessed visual data to the server. Upload the visual data to the server using a secure communication protocol. The input is the compressed visual data, and the output is the server that has received the data.

[0306] Step 5:

[0307] The server inputs the visual data into the generative AI model for analysis. Based on the deep learning algorithm, evaluate the presence and progression level of dental caries. The input is the visual data uploaded to the server, and the output is the numerical data of the analysis result.

[0308] Step 6:

[0309] The server sends feedback to the user based on the analysis results. The analyzed results are sent back to the home device in real time, allowing the user to check them immediately. The input is analyzed numerical data, and the output is diagnostic information displayed on the home device.

[0310] Step 7:

[0311] The user receives feedback and decides on the next course of action as needed. After reviewing the analysis results, the user then develops a treatment plan. The input is the diagnostic information displayed on the home device, and the output is the user's decision to schedule a treatment appointment.

[0312] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0313] This invention provides a more personalized diagnostic experience by combining an emotion recognition function with a dental diagnostic support system that allows users to understand the condition of their oral cavity. Specific embodiments of each element are described below.

[0314] This system begins with the user taking images of various parts of their oral cavity using a smartphone or digital camera. The user uploads the images to the system via a terminal. The terminal converts these images to the appropriate format and sends them to the server.

[0315] The server preprocesses the received images and inputs this data into an AI image analysis model. The model used here includes a multi-layer neural network and can detect areas potentially affected by tooth decay with high accuracy. The model analyzes the acquired data to determine the presence and progression of tooth decay. Based on the diagnosis, the server predicts the number of visits and costs required for treatment, based on past cases.

[0316] Furthermore, this system incorporates an emotion engine. This engine analyzes the user's voice or facial expression input to determine the user's emotional state. Based on this, the server adjusts how diagnostic and predictive results are presented according to the user's emotions. For example, if the user is feeling anxious, the results will be explained more carefully, ensuring optimal communication for the user.

[0317] For example, suppose a user uses the system to check the condition of their teeth. The analysis reveals a minor cavity, and it is predicted that treatment can be completed in a single visit. In this case, the emotion engine analyzes the user's voice and, if it detects that the user is slightly nervous, the server reads the results aloud in a dedicated, calming tone and provides additional reassuring messages as needed. In this way, the system helps users gain a sense of security through a diagnosis at home while planning an appropriate treatment.

[0318] This invention is expected to improve conventional dental diagnostic processes and enhance the user experience.

[0319] The following describes the processing flow.

[0320] Step 1:

[0321] The user takes images of various parts of their oral cavity using a smartphone or digital camera. Ensure sufficient lighting and capture all teeth clearly.

[0322] Step 2:

[0323] The device receives the captured images via a dedicated application and converts them to the appropriate format. This conversion process includes adjusting the image resolution and compressing the image.

[0324] Step 3:

[0325] The device uploads the converted image to the server. The image is transmitted using a secure communication protocol, thus maintaining data confidentiality.

[0326] Step 4:

[0327] The server preprocesses the received images. This process involves denoising and enhancing edges to facilitate recognition by the AI ​​analysis model.

[0328] Step 5:

[0329] The server inputs pre-processed images into an AI image analysis model. The model, based on a multi-layer neural network, determines the presence and progression of tooth decay.

[0330] Step 6:

[0331] After the analysis is complete, the server uses a machine learning model based on historical data to calculate the predicted number of treatments and costs. This result is statistically estimated based on treatment patterns.

[0332] Step 7:

[0333] The server simultaneously analyzes the audio and facial images received from the terminal via the emotion engine. The engine uses this data to determine the user's emotional state.

[0334] Step 8:

[0335] Depending on the user's emotional state, the server sends the diagnostic and predictive results to the user's device in a format appropriate to their needs. If the user appears anxious, the results are explained carefully in a gentle tone.

[0336] Step 9:

[0337] Users can review the information presented on their device and confidently make appointments and create treatment plans at dental clinics. This process allows users to have a diagnostic experience that takes their feelings into consideration.

[0338] (Example 2)

[0339] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0340] The objective of this invention is not only to enable users to easily diagnose the condition of their oral cavity at home, but also to consider the user's emotional state and provide individually optimized diagnostic information. This aims to alleviate user anxiety and doubts, and provide a more satisfying treatment experience.

[0341] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0342] In this invention, the server includes an analysis means that uses a generative model to analyze image data and determine the presence and progression of dental caries, a means that predicts the number of hospital visits and treatment costs required for treatment based on the analysis results, and a means that analyzes emotional information and adjusts the method of presenting the results. This makes it possible not only to provide highly accurate oral diagnosis results but also to provide information that takes into account the user's emotions.

[0343] An "image capture device" is a device used by a user to photograph the condition of their oral cavity, and is a device that includes an imaging function built into a general information device.

[0344] "Preprocessing" refers to the process of preparing acquired image data to make it easier to analyze, and includes steps such as noise reduction and contrast adjustment.

[0345] A "generative model" is an analytical model used to determine the presence or absence of dental caries and its progression, and it enables highly accurate diagnosis, particularly through the use of multi-layer algorithms.

[0346] "Analysis means" refers to a process within a system that uses a generative model to analyze image data and determine the patient's condition.

[0347] A "predictive method" is a procedure for predicting the number of hospital visits and treatment costs required for treatment, based on the analysis results.

[0348] "Emotional information" refers to data obtained from the user's voice or facial expressions, which is used to determine the user's emotional state through analysis.

[0349] "Means for analyzing emotional information" refers to a server-based process that processes a user's emotional information and determines their emotional state.

[0350] "Means for adjusting the method of presenting results" refers to functions that optimize how diagnostic results and predictions are communicated according to the user's emotional state.

[0351] This invention is a diagnostic support system that allows users to easily diagnose the condition of their oral cavity. The system includes an image capture device for users to take images of their oral cavity using a smartphone or digital camera. The user imports the captured images into their device and uploads them to a server via a dedicated application.

[0352] First, the terminal converts the received image into an appropriate format. This allows the server to handle the image efficiently. The converted image is then sent to the server using a secure communication protocol.

[0353] Next, the server preprocesses the received images. This preprocessing removes noise and adjusts contrast to generate image data suitable for analysis. The preprocessed image data is then input into a generative AI model using a multi-layer algorithm. This model is capable of accurately determining the presence and progression of dental caries.

[0354] Based on the analysis results, the server predicts the number of hospital visits and treatment costs required, referencing past cases. This prediction is optimized based on the user's emotional information. The server uses an emotion recognition engine to analyze emotional information from the user's voice and facial expressions, and reflects the results in how the diagnostic information is presented. Therefore, if the user is feeling anxious, the diagnostic results will be presented in a more careful and reassuring manner.

[0355] For example, if a user uses the system for the first time and a minor cavity is detected, the system predicts that the treatment can be completed in a single visit. If the emotion engine detects that the user is feeling anxious, the server will output a reassuring message such as, "The treatment is simple, so please don't worry." In this way, the system helps users gain a sense of security through home-based diagnosis and supports them in planning an appropriate treatment.

[0356] Examples of prompts for the generating AI model include: "Use intraoral images to check for the presence and progression of cavities, and adjust the diagnosis results by taking into account emotional information from the user's facial expressions and voice."

[0357] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0358] Step 1:

[0359] The user takes images of the inside of their mouth using a smartphone or digital camera. These images serve as input. It is desirable to take these images from multiple angles to record the condition of the oral cavity in detail.

[0360] Step 2:

[0361] The user imports the captured image into their device and uploads it to the system via a dedicated application. The input to this process is the captured image, and the output is the image file imported into the device. The device then prepares this image to facilitate conversion to an appropriate format (e.g., JPEG or PNG).

[0362] Step 3:

[0363] The terminal converts the captured image to an appropriate format. The input for this step is the captured image file, and the output is the converted image file. Converting the image format facilitates analysis on the server. The terminal then sends this converted image to the server.

[0364] Step 4:

[0365] The server preprocesses the received image data. The input for this step is the converted image file, and the output is the preprocessed image data. Specifically, the server performs denoising and contrast adjustment on the image to prepare it for use in the analysis model.

[0366] Step 5:

[0367] The server inputs pre-processed image data into the AI ​​model. The input for each step is the pre-processed image data, and the output is the analysis result. The AI ​​model uses this data to diagnose dental caries and determine the presence and progression of cavities.

[0368] Step 6:

[0369] The server makes predictions for treatment based on the analysis results. The input for this step is the analysis results, and the output is a prediction of the number of hospital visits required and the treatment costs. The server refers to past case data and derives predictions from similar cases.

[0370] Step 7:

[0371] After capturing an image, the user inputs voice and facial expression data into the system. The server analyzes this data using an emotion recognition engine. The input for this step is the user's voice and facial expression data, and the output is a judgment about the user's emotional state.

[0372] Step 8:

[0373] The server optimizes how the diagnostic results are presented based on the user's emotional state. The input for this step is the predicted results and the user's emotional information, while the output is a personalized presentation of the diagnostic results. For example, if the user is feeling anxious, the server provides reassurance by adding detailed explanations in a gentle tone.

[0374] (Application Example 2)

[0375] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0376] In modern dental diagnosis, patients often need to visit a medical facility in person, which can be time-consuming, laborious, and emotionally burdensome. Furthermore, traditional diagnostic processes often lack individualized care that takes into account the patient's emotional state, leading to anxiety and tension. There is a growing need for improved diagnostic accuracy and, consequently, a better user experience.

[0377] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0378] In this invention, the server includes means for acquiring images of multiple surfaces inside the oral cavity using an image acquisition device; means for preprocessing the acquired images to generate image information; means for analyzing the image information and determining the presence and progression of dental caries using a generative model; means for predicting the number of hospital visits and treatment costs required for treatment based on the analysis results; means for adjusting the presentation method of the analysis results using an emotion recognition engine that determines the emotional state from the user's voice or facial expressions; and means for outputting the prediction results to the device. As a result, the user can receive a highly accurate dental diagnosis from the comfort of their home and obtain personalized feedback tailored to their emotional state.

[0379] An "image acquisition device" is a device used to capture images of multiple surfaces inside the oral cavity.

[0380] "Image information" refers to data generated by preprocessing acquired images.

[0381] A "generative model" is an analytical algorithm used to determine the presence and progression of dental caries, and it includes multilayer neural networks.

[0382] "Analysis means" refers to a function that performs a process of determining the presence and progression of dental caries using image information.

[0383] An "emotion recognition engine" is a system that identifies the user's emotional state from their voice or facial expressions and incorporates that information into the way the diagnostic results are presented.

[0384] "Prediction results" refer to the estimated number of hospital visits and treatment costs required based on the analysis.

[0385] "Device" refers to equipment including a terminal or display for outputting prediction results to the user.

[0386] This invention describes a specific example of a system that efficiently diagnoses the oral condition of a user and enables personalized responses based on their emotions.

[0387] First, the user uses the image acquisition function of the computing device, which is an image acquisition device, to capture images of multiple surfaces inside the oral cavity. The acquired images are pre-processed by the terminal and sent to the server as image information. The server uses this image information to perform analysis using a generation AI model and a multi-layer neural network to determine the presence and progression of dental caries. To improve the accuracy of this analysis, the model employs pattern recognition technology that takes past data into consideration.

[0388] Based on the analysis results, the server uses a generated AI model to predict the number of hospital visits and treatment costs required. In this process, an emotion recognition engine is used to detect the user's emotional state through voice or facial expressions, and the presentation of the diagnostic results is adjusted accordingly. For example, if the user is feeling anxious, the server will add calming information to the presentation.

[0389] This allows users to obtain highly accurate diagnostic results at home and develop treatment plans with peace of mind. For example, if a user detects a minor cavity, the server may provide feedback such as, "Please rest assured that the treatment will be completed in a short time."

[0390] An example of a prompt to a generative AI model is, "Generate a calming phrase to use when the user is feeling anxious." This allows the system to respond in a way that is sensitive to the user's emotions.

[0391] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0392] Step 1:

[0393] The user captures images of multiple surfaces within the oral cavity using the image acquisition function of the computing device. The input is the raw intraoral images obtained from the camera, and the output is this image data. At this stage, the terminal performs basic contrast adjustment and noise reduction preprocessing.

[0394] Step 2:

[0395] The terminal sends pre-processed image data to the server. The input is the pre-processed image data, and the output is the image data that has been successfully received by the server. The server then confirms receipt of the data and performs a format check.

[0396] Step 3:

[0397] The server uses a generative AI model as an analysis tool to determine the presence and progression of dental caries from image data. Here, a multilayer neural network is applied to analyze the input image data. The output is a judgment regarding the presence and characteristics of tooth decay.

[0398] Step 4:

[0399] The server predicts the number of hospital visits and treatment costs required based on the analysis results. The input is the analysis results, and the output is the predicted number of hospital visits and costs based on a probabilistic model. The server records this information and prepares to communicate it appropriately to the user.

[0400] Step 5:

[0401] Similarly, the server uses an emotion recognition engine to analyze the user's voice input and identify their emotional state. The input is voice data from the user, and an emotion is determined using a voice analysis algorithm. The output is information about the user's emotional state.

[0402] Step 6:

[0403] The server uses prompts to optimally present diagnostic and predictive results based on the user's emotional state. Input consists of analysis results and emotional state information, and, if necessary, retrieves phrases to alleviate tension from an AI model. Output is personalized information presented to the user.

[0404] Step 7:

[0405] The user develops a treatment plan based on the information received from the device. Input is personalized diagnostic results and predictive information, while output is a reasonable treatment schedule based on their own health condition. The user can then consider this and confidently proceed to the next step.

[0406] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0407] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0408] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0409] [Third Embodiment]

[0410] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0411] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0412] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0413] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0414] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0415] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0416] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0417] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0418] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0419] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0420] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0421] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0422] The present invention aims to provide a dental diagnostic support system that allows users to easily and quickly understand the condition of their oral cavity at home. Specific embodiments of the present invention are described below.

[0423] This system begins with the user taking images of their upper, lower, left, and right teeth using an image-capturing device such as a smartphone or digital camera. Users must take the images under adequate lighting and ensure that all teeth are clearly visible. Since the captured images may not be suitable for analysis on the server as is, the system performs image format conversion and compression as needed, depending on the device.

[0424] Next, the device uploads the captured images to a server using a dedicated application or web portal. During this process, a secure communication protocol is used to safely transmit the data.

[0425] The server inputs the received images into an AI analysis model. The AI ​​model has been pre-trained with a large amount of dental image data and uses deep learning algorithms to identify areas in the images that are likely to have cavities. Through AI analysis, the locations and progression of cavities that are most likely to occur are extracted as numerical data.

[0426] Based on the analysis results, the server then determines the need for treatment. Using a machine learning model based on past treatment data, it predicts the number of hospital visits and treatment costs for a specific stage of progression. At this stage, comparisons with similar cases and statistical analysis are performed to improve prediction accuracy.

[0427] These prediction results are sent back to the device in real time, allowing the user to develop an appropriate treatment plan based on them. Ultimately, the user can use this information to make rational appointments at the dental clinic.

[0428] For example, suppose an image taken by a user is analyzed by an AI model, and an early-stage cavity is detected in the upper right molar. In this case, one visit to the dentist is recommended for treatment, with an estimated cost of approximately 5,000 yen, allowing the user to schedule their appointment appropriately. In this way, this system is expected to contribute to maintaining oral health while reducing the burden on the user.

[0429] The following describes the processing flow.

[0430] Step 1:

[0431] Users will use a smartphone or digital camera to take images of their upper, lower, left, and right teeth. Each image must be clear and sharp, and sufficient lighting should be ensured during shooting.

[0432] Step 2:

[0433] The device uploads the captured images to a dedicated application or web portal. The images are formatted and compressed as needed, and then prepared for transmission to the server.

[0434] Step 3:

[0435] The server receives the image sent from the terminal and verifies its appropriate format and quality. At this point, pre-processing such as adjusting the image resolution and reducing unnecessary noise is performed.

[0436] Step 4:

[0437] The server passes the pre-processed images to an AI image analysis model. The model uses a multi-layer neural network to analyze the images and identify areas that may have tooth decay.

[0438] Step 5:

[0439] The server analyzes data obtained from the AI ​​model to determine the presence and progression of tooth decay. The diagnosis integrates information about the location and size of the decay.

[0440] Step 6:

[0441] Based on the diagnostic results, the server uses a machine learning model based on historical data to predict the number of hospital visits and costs required for treatment. Statistical estimations are also performed based on similar cases.

[0442] Step 7:

[0443] The server sends the results to the terminal. This allows the user to receive information that enables them to properly develop a treatment plan.

[0444] Step 8:

[0445] Users can review the prediction results and use them to make an appointment at a dental clinic. They can also take and submit images again if necessary.

[0446] (Example 1)

[0447] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0448] This diagnostic system aims to solve the challenge of enabling users to easily and quickly understand their oral health status and create appropriate treatment plans without requiring them to expend much effort. Specifically, it is required to accurately identify areas potentially affected by tooth decay and provide the diagnostic results to the user in an easily usable format.

[0449] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0450] In this invention, the server includes means for acquiring images of multiple surfaces inside the oral cavity using an image capture function, means for preprocessing the acquired images to generate image information, and means for transmitting the image information using a dedicated application or web portal. This enables the user to quickly understand the health status of their oral cavity and to develop an appropriate treatment plan.

[0451] "Image capture function" refers to the function of imaging devices or equipment used to capture multiple surfaces within the oral cavity.

[0452] "Multiple surfaces within the oral cavity" refers to different parts and angles within the oral cavity, including individual teeth and gums.

[0453] "Image information" refers to data obtained by pre-processing captured images and converting them into a format suitable for analysis.

[0454] "Means of transmission" refers to technical methods for securely transmitting image information to a remote server via a dedicated application or web portal.

[0455] "Received image information" refers to the data in the state after the server has received the data sent from the user's terminal.

[0456] A "generative model" is a model that includes an algorithm that learns from a large amount of training data beforehand and performs data analysis and prediction.

[0457] A "deep learning algorithm" is a method that uses a multi-layered neural network structure to perform feature extraction and pattern recognition based on large amounts of data.

[0458] "Diagnostic data" refers to the results of analyzing image information to generate data regarding the presence and progression of tooth decay.

[0459] This invention provides a diagnostic support system that allows users to easily and quickly understand the condition of their oral cavity at home. First, the user uses a device with image capture capabilities, such as a smartphone or digital camera, to capture images of multiple surfaces in the oral cavity, i.e., the surfaces of the teeth and gums. The captured images are pre-processed on the device and generated as image information. Pre-processing includes image format conversion and compression.

[0460] Next, the terminal uses a dedicated application or web portal to send image information to the server via a secure communication protocol. On the server side, the received image information is input into a generative model. This generative AI model uses a deep learning algorithm and has been pre-trained on a large amount of dental images. As a result, the server can identify areas that may have cavities with high accuracy.

[0461] Next, the server predicts the progression of tooth decay, the number of visits required for treatment, and the treatment cost based on the analysis results. The server sends this information back to the terminal in real time, allowing the user to use the diagnostic results for self-management. For example, if the AI ​​detects an early-stage cavity in the upper right molar through an image taken by the user, one treatment is recommended, and the cost is estimated at 5,000 yen. This information is then used in the user's treatment plan.

[0462] An example of a prompt for a generated AI model is, "Please describe a system that uploads images of the inside of the mouth taken with a smartphone to an AI and automatically detects cavities." In this way, the system is expected to reduce the burden on users and support the maintenance of oral health.

[0463] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0464] Step 1:

[0465] The user takes images of their oral cavity. The user uses the image capture function of a smartphone or digital camera to take detailed images of their teeth and gums. During this process, sufficient lighting is ensured, and multiple surfaces of the oral cavity are captured from various angles. The input is the captured images. The output is visual image data.

[0466] Step 2:

[0467] The device performs image preprocessing. The device takes in the acquired image and converts or compresses it into a format suitable for analysis. For example, it may reduce the image size or remove noise. The input is the captured image data. The output is the preprocessed image information.

[0468] Step 3:

[0469] The terminal sends image information to the server. Data is uploaded to the server via a dedicated application or web portal using a secure communication protocol. The input is pre-processed image information. The output is the image information sent to the server.

[0470] Step 4:

[0471] The server inputs image information into an AI analysis model. The server takes in the image information and passes it to the generating AI model to perform the analysis. This analysis uses a deep learning algorithm to automatically identify areas with a high probability of tooth decay. The input is the image information received by the server. The output is the location information of the analyzed tooth decay.

[0472] Step 5:

[0473] The server generates diagnostic data. The server compiles the AI ​​analysis results and derives numerical data regarding the progression of the disease and the need for treatment. It also makes treatment predictions based on past data. The input is the output data of the AI ​​analysis model. The output consists of diagnostic data and treatment prediction information.

[0474] Step 6:

[0475] The server sends the diagnostic results to the terminal. The generated diagnostic data is sent back to the terminal in real time, allowing the user to quickly check the results. The input is the diagnostic data on the server. The output is the diagnostic results transferred to the terminal.

[0476] Step 7:

[0477] The user develops a treatment plan based on the diagnosis results. The user adjusts the treatment content and appointment schedule based on the received information, and makes appointments if necessary. The input is the diagnosis results displayed on the terminal. The output is the specific treatment plan.

[0478] (Application Example 1)

[0479] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0480] Conventional oral cavity diagnostic support systems often presented users with difficulties in image capture and understanding diagnostic results. Furthermore, the inconsistency in image quality due to user-initiated capture made accurate diagnosis challenging. A solution to these problems is needed.

[0481] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0482] In this invention, the server includes means for acquiring images of multiple parts of the oral cavity using an image acquisition device, means for preprocessing the acquired visual information to generate visual data, and analysis means using a generative model to analyze the visual data and determine the presence and progression level of tooth decay. As a result, with the support of a home-use device, users can easily photograph the inside of their mouths, and the analyzed results are provided in an easy-to-understand manner, enabling prompt and accurate dental treatment.

[0483] An "image acquisition device" is a device used to visually record the condition of the oral cavity, and its role is to allow the user to take images of the inside of their mouth.

[0484] "Visual information" refers to digital visual data captured by an image acquisition device, and is information that represents the condition of the oral cavity.

[0485] "Visual data" refers to digital data generated based on visual information, which has been converted into a format suitable for analysis and processing.

[0486] A "generative model" is an algorithm that analyzes visual data to determine the condition of teeth and the progression level of tooth decay; it is a computational model based on machine learning technology.

[0487] "Analysis means" refers to a device or program that uses a generative model to process visual data and perform a process to make judgments about the health status of teeth.

[0488] A "home-use device" is a device designed to support the user's daily tasks within their home, and in this invention, it is a device that assists with intraoral photography and presents the results of the analysis.

[0489] The system of the present invention is designed to allow users to easily monitor their oral health at home. First, the user takes images of their oral cavity using an image acquisition device via a home-use device. The home-use device facilitates the shooting process by guiding the user to the optimal shooting angle, for example, using dedicated voice assistance. The images are acquired as visual information and subsequently converted into visual data.

[0490] The server analyzes visual data through a generative model to determine the presence and progression of tooth decay. The generative model utilizes a multi-layer neural network, enabling highly accurate diagnoses based on large amounts of data. The analysis results are fed back to the user through a home device, allowing the user to quickly understand the results and plan the necessary treatment.

[0491] As a concrete example, suppose a user instructs a home-use device to "take a picture of the inside of the mouth and start the diagnosis." The home-use device assists with the photography and sends the captured image to a server. The server analyzes the received image, sends the analysis results back to the home-use device in real time, and notifies the user of the results. This entire process allows the user to receive an initial diagnosis at home before going to a specialist, enabling a quick and appropriate response.

[0492] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0493] Step 1:

[0494] The user instructs the home device to take intraoral photographs. The user initiates the process by using the prompt "Take intraoral photographs and begin diagnosis" and operating the home device. The input is the user's voice command, and the output is the home device starting camera preparation.

[0495] Step 2:

[0496] A home-use device activates the image acquisition system and takes images of the user's oral cavity. The terminal adjusts the optimal angle and captures images at appropriate brightness. The input is the lighting environment in the home and the user's eye position, and the output is a high-resolution intraoral image.

[0497] Step 3:

[0498] The acquired images are preprocessed as visual data. The image data captured by the device is converted to the optimal format (e.g., JPEG) and compressed as needed. The input is raw image data of the oral cavity, and the output is visual data for transmission to the server.

[0499] Step 4:

[0500] The terminal sends pre-processed visual data to the server. A secure communication protocol is used to upload the visual data to the server. The input is compressed visual data, and the output is the server that received the data.

[0501] Step 5:

[0502] The server inputs visual data into an AI model for analysis. Based on a deep learning algorithm, it evaluates the presence and progression level of tooth decay. The input is visual data uploaded to the server, and the output is numerical data of the analysis results.

[0503] Step 6:

[0504] The server sends feedback to the user based on the analysis results. The analyzed results are sent back to the home device in real time, allowing the user to check them immediately. The input is analyzed numerical data, and the output is diagnostic information displayed on the home device.

[0505] Step 7:

[0506] The user receives feedback and decides on the next course of action as needed. After reviewing the analysis results, the user then develops a treatment plan. The input is the diagnostic information displayed on the home device, and the output is the user's decision to schedule a treatment appointment.

[0507] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0508] This invention provides a more personalized diagnostic experience by combining an emotion recognition function with a dental diagnostic support system that allows users to understand the condition of their oral cavity. Specific embodiments of each element are described below.

[0509] This system begins with the user taking images of various parts of their oral cavity using a smartphone or digital camera. The user uploads the images to the system via a terminal. The terminal converts these images to the appropriate format and sends them to the server.

[0510] The server preprocesses the received images and inputs this data into an AI image analysis model. The model used here includes a multi-layer neural network and can detect areas potentially affected by tooth decay with high accuracy. The model analyzes the acquired data to determine the presence and progression of tooth decay. Based on the diagnosis, the server predicts the number of visits and costs required for treatment, based on past cases.

[0511] Furthermore, this system incorporates an emotion engine. This engine analyzes the user's voice or facial expression input to determine the user's emotional state. Based on this, the server adjusts how diagnostic and predictive results are presented according to the user's emotions. For example, if the user is feeling anxious, the results will be explained more carefully, ensuring optimal communication for the user.

[0512] For example, suppose a user uses the system to check the condition of their teeth. The analysis reveals a minor cavity, and it is predicted that treatment can be completed in a single visit. In this case, the emotion engine analyzes the user's voice and, if it detects that the user is slightly nervous, the server reads the results aloud in a dedicated, calming tone and provides additional reassuring messages as needed. In this way, the system helps users gain a sense of security through a diagnosis at home while planning an appropriate treatment.

[0513] This invention is expected to improve conventional dental diagnostic processes and enhance the user experience.

[0514] The following describes the processing flow.

[0515] Step 1:

[0516] The user takes images of various parts of their oral cavity using a smartphone or digital camera. Ensure sufficient lighting and capture all teeth clearly.

[0517] Step 2:

[0518] The device receives the captured images via a dedicated application and converts them to the appropriate format. This conversion process includes adjusting the image resolution and compressing the image.

[0519] Step 3:

[0520] The device uploads the converted image to the server. The image is transmitted using a secure communication protocol, thus maintaining data confidentiality.

[0521] Step 4:

[0522] The server preprocesses the received images. This process involves denoising and enhancing edges to facilitate recognition by the AI ​​analysis model.

[0523] Step 5:

[0524] The server inputs pre-processed images into an AI image analysis model. The model, based on a multi-layer neural network, determines the presence and progression of tooth decay.

[0525] Step 6:

[0526] After the analysis is complete, the server uses a machine learning model based on historical data to calculate the predicted number of treatments and costs. This result is statistically estimated based on treatment patterns.

[0527] Step 7:

[0528] The server simultaneously analyzes the audio and facial images received from the terminal via the emotion engine. The engine uses this data to determine the user's emotional state.

[0529] Step 8:

[0530] Depending on the user's emotional state, the server sends the diagnostic and predictive results to the user's device in a format appropriate to their needs. If the user appears anxious, the results are explained carefully in a gentle tone.

[0531] Step 9:

[0532] Users can review the information presented on their device and confidently make appointments and create treatment plans at dental clinics. This process allows users to have a diagnostic experience that takes their feelings into consideration.

[0533] (Example 2)

[0534] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0535] The objective of this invention is not only to enable users to easily diagnose the condition of their oral cavity at home, but also to consider the user's emotional state and provide individually optimized diagnostic information. This aims to alleviate user anxiety and doubts, and provide a more satisfying treatment experience.

[0536] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0537] In this invention, the server includes an analysis means that uses a generative model to analyze image data and determine the presence and progression of dental caries, a means that predicts the number of hospital visits and treatment costs required for treatment based on the analysis results, and a means that analyzes emotional information and adjusts the method of presenting the results. This makes it possible not only to provide highly accurate oral diagnosis results but also to provide information that takes into account the user's emotions.

[0538] An "image capture device" is a device used by a user to photograph the condition of their oral cavity, and is a device that includes an imaging function built into a general information device.

[0539] "Preprocessing" refers to the process of preparing acquired image data to make it easier to analyze, and includes steps such as noise reduction and contrast adjustment.

[0540] A "generative model" is an analytical model used to determine the presence or absence of dental caries and its progression, and it enables highly accurate diagnosis, particularly through the use of multi-layer algorithms.

[0541] "Analysis means" refers to a process within a system that uses a generative model to analyze image data and determine the patient's condition.

[0542] A "predictive method" is a procedure for predicting the number of hospital visits and treatment costs required for treatment, based on the analysis results.

[0543] "Emotional information" refers to data obtained from the user's voice or facial expressions, which is used to determine the user's emotional state through analysis.

[0544] "Means for analyzing emotional information" refers to a server-based process that processes a user's emotional information and determines their emotional state.

[0545] "Means for adjusting the method of presenting results" refers to functions that optimize how diagnostic results and predictions are communicated according to the user's emotional state.

[0546] This invention is a diagnostic support system that allows users to easily diagnose the condition of their oral cavity. The system includes an image capture device for users to take images of their oral cavity using a smartphone or digital camera. The user imports the captured images into their device and uploads them to a server via a dedicated application.

[0547] First, the terminal converts the received image into an appropriate format. This allows the server to handle the image efficiently. The converted image is then sent to the server using a secure communication protocol.

[0548] Next, the server preprocesses the received images. This preprocessing removes noise and adjusts contrast to generate image data suitable for analysis. The preprocessed image data is then input into a generative AI model using a multi-layer algorithm. This model is capable of accurately determining the presence and progression of dental caries.

[0549] Based on the analysis results, the server predicts the number of hospital visits and treatment costs required, referencing past cases. This prediction is optimized based on the user's emotional information. The server uses an emotion recognition engine to analyze emotional information from the user's voice and facial expressions, and reflects the results in how the diagnostic information is presented. Therefore, if the user is feeling anxious, the diagnostic results will be presented in a more careful and reassuring manner.

[0550] For example, if a user uses the system for the first time and a minor cavity is detected, the system predicts that the treatment can be completed in a single visit. If the emotion engine detects that the user is feeling anxious, the server will output a reassuring message such as, "The treatment is simple, so please don't worry." In this way, the system helps users gain a sense of security through home-based diagnosis and supports them in planning an appropriate treatment.

[0551] Examples of prompts for the generating AI model include: "Use intraoral images to check for the presence and progression of cavities, and adjust the diagnosis results by taking into account emotional information from the user's facial expressions and voice."

[0552] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0553] Step 1:

[0554] The user takes images of the inside of their mouth using a smartphone or digital camera. These images serve as input. It is desirable to take these images from multiple angles to record the condition of the oral cavity in detail.

[0555] Step 2:

[0556] The user imports the captured image into their device and uploads it to the system via a dedicated application. The input to this process is the captured image, and the output is the image file imported into the device. The device then prepares this image to facilitate conversion to an appropriate format (e.g., JPEG or PNG).

[0557] Step 3:

[0558] The terminal converts the captured image to an appropriate format. The input for this step is the captured image file, and the output is the converted image file. Converting the image format facilitates analysis on the server. The terminal then sends this converted image to the server.

[0559] Step 4:

[0560] The server preprocesses the received image data. The input for this step is the converted image file, and the output is the preprocessed image data. Specifically, the server performs denoising and contrast adjustment on the image to prepare it for use in the analysis model.

[0561] Step 5:

[0562] The server inputs pre-processed image data into the AI ​​model. The input for each step is the pre-processed image data, and the output is the analysis result. The AI ​​model uses this data to diagnose dental caries and determine the presence and progression of cavities.

[0563] Step 6:

[0564] The server makes predictions for treatment based on the analysis results. The input for this step is the analysis results, and the output is a prediction of the number of hospital visits required and the treatment costs. The server refers to past case data and derives predictions from similar cases.

[0565] Step 7:

[0566] After capturing an image, the user inputs voice and facial expression data into the system. The server analyzes this data using an emotion recognition engine. The input for this step is the user's voice and facial expression data, and the output is a judgment about the user's emotional state.

[0567] Step 8:

[0568] The server optimizes how the diagnostic results are presented based on the user's emotional state. The input for this step is the predicted results and the user's emotional information, while the output is a personalized presentation of the diagnostic results. For example, if the user is feeling anxious, the server provides reassurance by adding detailed explanations in a gentle tone.

[0569] (Application Example 2)

[0570] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0571] In modern dental diagnosis, patients often need to visit a medical facility in person, which can be time-consuming, laborious, and emotionally burdensome. Furthermore, traditional diagnostic processes often lack individualized care that takes into account the patient's emotional state, leading to anxiety and tension. There is a growing need for improved diagnostic accuracy and, consequently, a better user experience.

[0572] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0573] In this invention, the server includes means for acquiring images of multiple surfaces inside the oral cavity using an image acquisition device; means for preprocessing the acquired images to generate image information; means for analyzing the image information and determining the presence and progression of dental caries using a generative model; means for predicting the number of hospital visits and treatment costs required for treatment based on the analysis results; means for adjusting the presentation method of the analysis results using an emotion recognition engine that determines the emotional state from the user's voice or facial expressions; and means for outputting the prediction results to the device. As a result, the user can receive a highly accurate dental diagnosis from the comfort of their home and obtain personalized feedback tailored to their emotional state.

[0574] An "image acquisition device" is a device used to capture images of multiple surfaces inside the oral cavity.

[0575] "Image information" refers to data generated by preprocessing acquired images.

[0576] A "generative model" is an analytical algorithm used to determine the presence and progression of dental caries, and it includes multilayer neural networks.

[0577] "Analysis means" refers to a function that performs a process of determining the presence and progression of dental caries using image information.

[0578] An "emotion recognition engine" is a system that identifies the user's emotional state from their voice or facial expressions and incorporates that information into the way the diagnostic results are presented.

[0579] "Prediction results" refer to the estimated number of hospital visits and treatment costs required based on the analysis.

[0580] "Device" refers to equipment including a terminal or display for outputting prediction results to the user.

[0581] This invention describes a specific example of a system that efficiently diagnoses the oral condition of a user and enables personalized responses based on their emotions.

[0582] First, the user uses the image acquisition function of the computing device, which is an image acquisition device, to capture images of multiple surfaces inside the oral cavity. The acquired images are pre-processed by the terminal and sent to the server as image information. The server uses this image information to perform analysis using a generation AI model and a multi-layer neural network to determine the presence and progression of dental caries. To improve the accuracy of this analysis, the model employs pattern recognition technology that takes past data into consideration.

[0583] Based on the analysis results, the server uses a generated AI model to predict the number of hospital visits and treatment costs required. In this process, an emotion recognition engine is used to detect the user's emotional state through voice or facial expressions, and the presentation of the diagnostic results is adjusted accordingly. For example, if the user is feeling anxious, the server will add calming information to the presentation.

[0584] This allows users to obtain highly accurate diagnostic results at home and develop treatment plans with peace of mind. For example, if a user detects a minor cavity, the server may provide feedback such as, "Please rest assured that the treatment will be completed in a short time."

[0585] An example of a prompt to a generative AI model is, "Generate a calming phrase to use when the user is feeling anxious." This allows the system to respond in a way that is sensitive to the user's emotions.

[0586] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0587] Step 1:

[0588] The user captures images of multiple surfaces within the oral cavity using the image acquisition function of the computing device. The input is the raw intraoral images obtained from the camera, and the output is this image data. At this stage, the terminal performs basic contrast adjustment and noise reduction preprocessing.

[0589] Step 2:

[0590] The terminal sends pre-processed image data to the server. The input is the pre-processed image data, and the output is the image data that has been successfully received by the server. The server then confirms receipt of the data and performs a format check.

[0591] Step 3:

[0592] The server uses a generative AI model as an analysis tool to determine the presence and progression of dental caries from image data. Here, a multilayer neural network is applied to analyze the input image data. The output is a judgment regarding the presence and characteristics of tooth decay.

[0593] Step 4:

[0594] The server predicts the number of hospital visits and treatment costs required based on the analysis results. The input is the analysis results, and the output is the predicted number of hospital visits and costs based on a probabilistic model. The server records this information and prepares to communicate it appropriately to the user.

[0595] Step 5:

[0596] Similarly, the server uses an emotion recognition engine to analyze the user's voice input and identify their emotional state. The input is voice data from the user, and an emotion is determined using a voice analysis algorithm. The output is information about the user's emotional state.

[0597] Step 6:

[0598] The server uses prompts to optimally present diagnostic and predictive results based on the user's emotional state. Input consists of analysis results and emotional state information, and, if necessary, retrieves phrases to alleviate tension from an AI model. Output is personalized information presented to the user.

[0599] Step 7:

[0600] The user develops a treatment plan based on the information received from the device. Input is personalized diagnostic results and predictive information, while output is a reasonable treatment schedule based on their own health condition. The user can then consider this and confidently proceed to the next step.

[0601] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0602] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0603] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0604] [Fourth Embodiment]

[0605] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0606] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0607] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0608] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0609] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0610] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0611] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0612] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0613] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0614] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0615] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0616] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0617] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0618] The present invention aims to provide a dental diagnostic support system that allows users to easily and quickly understand the condition of their oral cavity at home. Specific embodiments of the present invention are described below.

[0619] This system begins with the user taking images of their upper, lower, left, and right teeth using an image-capturing device such as a smartphone or digital camera. Users must take the images under adequate lighting and ensure that all teeth are clearly visible. Since the captured images may not be suitable for analysis on the server as is, the system performs image format conversion and compression as needed, depending on the device.

[0620] Next, the device uploads the captured images to a server using a dedicated application or web portal. During this process, a secure communication protocol is used to safely transmit the data.

[0621] The server inputs the received images into an AI analysis model. The AI ​​model has been pre-trained with a large amount of dental image data and uses deep learning algorithms to identify areas in the images that are likely to have cavities. Through AI analysis, the locations and progression of cavities that are most likely to occur are extracted as numerical data.

[0622] Based on the analysis results, the server then determines the need for treatment. Using a machine learning model based on past treatment data, it predicts the number of hospital visits and treatment costs for a specific stage of progression. At this stage, comparisons with similar cases and statistical analysis are performed to improve prediction accuracy.

[0623] These prediction results are sent back to the device in real time, allowing the user to develop an appropriate treatment plan based on them. Ultimately, the user can use this information to make rational appointments at the dental clinic.

[0624] For example, suppose an image taken by a user is analyzed by an AI model, and an early-stage cavity is detected in the upper right molar. In this case, one visit to the dentist is recommended for treatment, with an estimated cost of approximately 5,000 yen, allowing the user to schedule their appointment appropriately. In this way, this system is expected to contribute to maintaining oral health while reducing the burden on the user.

[0625] The following describes the processing flow.

[0626] Step 1:

[0627] Users will use a smartphone or digital camera to take images of their upper, lower, left, and right teeth. Each image must be clear and sharp, and sufficient lighting should be ensured during shooting.

[0628] Step 2:

[0629] The device uploads the captured images to a dedicated application or web portal. The images are formatted and compressed as needed, and then prepared for transmission to the server.

[0630] Step 3:

[0631] The server receives the image sent from the terminal and verifies its appropriate format and quality. At this point, pre-processing such as adjusting the image resolution and reducing unnecessary noise is performed.

[0632] Step 4:

[0633] The server passes the pre-processed images to an AI image analysis model. The model uses a multi-layer neural network to analyze the images and identify areas that may have tooth decay.

[0634] Step 5:

[0635] The server analyzes data obtained from the AI ​​model to determine the presence and progression of tooth decay. The diagnosis integrates information about the location and size of the decay.

[0636] Step 6:

[0637] Based on the diagnostic results, the server uses a machine learning model based on historical data to predict the number of hospital visits and costs required for treatment. Statistical estimations are also performed based on similar cases.

[0638] Step 7:

[0639] The server sends the results to the terminal. This allows the user to receive information that enables them to properly develop a treatment plan.

[0640] Step 8:

[0641] Users can review the prediction results and use them to make an appointment at a dental clinic. They can also take and submit images again if necessary.

[0642] (Example 1)

[0643] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0644] This diagnostic system aims to solve the challenge of enabling users to easily and quickly understand their oral health status and create appropriate treatment plans without requiring them to expend much effort. Specifically, it is required to accurately identify areas potentially affected by tooth decay and provide the diagnostic results to the user in an easily usable format.

[0645] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0646] In this invention, the server includes means for acquiring images of multiple surfaces inside the oral cavity using an image capture function, means for preprocessing the acquired images to generate image information, and means for transmitting the image information using a dedicated application or web portal. This enables the user to quickly understand the health status of their oral cavity and to develop an appropriate treatment plan.

[0647] "Image capture function" refers to the function of imaging devices or equipment used to capture multiple surfaces within the oral cavity.

[0648] "Multiple surfaces within the oral cavity" refers to different parts and angles within the oral cavity, including individual teeth and gums.

[0649] "Image information" refers to data obtained by pre-processing captured images and converting them into a format suitable for analysis.

[0650] "Means of transmission" refers to technical methods for securely transmitting image information to a remote server via a dedicated application or web portal.

[0651] "Received image information" refers to the data in the state after the server has received the data sent from the user's terminal.

[0652] A "generative model" is a model that includes an algorithm that learns from a large amount of training data beforehand and performs data analysis and prediction.

[0653] A "deep learning algorithm" is a method that uses a multi-layered neural network structure to perform feature extraction and pattern recognition based on large amounts of data.

[0654] "Diagnostic data" refers to the results of analyzing image information to generate data regarding the presence and progression of tooth decay.

[0655] This invention provides a diagnostic support system that allows users to easily and quickly understand the condition of their oral cavity at home. First, the user uses a device with image capture capabilities, such as a smartphone or digital camera, to capture images of multiple surfaces in the oral cavity, i.e., the surfaces of the teeth and gums. The captured images are pre-processed on the device and generated as image information. Pre-processing includes image format conversion and compression.

[0656] Next, the terminal uses a dedicated application or web portal to send image information to the server via a secure communication protocol. On the server side, the received image information is input into a generative model. This generative AI model uses a deep learning algorithm and has been pre-trained on a large amount of dental images. As a result, the server can identify areas that may have cavities with high accuracy.

[0657] Next, the server predicts the progression of tooth decay, the number of visits required for treatment, and the treatment cost based on the analysis results. The server sends this information back to the terminal in real time, allowing the user to use the diagnostic results for self-management. For example, if the AI ​​detects an early-stage cavity in the upper right molar through an image taken by the user, one treatment is recommended, and the cost is estimated at 5,000 yen. This information is then used in the user's treatment plan.

[0658] An example of a prompt for a generated AI model is, "Please describe a system that uploads images of the inside of the mouth taken with a smartphone to an AI and automatically detects cavities." In this way, the system is expected to reduce the burden on users and support the maintenance of oral health.

[0659] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0660] Step 1:

[0661] The user takes images of their oral cavity. The user uses the image capture function of a smartphone or digital camera to take detailed images of their teeth and gums. During this process, sufficient lighting is ensured, and multiple surfaces of the oral cavity are captured from various angles. The input is the captured images. The output is visual image data.

[0662] Step 2:

[0663] The device performs image preprocessing. The device takes in the acquired image and converts or compresses it into a format suitable for analysis. For example, it may reduce the image size or remove noise. The input is the captured image data. The output is the preprocessed image information.

[0664] Step 3:

[0665] The terminal sends image information to the server. Data is uploaded to the server via a dedicated application or web portal using a secure communication protocol. The input is pre-processed image information. The output is the image information sent to the server.

[0666] Step 4:

[0667] The server inputs image information into an AI analysis model. The server takes in the image information and passes it to the generating AI model to perform the analysis. This analysis uses a deep learning algorithm to automatically identify areas with a high probability of tooth decay. The input is the image information received by the server. The output is the location information of the analyzed tooth decay.

[0668] Step 5:

[0669] The server generates diagnostic data. The server compiles the AI ​​analysis results and derives numerical data regarding the progression of the disease and the need for treatment. It also makes treatment predictions based on past data. The input is the output data of the AI ​​analysis model. The output consists of diagnostic data and treatment prediction information.

[0670] Step 6:

[0671] The server sends the diagnostic results to the terminal. The generated diagnostic data is sent back to the terminal in real time, allowing the user to quickly check the results. The input is the diagnostic data on the server. The output is the diagnostic results transferred to the terminal.

[0672] Step 7:

[0673] The user develops a treatment plan based on the diagnosis results. The user adjusts the treatment content and appointment schedule based on the received information, and makes appointments if necessary. The input is the diagnosis results displayed on the terminal. The output is the specific treatment plan.

[0674] (Application Example 1)

[0675] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0676] Conventional oral cavity diagnostic support systems often presented users with difficulties in image capture and understanding diagnostic results. Furthermore, the inconsistency in image quality due to user-initiated capture made accurate diagnosis challenging. A solution to these problems is needed.

[0677] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0678] In this invention, the server includes means for acquiring images of multiple parts of the oral cavity using an image acquisition device, means for preprocessing the acquired visual information to generate visual data, and analysis means using a generative model to analyze the visual data and determine the presence and progression level of tooth decay. As a result, with the support of a home-use device, users can easily photograph the inside of their mouths, and the analyzed results are provided in an easy-to-understand manner, enabling prompt and accurate dental treatment.

[0679] An "image acquisition device" is a device used to visually record the condition of the oral cavity, and its role is to allow the user to take images of the inside of their mouth.

[0680] "Visual information" refers to digital visual data captured by an image acquisition device, and is information that represents the condition of the oral cavity.

[0681] "Visual data" refers to digital data generated based on visual information, which has been converted into a format suitable for analysis and processing.

[0682] A "generative model" is an algorithm that analyzes visual data to determine the condition of teeth and the progression level of tooth decay; it is a computational model based on machine learning technology.

[0683] "Analysis means" refers to a device or program that uses a generative model to process visual data and perform a process to make judgments about the health status of teeth.

[0684] A "home-use device" is a device designed to support the user's daily tasks within their home, and in this invention, it is a device that assists with intraoral photography and presents the results of the analysis.

[0685] The system of the present invention is designed to allow users to easily monitor their oral health at home. First, the user takes images of their oral cavity using an image acquisition device via a home-use device. The home-use device facilitates the shooting process by guiding the user to the optimal shooting angle, for example, using dedicated voice assistance. The images are acquired as visual information and subsequently converted into visual data.

[0686] The server analyzes visual data through a generative model to determine the presence and progression of tooth decay. The generative model utilizes a multi-layer neural network, enabling highly accurate diagnoses based on large amounts of data. The analysis results are fed back to the user through a home device, allowing the user to quickly understand the results and plan the necessary treatment.

[0687] As a concrete example, suppose a user instructs a home-use device to "take a picture of the inside of the mouth and start the diagnosis." The home-use device assists with the photography and sends the captured image to a server. The server analyzes the received image, sends the analysis results back to the home-use device in real time, and notifies the user of the results. This entire process allows the user to receive an initial diagnosis at home before going to a specialist, enabling a quick and appropriate response.

[0688] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0689] Step 1:

[0690] The user instructs the home device to take intraoral photographs. The user initiates the process by using the prompt "Take intraoral photographs and begin diagnosis" and operating the home device. The input is the user's voice command, and the output is the home device starting camera preparation.

[0691] Step 2:

[0692] A home-use device activates the image acquisition system and takes images of the user's oral cavity. The terminal adjusts the optimal angle and captures images at appropriate brightness. The input is the lighting environment in the home and the user's eye position, and the output is a high-resolution intraoral image.

[0693] Step 3:

[0694] The acquired images are preprocessed as visual data. The image data captured by the device is converted to the optimal format (e.g., JPEG) and compressed as needed. The input is raw image data of the oral cavity, and the output is visual data for transmission to the server.

[0695] Step 4:

[0696] The terminal sends pre-processed visual data to the server. A secure communication protocol is used to upload the visual data to the server. The input is compressed visual data, and the output is the server that received the data.

[0697] Step 5:

[0698] The server inputs visual data into an AI model for analysis. Based on a deep learning algorithm, it evaluates the presence and progression level of tooth decay. The input is visual data uploaded to the server, and the output is numerical data of the analysis results.

[0699] Step 6:

[0700] The server sends feedback to the user based on the analysis results. The analyzed results are sent back to the home device in real time, allowing the user to check them immediately. The input is analyzed numerical data, and the output is diagnostic information displayed on the home device.

[0701] Step 7:

[0702] The user receives feedback and decides on the next course of action as needed. After reviewing the analysis results, the user then develops a treatment plan. The input is the diagnostic information displayed on the home device, and the output is the user's decision to schedule a treatment appointment.

[0703] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0704] This invention provides a more personalized diagnostic experience by combining an emotion recognition function with a dental diagnostic support system that allows users to understand the condition of their oral cavity. Specific embodiments of each element are described below.

[0705] This system begins with the user taking images of various parts of their oral cavity using a smartphone or digital camera. The user uploads the images to the system via a terminal. The terminal converts these images to the appropriate format and sends them to the server.

[0706] The server preprocesses the received images and inputs this data into an AI image analysis model. The model used here includes a multi-layer neural network and can detect areas potentially affected by tooth decay with high accuracy. The model analyzes the acquired data to determine the presence and progression of tooth decay. Based on the diagnosis, the server predicts the number of visits and costs required for treatment, based on past cases.

[0707] Furthermore, this system incorporates an emotion engine. This engine analyzes the user's voice or facial expression input to determine the user's emotional state. Based on this, the server adjusts how diagnostic and predictive results are presented according to the user's emotions. For example, if the user is feeling anxious, the results will be explained more carefully, ensuring optimal communication for the user.

[0708] For example, suppose a user uses the system to check the condition of their teeth. The analysis reveals a minor cavity, and it is predicted that treatment can be completed in a single visit. In this case, the emotion engine analyzes the user's voice and, if it detects that the user is slightly nervous, the server reads the results aloud in a dedicated, calming tone and provides additional reassuring messages as needed. In this way, the system helps users gain a sense of security through a diagnosis at home while planning an appropriate treatment.

[0709] This invention is expected to improve conventional dental diagnostic processes and enhance the user experience.

[0710] The following describes the processing flow.

[0711] Step 1:

[0712] The user takes images of various parts of their oral cavity using a smartphone or digital camera. Ensure sufficient lighting and capture all teeth clearly.

[0713] Step 2:

[0714] The device receives the captured images via a dedicated application and converts them to the appropriate format. This conversion process includes adjusting the image resolution and compressing the image.

[0715] Step 3:

[0716] The device uploads the converted image to the server. The image is transmitted using a secure communication protocol, thus maintaining data confidentiality.

[0717] Step 4:

[0718] The server preprocesses the received images. This process involves denoising and enhancing edges to facilitate recognition by the AI ​​analysis model.

[0719] Step 5:

[0720] The server inputs pre-processed images into an AI image analysis model. The model, based on a multi-layer neural network, determines the presence and progression of tooth decay.

[0721] Step 6:

[0722] After the analysis is complete, the server uses a machine learning model based on historical data to calculate the predicted number of treatments and costs. This result is statistically estimated based on treatment patterns.

[0723] Step 7:

[0724] The server simultaneously analyzes the audio and facial images received from the terminal via the emotion engine. The engine uses this data to determine the user's emotional state.

[0725] Step 8:

[0726] Depending on the user's emotional state, the server sends the diagnostic and predictive results to the user's device in a format appropriate to their needs. If the user appears anxious, the results are explained carefully in a gentle tone.

[0727] Step 9:

[0728] Users can review the information presented on their device and confidently make appointments and create treatment plans at dental clinics. This process allows users to have a diagnostic experience that takes their feelings into consideration.

[0729] (Example 2)

[0730] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0731] The objective of this invention is not only to enable users to easily diagnose the condition of their oral cavity at home, but also to consider the user's emotional state and provide individually optimized diagnostic information. This aims to alleviate user anxiety and doubts, and provide a more satisfying treatment experience.

[0732] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0733] In this invention, the server includes an analysis means that uses a generative model to analyze image data and determine the presence and progression of dental caries, a means that predicts the number of hospital visits and treatment costs required for treatment based on the analysis results, and a means that analyzes emotional information and adjusts the method of presenting the results. This makes it possible not only to provide highly accurate oral diagnosis results but also to provide information that takes into account the user's emotions.

[0734] An "image capture device" is a device used by a user to photograph the condition of their oral cavity, and is a device that includes an imaging function built into a general information device.

[0735] "Preprocessing" refers to the process of preparing acquired image data to make it easier to analyze, and includes steps such as noise reduction and contrast adjustment.

[0736] A "generative model" is an analytical model used to determine the presence or absence of dental caries and its progression, and it enables highly accurate diagnosis, particularly through the use of multi-layer algorithms.

[0737] "Analysis means" refers to a process within a system that uses a generative model to analyze image data and determine the patient's condition.

[0738] A "predictive method" is a procedure for predicting the number of hospital visits and treatment costs required for treatment, based on the analysis results.

[0739] "Emotional information" refers to data obtained from the user's voice or facial expressions, which is used to determine the user's emotional state through analysis.

[0740] "Means for analyzing emotional information" refers to a server-based process that processes a user's emotional information and determines their emotional state.

[0741] "Means for adjusting the method of presenting results" refers to functions that optimize how diagnostic results and predictions are communicated according to the user's emotional state.

[0742] This invention is a diagnostic support system that allows users to easily diagnose the condition of their oral cavity. The system includes an image capture device for users to take images of their oral cavity using a smartphone or digital camera. The user imports the captured images into their device and uploads them to a server via a dedicated application.

[0743] First, the terminal converts the received image into an appropriate format. This allows the server to handle the image efficiently. The converted image is then sent to the server using a secure communication protocol.

[0744] Next, the server preprocesses the received images. This preprocessing removes noise and adjusts contrast to generate image data suitable for analysis. The preprocessed image data is then input into a generative AI model using a multi-layer algorithm. This model is capable of accurately determining the presence and progression of dental caries.

[0745] Based on the analysis results, the server predicts the number of hospital visits and treatment costs required, referencing past cases. This prediction is optimized based on the user's emotional information. The server uses an emotion recognition engine to analyze emotional information from the user's voice and facial expressions, and reflects the results in how the diagnostic information is presented. Therefore, if the user is feeling anxious, the diagnostic results will be presented in a more careful and reassuring manner.

[0746] For example, if a user uses the system for the first time and a minor cavity is detected, the system predicts that the treatment can be completed in a single visit. If the emotion engine detects that the user is feeling anxious, the server will output a reassuring message such as, "The treatment is simple, so please don't worry." In this way, the system helps users gain a sense of security through home-based diagnosis and supports them in planning an appropriate treatment.

[0747] Examples of prompts for the generating AI model include: "Use intraoral images to check for the presence and progression of cavities, and adjust the diagnosis results by taking into account emotional information from the user's facial expressions and voice."

[0748] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0749] Step 1:

[0750] The user takes images of the inside of their mouth using a smartphone or digital camera. These images serve as input. It is desirable to take these images from multiple angles to record the condition of the oral cavity in detail.

[0751] Step 2:

[0752] The user imports the captured image into their device and uploads it to the system via a dedicated application. The input to this process is the captured image, and the output is the image file imported into the device. The device then prepares this image to facilitate conversion to an appropriate format (e.g., JPEG or PNG).

[0753] Step 3:

[0754] The terminal converts the captured image to an appropriate format. The input for this step is the captured image file, and the output is the converted image file. Converting the image format facilitates analysis on the server. The terminal then sends this converted image to the server.

[0755] Step 4:

[0756] The server preprocesses the received image data. The input for this step is the converted image file, and the output is the preprocessed image data. Specifically, the server performs denoising and contrast adjustment on the image to prepare it for use in the analysis model.

[0757] Step 5:

[0758] The server inputs pre-processed image data into the AI ​​model. The input for each step is the pre-processed image data, and the output is the analysis result. The AI ​​model uses this data to diagnose dental caries and determine the presence and progression of cavities.

[0759] Step 6:

[0760] The server makes predictions for treatment based on the analysis results. The input for this step is the analysis results, and the output is a prediction of the number of hospital visits required and the treatment costs. The server refers to past case data and derives predictions from similar cases.

[0761] Step 7:

[0762] After capturing an image, the user inputs voice and facial expression data into the system. The server analyzes this data using an emotion recognition engine. The input for this step is the user's voice and facial expression data, and the output is a judgment about the user's emotional state.

[0763] Step 8:

[0764] The server optimizes how the diagnostic results are presented based on the user's emotional state. The input for this step is the predicted results and the user's emotional information, while the output is a personalized presentation of the diagnostic results. For example, if the user is feeling anxious, the server provides reassurance by adding detailed explanations in a gentle tone.

[0765] (Application Example 2)

[0766] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0767] In modern dental diagnosis, patients often need to visit a medical facility in person, which can be time-consuming, laborious, and emotionally burdensome. Furthermore, traditional diagnostic processes often lack individualized care that takes into account the patient's emotional state, leading to anxiety and tension. There is a growing need for improved diagnostic accuracy and, consequently, a better user experience.

[0768] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0769] In this invention, the server includes means for acquiring images of multiple surfaces inside the oral cavity using an image acquisition device; means for preprocessing the acquired images to generate image information; means for analyzing the image information and determining the presence and progression of dental caries using a generative model; means for predicting the number of hospital visits and treatment costs required for treatment based on the analysis results; means for adjusting the presentation method of the analysis results using an emotion recognition engine that determines the emotional state from the user's voice or facial expressions; and means for outputting the prediction results to the device. As a result, the user can receive a highly accurate dental diagnosis from the comfort of their home and obtain personalized feedback tailored to their emotional state.

[0770] An "image acquisition device" is a device used to capture images of multiple surfaces inside the oral cavity.

[0771] "Image information" refers to data generated by preprocessing acquired images.

[0772] A "generative model" is an analytical algorithm used to determine the presence and progression of dental caries, and it includes multilayer neural networks.

[0773] "Analysis means" refers to a function that performs a process of determining the presence and progression of dental caries using image information.

[0774] An "emotion recognition engine" is a system that identifies the user's emotional state from their voice or facial expressions and incorporates that information into the way the diagnostic results are presented.

[0775] "Prediction results" refer to the estimated number of hospital visits and treatment costs required based on the analysis.

[0776] "Device" refers to equipment including a terminal or display for outputting prediction results to the user.

[0777] This invention describes a specific example of a system that efficiently diagnoses the oral condition of a user and enables personalized responses based on their emotions.

[0778] First, the user uses the image acquisition function of the computing device, which is an image acquisition device, to capture images of multiple surfaces inside the oral cavity. The acquired images are pre-processed by the terminal and sent to the server as image information. The server uses this image information to perform analysis using a generation AI model and a multi-layer neural network to determine the presence and progression of dental caries. To improve the accuracy of this analysis, the model employs pattern recognition technology that takes past data into consideration.

[0779] Based on the analysis results, the server uses a generated AI model to predict the number of hospital visits and treatment costs required. In this process, an emotion recognition engine is used to detect the user's emotional state through voice or facial expressions, and the presentation of the diagnostic results is adjusted accordingly. For example, if the user is feeling anxious, the server will add calming information to the presentation.

[0780] This allows users to obtain highly accurate diagnostic results at home and develop treatment plans with peace of mind. For example, if a user detects a minor cavity, the server may provide feedback such as, "Please rest assured that the treatment will be completed in a short time."

[0781] An example of a prompt to a generative AI model is, "Generate a calming phrase to use when the user is feeling anxious." This allows the system to respond in a way that is sensitive to the user's emotions.

[0782] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0783] Step 1:

[0784] The user captures images of multiple surfaces within the oral cavity using the image acquisition function of the computing device. The input is the raw intraoral images obtained from the camera, and the output is this image data. At this stage, the terminal performs basic contrast adjustment and noise reduction preprocessing.

[0785] Step 2:

[0786] The terminal sends pre-processed image data to the server. The input is the pre-processed image data, and the output is the image data that has been successfully received by the server. The server then confirms receipt of the data and performs a format check.

[0787] Step 3:

[0788] The server uses a generative AI model as an analysis tool to determine the presence and progression of dental caries from image data. Here, a multilayer neural network is applied to analyze the input image data. The output is a judgment regarding the presence and characteristics of tooth decay.

[0789] Step 4:

[0790] The server predicts the number of hospital visits and treatment costs required based on the analysis results. The input is the analysis results, and the output is the predicted number of hospital visits and costs based on a probabilistic model. The server records this information and prepares to communicate it appropriately to the user.

[0791] Step 5:

[0792] Similarly, the server uses an emotion recognition engine to analyze the user's voice input and identify their emotional state. The input is voice data from the user, and an emotion is determined using a voice analysis algorithm. The output is information about the user's emotional state.

[0793] Step 6:

[0794] The server uses prompts to optimally present diagnostic and predictive results based on the user's emotional state. Input consists of analysis results and emotional state information, and, if necessary, retrieves phrases to alleviate tension from an AI model. Output is personalized information presented to the user.

[0795] Step 7:

[0796] The user develops a treatment plan based on the information received from the device. Input is personalized diagnostic results and predictive information, while output is a reasonable treatment schedule based on their own health condition. The user can then consider this and confidently proceed to the next step.

[0797] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0798] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0799] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0800] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0801] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0802] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0803] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0804] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0805] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0806] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0807] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0808] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0809] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0810] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0811] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0812] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0813] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0814] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0815] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0816] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0817] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0818] The following is further disclosed regarding the embodiments described above.

[0819] (Claim 1)

[0820] A means for acquiring images of multiple surfaces inside the oral cavity using an image acquisition device,

[0821] A means for preprocessing acquired images to generate image data,

[0822] An analytical means using a generative model that analyzes image data to determine the presence and progression of dental caries,

[0823] Based on the analysis results, a means to predict the number of hospital visits and treatment costs required for treatment,

[0824] A means of outputting the prediction results to the terminal,

[0825] A system that includes this.

[0826] (Claim 2)

[0827] The system according to claim 1, characterized in that the image capture device has the camera function of a smart device.

[0828] (Claim 3)

[0829] The system according to claim 1, characterized in that the analysis means includes a multilayer neural network.

[0830] "Example 1"

[0831] (Claim 1)

[0832] A means for acquiring images of multiple surfaces inside the oral cavity using an image capture function,

[0833] A means for preprocessing acquired images to generate image information,

[0834] Means for transmitting image information using a dedicated application or web portal,

[0835] An analytical means using a generative model to analyze received image information and identify areas of potential caries,

[0836] Based on the analysis results, a means to predict the number of hospital visits and treatment costs required for treatment,

[0837] A means for outputting diagnostic data based on prediction results and analysis to a terminal,

[0838] A system that includes this.

[0839] (Claim 2)

[0840] The system according to claim 1, characterized in that the image capture function has the function of capturing images on a multipurpose device.

[0841] (Claim 3)

[0842] The system according to claim 1, characterized in that the analysis means includes a deep learning algorithm.

[0843] "Application Example 1"

[0844] (Claim 1)

[0845] A means for acquiring images of multiple parts of the oral cavity using an image acquisition device,

[0846] A means for preprocessing acquired visual information to generate visual data,

[0847] An analytical method using a generative model that analyzes visual data to determine the presence and progression level of tooth decay,

[0848] Based on the analysis results, a means to predict the number of visits required for treatment and the treatment costs,

[0849] A means of displaying the prediction results on the terminal,

[0850] A means to photograph the inside of the mouth with the support of home appliances and to support the interpretation of the analysis results,

[0851] A system that includes this.

[0852] (Claim 2)

[0853] The system according to claim 1, characterized in that the image acquisition device has a shooting function for a personal mobile terminal.

[0854] (Claim 3)

[0855] The system according to claim 1, characterized in that the analysis means includes a multilayer neural network.

[0856] "Example 2 of combining an emotion engine"

[0857] (Claim 1)

[0858] A means for acquiring images of multiple surfaces inside the oral cavity using an image acquisition device,

[0859] A means for preprocessing acquired images to generate image data,

[0860] An analytical means using a generative model that analyzes image data to determine the presence and progression of dental caries,

[0861] Based on the analysis results, a means to predict the number of hospital visits and treatment costs required for treatment,

[0862] A means of analyzing emotional information and adjusting the method of presenting the results,

[0863] A means of outputting the prediction results to the terminal,

[0864] A system that includes this.

[0865] (Claim 2)

[0866] The system according to claim 1, characterized in that the image capture device has the imaging function of a general information device.

[0867] (Claim 3)

[0868] The system according to claim 1, characterized in that the analysis means includes a multilayer algorithm.

[0869] "Application example 2 when combining with an emotional engine"

[0870] (Claim 1)

[0871] A means for acquiring images of multiple surfaces inside the oral cavity using an image acquisition device,

[0872] A means for preprocessing acquired images to generate image information,

[0873] An analytical means using a generative model that analyzes image information to determine the presence and progression of dental caries,

[0874] Based on the analysis results, a means to predict the number of hospital visits and treatment costs required for treatment,

[0875] A means for adjusting the presentation method of analysis results using an emotion recognition engine that determines the emotional state from the user's voice or facial expressions,

[0876] Means for outputting prediction results to the device,

[0877] A system that includes this.

[0878] (Claim 2)

[0879] The system according to claim 1, characterized in that the image acquisition device has an image capture function of the computing device.

[0880] (Claim 3)

[0881] The system according to claim 1, characterized in that the analysis means and emotion recognition engine include a multilayer neural network. [Explanation of symbols]

[0882] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for acquiring images of multiple surfaces inside the oral cavity using an image acquisition device, A means for preprocessing acquired images to generate image data, An analytical means using a generative model that analyzes image data to determine the presence and progression of dental caries, Based on the analysis results, a means to predict the number of hospital visits and treatment costs required for treatment, A means of outputting the prediction results to the terminal, A system that includes this.

2. The system according to claim 1, characterized in that the image capture device has the camera function of a smart device.

3. The system according to claim 1, characterized in that the analysis means includes a multilayer neural network.

Images