System

Abstract

A system is provided.SOLUTION: A system comprising: means for acquiring images of teeth taken by a user; image processing means including a AI algorithm for analyzing the acquired images and diagnosing dental health; means for providing a brushing regimen assessment and recommendations to the user based on the diagnostic results; and means for introducing the user to a nearby dental office if the diagnostic results indicate that a specialist is needed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Regular dental checkups are necessary to maintain dental health, but many people find it difficult to visit a dentist due to busy lifestyles or various other reasons. Many people also lack knowledge of effective tooth brushing techniques or how to choose the right toothbrush, which can lead to poor dental health. Furthermore, if there is a dental problem, it is important to receive a diagnosis from a specialist as soon as possible, but making that decision on your own can be difficult. Therefore, there is a need for a system that allows people to easily check their dental health at home and receive appropriate advice without having to go to a dentist. [Means for solving the problem]

[0005] The present invention is a system equipped with image processing means including an AI algorithm that acquires images of teeth taken by a user and analyzes the acquired images to diagnose the health condition of the teeth. The system also includes means for providing the user with an evaluation and recommendation of their brushing method based on the diagnosis results, and means for referring the user to a nearby dental clinic if the diagnosis results indicate that a specialist diagnosis is necessary. This allows the user to easily check their dental health at home, receive appropriate advice, and, if necessary, receive a specialist diagnosis. The system also includes means for determining when the user's toothbrush should be replaced based on the diagnosis results and recommending an appropriate type of toothbrush, thereby providing more precise advice. By acquiring the user's location information and locating and referring the user to a nearby dental clinic based on the user's location information if the diagnosis results indicate that a specialist diagnosis is necessary, the system can provide prompt and appropriate dental care services.

[0006] "User" refers to an individual who intends to use the system to check their dental health.

[0007] "Tooth images" are photographs taken by the user of the inside of their own mouth, which are detailed records of the condition of their teeth and gums.

[0008] "Means for acquiring" refers to a method or device for capturing user-taken tooth images into the system.

[0009] "Image processing means" refers to a device or method that includes algorithms and software for analyzing dental images to diagnose dental health.

[0010] "AI algorithm" refers to an algorithm that uses artificial intelligence technology to analyze dental images and diagnose health conditions.

[0011] "Diagnosis Results" refers to information about the user's dental health obtained based on dental images analyzed by an AI algorithm.

[0012] The term "means for evaluating brushing method" refers to a device or method that evaluates the user's current brushing method based on the diagnostic results and generates improvements and recommendations.

[0013] "Recommendations" refers to advice or guidance provided to a user based on diagnostic results.

[0014] "Consult a dentist" refers to a situation where the user's dental health is serious and a professional dental examination and treatment is recommended.

[0015] The "means for introducing a nearby dental clinic" refers to a device or method for selecting the most suitable dental clinic based on the user's location information and introducing it to the user when it is determined that a specialist diagnosis is necessary.

[0016] The term "means for determining when to replace a toothbrush in use" refers to a device or method for determining when to replace a toothbrush used by a user based on the diagnostic results.

[0017] The term "means for recommending an appropriate type of toothbrush" refers to a device or method that suggests a type of toothbrush that is suited to the user's dental health condition.

[0018] "Means for obtaining location information" refers to a device or method for obtaining the user's current location. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] The present invention is a system that allows users to easily check their dental health at home and receive appropriate advice. This system acquires images of the user's teeth and is equipped with an AI algorithm that analyzes the acquired images to diagnose the dental health. Below, we will explain in detail the embodiments of the present invention.

[0041] Image acquisition

[0042] The user takes photos of their teeth and gums in their oral cavity using a smartphone or digital camera. At least five images are taken from different angles, particularly of the front teeth, back teeth, and gums. The app guides the user through the photo-taking process, making it easy for them to obtain the appropriate images. The captured images are then saved on the device.

[0043] Uploading an image

[0044] The device uploads the captured images to a cloud server via the Internet, which securely and quickly transmits the images to the server, which receives them and uses them for analysis.

[0045] Image Processing and Diagnosis

[0046] The server analyzes the received images using an AI algorithm. The AI ​​algorithm processes the tooth images and diagnoses the health of the teeth, including cavities, tartar, and gum inflammation. The analysis results are stored on the server and managed for each user, allowing users to check the health of their own teeth.

[0047] Providing diagnostic results

[0048] Based on the diagnostic results, the server evaluates the user's brushing method and generates recommendations for improvements. It also recommends the appropriate toothbrush replacement timing and type based on the diagnostic results. This information is sent to the device and notified to the user. The user can receive advice through the app and learn more effective brushing methods.

[0049] Specialist referrals

[0050] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information. Based on the obtained location information, the server searches for nearby dental clinics and lists appropriate dental clinics. This allows the user to quickly receive a specialist diagnosis even if a serious health problem is detected.

[0051] Specific examples

[0052] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses that there is early-stage decay in the front teeth. As a result, specific advice on brushing techniques (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specialized for preventing cavities) are provided. Furthermore, if the diagnosis determines that a specialist diagnosis is necessary, the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[0053] This allows users to easily check the health of their teeth at home and receive necessary advice and information on how to seek a diagnosis from a specialist. Using this system will greatly simplify dental health management for users and enable early detection and treatment.

[0054] The processing flow will be explained below.

[0055] Step 1:

[0056] The user launches the app and takes photos of their teeth and gums in their mouth. Following the in-app guide, they take at least five images from different angles, including the front teeth, back teeth, and gums. The images are then saved on the device.

[0057] Step 2:

[0058] The device uploads the captured images to a cloud server, where they are sent securely and quickly via the Internet.

[0059] Step 3:

[0060] The server receives the uploaded images and passes them to the image processing means, where they are processed by AI algorithms for analysis.

[0061] Step 4:

[0062] The server uses an AI algorithm to analyze the images and diagnose the health of the teeth. The analysis results include information on cavities, tartar, gum inflammation, etc. The diagnosis results are stored on the server and managed for each user.

[0063] Step 5:

[0064] The server evaluates the user's brushing method based on the diagnostic results, assesses the effectiveness of the current brushing method, and identifies areas for improvement.

[0065] Step 6:

[0066] The server generates information based on the diagnosis results that recommends the appropriate time and type of toothbrush to replace, and provides the user with information on effective brushing methods and recommended toothbrushes.

[0067] Step 7:

[0068] The server sends the generated advice information to the device, and the user receives an evaluation of their brushing method, areas for improvement, when to replace their toothbrush, and notifications of recommended toothbrushes through the app.

[0069] Step 8:

[0070] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information and uses the obtained location information to search for dental clinics near the user's location.

[0071] Step 9:

[0072] The server creates a list of suitable dental clinics from the search results and sends it to the device. The user receives information about nearby dental clinics via the app and can take next steps, such as making an appointment.

[0073] Through these steps, the system allows users to easily understand the state of their dental health and receive appropriate guidance. If necessary, it also provides information to promptly seek a diagnosis from a specialist.

[0074] Example 1

[0075] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0076] Conventional dental diagnostic systems require users to visit a dentist regularly, making it difficult to easily check the health of their teeth at home and receive appropriate advice. While there are applications for checking dental health, they lack the ability to respond quickly when a professional diagnosis is required. Furthermore, the lack of specific brushing methods or recommendations for appropriate toothbrushes based on the diagnosis results makes it difficult for users to efficiently perform oral care.

[0077] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0078] In this invention, the server includes means for acquiring tooth images taken by a user, means for saving the captured tooth images in a digital device, means for uploading the saved images to a cloud computing environment via the Internet, image processing means including a generative AI algorithm for analyzing the acquired images and diagnosing the dental health condition, means for providing an evaluation of the user's brushing method and suggestions for improvement based on the diagnosis results, means for recommending the appropriate type of toothbrush and replacement timing to the user based on the diagnosis results, and means for acquiring the user's location information and referring the user to a nearby specialist facility if the diagnosis results determine that a specialist diagnosis is necessary. This allows users to easily check their oral health at home and quickly obtain necessary advice and specialist information.

[0079] "User" refers to a person who uses the system to check their dental health and receive advice and diagnostic results.

[0080] "Digital devices" refers to electronic devices used to capture and record images, such as smartphones and digital cameras.

[0081] "Cloud computing environment" refers to remote server storage and processing power provided over the Internet.

[0082] "Generative AI algorithm" refers to a program that uses AI technology to analyze data and diagnose dental health.

[0083] "Image processing means" refers to the technical process for analyzing the acquired images and generating a diagnostic result.

[0084] "Means for evaluating brushing methods and providing points for improvement" refers to a system function that evaluates the user's tooth brushing method based on the diagnostic results and specifically suggests points for improvement.

[0085] "Means for recommending toothbrush types and replacement timing" refers to a system function that indicates to the user the appropriate toothbrush selection and replacement timing based on the diagnostic results.

[0086] "When it is determined that a specialist diagnosis is necessary" refers to a situation where the analysis results of the AI ​​algorithm indicate a serious oral problem and a specialist diagnosis is recommended.

[0087] "Means for obtaining location information and introducing nearby specialist facilities" refers to a system function that identifies the user's current geographic location and searches for and recommends specialist facilities such as dental clinics in the vicinity.

[0088] This invention is a system that allows users to easily check their dental health at home and receive appropriate advice. This system acquires images of the user's teeth and includes a generative AI algorithm that analyzes the acquired images to diagnose the dental health. Below, we will explain in detail the embodiments of the invention.

[0089] Image acquisition

[0090] The user uses a smartphone or digital camera (hereinafter referred to as "digital device") to take photos of their teeth and gums in their mouth. They launch the application and follow the in-app guide to take photos from the appropriate angle and position. At least five images should be taken, including front teeth, back teeth, gums, etc. The captured images are saved in the device's internal storage.

[0091] Uploading an image

[0092] The device uploads the captured images to a cloud computing environment (hereafter referred to as the cloud server) via the Internet. When uploading, the device checks for Internet connectivity and compresses and encrypts the image files. The device uses the upload API to send the images to the cloud server, and the server confirms receipt.

[0093] Image Processing and Diagnosis

[0094] The server analyzes the received images using a generative AI algorithm, which specifically includes the following processes:

[0095] 1. Decode the received image file and restore the original image.

[0096] 2. Pre-process the image to adjust the sharpness and contrast.

[0097] 3. An AI algorithm analyzes the images and diagnoses dental health conditions such as cavities, tartar, and gum inflammation. The results are stored in a database for each user.

[0098] Providing diagnostic results

[0099] The server generates a diagnosis based on the image analysis, which includes the following information:

[0100] 1. Detailed information such as whether or not you have cavities, their progression, whether or not you have tartar, and whether or not you have gum inflammation.

[0101] 2. Evaluate your brushing technique and suggest areas for improvement, specifically, advice on which areas to brush more carefully.

[0102] 3. Recommended types of toothbrushes and when to replace them.

[0103] This information is sent from the server to the terminal and notified to the user.

[0104] Specialist referrals

[0105] If the server determines that a diagnosis by a specialist is necessary based on the diagnostic results, it performs the following process:

[0106] 1. Use the device's location sensor to obtain the user's location information.

[0107] 2. Search for nearby dental clinics based on location information and list suitable dental clinics.

[0108] 3. The created list is sent to the terminal and notified to the user.

[0109] Specific examples

[0110] For example, if a user uses an app to take a picture of their teeth and upload it, the server would do the following:

[0111] 1. The user follows the in-app guide to take five intraoral images from different angles. The device stores these images in its internal storage.

[0112] 2. When you press a button in the app, the device will encrypt the captured image and upload it to the cloud server. A progress bar will be displayed until the transfer is complete.

[0113] 3. The server decodes the received image and converts it into a format for analysis. A generative AI algorithm scans the image and detects cavities, tartar, gum inflammation, etc. For example, an early stage of cavities was detected in a front tooth.

[0114] 4. Based on the analysis results, the server generates information on how to improve the user's brushing method and recommends toothbrushes. For example, it suggests "brushing the insides of your front teeth more carefully and using a toothbrush specifically designed to prevent cavities."

[0115] 5. The server obtains the user's location information and searches for nearby dental clinics. For example, it creates a list of "XX Hospital, △△ Dental Clinic" and sends it to the terminal.

[0116] Prompt Sentence Examples

[0117] Here is an example prompt:

[0118] Example prompt 1:

[0119] "Please analyze the images of my teeth and tell me if I have cavities. I can also learn the correct way to brush my teeth and recommend a toothbrush."

[0120] Example prompt 2:

[0121] "Based on my dental diagnosis, if I need to see a specialist, please tell me the nearest dental clinic based on my location."

[0122] In this way, users can easily check their oral health at home and quickly obtain necessary advice and information from specialists.

[0123] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0124] Step 1:

[0125] A user takes a photograph of the teeth and gums in their mouth using a digital device (smartphone or digital camera).

[0126] Input: A video of the user's mouth.

[0127] How it works: Users launch the app and follow the in-app guide to take at least five images from different angles, which provides detailed images of the front teeth, back teeth, and gums.

[0128] Output: Multiple image files of the teeth (JPEG, PNG, etc.).

[0129] Step 2:

[0130] The device saves the captured image file to its internal storage.

[0131] Input: The image file taken in step 1.

[0132] What it does: The device automatically saves the image in a specific folder for further processing, and adds a timestamp to the file name for easy identification.

[0133] Output: Image file saved on the device.

[0134] Step 3:

[0135] The device uploads the image file to the cloud server.

[0136] Input: The image file saved in step 2.

[0137] What it does: The device checks for an internet connection, compresses and encrypts the image file, then sends it to the cloud server via the upload API, where it confirms receipt. A progress bar is displayed in the app.

[0138] Output: Encrypted image files stored in the cloud server.

[0139] Step 4:

[0140] The server decodes the received image file and performs image preprocessing.

[0141] Input: The encrypted image file uploaded to the cloud server in step 3.

[0142] What it does: The server decodes and restores the image file, then performs preprocessing such as adjusting sharpness and contrast to improve the image quality and make it easier to analyze.

[0143] Output: Preprocessed tooth image.

[0144] Step 5:

[0145] The server uses a generative AI algorithm to analyze the images and diagnose the health of the teeth.

[0146] Input: Image files preprocessed in step 4.

[0147] How it works: Generative AI algorithms scan images to detect cavities, tartar, gum inflammation, etc. The algorithms are trained on large medical datasets and therefore produce highly accurate diagnoses.

[0148] Output: Dental health diagnosis (e.g., presence of cavities, tartar, and gum inflammation).

[0149] Step 6:

[0150] Based on the diagnostic results, the server evaluates the user's brushing method, recommends areas for improvement, and recommends the appropriate type of toothbrush and when to replace it.

[0151] Input: The diagnostic results obtained in step 5.

[0152] Specific operation: The server evaluates the user's brushing method based on the analysis results and suggests areas for improvement, such as brushing specific areas more carefully. It also generates information on toothbrushes and replacement times based on the user's situation.

[0153] Output: Brushing advice and toothbrush recommendations for the user.

[0154] Step 7:

[0155] The server acquires the user's location information, and if the diagnosis indicates that a specialist's diagnosis is necessary, the server will refer the user to a nearby specialist facility.

[0156] Input: The diagnostic results generated in step 6 and the user's location information.

[0157] Specific operation: The server uses the user's location sensor to obtain their current geographic location, searches for nearby dental clinics based on the location information, and creates a list of suitable dental clinics.

[0158] Output: A list of nearby dental clinics provided to the user.

[0159] In this way, the present invention provides a system that allows users to easily check their dental health at home and quickly obtain necessary advice and information from specialists.

[0160] (Application example 1)

[0161] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0162] Conventional methods for checking dental health required regular visits to a dental clinic to receive a diagnosis from a specialist, which was time-consuming and laborious. Furthermore, selecting the appropriate dental care products required specialized knowledge, making it difficult for average users. This led to neglecting dental health management and often overlooking preventable problems. Furthermore, there was a need for a system that could instantly recommend and purchase appropriate dental care products based on diagnosis results through an online shopping function.

[0163] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0164] In this invention, the server includes: a means for acquiring images of teeth taken by a user; an image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; a means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; a means for referring the user to a nearby dental clinic if the diagnosis results indicate that a specialist diagnosis is necessary; and a means for recommending appropriate dental care products through an online shopping function based on the diagnosis results and providing a purchase link. This allows users to easily check the health condition of their teeth from home and obtain appropriate dental care products based on the diagnosis results. Furthermore, if a specialist diagnosis is necessary, appropriate information can be obtained quickly, making it easier to access a dental clinic.

[0165] "User" refers to an individual who uses the system to take images of their teeth and check their health.

[0166] The "image acquisition means" refers to a means including a device such as a smartphone or a digital camera for acquiring images of teeth taken by a user.

[0167] "Image processing means" refers to means including an AI algorithm that analyzes the acquired images and diagnoses the health of the teeth, such as detecting cavities, tartar, and gum inflammation.

[0168] The "diagnosis result providing means" is a means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results, thereby enabling the user to learn appropriate tooth brushing methods.

[0169] The "specialist referral means" is a means for referring the user to a nearby dental clinic when the diagnosis result indicates that a specialist diagnosis is necessary. This means is performed based on the user's location information.

[0170] "Online Shopping Feature" refers to a means of recommending appropriate dental care products based on the diagnosis results and providing a purchase link, allowing the user to immediately purchase the recommended products.

[0171] "Dental care products" refers to toothbrushes, toothpastes, mouthwashes, etc. for preventing cavities and removing tartar that are recommended based on the diagnosis results.

[0172] "Diagnosis results" refer to the evaluation results of dental health analyzed by the AI ​​algorithm, which determine the presence or absence of cavities, tartar, gum inflammation, etc.

[0173] Overall system configuration

[0174] This system allows users to easily check their dental health at home and instantly purchase appropriate dental care products. The system mainly consists of the following elements: image acquisition means, image processing means, diagnostic result provision means, specialist referral means, and online shopping function.

[0175] Hardware and software configuration

[0176] User device: A device with a camera and internet connection, such as a smartphone or tablet. The user device takes pictures and uploads them to the server.

[0177] Server: Cloud server. A central server that integrates image processing, diagnostic result provision, specialist referral, and online shopping functions.

[0178] Image processing software: Python libraries such as OpenCV and Keras, which allow captured images to be pre-processed and the AI ​​algorithms to make the diagnosis.

[0179] Communication software: Data is sent and received between the user device and the cloud server using HTTP libraries such as Requests.

[0180] Data Flow and Processing

[0181] 1. Image Acquisition:

[0182] The user takes multiple images of the oral cavity using a user device such as a smartphone, taking pictures of different areas of the mouth, particularly the front teeth, back teeth, and gums, and uses the in-app guide to ensure clear images are obtained.

[0183] 2. Upload your image:

[0184] The user device uploads the captured images to a cloud server via the Internet, and the image data is securely transmitted using communication software.

[0185] 3. Image Analysis and Diagnosis:

[0186] The cloud server analyzes the received images using AI algorithms, preprocessing the image data using image processing software, and then using an AI model (built with Keras) to diagnose dental health conditions such as cavities, tartar, and gum inflammation.

[0187] 4. Providing diagnostic results and recommendations:

[0188] The server provides the user with an evaluation and recommendations on brushing methods based on the diagnostic results. It also recommends appropriate dental care products (e.g., anti-cavity toothbrushes) based on the diagnostic results and provides a link to purchase them through an online shopping function.

[0189] 5. Specialist referrals:

[0190] If the diagnosis indicates that a specialist consultation is required, the server obtains the user's location and searches for a nearby dental clinic to refer the patient to, using an appropriate geographical information API.

[0191] Specific examples

[0192] Example 1: A user uses an app to take and upload a picture of their front teeth. This image is sent to a cloud server and analyzed by an AI algorithm. The diagnosis is that the patient has "early stages of cavities," and the app recommends the user to use an "anti-cavity toothbrush" and "anti-cavity gourmet paste." If the app determines that a dentist's diagnosis is necessary, the app will refer the user to a nearby dental clinic based on the user's location.

[0193] Prompt Sentence Examples

[0194] “I used the app to take a photo of my front teeth and uploaded it. The diagnosis was that I had early stages of tooth decay. What toothbrush should I buy?”

[0195] This system allows users to easily check the health of their teeth from the comfort of their own home, instantly purchase the necessary dental care products, and quickly receive a diagnosis from a specialist.

[0196] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0197] Step 1:

[0198] Taking an image

[0199] The user uses the smartphone camera to take multiple images of the teeth and gums in the oral cavity. These images include different areas such as the front teeth, back teeth, and gums. The input here is raw image data captured through the smartphone camera, and the output is a JPEG image file saved on the user's device. Specifically, the user launches the camera app and follows the in-app guide to adjust the shooting location while taking the image.

[0200] Step 2:

[0201] Uploading an image

[0202] Images taken on the user's device are selected and uploaded to the cloud server through the application. In this process, the user selects an image and taps the "Upload" button. The input is the image file stored on the user's device, and the output is the image file received by the cloud server. Specifically, the application generates an HTTP request to send the selected image file to the server. This is done using communication software (e.g., the Requests library).

[0203] Step 3:

[0204] Image preprocessing

[0205] The server preprocesses the received images, using the OpenCV library to resize and convert colors (e.g., to RGB). The input is a JPEG image file received by the cloud server, and the output is image data converted into a format suitable for AI algorithms. Specifically, the image is resized to 256x256 pixels, color converted, and then the data is normalized.

[0206] Step 4:

[0207] Image Analysis and Diagnosis

[0208] The server inputs the preprocessed images into an AI algorithm for analysis. Here, an AI model built with Keras is used. The input is the preprocessed image data, and the output is the diagnosis result (e.g., whether or not there is tooth decay, tartar, or gum inflammation). Specifically, the AI ​​model analyzes the image data and calculates the confidence level for each class (e.g., tooth decay, tartar, gum inflammation, etc.).

[0209] Step 5:

[0210] Providing diagnostic results and recommendations

[0211] The server evaluates and recommends brushing methods to the user based on the diagnostic results. It also recommends appropriate dental care products (e.g., anti-cavity toothbrushes) based on the diagnostic results and provides a purchase link through an online shopping function. The input is the diagnostic results obtained from the AI ​​model, and the output is recommendations and purchase links sent to the user. Specifically, it analyzes the diagnostic results, generates appropriate recommendations and product links, and outputs them to the user's application.

[0212] Step 6:

[0213] Specialist referrals

[0214] If the diagnosis result indicates that a specialist diagnosis is required, the server obtains the user's location information, searches for nearby dental clinics, and refers the user to them. The input is the diagnosis result and the user's location information, and the output is information about nearby dental clinics that is notified to the user. Specifically, the server uses a geographic information API to search for the most suitable dental clinic based on the location information, and provides that information to the user.

[0215] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0216] The present invention is a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. The system is equipped with an AI algorithm that acquires images of the user's teeth, analyzes the acquired images, and diagnoses the dental health condition, as well as an emotion engine that recognizes the user's emotions.

[0217] Image acquisition

[0218] The user takes photos of their teeth and gums in their oral cavity using a smartphone or digital camera. At least five images are taken from different angles, particularly of the front teeth, back teeth, and gums. The app guides the user through the photo-taking process, making it easy for them to obtain the appropriate images. The captured images are then saved on the device.

[0219] Uploading an image

[0220] The device uploads the captured images to a cloud server via the Internet, which securely and quickly transmits the images to the server, which receives them and uses them for analysis.

[0221] Image Processing and Diagnosis

[0222] The server analyzes the received images using an AI algorithm. The AI ​​algorithm processes the tooth images and diagnoses the health of the teeth, including cavities, tartar, and gum inflammation. The analysis results are stored on the server and managed for each user, allowing users to check the health of their own teeth.

[0223] emotion recognition

[0224] Before providing a diagnosis result, the server analyzes the user's emotional state using an emotion engine. The emotion engine reads emotions from the user's facial expressions and tone of voice to determine their current state of mind. This allows the server to provide feedback according to the user's emotional state.

[0225] Providing diagnostic results

[0226] Based on the diagnosis results, the server evaluates the user's brushing method and generates recommendations for improvements. Furthermore, depending on the diagnosis results, it recommends the appropriate toothbrush replacement timing and type. Taking into account the analysis results of the emotion engine, it provides encouraging and comforting messages according to the user's feelings.

[0227] For example, if the diagnosis is good, send a positive message such as, "You're polishing really well! Keep it up!"; if the diagnosis is bad, send a comforting message such as, "You need to improve a little. Don't worry, we're here to support you!"

[0228] Specialist referrals

[0229] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information. Based on the obtained location information, the server searches for nearby dental clinics and lists appropriate dental clinics. This allows the user to quickly receive a specialist diagnosis even if a serious health problem is detected.

[0230] Specific examples

[0231] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses the presence of early-stage cavities in the front teeth. After analyzing the image, the server then uses its emotion engine to analyze the user's emotional state and determines that the user is feeling anxious. The server then provides specific brushing advice (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specifically designed to prevent cavities), while also sending a comforting message: "Don't worry. We'll improve this together over time." Furthermore, the diagnosis indicates the need for a specialist consultation, so the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[0232] This allows users to easily check the health status of their teeth at home and receive appropriate advice. Feedback is also provided by the emotion engine, providing information to promptly seek a diagnosis from a specialist if necessary. Using this system significantly simplifies dental health management for users, while also providing psychological support.

[0233] The processing flow will be explained below.

[0234] Step 1:

[0235] The user launches the app and takes photos of their teeth and gums in their mouth. Following the in-app guide, they take at least five images from different angles, including the front teeth, back teeth, and gums. The images are then saved on the device.

[0236] Step 2:

[0237] The device uploads the captured images to a cloud server, where they are sent securely and quickly via the Internet.

[0238] Step 3:

[0239] The server receives the uploaded images and passes them to the image processing means, where they are processed by AI algorithms for analysis.

[0240] Step 4:

[0241] The server uses an AI algorithm to analyze the images and diagnose the health of the teeth. The analysis results include information on cavities, tartar, gum inflammation, etc. The diagnosis results are stored on the server and managed for each user.

[0242] Step 5:

[0243] The server evaluates the user's brushing method based on the diagnostic results, assesses the effectiveness of the current brushing method, and identifies areas for improvement.

[0244] Step 6:

[0245] The server generates information based on the diagnosis results that recommends the appropriate time and type of toothbrush to replace, and provides the user with information on effective brushing methods and recommended toothbrushes.

[0246] Step 7:

[0247] The server uses an emotion engine to analyze the user's emotional state before providing a diagnosis, reading emotions from the user's facial expressions and tone of voice to determine their current state of mind.

[0248] Step 8:

[0249] The server adjusts the generated advice information according to the user's emotional state and sends it to the device. The user receives an evaluation of their brushing technique, areas for improvement, notification of when to replace their toothbrush, and notifications of recommended toothbrushes through the app. The app also provides encouraging and comforting messages that match the user's emotional state.

[0250] Step 9:

[0251] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information and uses the obtained location information to search for dental clinics near the user's location.

[0252] Step 10:

[0253] The server creates a list of suitable dental clinics from the search results and sends it to the device. The user receives information about nearby dental clinics via the app and can take next steps, such as making an appointment.

[0254] Through this series of steps, users can easily check their dental health at home, receive appropriate guidance and psychological support, and be provided with information to quickly seek a specialist diagnosis if necessary.

[0255] Example 2

[0256] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0257] Conventional dental examination systems make it difficult for users to easily check their dental health at home and receive appropriate advice based on the results. They also fail to provide feedback that takes into account the user's emotional state, and lack sufficient psychological support. Furthermore, it is difficult for users to quickly find an appropriate dental clinic when a specialist diagnosis is required.

[0258] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0259] In this invention, the server includes: means for acquiring images of teeth taken by a user; image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; means including an emotion engine for analyzing the user's emotional state and providing feedback according to the emotion; and means for referring the user to a nearby dental clinic if the diagnosis results indicate that a specialist diagnosis is necessary. This enables a user to easily check the dental health condition at home and receive a referral to a specialist along with emotionally sensitive feedback.

[0260] "Means for acquiring images of teeth taken by a user" refers to a mechanism that allows the system to receive image data of teeth taken by a user using a smartphone or digital camera and make it available for storage or processing.

[0261] "Image processing means including an AI algorithm that analyzes acquired images and diagnoses the health condition of teeth" refers to a combination of software and hardware that uses artificial intelligence technology to analyze received dental image data and determine health conditions such as cavities, tartar, and gum inflammation.

[0262] "Means of providing users with an evaluation and recommendations on brushing methods based on the diagnostic results" refers to a function that provides specific guidance to users on appropriate brushing methods, areas for improvement, the type of toothbrush they should use, etc., based on the results of a dental health diagnosis by an AI algorithm.

[0263] "Means including an emotion engine that analyzes the user's emotional state and provides feedback according to that emotion" refers to a combination of software and hardware that reads the user's current emotion by analyzing the user's facial expression and tone of voice, and generates feedback or messages appropriate to that emotion.

[0264] "Means for referring the user to a nearby dental clinic when the diagnostic results indicate that a specialist diagnosis is necessary" refers to a mechanism by which the system refers to the user's diagnostic results, and when it determines that a specialist diagnosis is necessary, searches for the nearest appropriate dental clinic using the user's geographical location information and provides a list or reservation information.

[0265] The present invention provides a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. This system is implemented using the following hardware and software.

[0266] Hardware Configuration

[0267] 1. Smartphone or digital camera: Used by the user to take images of teeth. Equipped with a camera function and internet connection.

[0268] 2. Cloud Server: Analyzes the received images, generates and stores diagnostic results. This server is equipped with advanced computing resources and storage.

[0269] 3. Device: A smartphone or tablet used by a user to take pictures, upload them, and receive feedback.

[0270] Software Configuration

[0271] 1. Dedicated application: An application that allows users to take images of their own teeth and upload them to a cloud server.

[0272] 2. AI Algorithm: An artificial intelligence program that runs on a cloud server and analyzes received dental images to diagnose health conditions such as cavities, tartar, and gum inflammation.

[0273] 3. Emotion Engine: Another AI program running on a cloud server that analyzes the user's emotions from their facial expressions and tone of voice and generates appropriate feedback.

[0274] Operation explanation

[0275] Users take pictures of their teeth and gums using a smartphone or digital camera with a dedicated application installed. At least five images are taken from different angles, including the front teeth, back teeth, and gums. A shooting guide is displayed in the app, making it easy for users to take the appropriate images. These images are temporarily saved on the device.

[0276] The device then uploads these images to a cloud server, where they are encrypted and securely transmitted. The server then passes the images to an AI algorithm, which analyzes them and diagnoses dental health conditions such as cavities, tartar, and gum inflammation. The results are stored in a database for each user, allowing them to view them later.

[0277] Before providing a diagnosis, the server analyzes the user's emotions using an emotion engine. The emotion engine determines the user's current emotional state from the facial expressions and tone of voice provided by the user through the camera. Based on this, the server generates messages and feedback according to the user's emotional state.

[0278] Finally, the server sends the diagnosis results along with an evaluation of the user's brushing technique, along with recommendations for improvement and suggestions for improvement. For example, it may recommend using fluoride toothpaste or adopting a specific brushing technique to prevent cavities. Depending on the results of the emotion engine, a comforting message such as "Don't worry. We'll work together to improve over time" may also be sent.

[0279] If a specialist consultation is required, the server obtains the user's location information and searches for and lists nearby dental clinics, allowing the user to quickly find the appropriate clinic and use the appointment link to receive a prompt diagnosis.

[0280] Examples of concrete examples and prompts

[0281] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses the presence of early-stage cavities in the front teeth. After analyzing the image, the server then uses its emotion engine to analyze the user's emotional state and determines that the user is feeling anxious. The server then provides specific brushing advice (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specifically designed to prevent cavities), while also sending a comforting message: "Don't worry. We'll improve this together over time." Furthermore, the diagnosis indicates the need for a specialist consultation, so the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[0282] Example prompt (input to generative AI model):

[0283] Analyze the tooth images uploaded by the user and generate a diagnosis. Based on the analysis, provide specific brushing improvement suggestions and recommend suitable toothbrushes. Also, analyze the user's emotional state and generate messages to alleviate their concerns.

[0284] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0285] Step 1:

[0286] The user launches the dedicated application and takes pictures of their teeth using a smartphone or digital camera. The user follows the in-app guide to take at least five images from different angles, including the front teeth, back teeth, and gums. The guide displays instructions for taking appropriate images. The captured images are temporarily stored on the device.

[0287] Specific behavior:

[0288] 1. The user launches the application and taps the "Take Image" button.

[0289] 2. The device will start the camera and display a shooting guide.

[0290] 3. The user follows the guide to take images of each part of the body and taps "Save" to save them to the device.

[0291] Input: User actions and camera guide

[0292] Output: Saved tooth image data

[0293] Step 2:

[0294] The device uploads the saved images to the cloud server. The images are encrypted and sent securely. After the upload is complete, the device notifies the user.

[0295] Specific behavior:

[0296] 1. Select the image saved on your device and tap the "Upload" button.

[0297] 2. The device encrypts the image and sends it to a server over the Internet.

[0298] 3. The server receives the image, verifies the status, and notifies the device of a successful upload.

[0299] Input: Stored tooth image data, user command

[0300] Output: Image data uploaded to the cloud server

[0301] Step 3:

[0302] The server then passes the received image to an AI algorithm to begin analysis. The AI ​​algorithm analyzes the image and diagnoses health conditions such as cavities, tartar, and gum inflammation. The analysis results are then stored in a database.

[0303] Specific behavior:

[0304] 1. The server inputs the received image data into the AI ​​algorithm.

[0305] 2. AI algorithms analyze images and diagnose cavities, tartar, and gum inflammation.

[0306] 3. Diagnostic results are stored in a database for each user.

[0307] Input: Uploaded image data

[0308] Output: Save diagnostic results to database

[0309] Step 4:

[0310] The server uses an emotion engine to analyze the user's emotional state before providing a diagnosis. It determines emotions from the user's camera image and tone of voice and stores the results.

[0311] Specific behavior:

[0312] 1. The server passes the user's camera image and voice data to the emotion engine.

[0313] 2. The emotion engine analyzes this data and identifies the user's emotional state.

[0314] 3. The results of the sentiment analysis are generated and stored in a database for each user.

[0315] Input: Camera image and audio data

[0316] Output: Emotion analysis results

[0317] Step 5:

[0318] The server generates feedback based on the diagnosis and emotion analysis results, including recommendations for brushing method improvements and messages based on the user's emotional state. This feedback is then sent to the device.

[0319] Specific behavior:

[0320] 1. The server generates a feedback message based on the diagnosis and emotion analysis results.

[0321] 2. Simultaneously generate improvements and specific recommendations for brushing methods.

[0322] 3. Send feedback to the device and notify the user.

[0323] Input: Diagnosis results, emotion analysis results

[0324] Output: Feedback messages and brushing advice

[0325] Step 6:

[0326] If the diagnosis indicates that a specialist consultation is necessary, the server acquires the user's location information, searches for and lists nearby dental clinics, and sends the information to the user.

[0327] Specific behavior:

[0328] 1. The server obtains the user's location information and searches for nearby dental clinics.

[0329] 2. Create a list of suitable dental clinics and create a link to book appointments.

[0330] 3. This information is sent to the terminal and notified to the user.

[0331] Input: Diagnostic results, user location information

[0332] Output: A list of dental clinics and a link to book an appointment

[0333] (Application example 2)

[0334] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0335] In today's busy lifestyles, there is a need for a system that allows users to easily check their dental health at home and receive appropriate feedback and a prompt diagnosis. However, existing systems lack the functionality to provide feedback based on emotional state or recommend experts, making it difficult to provide effective advice while alleviating users' concerns.

[0336] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for acquiring images of teeth taken by the user; image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; means for referring the user to a nearby medical institution if the diagnosis results indicate that a professional diagnosis is necessary; and means for recognizing the user's emotional state and providing feedback according to the user's emotion. This allows the user to easily check the health condition of their teeth at home and receive appropriate advice and support according to their emotion.

[0337] "User" refers to the person who uses the system, and in this case, refers to an individual who wants to check and manage their dental health status.

[0338] A "tooth image" is a photograph of the teeth and gums inside the oral cavity taken by the user.

[0339] The "means for acquiring" is a function for importing images of teeth taken by the user into the system.

[0340] "Image processing means" is a function for analyzing acquired tooth images using an AI algorithm.

[0341] "AI algorithm" refers to an analytical method that uses artificial intelligence, and refers to the function of analyzing images of teeth to diagnose their health condition.

[0342] "Diagnosis" refers to the evaluation of dental health based on the analysis results.

[0343] "Health status" refers to the state of the teeth and gums, and whether or not there are any symptoms such as cavities or inflammation.

[0344] "Brushing technique" refers to the method of brushing your teeth effectively, such as how you brush your teeth and the type of toothbrush you use.

[0345] "Evaluation" refers to determining whether the current brushing method is appropriate based on the diagnostic results.

[0346] "Recommendations" are specific advice or suggestions for improvement provided to the user based on the diagnostic results.

[0347] A "professional" is someone who has specialized knowledge and qualifications related to teeth and oral health, such as a dentist or dental hygienist.

[0348] "Diagnosis required" means that the system's analysis results indicate that the user's dental health condition requires a specialist examination.

[0349] "Nearby medical institution" refers to a medical facility such as a dental clinic that is located within easy access from the user's current location.

[0350] "Means for referral" is a function that guides the user to an appropriate medical institution based on the diagnosis results.

[0351] "Emotional state" refers to the user's psychological state or mood, which in this case is determined from facial expressions and tone of voice.

[0352] "Feedback" is a general term for comments and advice provided to users based on diagnostic and analytical results.

[0353] This invention is a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. This system supports users' dental health management by analyzing dental images taken by the user and providing appropriate advice and emotional feedback based on the diagnosis results.

[0354] The system includes the following means:

[0355] 1. Image acquisition method:

[0356] The user takes images of their teeth using a smartphone or digital camera. The images are taken from at least five angles: front, left side, right side, top, and bottom. This allows detailed image data to be obtained. The captured image data is saved on the user's device.

[0357] 2. Image upload method:

[0358] The user's device uploads the captured images to a cloud server via the Internet, where the image data is received securely and quickly.

[0359] 3. Imaging and diagnostic tools:

[0360] The cloud server analyzes the received images using an AI algorithm, which diagnoses the health of the teeth (e.g., cavities, tartar, gum inflammation, etc.) and stores the results on the server.

[0361] 4. Emotion recognition means:

[0362] Before providing a diagnosis, the server analyzes the user's emotional state using an emotion engine, which identifies emotions from the user's facial expressions and tone of voice to determine the user's current state of mind.

[0363] 5. Feedback can be provided by:

[0364] The server provides optimal feedback to the user based on the diagnosis and emotion recognition results. The feedback includes an evaluation of brushing methods, areas for improvement, and recommendations. It also recommends the type and timing of toothbrush replacement based on the diagnosis results. It also provides encouraging and comforting messages based on the emotion engine results.

[0365] 6. Specialist referral methods:

[0366] If the diagnosis results indicate that a specialist diagnosis is necessary, the server obtains the user's location information and searches for and refers the user to a nearby medical institution. This allows the user to receive a prompt, professional diagnosis even if a serious health problem is detected.

[0367] Hardware and software used

[0368] Hardware:

[0369] Smartphone or digital camera (for taking pictures)

[0370] Internet-connected device (image upload)

[0371] software:

[0372] OpenCV (image processing)

[0373] EmotionRecognizer

[0374] AI Diagnostic (Diagnosis by AI algorithm)

[0375] requests (REST API calls)

[0376] Specific examples

[0377] For example, a user can use the app to take pictures of their teeth from the front, left side, right side, top, and bottom, and upload them to a cloud server. The server then uses AI to analyze the received images and diagnoses the presence of early-stage cavities in the front teeth. The emotion engine then analyzes the user's emotional state and determines that the user is feeling anxious. The app then provides specific brushing advice and recommended toothbrushes, along with a comforting message: "Don't worry. We'll take the time to improve together." Furthermore, if the diagnosis indicates that a professional diagnosis is necessary, the app searches for nearby dental clinics based on the user's location and provides information on appropriate dental clinics.

[0378] Prompt Sentence Examples

[0379] A user takes photos of their teeth with their smartphone from the following angles: front, left side, right side, top, and bottom. Upload these images to a cloud server and use an AI algorithm to analyze the health of their teeth. Additionally, recognize emotions based on the user's facial expressions and voice and provide feedback based on the following criteria:

[0380] If a user is feeling anxious, provide comforting messages such as, "Don't worry. We'll improve this together over time."

[0381] If the diagnosis is positive, a positive message such as "You're polishing very well! Keep it up!" is provided.

[0382] If the diagnosis is negative, provide comforting messages such as, "You need to improve a little. It's okay, we're here to help!"

[0383] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0384] Step 1:

[0385] The user takes images of the teeth using a smartphone or digital camera. The images are taken from at least five angles: front, left side, right side, top, and bottom. The user follows the guide within the app to obtain the appropriate images. This is the input, and the images are saved in the smartphone's storage as the output. Specifically, the user launches the camera app and takes photos from each angle while following the guide display.

[0386] Step 2:

[0387] The user's device uploads the captured tooth images to the cloud server via the Internet. The input is the image stored on the device, and the output is the image stored on the cloud server. Specifically, the device sends the image file to the cloud server using a REST API.

[0388] Step 3:

[0389] The server receives images uploaded to the cloud server and analyzes each image using an AI algorithm. The input is the uploaded image data, and the output is a diagnosis of the dental health condition (e.g., cavities, tartar, gum inflammation, etc.). Specifically, the server preprocesses the images using an image processing library (e.g., OpenCV) and runs a deep learning model to evaluate the dental condition.

[0390] Step 4:

[0391] The server extracts emotions from the user's facial expressions and tone of voice to recognize the user's emotional state based on the diagnosis results. The input is the diagnosis results and the user's facial expressions and voice data, and the output is a judgment of the user's emotional state (e.g., anxiety, relief, etc.). Specifically, the server uses an emotion recognition engine (e.g., EmotionRecognizer) to analyze the facial images and voice data.

[0392] Step 5:

[0393] The server generates optimal feedback for the user based on the diagnosis results and emotion recognition results. The input is the diagnosis results and emotion determination results, and the output is advice or a message to be provided to the user. Specifically, the server generates a positive message if the diagnosis results are good, and a comforting message if the user is very anxious, and sends it to the user.

[0394] Step 6:

[0395] If the diagnosis result indicates that a specialist diagnosis is necessary, the server obtains the user's location information. The input is the diagnosis result and the user's location information, and the output is a list of nearby medical institutions. Specifically, the server obtains the user's location using a location information API and searches for the nearest medical institution based on that location.

[0396] Step 7:

[0397] The server introduces nearby medical institutions to the user. The input is the location information of the user who is deemed to need a specialist diagnosis and a list of medical institutions, and the output is a notification message including the referral details. Specifically, the server sends information about appropriate medical institutions to the user's device and displays it as a notification.

[0398] These steps allow users to easily check their dental health status at home and receive appropriate advice and support according to their emotions.

[0399] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0400] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0401] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0402] [Second embodiment]

[0403] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0404] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0405] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0406] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0407] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0408] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0409] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0410] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0411] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0412] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0413] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0414] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0415] The present invention is a system that allows users to easily check their dental health at home and receive appropriate advice. This system acquires images of the user's teeth and is equipped with an AI algorithm that analyzes the acquired images to diagnose the dental health. Below, we will explain in detail the embodiments of the present invention.

[0416] Image acquisition

[0417] The user takes photos of their teeth and gums in their oral cavity using a smartphone or digital camera. At least five images are taken from different angles, particularly of the front teeth, back teeth, and gums. The app guides the user through the photo-taking process, making it easy for them to obtain the appropriate images. The captured images are then saved on the device.

[0418] Uploading an image

[0419] The device uploads the captured images to a cloud server via the Internet, which securely and quickly transmits the images to the server, which receives them and uses them for analysis.

[0420] Image Processing and Diagnosis

[0421] The server analyzes the received images using an AI algorithm. The AI ​​algorithm processes the tooth images and diagnoses the health of the teeth, including cavities, tartar, and gum inflammation. The analysis results are stored on the server and managed for each user, allowing users to check the health of their own teeth.

[0422] Providing diagnostic results

[0423] Based on the diagnostic results, the server evaluates the user's brushing method and generates recommendations for improvements. It also recommends the appropriate toothbrush replacement timing and type based on the diagnostic results. This information is sent to the device and notified to the user. The user can receive advice through the app and learn more effective brushing methods.

[0424] Specialist referrals

[0425] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information. Based on the obtained location information, the server searches for nearby dental clinics and lists appropriate dental clinics. This allows the user to quickly receive a specialist diagnosis even if a serious health problem is detected.

[0426] Specific examples

[0427] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses that there is early-stage decay in the front teeth. As a result, specific advice on brushing techniques (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specialized for preventing cavities) are provided. Furthermore, if the diagnosis determines that a specialist diagnosis is necessary, the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[0428] This allows users to easily check the health of their teeth at home and receive necessary advice and information on how to seek a diagnosis from a specialist. Using this system will greatly simplify dental health management for users and enable early detection and treatment.

[0429] The processing flow will be explained below.

[0430] Step 1:

[0431] The user launches the app and takes photos of their teeth and gums in their mouth. Following the in-app guide, they take at least five images from different angles, including the front teeth, back teeth, and gums. The images are then saved on the device.

[0432] Step 2:

[0433] The device uploads the captured images to a cloud server, where they are sent securely and quickly via the Internet.

[0434] Step 3:

[0435] The server receives the uploaded images and passes them to the image processing means, where they are processed by AI algorithms for analysis.

[0436] Step 4:

[0437] The server uses an AI algorithm to analyze the images and diagnose the health of the teeth. The analysis results include information on cavities, tartar, gum inflammation, etc. The diagnosis results are stored on the server and managed for each user.

[0438] Step 5:

[0439] The server evaluates the user's brushing method based on the diagnostic results, assesses the effectiveness of the current brushing method, and identifies areas for improvement.

[0440] Step 6:

[0441] The server generates information based on the diagnosis results that recommends the appropriate time and type of toothbrush to replace, and provides the user with information on effective brushing methods and recommended toothbrushes.

[0442] Step 7:

[0443] The server sends the generated advice information to the device, and the user receives an evaluation of their brushing method, areas for improvement, when to replace their toothbrush, and notifications of recommended toothbrushes through the app.

[0444] Step 8:

[0445] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information and uses the obtained location information to search for dental clinics near the user's location.

[0446] Step 9:

[0447] The server creates a list of suitable dental clinics from the search results and sends it to the device. The user receives information about nearby dental clinics via the app and can take next steps, such as making an appointment.

[0448] Through these steps, the system allows users to easily understand the state of their dental health and receive appropriate guidance. If necessary, it also provides information to promptly seek a diagnosis from a specialist.

[0449] Example 1

[0450] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0451] Conventional dental diagnostic systems require users to visit a dentist regularly, making it difficult to easily check the health of their teeth at home and receive appropriate advice. While there are applications for checking dental health, they lack the ability to respond quickly when a professional diagnosis is required. Furthermore, the lack of specific brushing methods or recommendations for appropriate toothbrushes based on the diagnosis results makes it difficult for users to efficiently perform oral care.

[0452] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0453] In this invention, the server includes means for acquiring tooth images taken by a user, means for saving the captured tooth images in a digital device, means for uploading the saved images to a cloud computing environment via the Internet, image processing means including a generative AI algorithm for analyzing the acquired images and diagnosing the dental health condition, means for providing an evaluation of the user's brushing method and suggestions for improvement based on the diagnosis results, means for recommending the appropriate type of toothbrush and replacement timing to the user based on the diagnosis results, and means for acquiring the user's location information and referring the user to a nearby specialist facility if the diagnosis results determine that a specialist diagnosis is necessary. This allows users to easily check their oral health at home and quickly obtain necessary advice and specialist information.

[0454] "User" refers to a person who uses the system to check their dental health and receive advice and diagnostic results.

[0455] "Digital devices" refers to electronic devices used to capture and record images, such as smartphones and digital cameras.

[0456] "Cloud computing environment" refers to remote server storage and processing power provided over the Internet.

[0457] "Generative AI algorithm" refers to a program that uses AI technology to analyze data and diagnose dental health.

[0458] "Image processing means" refers to the technical process for analyzing the acquired images and generating a diagnostic result.

[0459] "Means for evaluating brushing methods and providing points for improvement" refers to a system function that evaluates the user's tooth brushing method based on the diagnostic results and specifically suggests points for improvement.

[0460] "Means for recommending toothbrush types and replacement timing" refers to a system function that indicates to the user the appropriate toothbrush selection and replacement timing based on the diagnostic results.

[0461] "When it is determined that a specialist diagnosis is necessary" refers to a situation where the analysis results of the AI ​​algorithm indicate a serious oral problem and a specialist diagnosis is recommended.

[0462] "Means for obtaining location information and introducing nearby specialist facilities" refers to a system function that identifies the user's current geographic location and searches for and recommends specialist facilities such as dental clinics in the vicinity.

[0463] This invention is a system that allows users to easily check their dental health at home and receive appropriate advice. This system acquires images of the user's teeth and includes a generative AI algorithm that analyzes the acquired images to diagnose the dental health. Below, we will explain in detail the embodiments of the invention.

[0464] Image acquisition

[0465] The user uses a smartphone or digital camera (hereinafter referred to as "digital device") to take photos of their teeth and gums in their mouth. They launch the application and follow the in-app guide to take photos from the appropriate angle and position. At least five images should be taken, including front teeth, back teeth, gums, etc. The captured images are saved in the device's internal storage.

[0466] Uploading an image

[0467] The device uploads the captured images to a cloud computing environment (hereafter referred to as the cloud server) via the Internet. When uploading, the device checks for Internet connectivity and compresses and encrypts the image files. The device uses the upload API to send the images to the cloud server, and the server confirms receipt.

[0468] Image Processing and Diagnosis

[0469] The server analyzes the received images using a generative AI algorithm, which specifically includes the following processes:

[0470] 1. Decode the received image file and restore the original image.

[0471] 2. Pre-process the image to adjust the sharpness and contrast.

[0472] 3. An AI algorithm analyzes the images and diagnoses dental health conditions such as cavities, tartar, and gum inflammation. The results are stored in a database for each user.

[0473] Providing diagnostic results

[0474] The server generates a diagnosis based on the image analysis, which includes the following information:

[0475] 1. Detailed information such as whether or not you have cavities, their progression, whether or not you have tartar, and whether or not you have gum inflammation.

[0476] 2. Evaluate your brushing technique and suggest areas for improvement, specifically, advice on which areas to brush more carefully.

[0477] 3. Recommended types of toothbrushes and when to replace them.

[0478] This information is sent from the server to the terminal and notified to the user.

[0479] Specialist referrals

[0480] If the server determines that a diagnosis by a specialist is necessary based on the diagnostic results, it performs the following process:

[0481] 1. Use the device's location sensor to obtain the user's location information.

[0482] 2. Search for nearby dental clinics based on location information and list suitable dental clinics.

[0483] 3. The created list is sent to the terminal and notified to the user.

[0484] Specific examples

[0485] For example, if a user uses an app to take a picture of their teeth and upload it, the server would do the following:

[0486] 1. The user follows the in-app guide to take five intraoral images from different angles. The device stores these images in its internal storage.

[0487] 2. When you press a button in the app, the device will encrypt the captured image and upload it to the cloud server. A progress bar will be displayed until the transfer is complete.

[0488] 3. The server decodes the received image and converts it into a format for analysis. A generative AI algorithm scans the image and detects cavities, tartar, gum inflammation, etc. For example, an early stage of cavities was detected in a front tooth.

[0489] 4. Based on the analysis results, the server generates information on how to improve the user's brushing method and recommends toothbrushes. For example, it suggests "brushing the insides of your front teeth more carefully and using a toothbrush specifically designed to prevent cavities."

[0490] 5. The server obtains the user's location information and searches for nearby dental clinics. For example, it creates a list of "XX Hospital, △△ Dental Clinic" and sends it to the terminal.

[0491] Prompt Sentence Examples

[0492] Here is an example prompt:

[0493] Example prompt 1:

[0494] "Please analyze the images of my teeth and tell me if I have cavities. I can also learn the correct way to brush my teeth and recommend a toothbrush."

[0495] Example prompt 2:

[0496] "Based on my dental diagnosis, if I need to see a specialist, please tell me the nearest dental clinic based on my location."

[0497] In this way, users can easily check their oral health at home and quickly obtain necessary advice and information from specialists.

[0498] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0499] Step 1:

[0500] A user takes a photograph of the teeth and gums in their mouth using a digital device (smartphone or digital camera).

[0501] Input: A video of the user's mouth.

[0502] How it works: Users launch the app and follow the in-app guide to take at least five images from different angles, which provides detailed images of the front teeth, back teeth, and gums.

[0503] Output: Multiple image files of the teeth (JPEG, PNG, etc.).

[0504] Step 2:

[0505] The device saves the captured image file to its internal storage.

[0506] Input: The image file taken in step 1.

[0507] What it does: The device automatically saves the image in a specific folder for further processing, and adds a timestamp to the file name for easy identification.

[0508] Output: Image file saved on the device.

[0509] Step 3:

[0510] The device uploads the image file to the cloud server.

[0511] Input: The image file saved in step 2.

[0512] What it does: The device checks for an internet connection, compresses and encrypts the image file, then sends it to the cloud server via the upload API, where it confirms receipt. A progress bar is displayed in the app.

[0513] Output: Encrypted image files stored in the cloud server.

[0514] Step 4:

[0515] The server decodes the received image file and performs image preprocessing.

[0516] Input: The encrypted image file uploaded to the cloud server in step 3.

[0517] What it does: The server decodes and restores the image file, then performs preprocessing such as adjusting sharpness and contrast to improve the image quality and make it easier to analyze.

[0518] Output: Preprocessed tooth image.

[0519] Step 5:

[0520] The server uses a generative AI algorithm to analyze the images and diagnose the health of the teeth.

[0521] Input: Image files preprocessed in step 4.

[0522] How it works: Generative AI algorithms scan images to detect cavities, tartar, gum inflammation, etc. The algorithms are trained on large medical datasets and therefore produce highly accurate diagnoses.

[0523] Output: Dental health diagnosis (e.g., presence of cavities, tartar, and gum inflammation).

[0524] Step 6:

[0525] Based on the diagnostic results, the server evaluates the user's brushing method, recommends areas for improvement, and recommends the appropriate type of toothbrush and when to replace it.

[0526] Input: The diagnostic results obtained in step 5.

[0527] Specific operation: The server evaluates the user's brushing method based on the analysis results and suggests areas for improvement, such as brushing specific areas more carefully. It also generates information on toothbrushes and replacement times based on the user's situation.

[0528] Output: Brushing advice and toothbrush recommendations for the user.

[0529] Step 7:

[0530] The server acquires the user's location information, and if the diagnosis indicates that a specialist's diagnosis is necessary, the server will refer the user to a nearby specialist facility.

[0531] Input: The diagnostic results generated in step 6 and the user's location information.

[0532] Specific operation: The server uses the user's location sensor to obtain their current geographic location, searches for nearby dental clinics based on the location information, and creates a list of suitable dental clinics.

[0533] Output: A list of nearby dental clinics provided to the user.

[0534] In this way, the present invention provides a system that allows users to easily check their dental health at home and quickly obtain necessary advice and information from specialists.

[0535] (Application example 1)

[0536] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0537] Conventional methods for checking dental health required regular visits to a dental clinic to receive a diagnosis from a specialist, which was time-consuming and laborious. Furthermore, selecting the appropriate dental care products required specialized knowledge, making it difficult for average users. This led to neglecting dental health management and often overlooking preventable problems. Furthermore, there was a need for a system that could instantly recommend and purchase appropriate dental care products based on diagnosis results through an online shopping function.

[0538] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0539] In this invention, the server includes: a means for acquiring images of teeth taken by a user; an image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; a means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; a means for referring the user to a nearby dental clinic if the diagnosis results indicate that a specialist diagnosis is necessary; and a means for recommending appropriate dental care products through an online shopping function based on the diagnosis results and providing a purchase link. This allows users to easily check the health condition of their teeth from home and obtain appropriate dental care products based on the diagnosis results. Furthermore, if a specialist diagnosis is necessary, appropriate information can be obtained quickly, making it easier to access a dental clinic.

[0540] "User" refers to an individual who uses the system to take images of their teeth and check their health.

[0541] The "image acquisition means" refers to a means including a device such as a smartphone or a digital camera for acquiring images of teeth taken by a user.

[0542] "Image processing means" refers to means including an AI algorithm that analyzes the acquired images and diagnoses the health of the teeth, such as detecting cavities, tartar, and gum inflammation.

[0543] The "diagnosis result providing means" is a means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results, thereby enabling the user to learn appropriate tooth brushing methods.

[0544] The "specialist referral means" is a means for referring the user to a nearby dental clinic when the diagnosis result indicates that a specialist diagnosis is necessary. This means is performed based on the user's location information.

[0545] "Online Shopping Feature" refers to a means of recommending appropriate dental care products based on the diagnosis results and providing a purchase link, allowing the user to immediately purchase the recommended products.

[0546] "Dental care products" refers to toothbrushes, toothpastes, mouthwashes, etc. for preventing cavities and removing tartar that are recommended based on the diagnosis results.

[0547] "Diagnosis results" refer to the evaluation results of dental health analyzed by the AI ​​algorithm, which determine the presence or absence of cavities, tartar, gum inflammation, etc.

[0548] Overall system configuration

[0549] This system allows users to easily check their dental health at home and instantly purchase appropriate dental care products. The system mainly consists of the following elements: image acquisition means, image processing means, diagnostic result provision means, specialist referral means, and online shopping function.

[0550] Hardware and software configuration

[0551] User device: A device with a camera and internet connection, such as a smartphone or tablet. The user device takes pictures and uploads them to the server.

[0552] Server: Cloud server. A central server that integrates image processing, diagnostic result provision, specialist referral, and online shopping functions.

[0553] Image processing software: Python libraries such as OpenCV and Keras, which allow captured images to be pre-processed and the AI ​​algorithms to make the diagnosis.

[0554] Communication software: Data is sent and received between the user device and the cloud server using HTTP libraries such as Requests.

[0555] Data Flow and Processing

[0556] 1. Image Acquisition:

[0557] The user takes multiple images of the oral cavity using a user device such as a smartphone, taking pictures of different areas of the mouth, particularly the front teeth, back teeth, and gums, and uses the in-app guide to ensure clear images are obtained.

[0558] 2. Upload your image:

[0559] The user device uploads the captured images to a cloud server via the Internet, and the image data is securely transmitted using communication software.

[0560] 3. Image Analysis and Diagnosis:

[0561] The cloud server analyzes the received images using AI algorithms, preprocessing the image data using image processing software, and then using an AI model (built with Keras) to diagnose dental health conditions such as cavities, tartar, and gum inflammation.

[0562] 4. Providing diagnostic results and recommendations:

[0563] The server provides the user with an evaluation and recommendations on brushing methods based on the diagnostic results. It also recommends appropriate dental care products (e.g., anti-cavity toothbrushes) based on the diagnostic results and provides a link to purchase them through an online shopping function.

[0564] 5. Specialist referrals:

[0565] If the diagnosis indicates that a specialist consultation is required, the server obtains the user's location and searches for a nearby dental clinic to refer the patient to, using an appropriate geographical information API.

[0566] Specific examples

[0567] Example 1: A user uses an app to take and upload a picture of their front teeth. This image is sent to a cloud server and analyzed by an AI algorithm. The diagnosis is that the patient has "early stages of cavities," and the app recommends the user to use an "anti-cavity toothbrush" and "anti-cavity gourmet paste." If the app determines that a dentist's diagnosis is necessary, the app will refer the user to a nearby dental clinic based on the user's location.

[0568] Prompt Sentence Examples

[0569] "I used the app to take a photo of my front teeth and uploaded it. The diagnosis was 'early stage of tooth decay.' What toothbrush should I buy?"

[0570] This system allows users to easily check the health of their teeth from the comfort of their own home, instantly purchase the necessary dental care products, and quickly receive a diagnosis from a specialist.

[0571] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0572] Step 1:

[0573] Taking an image

[0574] The user uses the smartphone camera to take multiple images of the teeth and gums in the oral cavity. These images include different areas such as the front teeth, back teeth, and gums. The input here is raw image data captured through the smartphone camera, and the output is a JPEG image file saved on the user's device. Specifically, the user launches the camera app and follows the in-app guide to adjust the shooting location while taking the image.

[0575] Step 2:

[0576] Uploading an image

[0577] Images taken on the user's device are selected and uploaded to the cloud server through the application. In this process, the user selects an image and taps the "Upload" button. The input is the image file stored on the user's device, and the output is the image file received by the cloud server. Specifically, the application generates an HTTP request to send the selected image file to the server. This is done using communication software (e.g., the Requests library).

[0578] Step 3:

[0579] Image preprocessing

[0580] The server preprocesses the received images, using the OpenCV library to resize and convert colors (e.g., to RGB). The input is a JPEG image file received by the cloud server, and the output is image data converted into a format suitable for AI algorithms. Specifically, the image is resized to 256x256 pixels, color converted, and then the data is normalized.

[0581] Step 4:

[0582] Image Analysis and Diagnosis

[0583] The server inputs the preprocessed images into an AI algorithm for analysis. Here, an AI model built with Keras is used. The input is the preprocessed image data, and the output is the diagnosis result (e.g., whether or not there is tooth decay, tartar, or gum inflammation). Specifically, the AI ​​model analyzes the image data and calculates the confidence level for each class (e.g., tooth decay, tartar, gum inflammation, etc.).

[0584] Step 5:

[0585] Providing diagnostic results and recommendations

[0586] The server evaluates and recommends brushing methods to the user based on the diagnostic results. It also recommends appropriate dental care products (e.g., anti-cavity toothbrushes) based on the diagnostic results and provides a purchase link through an online shopping function. The input is the diagnostic results obtained from the AI ​​model, and the output is recommendations and purchase links sent to the user. Specifically, it analyzes the diagnostic results, generates appropriate recommendations and product links, and outputs them to the user's application.

[0587] Step 6:

[0588] Specialist referrals

[0589] If the diagnosis result indicates that a specialist diagnosis is required, the server obtains the user's location information, searches for nearby dental clinics, and refers the user to them. The input is the diagnosis result and the user's location information, and the output is information about nearby dental clinics that is notified to the user. Specifically, the server uses a geographic information API to search for the most suitable dental clinic based on the location information, and provides that information to the user.

[0590] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0591] The present invention is a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. The system is equipped with an AI algorithm that acquires images of the user's teeth, analyzes the acquired images, and diagnoses the dental health condition, as well as an emotion engine that recognizes the user's emotions.

[0592] Image acquisition

[0593] The user takes photos of their teeth and gums in their oral cavity using a smartphone or digital camera. At least five images are taken from different angles, particularly of the front teeth, back teeth, and gums. The app guides the user through the photo-taking process, making it easy for them to obtain the appropriate images. The captured images are then saved on the device.

[0594] Uploading an image

[0595] The device uploads the captured images to a cloud server via the Internet, which securely and quickly transmits the images to the server, which receives them and uses them for analysis.

[0596] Image Processing and Diagnosis

[0597] The server analyzes the received images using an AI algorithm. The AI ​​algorithm processes the tooth images and diagnoses the health of the teeth, including cavities, tartar, and gum inflammation. The analysis results are stored on the server and managed for each user, allowing users to check the health of their own teeth.

[0598] emotion recognition

[0599] Before providing a diagnosis result, the server analyzes the user's emotional state using an emotion engine. The emotion engine reads emotions from the user's facial expressions and tone of voice to determine their current state of mind. This allows the server to provide feedback according to the user's emotional state.

[0600] Providing diagnostic results

[0601] Based on the diagnosis results, the server evaluates the user's brushing method and generates recommendations for improvements. Furthermore, depending on the diagnosis results, it recommends the appropriate toothbrush replacement timing and type. Taking into account the analysis results of the emotion engine, it provides encouraging and comforting messages according to the user's feelings.

[0602] For example, if the diagnosis is good, send a positive message such as, "You're polishing really well! Keep it up!"; if the diagnosis is bad, send a comforting message such as, "You need to improve a little. Don't worry, we're here to support you!"

[0603] Specialist referrals

[0604] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information. Based on the obtained location information, the server searches for nearby dental clinics and lists appropriate dental clinics. This allows the user to quickly receive a specialist diagnosis even if a serious health problem is detected.

[0605] Specific examples

[0606] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses the presence of early-stage cavities in the front teeth. After analyzing the image, the server then uses its emotion engine to analyze the user's emotional state and determines that the user is feeling anxious. The server then provides specific brushing advice (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specifically designed to prevent cavities), while also sending a comforting message: "Don't worry. We'll improve this together over time." Furthermore, the diagnosis indicates the need for a specialist consultation, so the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[0607] This allows users to easily check the health status of their teeth at home and receive appropriate advice. Feedback is also provided by the emotion engine, providing information to promptly seek a diagnosis from a specialist if necessary. Using this system significantly simplifies dental health management for users, while also providing psychological support.

[0608] The processing flow will be explained below.

[0609] Step 1:

[0610] The user launches the app and takes photos of their teeth and gums in their mouth. Following the in-app guide, they take at least five images from different angles, including the front teeth, back teeth, and gums. The images are then saved on the device.

[0611] Step 2:

[0612] The device uploads the captured images to a cloud server, where they are sent securely and quickly via the Internet.

[0613] Step 3:

[0614] The server receives the uploaded images and passes them to the image processing means, where they are processed by AI algorithms for analysis.

[0615] Step 4:

[0616] The server uses an AI algorithm to analyze the images and diagnose the health of the teeth. The analysis results include information on cavities, tartar, gum inflammation, etc. The diagnosis results are stored on the server and managed for each user.

[0617] Step 5:

[0618] The server evaluates the user's brushing method based on the diagnostic results, assesses the effectiveness of the current brushing method, and identifies areas for improvement.

[0619] Step 6:

[0620] The server generates information based on the diagnosis results that recommends the appropriate time and type of toothbrush to replace, and provides the user with information on effective brushing methods and recommended toothbrushes.

[0621] Step 7:

[0622] The server uses an emotion engine to analyze the user's emotional state before providing a diagnosis, reading emotions from the user's facial expressions and tone of voice to determine their current state of mind.

[0623] Step 8:

[0624] The server adjusts the generated advice information according to the user's emotional state and sends it to the device. The user receives an evaluation of their brushing technique, areas for improvement, notification of when to replace their toothbrush, and notifications of recommended toothbrushes through the app. The app also provides encouraging and comforting messages that match the user's emotional state.

[0625] Step 9:

[0626] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information and uses the obtained location information to search for dental clinics near the user's location.

[0627] Step 10:

[0628] The server creates a list of suitable dental clinics from the search results and sends it to the device. The user receives information about nearby dental clinics via the app and can take next steps, such as making an appointment.

[0629] Through this series of steps, users can easily check their dental health at home, receive appropriate guidance and psychological support, and be provided with information to quickly seek a specialist diagnosis if necessary.

[0630] Example 2

[0631] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0632] Conventional dental examination systems make it difficult for users to easily check their dental health at home and receive appropriate advice based on the results. They also fail to provide feedback that takes into account the user's emotional state, and lack sufficient psychological support. Furthermore, it is difficult for users to quickly find an appropriate dental clinic when a specialist diagnosis is required.

[0633] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0634] In this invention, the server includes: means for acquiring images of teeth taken by a user; image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; means including an emotion engine for analyzing the user's emotional state and providing feedback according to the emotion; and means for referring the user to a nearby dental clinic if the diagnosis results indicate that a specialist diagnosis is necessary. This enables a user to easily check the dental health condition at home and receive a referral to a specialist along with emotionally sensitive feedback.

[0635] "Means for acquiring images of teeth taken by a user" refers to a mechanism that allows the system to receive image data of teeth taken by a user using a smartphone or digital camera and make it available for storage or processing.

[0636] "Image processing means including an AI algorithm that analyzes acquired images and diagnoses the health condition of teeth" refers to a combination of software and hardware that uses artificial intelligence technology to analyze received dental image data and determine health conditions such as cavities, tartar, and gum inflammation.

[0637] "Means of providing users with an evaluation and recommendations on brushing methods based on the diagnostic results" refers to a function that provides specific guidance to users on appropriate brushing methods, areas for improvement, the type of toothbrush they should use, etc., based on the results of a dental health diagnosis by an AI algorithm.

[0638] "Means including an emotion engine that analyzes the user's emotional state and provides feedback according to that emotion" refers to a combination of software and hardware that reads the user's current emotion by analyzing the user's facial expression and tone of voice, and generates feedback or messages appropriate to that emotion.

[0639] "Means for referring the user to a nearby dental clinic when the diagnostic results indicate that a specialist diagnosis is necessary" refers to a mechanism by which the system refers to the user's diagnostic results, and when it determines that a specialist diagnosis is necessary, searches for the nearest appropriate dental clinic using the user's geographical location information and provides a list or reservation information.

[0640] The present invention provides a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. This system is implemented using the following hardware and software.

[0641] Hardware Configuration

[0642] 1. Smartphone or digital camera: Used by the user to take images of teeth. Equipped with a camera function and internet connection.

[0643] 2. Cloud Server: Analyzes the received images, generates and stores diagnostic results. This server is equipped with advanced computing resources and storage.

[0644] 3. Device: A smartphone or tablet used by a user to take pictures, upload them, and receive feedback.

[0645] Software Configuration

[0646] 1. Dedicated application: An application that allows users to take images of their own teeth and upload them to a cloud server.

[0647] 2. AI Algorithm: An artificial intelligence program that runs on a cloud server and analyzes received dental images to diagnose health conditions such as cavities, tartar, and gum inflammation.

[0648] 3. Emotion Engine: Another AI program running on a cloud server that analyzes the user's emotions from their facial expressions and tone of voice and generates appropriate feedback.

[0649] Operation explanation

[0650] Users take pictures of their teeth and gums using a smartphone or digital camera with a dedicated application installed. At least five images are taken from different angles, including the front teeth, back teeth, and gums. A shooting guide is displayed in the app, making it easy for users to take the appropriate images. These images are temporarily saved on the device.

[0651] The device then uploads these images to a cloud server, where they are encrypted and securely transmitted. The server then passes the images to an AI algorithm, which analyzes them and diagnoses dental health conditions such as cavities, tartar, and gum inflammation. The results are stored in a database for each user, allowing them to view them later.

[0652] Before providing a diagnosis, the server analyzes the user's emotions using an emotion engine. The emotion engine determines the user's current emotional state from the facial expressions and tone of voice provided by the user through the camera. Based on this, the server generates messages and feedback according to the user's emotional state.

[0653] Finally, the server sends the diagnosis results along with an evaluation of the user's brushing technique, along with recommendations for improvement and suggestions for improvement. For example, it may recommend using fluoride toothpaste or adopting a specific brushing technique to prevent cavities. Depending on the results of the emotion engine, a comforting message such as "Don't worry. We'll work together to improve over time" may also be sent.

[0654] If a specialist consultation is required, the server obtains the user's location information and searches for and lists nearby dental clinics, allowing the user to quickly find the appropriate clinic and use the appointment link to receive a prompt diagnosis.

[0655] Examples of concrete examples and prompts

[0656] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses the presence of early-stage cavities in the front teeth. After analyzing the image, the server then uses its emotion engine to analyze the user's emotional state and determines that the user is feeling anxious. The server then provides specific brushing advice (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specifically designed to prevent cavities), while also sending a comforting message: "Don't worry. We'll improve this together over time." Furthermore, the diagnosis indicates the need for a specialist consultation, so the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[0657] Example prompt (input to generative AI model):

[0658] Analyze the tooth images uploaded by the user and generate a diagnosis. Based on the analysis, provide specific brushing improvement suggestions and recommend suitable toothbrushes. Also, analyze the user's emotional state and generate messages to alleviate their concerns.

[0659] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0660] Step 1:

[0661] The user launches the dedicated application and takes pictures of their teeth using a smartphone or digital camera. The user follows the in-app guide to take at least five images from different angles, including the front teeth, back teeth, and gums. The guide displays instructions for taking appropriate images. The captured images are temporarily stored on the device.

[0662] Specific behavior:

[0663] 1. The user launches the application and taps the "Take Image" button.

[0664] 2. The device will start the camera and display a shooting guide.

[0665] 3. The user follows the guide to take images of each part of the body and taps "Save" to save them to the device.

[0666] Input: User actions and camera guide

[0667] Output: Saved tooth image data

[0668] Step 2:

[0669] The device uploads the saved images to the cloud server. The images are encrypted and sent securely. After the upload is complete, the device notifies the user.

[0670] Specific behavior:

[0671] 1. Select the image saved on your device and tap the "Upload" button.

[0672] 2. The device encrypts the image and sends it to a server over the Internet.

[0673] 3. The server receives the image, verifies the status, and notifies the device of a successful upload.

[0674] Input: Stored tooth image data, user command

[0675] Output: Image data uploaded to the cloud server

[0676] Step 3:

[0677] The server then passes the received image to an AI algorithm to begin analysis. The AI ​​algorithm analyzes the image and diagnoses health conditions such as cavities, tartar, and gum inflammation. The analysis results are then stored in a database.

[0678] Specific behavior:

[0679] 1. The server inputs the received image data into the AI ​​algorithm.

[0680] 2. AI algorithms analyze images and diagnose cavities, tartar, and gum inflammation.

[0681] 3. Diagnostic results are stored in a database for each user.

[0682] Input: Uploaded image data

[0683] Output: Save diagnostic results to database

[0684] Step 4:

[0685] The server uses an emotion engine to analyze the user's emotional state before providing a diagnosis. It determines emotions from the user's camera image and tone of voice and stores the results.

[0686] Specific behavior:

[0687] 1. The server passes the user's camera image and voice data to the emotion engine.

[0688] 2. The emotion engine analyzes this data and identifies the user's emotional state.

[0689] 3. The results of the sentiment analysis are generated and stored in a database for each user.

[0690] Input: Camera image and audio data

[0691] Output: Emotion analysis results

[0692] Step 5:

[0693] The server generates feedback based on the diagnosis and emotion analysis results, including recommendations for brushing method improvements and messages based on the user's emotional state. This feedback is then sent to the device.

[0694] Specific behavior:

[0695] 1. The server generates a feedback message based on the diagnosis and emotion analysis results.

[0696] 2. Simultaneously generate improvements and specific recommendations for brushing methods.

[0697] 3. Send feedback to the device and notify the user.

[0698] Input: Diagnosis results, emotion analysis results

[0699] Output: Feedback messages and brushing advice

[0700] Step 6:

[0701] If the diagnosis indicates that a specialist consultation is necessary, the server acquires the user's location information, searches for and lists nearby dental clinics, and sends the information to the user.

[0702] Specific behavior:

[0703] 1. The server obtains the user's location information and searches for nearby dental clinics.

[0704] 2. Create a list of suitable dental clinics and create a link to book appointments.

[0705] 3. This information is sent to the terminal and notified to the user.

[0706] Input: Diagnostic results, user location information

[0707] Output: A list of dental clinics and a link to book an appointment

[0708] (Application example 2)

[0709] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0710] In today's busy lifestyles, there is a need for a system that allows users to easily check their dental health at home and receive appropriate feedback and a prompt diagnosis. However, existing systems lack the functionality to provide feedback based on emotional state or recommend experts, making it difficult to provide effective advice while alleviating users' concerns.

[0711] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for acquiring images of teeth taken by the user; image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; means for referring the user to a nearby medical institution if the diagnosis results indicate that a professional diagnosis is necessary; and means for recognizing the user's emotional state and providing feedback according to the user's emotion. This allows the user to easily check the health condition of their teeth at home and receive appropriate advice and support according to their emotion.

[0712] "User" refers to the person who uses the system, and in this case, refers to an individual who wants to check and manage their dental health status.

[0713] A "tooth image" is a photograph of the teeth and gums inside the oral cavity taken by the user.

[0714] The "means for acquiring" is a function for importing images of teeth taken by the user into the system.

[0715] "Image processing means" is a function for analyzing acquired tooth images using an AI algorithm.

[0716] "AI algorithm" refers to an analytical method that uses artificial intelligence, and refers to the function of analyzing images of teeth to diagnose their health condition.

[0717] "Diagnosis" refers to the evaluation of dental health based on the analysis results.

[0718] "Health status" refers to the state of the teeth and gums, and whether or not there are any symptoms such as cavities or inflammation.

[0719] "Brushing technique" refers to the method of brushing your teeth effectively, such as how you brush your teeth and the type of toothbrush you use.

[0720] "Evaluation" refers to determining whether the current brushing method is appropriate based on the diagnostic results.

[0721] "Recommendations" are specific advice or suggestions for improvement provided to the user based on the diagnostic results.

[0722] A "professional" is someone who has specialized knowledge and qualifications related to teeth and oral health, such as a dentist or dental hygienist.

[0723] "Diagnosis required" means that the system's analysis results indicate that the user's dental health condition requires a specialist examination.

[0724] "Nearby medical institution" refers to a medical facility such as a dental clinic that is located within easy access from the user's current location.

[0725] "Means for referral" is a function that guides the user to an appropriate medical institution based on the diagnosis results.

[0726] "Emotional state" refers to the user's psychological state or mood, which in this case is determined from facial expressions and tone of voice.

[0727] "Feedback" is a general term for comments and advice provided to users based on diagnostic and analytical results.

[0728] This invention is a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. This system supports users' dental health management by analyzing dental images taken by the user and providing appropriate advice and emotional feedback based on the diagnosis results.

[0729] The system includes the following means:

[0730] 1. Image acquisition method:

[0731] The user takes images of their teeth using a smartphone or digital camera. The images are taken from at least five angles: front, left side, right side, top, and bottom. This allows detailed image data to be obtained. The captured image data is saved on the user's device.

[0732] 2. Image upload method:

[0733] The user's device uploads the captured images to a cloud server via the Internet, where the image data is received securely and quickly.

[0734] 3. Imaging and diagnostic tools:

[0735] The cloud server analyzes the received images using an AI algorithm, which diagnoses the health of the teeth (e.g., cavities, tartar, gum inflammation, etc.) and stores the results on the server.

[0736] 4. Emotion recognition means:

[0737] Before providing a diagnosis, the server analyzes the user's emotional state using an emotion engine, which identifies emotions from the user's facial expressions and tone of voice to determine the user's current state of mind.

[0738] 5. Feedback can be provided by:

[0739] The server provides optimal feedback to the user based on the diagnosis and emotion recognition results. The feedback includes an evaluation of brushing methods, areas for improvement, and recommendations. It also recommends the type and timing of toothbrush replacement based on the diagnosis results. It also provides encouraging and comforting messages based on the emotion engine results.

[0740] 6. Specialist referral methods:

[0741] If the diagnosis results indicate that a specialist diagnosis is necessary, the server obtains the user's location information and searches for and refers the user to a nearby medical institution. This allows the user to receive a prompt, professional diagnosis even if a serious health problem is detected.

[0742] Hardware and software used

[0743] Hardware:

[0744] Smartphone or digital camera (for taking pictures)

[0745] Internet-connected device (image upload)

[0746] software:

[0747] OpenCV (image processing)

[0748] EmotionRecognizer

[0749] AI Diagnostic (Diagnosis by AI algorithm)

[0750] requests (REST API calls)

[0751] Specific examples

[0752] For example, a user can use the app to take pictures of their teeth from the front, left side, right side, top, and bottom, and upload them to a cloud server. The server then uses AI to analyze the received images and diagnoses the presence of early-stage cavities in the front teeth. The emotion engine then analyzes the user's emotional state and determines that the user is feeling anxious. The app then provides specific brushing advice and recommended toothbrushes, along with a comforting message: "Don't worry. We'll take the time to improve together." Furthermore, if the diagnosis indicates that a professional diagnosis is necessary, the app searches for nearby dental clinics based on the user's location and provides information on appropriate dental clinics.

[0753] Prompt Sentence Examples

[0754] A user takes photos of their teeth with their smartphone from the following angles: front, left side, right side, top, and bottom. Upload these images to a cloud server and use an AI algorithm to analyze the health of their teeth. Additionally, recognize emotions based on the user's facial expressions and voice and provide feedback based on the following criteria:

[0755] If a user is feeling anxious, provide comforting messages such as, "Don't worry. We'll improve this together over time."

[0756] If the diagnosis is positive, a positive message such as "You're polishing very well! Keep it up!" is provided.

[0757] If the diagnosis is negative, provide comforting messages such as, "You need to improve a little. It's okay, we're here to help!"

[0758] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0759] Step 1:

[0760] The user takes images of the teeth using a smartphone or digital camera. The images are taken from at least five angles: front, left side, right side, top, and bottom. The user follows the guide within the app to obtain the appropriate images. This is the input, and the images are saved in the smartphone's storage as the output. Specifically, the user launches the camera app and takes photos from each angle while following the guide display.

[0761] Step 2:

[0762] The user's device uploads the captured tooth images to the cloud server via the Internet. The input is the image stored on the device, and the output is the image stored on the cloud server. Specifically, the device sends the image file to the cloud server using a REST API.

[0763] Step 3:

[0764] The server receives images uploaded to the cloud server and analyzes each image using an AI algorithm. The input is the uploaded image data, and the output is a diagnosis of the dental health condition (e.g., cavities, tartar, gum inflammation, etc.). Specifically, the server preprocesses the images using an image processing library (e.g., OpenCV) and runs a deep learning model to evaluate the dental condition.

[0765] Step 4:

[0766] The server extracts emotions from the user's facial expressions and tone of voice to recognize the user's emotional state based on the diagnosis results. The input is the diagnosis results and the user's facial expressions and voice data, and the output is a judgment of the user's emotional state (e.g., anxiety, relief, etc.). Specifically, the server uses an emotion recognition engine (e.g., EmotionRecognizer) to analyze the facial images and voice data.

[0767] Step 5:

[0768] The server generates optimal feedback for the user based on the diagnosis results and emotion recognition results. The input is the diagnosis results and emotion determination results, and the output is advice or a message to be provided to the user. Specifically, the server generates a positive message if the diagnosis results are good, and a comforting message if the user is very anxious, and sends it to the user.

[0769] Step 6:

[0770] If the diagnosis result indicates that a specialist diagnosis is necessary, the server obtains the user's location information. The input is the diagnosis result and the user's location information, and the output is a list of nearby medical institutions. Specifically, the server obtains the user's location using a location information API and searches for the nearest medical institution based on that location.

[0771] Step 7:

[0772] The server introduces nearby medical institutions to the user. The input is the location information of the user who is deemed to need a specialist diagnosis and a list of medical institutions, and the output is a notification message including the referral details. Specifically, the server sends information about appropriate medical institutions to the user's device and displays it as a notification.

[0773] These steps allow users to easily check their dental health status at home and receive appropriate advice and support according to their emotions.

[0774] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0775] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0776] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0777] [Third embodiment]

[0778] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0779] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0780] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0781] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0782] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0783] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0784] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0785] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0786] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0787] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0788] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0789] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0790] The present invention is a system that allows users to easily check their dental health at home and receive appropriate advice. This system acquires images of the user's teeth and is equipped with an AI algorithm that analyzes the acquired images to diagnose the dental health. Below, we will explain in detail the embodiments of the present invention.

[0791] Image acquisition

[0792] The user takes photos of their teeth and gums in their oral cavity using a smartphone or digital camera. At least five images are taken from different angles, particularly of the front teeth, back teeth, and gums. The app guides the user through the photo-taking process, making it easy for them to obtain the appropriate images. The captured images are then saved on the device.

[0793] Uploading an image

[0794] The device uploads the captured images to a cloud server via the Internet, which securely and quickly transmits the images to the server, which receives them and uses them for analysis.

[0795] Image Processing and Diagnosis

[0796] The server analyzes the received images using an AI algorithm. The AI ​​algorithm processes the tooth images and diagnoses the health of the teeth, including cavities, tartar, and gum inflammation. The analysis results are stored on the server and managed for each user, allowing users to check the health of their own teeth.

[0797] Providing diagnostic results

[0798] Based on the diagnostic results, the server evaluates the user's brushing method and generates recommendations for improvements. It also recommends the appropriate toothbrush replacement timing and type based on the diagnostic results. This information is sent to the device and notified to the user. The user can receive advice through the app and learn more effective brushing methods.

[0799] Specialist referrals

[0800] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information. Based on the obtained location information, the server searches for nearby dental clinics and lists appropriate dental clinics. This allows the user to quickly receive a specialist diagnosis even if a serious health problem is detected.

[0801] Specific examples

[0802] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses that there is early-stage decay in the front teeth. As a result, specific advice on brushing techniques (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specialized for preventing cavities) are provided. Furthermore, if the diagnosis determines that a specialist diagnosis is necessary, the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[0803] This allows users to easily check the health of their teeth at home and receive necessary advice and information on how to seek a diagnosis from a specialist. Using this system will greatly simplify dental health management for users and enable early detection and treatment.

[0804] The processing flow will be explained below.

[0805] Step 1:

[0806] The user launches the app and takes photos of their teeth and gums in their mouth. Following the in-app guide, they take at least five images from different angles, including the front teeth, back teeth, and gums. The images are then saved on the device.

[0807] Step 2:

[0808] The device uploads the captured images to a cloud server, where they are sent securely and quickly via the Internet.

[0809] Step 3:

[0810] The server receives the uploaded images and passes them to the image processing means, where they are processed by AI algorithms for analysis.

[0811] Step 4:

[0812] The server uses an AI algorithm to analyze the images and diagnose the health of the teeth. The analysis results include information on cavities, tartar, gum inflammation, etc. The diagnosis results are stored on the server and managed for each user.

[0813] Step 5:

[0814] The server evaluates the user's brushing method based on the diagnostic results, assesses the effectiveness of the current brushing method, and identifies areas for improvement.

[0815] Step 6:

[0816] The server generates information based on the diagnosis results that recommends the appropriate time and type of toothbrush to replace, and provides the user with information on effective brushing methods and recommended toothbrushes.

[0817] Step 7:

[0818] The server sends the generated advice information to the device, and the user receives an evaluation of their brushing method, areas for improvement, when to replace their toothbrush, and notifications of recommended toothbrushes through the app.

[0819] Step 8:

[0820] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information and uses the obtained location information to search for dental clinics near the user's location.

[0821] Step 9:

[0822] The server creates a list of suitable dental clinics from the search results and sends it to the device. The user receives information about nearby dental clinics via the app and can take next steps, such as making an appointment.

[0823] Through these steps, the system allows users to easily understand the state of their dental health and receive appropriate guidance. If necessary, it also provides information to promptly seek a diagnosis from a specialist.

[0824] Example 1

[0825] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0826] Conventional dental diagnostic systems require users to visit a dentist regularly, making it difficult to easily check the health of their teeth at home and receive appropriate advice. While there are applications for checking dental health, they lack the ability to respond quickly when a professional diagnosis is required. Furthermore, the lack of specific brushing methods or recommendations for appropriate toothbrushes based on the diagnosis results makes it difficult for users to efficiently perform oral care.

[0827] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0828] In this invention, the server includes means for acquiring tooth images taken by a user, means for saving the captured tooth images in a digital device, means for uploading the saved images to a cloud computing environment via the Internet, image processing means including a generative AI algorithm for analyzing the acquired images and diagnosing the dental health condition, means for providing an evaluation of the user's brushing method and suggestions for improvement based on the diagnosis results, means for recommending the appropriate type of toothbrush and replacement timing to the user based on the diagnosis results, and means for acquiring the user's location information and referring the user to a nearby specialist facility if the diagnosis results determine that a specialist diagnosis is necessary. This allows users to easily check their oral health at home and quickly obtain necessary advice and specialist information.

[0829] "User" refers to a person who uses the system to check their dental health and receive advice and diagnostic results.

[0830] "Digital devices" refers to electronic devices used to capture and record images, such as smartphones and digital cameras.

[0831] "Cloud computing environment" refers to remote server storage and processing power provided over the Internet.

[0832] "Generative AI algorithm" refers to a program that uses AI technology to analyze data and diagnose dental health.

[0833] "Image processing means" refers to the technical process for analyzing the acquired images and generating a diagnostic result.

[0834] "Means for evaluating brushing methods and providing points for improvement" refers to a system function that evaluates the user's tooth brushing method based on the diagnostic results and specifically suggests points for improvement.

[0835] "Means for recommending toothbrush types and replacement timing" refers to a system function that indicates to the user the appropriate toothbrush selection and replacement timing based on the diagnostic results.

[0836] "When it is determined that a specialist diagnosis is necessary" refers to a situation where the analysis results of the AI ​​algorithm indicate a serious oral problem and a specialist diagnosis is recommended.

[0837] "Means for obtaining location information and introducing nearby specialist facilities" refers to a system function that identifies the user's current geographic location and searches for and recommends specialist facilities such as dental clinics in the vicinity.

[0838] This invention is a system that allows users to easily check their dental health at home and receive appropriate advice. This system acquires images of the user's teeth and includes a generative AI algorithm that analyzes the acquired images to diagnose the dental health. Below, we will explain in detail the embodiments of the invention.

[0839] Image acquisition

[0840] The user uses a smartphone or digital camera (hereinafter referred to as "digital device") to take photos of their teeth and gums in their mouth. They launch the application and follow the in-app guide to take photos from the appropriate angle and position. At least five images should be taken, including front teeth, back teeth, gums, etc. The captured images are saved in the device's internal storage.

[0841] Uploading an image

[0842] The device uploads the captured images to a cloud computing environment (hereafter referred to as the cloud server) via the Internet. When uploading, the device checks for Internet connectivity and compresses and encrypts the image files. The device uses the upload API to send the images to the cloud server, and the server confirms receipt.

[0843] Image Processing and Diagnosis

[0844] The server analyzes the received images using a generative AI algorithm, which specifically includes the following processes:

[0845] 1. Decode the received image file and restore the original image.

[0846] 2. Pre-process the image to adjust the sharpness and contrast.

[0847] 3. An AI algorithm analyzes the images and diagnoses dental health conditions such as cavities, tartar, and gum inflammation. The results are stored in a database for each user.

[0848] Providing diagnostic results

[0849] The server generates a diagnosis based on the image analysis, which includes the following information:

[0850] 1. Detailed information such as whether or not you have cavities, their progression, whether or not you have tartar, and whether or not you have gum inflammation.

[0851] 2. Evaluate your brushing technique and suggest areas for improvement, specifically, advice on which areas to brush more carefully.

[0852] 3. Recommended types of toothbrushes and when to replace them.

[0853] This information is sent from the server to the terminal and notified to the user.

[0854] Specialist referrals

[0855] If the server determines that a diagnosis by a specialist is necessary based on the diagnostic results, it performs the following process:

[0856] 1. Use the device's location sensor to obtain the user's location information.

[0857] 2. Search for nearby dental clinics based on location information and list suitable dental clinics.

[0858] 3. The created list is sent to the terminal and notified to the user.

[0859] Specific examples

[0860] For example, if a user uses an app to take a picture of their teeth and upload it, the server would do the following:

[0861] 1. The user follows the in-app guide to take five intraoral images from different angles. The device stores these images in its internal storage.

[0862] 2. When you press a button in the app, the device will encrypt the captured image and upload it to the cloud server. A progress bar will be displayed until the transfer is complete.

[0863] 3. The server decodes the received image and converts it into a format for analysis. A generative AI algorithm scans the image and detects cavities, tartar, gum inflammation, etc. For example, an early stage of cavities was detected in a front tooth.

[0864] 4. Based on the analysis results, the server generates information on how to improve the user's brushing method and recommends toothbrushes. For example, it suggests "brushing the insides of your front teeth more carefully and using a toothbrush specifically designed to prevent cavities."

[0865] 5. The server obtains the user's location information and searches for nearby dental clinics. For example, it creates a list of "XX Hospital, △△ Dental Clinic" and sends it to the terminal.

[0866] Prompt Sentence Examples

[0867] Here is an example prompt:

[0868] Example prompt 1:

[0869] "Please analyze the images of my teeth and tell me if I have cavities. I can also learn the correct way to brush my teeth and recommend a toothbrush."

[0870] Example prompt 2:

[0871] "Based on my dental diagnosis, if I need to see a specialist, please tell me the nearest dental clinic based on my location."

[0872] In this way, users can easily check their oral health at home and quickly obtain necessary advice and information from specialists.

[0873] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0874] Step 1:

[0875] A user takes a photograph of the teeth and gums in their mouth using a digital device (smartphone or digital camera).

[0876] Input: A video of the user's mouth.

[0877] How it works: Users launch the app and follow the in-app guide to take at least five images from different angles, which provides detailed images of the front teeth, back teeth, and gums.

[0878] Output: Multiple image files of the teeth (JPEG, PNG, etc.).

[0879] Step 2:

[0880] The device saves the captured image file to its internal storage.

[0881] Input: The image file taken in step 1.

[0882] What it does: The device automatically saves the image in a specific folder for further processing, and adds a timestamp to the file name for easy identification.

[0883] Output: Image file saved on the device.

[0884] Step 3:

[0885] The device uploads the image file to the cloud server.

[0886] Input: The image file saved in step 2.

[0887] What it does: The device checks for an internet connection, compresses and encrypts the image file, then sends it to the cloud server via the upload API, where it confirms receipt. A progress bar is displayed in the app.

[0888] Output: Encrypted image files stored in the cloud server.

[0889] Step 4:

[0890] The server decodes the received image file and performs image preprocessing.

[0891] Input: The encrypted image file uploaded to the cloud server in step 3.

[0892] What it does: The server decodes and restores the image file, then performs preprocessing such as adjusting sharpness and contrast to improve the image quality and make it easier to analyze.

[0893] Output: Preprocessed tooth image.

[0894] Step 5:

[0895] The server uses a generative AI algorithm to analyze the images and diagnose the health of the teeth.

[0896] Input: Image files preprocessed in step 4.

[0897] How it works: Generative AI algorithms scan images to detect cavities, tartar, gum inflammation, etc. The algorithms are trained on large medical datasets and therefore produce highly accurate diagnoses.

[0898] Output: Dental health diagnosis (e.g., presence of cavities, tartar, and gum inflammation).

[0899] Step 6:

[0900] Based on the diagnostic results, the server evaluates the user's brushing method, recommends areas for improvement, and recommends the appropriate type of toothbrush and when to replace it.

[0901] Input: The diagnostic results obtained in step 5.

[0902] Specific operation: The server evaluates the user's brushing method based on the analysis results and suggests areas for improvement, such as brushing specific areas more carefully. It also generates information on toothbrushes and replacement times based on the user's situation.

[0903] Output: Brushing advice and toothbrush recommendations for the user.

[0904] Step 7:

[0905] The server acquires the user's location information, and if the diagnosis indicates that a specialist's diagnosis is necessary, the server will refer the user to a nearby specialist facility.

[0906] Input: The diagnostic results generated in step 6 and the user's location information.

[0907] Specific operation: The server uses the user's location sensor to obtain their current geographic location, searches for nearby dental clinics based on the location information, and creates a list of suitable dental clinics.

[0908] Output: A list of nearby dental clinics provided to the user.

[0909] In this way, the present invention provides a system that allows users to easily check their dental health at home and quickly obtain necessary advice and information from specialists.

[0910] (Application example 1)

[0911] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0912] Conventional methods for checking dental health required regular visits to a dental clinic to receive a diagnosis from a specialist, which was time-consuming and laborious. Furthermore, selecting the appropriate dental care products required specialized knowledge, making it difficult for average users. This led to neglecting dental health management and often overlooking preventable problems. Furthermore, there was a need for a system that could instantly recommend and purchase appropriate dental care products based on diagnosis results through an online shopping function.

[0913] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0914] In this invention, the server includes: a means for acquiring images of teeth taken by a user; an image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; a means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; a means for referring the user to a nearby dental clinic if the diagnosis results indicate that a specialist diagnosis is necessary; and a means for recommending appropriate dental care products through an online shopping function based on the diagnosis results and providing a purchase link. This allows users to easily check the health condition of their teeth from home and obtain appropriate dental care products based on the diagnosis results. Furthermore, if a specialist diagnosis is necessary, appropriate information can be obtained quickly, making it easier to access a dental clinic.

[0915] "User" refers to an individual who uses the system to take images of their teeth and check their health.

[0916] The "image acquisition means" refers to a means including a device such as a smartphone or a digital camera for acquiring images of teeth taken by a user.

[0917] "Image processing means" refers to means including an AI algorithm that analyzes the acquired images and diagnoses the health of the teeth, such as detecting cavities, tartar, and gum inflammation.

[0918] The "diagnosis result providing means" is a means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results, thereby enabling the user to learn appropriate tooth brushing methods.

[0919] The "specialist referral means" is a means for referring the user to a nearby dental clinic when the diagnosis result indicates that a specialist diagnosis is necessary. This means is performed based on the user's location information.

[0920] "Online Shopping Feature" refers to a means of recommending appropriate dental care products based on the diagnosis results and providing a purchase link, allowing the user to immediately purchase the recommended products.

[0921] "Dental care products" refers to toothbrushes, toothpastes, mouthwashes, etc. for preventing cavities and removing tartar that are recommended based on the diagnosis results.

[0922] "Diagnosis results" refer to the evaluation results of dental health analyzed by the AI ​​algorithm, which determine the presence or absence of cavities, tartar, gum inflammation, etc.

[0923] Overall system configuration

[0924] This system allows users to easily check their dental health at home and instantly purchase appropriate dental care products. The system mainly consists of the following elements: image acquisition means, image processing means, diagnostic result provision means, specialist referral means, and online shopping function.

[0925] Hardware and software configuration

[0926] User device: A device with a camera and internet connection, such as a smartphone or tablet. The user device takes pictures and uploads them to the server.

[0927] Server: Cloud server. A central server that integrates image processing, diagnostic result provision, specialist referral, and online shopping functions.

[0928] Image processing software: Python libraries such as OpenCV and Keras, which allow captured images to be pre-processed and the AI ​​algorithms to make the diagnosis.

[0929] Communication software: Data is sent and received between the user device and the cloud server using HTTP libraries such as Requests.

[0930] Data Flow and Processing

[0931] 1. Image Acquisition:

[0932] The user takes multiple images of the oral cavity using a user device such as a smartphone, taking pictures of different areas of the mouth, particularly the front teeth, back teeth, and gums, and uses the in-app guide to ensure clear images are obtained.

[0933] 2. Upload your image:

[0934] The user device uploads the captured images to a cloud server via the Internet, and the image data is securely transmitted using communication software.

[0935] 3. Image Analysis and Diagnosis:

[0936] The cloud server analyzes the received images using AI algorithms, preprocessing the image data using image processing software, and then using an AI model (built with Keras) to diagnose dental health conditions such as cavities, tartar, and gum inflammation.

[0937] 4. Providing diagnostic results and recommendations:

[0938] The server provides the user with an evaluation and recommendations on brushing methods based on the diagnostic results. It also recommends appropriate dental care products (e.g., anti-cavity toothbrushes) based on the diagnostic results and provides a link to purchase them through an online shopping function.

[0939] 5. Specialist referrals:

[0940] If the diagnosis indicates that a specialist consultation is required, the server obtains the user's location and searches for a nearby dental clinic to refer the patient to, using an appropriate geographical information API.

[0941] Specific examples

[0942] Example 1: A user uses an app to take and upload a picture of their front teeth. This image is sent to a cloud server and analyzed by an AI algorithm. The diagnosis is that the patient has "early stages of cavities," and the app recommends the user to use an "anti-cavity toothbrush" and "anti-cavity gourmet paste." If the app determines that a dentist's diagnosis is necessary, the app will refer the user to a nearby dental clinic based on the user's location.

[0943] Prompt Sentence Examples

[0944] "I used the app to take a photo of my front teeth and uploaded it. The diagnosis was 'early stage of tooth decay.' What toothbrush should I buy?"

[0945] This system allows users to easily check the health of their teeth from the comfort of their own home, instantly purchase the necessary dental care products, and quickly receive a diagnosis from a specialist.

[0946] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0947] Step 1:

[0948] Taking an image

[0949] The user uses the smartphone camera to take multiple images of the teeth and gums in the oral cavity. These images include different areas such as the front teeth, back teeth, and gums. The input here is raw image data captured through the smartphone camera, and the output is a JPEG image file saved on the user's device. Specifically, the user launches the camera app and follows the in-app guide to adjust the shooting location while taking the image.

[0950] Step 2:

[0951] Uploading an image

[0952] Images taken on the user's device are selected and uploaded to the cloud server through the application. In this process, the user selects an image and taps the "Upload" button. The input is the image file stored on the user's device, and the output is the image file received by the cloud server. Specifically, the application generates an HTTP request to send the selected image file to the server. This is done using communication software (e.g., the Requests library).

[0953] Step 3:

[0954] Image preprocessing

[0955] The server preprocesses the received images, using the OpenCV library to resize and convert colors (e.g., to RGB). The input is a JPEG image file received by the cloud server, and the output is image data converted into a format suitable for AI algorithms. Specifically, the image is resized to 256x256 pixels, color converted, and then the data is normalized.

[0956] Step 4:

[0957] Image Analysis and Diagnosis

[0958] The server inputs the preprocessed images into an AI algorithm for analysis. Here, an AI model built with Keras is used. The input is the preprocessed image data, and the output is the diagnosis result (e.g., whether or not there is tooth decay, tartar, or gum inflammation). Specifically, the AI ​​model analyzes the image data and calculates the confidence level for each class (e.g., tooth decay, tartar, gum inflammation, etc.).

[0959] Step 5:

[0960] Providing diagnostic results and recommendations

[0961] The server evaluates and recommends brushing methods to the user based on the diagnostic results. It also recommends appropriate dental care products (e.g., anti-cavity toothbrushes) based on the diagnostic results and provides a purchase link through an online shopping function. The input is the diagnostic results obtained from the AI ​​model, and the output is recommendations and purchase links sent to the user. Specifically, it analyzes the diagnostic results, generates appropriate recommendations and product links, and outputs them to the user's application.

[0962] Step 6:

[0963] Specialist referrals

[0964] If the diagnosis result indicates that a specialist diagnosis is required, the server obtains the user's location information, searches for nearby dental clinics, and refers the user to them. The input is the diagnosis result and the user's location information, and the output is information about nearby dental clinics that is notified to the user. Specifically, the server uses a geographic information API to search for the most suitable dental clinic based on the location information, and provides that information to the user.

[0965] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0966] The present invention is a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. The system is equipped with an AI algorithm that acquires images of the user's teeth, analyzes the acquired images, and diagnoses the dental health condition, as well as an emotion engine that recognizes the user's emotions.

[0967] Image acquisition

[0968] The user takes photos of their teeth and gums in their oral cavity using a smartphone or digital camera. At least five images are taken from different angles, particularly of the front teeth, back teeth, and gums. The app guides the user through the photo-taking process, making it easy for them to obtain the appropriate images. The captured images are then saved on the device.

[0969] Uploading an image

[0970] The device uploads the captured images to a cloud server via the Internet, which securely and quickly transmits the images to the server, which receives them and uses them for analysis.

[0971] Image Processing and Diagnosis

[0972] The server analyzes the received images using an AI algorithm. The AI ​​algorithm processes the tooth images and diagnoses the health of the teeth, including cavities, tartar, and gum inflammation. The analysis results are stored on the server and managed for each user, allowing users to check the health of their own teeth.

[0973] emotion recognition

[0974] Before providing a diagnosis result, the server analyzes the user's emotional state using an emotion engine. The emotion engine reads emotions from the user's facial expressions and tone of voice to determine their current state of mind. This allows the server to provide feedback according to the user's emotional state.

[0975] Providing diagnostic results

[0976] Based on the diagnosis results, the server evaluates the user's brushing method and generates recommendations for improvements. Furthermore, depending on the diagnosis results, it recommends the appropriate toothbrush replacement timing and type. Taking into account the analysis results of the emotion engine, it provides encouraging and comforting messages according to the user's feelings.

[0977] For example, if the diagnosis is good, send a positive message such as, "You're polishing really well! Keep it up!"; if the diagnosis is bad, send a comforting message such as, "You need to improve a little. Don't worry, we're here to support you!"

[0978] Specialist referrals

[0979] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information. Based on the obtained location information, the server searches for nearby dental clinics and lists appropriate dental clinics. This allows the user to quickly receive a specialist diagnosis even if a serious health problem is detected.

[0980] Specific examples

[0981] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses the presence of early-stage cavities in the front teeth. After analyzing the image, the server then uses its emotion engine to analyze the user's emotional state and determines that the user is feeling anxious. The server then provides specific brushing advice (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specifically designed to prevent cavities), while also sending a comforting message: "Don't worry. We'll improve this together over time." Furthermore, the diagnosis indicates the need for a specialist consultation, so the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[0982] This allows users to easily check the health status of their teeth at home and receive appropriate advice. Feedback is also provided by the emotion engine, providing information to promptly seek a diagnosis from a specialist if necessary. Using this system significantly simplifies dental health management for users, while also providing psychological support.

[0983] The processing flow will be explained below.

[0984] Step 1:

[0985] The user launches the app and takes photos of their teeth and gums in their mouth. Following the in-app guide, they take at least five images from different angles, including the front teeth, back teeth, and gums. The images are then saved on the device.

[0986] Step 2:

[0987] The device uploads the captured images to a cloud server, where they are sent securely and quickly via the Internet.

[0988] Step 3:

[0989] The server receives the uploaded images and passes them to the image processing means, where they are processed by AI algorithms for analysis.

[0990] Step 4:

[0991] The server uses an AI algorithm to analyze the images and diagnose the health of the teeth. The analysis results include information on cavities, tartar, gum inflammation, etc. The diagnosis results are stored on the server and managed for each user.

[0992] Step 5:

[0993] The server evaluates the user's brushing method based on the diagnostic results, assesses the effectiveness of the current brushing method, and identifies areas for improvement.

[0994] Step 6:

[0995] The server generates information based on the diagnosis results that recommends the appropriate time and type of toothbrush to replace, and provides the user with information on effective brushing methods and recommended toothbrushes.

[0996] Step 7:

[0997] The server uses an emotion engine to analyze the user's emotional state before providing a diagnosis, reading emotions from the user's facial expressions and tone of voice to determine their current state of mind.

[0998] Step 8:

[0999] The server adjusts the generated advice information according to the user's emotional state and sends it to the device. The user receives an evaluation of their brushing technique, areas for improvement, notification of when to replace their toothbrush, and notifications of recommended toothbrushes through the app. The app also provides encouraging and comforting messages that match the user's emotional state.

[1000] Step 9:

[1001] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information and uses the obtained location information to search for dental clinics near the user's location.

[1002] Step 10:

[1003] The server creates a list of suitable dental clinics from the search results and sends it to the device. The user receives information about nearby dental clinics via the app and can take next steps, such as making an appointment.

[1004] Through this series of steps, users can easily check their dental health at home, receive appropriate guidance and psychological support, and be provided with information to quickly seek a specialist diagnosis if necessary.

[1005] Example 2

[1006] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1007] Conventional dental examination systems make it difficult for users to easily check their dental health at home and receive appropriate advice based on the results. They also fail to provide feedback that takes into account the user's emotional state, and lack sufficient psychological support. Furthermore, it is difficult for users to quickly find an appropriate dental clinic when a specialist diagnosis is required.

[1008] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1009] In this invention, the server includes: means for acquiring images of teeth taken by a user; image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; means including an emotion engine for analyzing the user's emotional state and providing feedback according to the emotion; and means for referring the user to a nearby dental clinic if the diagnosis results indicate that a specialist diagnosis is necessary. This enables a user to easily check the dental health condition at home and receive a referral to a specialist along with emotionally sensitive feedback.

[1010] "Means for acquiring images of teeth taken by a user" refers to a mechanism that allows the system to receive image data of teeth taken by a user using a smartphone or digital camera and make it available for storage or processing.

[1011] "Image processing means including an AI algorithm that analyzes acquired images and diagnoses the health condition of teeth" refers to a combination of software and hardware that uses artificial intelligence technology to analyze received dental image data and determine health conditions such as cavities, tartar, and gum inflammation.

[1012] "Means of providing users with an evaluation and recommendations on brushing methods based on the diagnostic results" refers to a function that provides specific guidance to users on appropriate brushing methods, areas for improvement, the type of toothbrush they should use, etc., based on the results of a dental health diagnosis by an AI algorithm.

[1013] "Means including an emotion engine that analyzes the user's emotional state and provides feedback according to that emotion" refers to a combination of software and hardware that reads the user's current emotion by analyzing the user's facial expression and tone of voice, and generates feedback or messages appropriate to that emotion.

[1014] "Means for referring the user to a nearby dental clinic when the diagnostic results indicate that a specialist diagnosis is necessary" refers to a mechanism by which the system refers to the user's diagnostic results, and when it determines that a specialist diagnosis is necessary, searches for the nearest appropriate dental clinic using the user's geographical location information and provides a list or reservation information.

[1015] The present invention provides a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. This system is implemented using the following hardware and software.

[1016] Hardware Configuration

[1017] 1. Smartphone or digital camera: Used by the user to take images of teeth. Equipped with a camera function and internet connection.

[1018] 2. Cloud Server: Analyzes the received images, generates and stores diagnostic results. This server is equipped with advanced computing resources and storage.

[1019] 3. Device: A smartphone or tablet used by a user to take pictures, upload them, and receive feedback.

[1020] Software Configuration

[1021] 1. Dedicated application: An application that allows users to take images of their own teeth and upload them to a cloud server.

[1022] 2. AI Algorithm: An artificial intelligence program that runs on a cloud server and analyzes received dental images to diagnose health conditions such as cavities, tartar, and gum inflammation.

[1023] 3. Emotion Engine: Another AI program running on a cloud server that analyzes the user's emotions from their facial expressions and tone of voice and generates appropriate feedback.

[1024] Operation explanation

[1025] Users take pictures of their teeth and gums using a smartphone or digital camera with a dedicated application installed. At least five images are taken from different angles, including the front teeth, back teeth, and gums. A shooting guide is displayed in the app, making it easy for users to take the appropriate images. These images are temporarily saved on the device.

[1026] The device then uploads these images to a cloud server, where they are encrypted and securely transmitted. The server then passes the images to an AI algorithm, which analyzes them and diagnoses dental health conditions such as cavities, tartar, and gum inflammation. The results are stored in a database for each user, allowing them to view them later.

[1027] Before providing a diagnosis, the server analyzes the user's emotions using an emotion engine. The emotion engine determines the user's current emotional state from the facial expressions and tone of voice provided by the user through the camera. Based on this, the server generates messages and feedback according to the user's emotional state.

[1028] Finally, the server sends the diagnosis results along with an evaluation of the user's brushing technique, along with recommendations for improvement and suggestions for improvement. For example, it may recommend using fluoride toothpaste or adopting a specific brushing technique to prevent cavities. Depending on the results of the emotion engine, a comforting message such as "Don't worry. We'll work together to improve over time" may also be sent.

[1029] If a specialist consultation is required, the server obtains the user's location information and searches for and lists nearby dental clinics, allowing the user to quickly find the appropriate clinic and use the appointment link to receive a prompt diagnosis.

[1030] Examples of concrete examples and prompts

[1031] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses the presence of early-stage cavities in the front teeth. After analyzing the image, the server then uses its emotion engine to analyze the user's emotional state and determines that the user is feeling anxious. The server then provides specific brushing advice (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specifically designed to prevent cavities), while also sending a comforting message: "Don't worry. We'll improve this together over time." Furthermore, the diagnosis indicates the need for a specialist consultation, so the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[1032] Example prompt (input to generative AI model):

[1033] Analyze the tooth images uploaded by the user and generate a diagnosis. Based on the analysis, provide specific brushing improvement suggestions and recommend suitable toothbrushes. Also, analyze the user's emotional state and generate messages to alleviate their concerns.

[1034] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1035] Step 1:

[1036] The user launches the dedicated application and takes pictures of their teeth using a smartphone or digital camera. The user follows the in-app guide to take at least five images from different angles, including the front teeth, back teeth, and gums. The guide displays instructions for taking appropriate images. The captured images are temporarily stored on the device.

[1037] Specific behavior:

[1038] 1. The user launches the application and taps the "Take Image" button.

[1039] 2. The device will start the camera and display a shooting guide.

[1040] 3. The user follows the guide to take images of each part of the body and taps "Save" to save them to the device.

[1041] Input: User actions and camera guide

[1042] Output: Saved tooth image data

[1043] Step 2:

[1044] The device uploads the saved images to the cloud server. The images are encrypted and sent securely. After the upload is complete, the device notifies the user.

[1045] Specific behavior:

[1046] 1. Select the image saved on your device and tap the "Upload" button.

[1047] 2. The device encrypts the image and sends it to a server over the Internet.

[1048] 3. The server receives the image, verifies the status, and notifies the device of a successful upload.

[1049] Input: Stored tooth image data, user command

[1050] Output: Image data uploaded to the cloud server

[1051] Step 3:

[1052] The server then passes the received image to an AI algorithm to begin analysis. The AI ​​algorithm analyzes the image and diagnoses health conditions such as cavities, tartar, and gum inflammation. The analysis results are then stored in a database.

[1053] Specific behavior:

[1054] 1. The server inputs the received image data into the AI ​​algorithm.

[1055] 2. AI algorithms analyze images and diagnose cavities, tartar, and gum inflammation.

[1056] 3. Diagnostic results are stored in a database for each user.

[1057] Input: Uploaded image data

[1058] Output: Save diagnostic results to database

[1059] Step 4:

[1060] The server uses an emotion engine to analyze the user's emotional state before providing a diagnosis. It determines emotions from the user's camera image and tone of voice and stores the results.

[1061] Specific behavior:

[1062] 1. The server passes the user's camera image and voice data to the emotion engine.

[1063] 2. The emotion engine analyzes this data and identifies the user's emotional state.

[1064] 3. The results of the sentiment analysis are generated and stored in a database for each user.

[1065] Input: Camera image and audio data

[1066] Output: Emotion analysis results

[1067] Step 5:

[1068] The server generates feedback based on the diagnosis and emotion analysis results, including recommendations for brushing method improvements and messages based on the user's emotional state. This feedback is then sent to the device.

[1069] Specific behavior:

[1070] 1. The server generates a feedback message based on the diagnosis and emotion analysis results.

[1071] 2. Simultaneously generate improvements and specific recommendations for brushing methods.

[1072] 3. Send feedback to the device and notify the user.

[1073] Input: Diagnosis results, emotion analysis results

[1074] Output: Feedback messages and brushing advice

[1075] Step 6:

[1076] If the diagnosis indicates that a specialist consultation is necessary, the server acquires the user's location information, searches for and lists nearby dental clinics, and sends the information to the user.

[1077] Specific behavior:

[1078] 1. The server obtains the user's location information and searches for nearby dental clinics.

[1079] 2. Create a list of suitable dental clinics and create a link to book appointments.

[1080] 3. This information is sent to the terminal and notified to the user.

[1081] Input: Diagnostic results, user location information

[1082] Output: A list of dental clinics and a link to book an appointment

[1083] (Application example 2)

[1084] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1085] In today's busy lifestyles, there is a need for a system that allows users to easily check their dental health at home and receive appropriate feedback and a prompt diagnosis. However, existing systems lack the functionality to provide feedback based on emotional state or recommend experts, making it difficult to provide effective advice while alleviating users' concerns.

[1086] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for acquiring images of teeth taken by the user; image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; means for referring the user to a nearby medical institution if the diagnosis results indicate that a professional diagnosis is necessary; and means for recognizing the user's emotional state and providing feedback according to the user's emotion. This allows the user to easily check the health condition of their teeth at home and receive appropriate advice and support according to their emotion.

[1087] "User" refers to the person who uses the system, and in this case, refers to an individual who wants to check and manage their dental health status.

[1088] A "tooth image" is a photograph of the teeth and gums inside the oral cavity taken by the user.

[1089] The "means for acquiring" is a function for importing images of teeth taken by the user into the system.

[1090] "Image processing means" is a function for analyzing acquired tooth images using an AI algorithm.

[1091] "AI algorithm" refers to an analytical method that uses artificial intelligence, and refers to the function of analyzing images of teeth to diagnose their health condition.

[1092] "Diagnosis" refers to the evaluation of dental health based on the analysis results.

[1093] "Health status" refers to the state of the teeth and gums, and whether or not there are any symptoms such as cavities or inflammation.

[1094] "Brushing technique" refers to the method of brushing your teeth effectively, such as how you brush your teeth and the type of toothbrush you use.

[1095] "Evaluation" refers to determining whether the current brushing method is appropriate based on the diagnostic results.

[1096] "Recommendations" are specific advice or suggestions for improvement provided to the user based on the diagnostic results.

[1097] A "professional" is someone who has specialized knowledge and qualifications related to teeth and oral health, such as a dentist or dental hygienist.

[1098] "Diagnosis required" means that the system's analysis results indicate that the user's dental health condition requires a specialist examination.

[1099] "Nearby medical institution" refers to a medical facility such as a dental clinic that is located within easy access from the user's current location.

[1100] "Means for referral" is a function that guides the user to an appropriate medical institution based on the diagnosis results.

[1101] "Emotional state" refers to the user's psychological state or mood, which in this case is determined from facial expressions and tone of voice.

[1102] "Feedback" is a general term for comments and advice provided to users based on diagnostic and analytical results.

[1103] This invention is a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. This system supports users' dental health management by analyzing dental images taken by the user and providing appropriate advice and emotional feedback based on the diagnosis results.

[1104] The system includes the following means:

[1105] 1. Image acquisition method:

[1106] The user takes images of their teeth using a smartphone or digital camera. The images are taken from at least five angles: front, left side, right side, top, and bottom. This allows detailed image data to be obtained. The captured image data is saved on the user's device.

[1107] 2. Image upload method:

[1108] The user's device uploads the captured images to a cloud server via the Internet, where the image data is received securely and quickly.

[1109] 3. Imaging and diagnostic tools:

[1110] The cloud server analyzes the received images using an AI algorithm, which diagnoses the health of the teeth (e.g., cavities, tartar, gum inflammation, etc.) and stores the results on the server.

[1111] 4. Emotion recognition means:

[1112] Before providing a diagnosis, the server analyzes the user's emotional state using an emotion engine, which identifies emotions from the user's facial expressions and tone of voice to determine the user's current state of mind.

[1113] 5. Feedback can be provided by:

[1114] The server provides optimal feedback to the user based on the diagnosis and emotion recognition results. The feedback includes an evaluation of brushing methods, areas for improvement, and recommendations. It also recommends the type and timing of toothbrush replacement based on the diagnosis results. It also provides encouraging and comforting messages based on the emotion engine results.

[1115] 6. Specialist referral methods:

[1116] If the diagnosis results indicate that a specialist diagnosis is necessary, the server obtains the user's location information and searches for and refers the user to a nearby medical institution. This allows the user to receive a prompt, professional diagnosis even if a serious health problem is detected.

[1117] Hardware and software used

[1118] Hardware:

[1119] Smartphone or digital camera (for taking pictures)

[1120] Internet-connected device (image upload)

[1121] software:

[1122] OpenCV (image processing)

[1123] EmotionRecognizer

[1124] AI Diagnostic (Diagnosis by AI algorithm)

[1125] requests (REST API calls)

[1126] Specific examples

[1127] For example, a user can use the app to take pictures of their teeth from the front, left side, right side, top, and bottom, and upload them to a cloud server. The server then uses AI to analyze the received images and diagnoses the presence of early-stage cavities in the front teeth. The emotion engine then analyzes the user's emotional state and determines that the user is feeling anxious. The app then provides specific brushing advice and recommended toothbrushes, along with a comforting message: "Don't worry. We'll take the time to improve together." Furthermore, if the diagnosis indicates that a professional diagnosis is necessary, the app searches for nearby dental clinics based on the user's location and provides information on appropriate dental clinics.

[1128] Prompt Sentence Examples

[1129] A user takes photos of their teeth with their smartphone from the following angles: front, left side, right side, top, and bottom. Upload these images to a cloud server and use an AI algorithm to analyze the health of their teeth. Additionally, recognize emotions based on the user's facial expressions and voice and provide feedback based on the following criteria:

[1130] If a user is feeling anxious, provide comforting messages such as, "Don't worry. We'll improve this together over time."

[1131] If the diagnosis is positive, a positive message such as "You're polishing very well! Keep it up!" is provided.

[1132] If the diagnosis is negative, provide comforting messages such as, "You need to improve a little. It's okay, we're here to help!"

[1133] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1134] Step 1:

[1135] The user takes images of the teeth using a smartphone or digital camera. The images are taken from at least five angles: front, left side, right side, top, and bottom. The user follows the guide within the app to obtain the appropriate images. This is the input, and the images are saved in the smartphone's storage as the output. Specifically, the user launches the camera app and takes photos from each angle while following the guide display.

[1136] Step 2:

[1137] The user's device uploads the captured tooth images to the cloud server via the Internet. The input is the image stored on the device, and the output is the image stored on the cloud server. Specifically, the device sends the image file to the cloud server using a REST API.

[1138] Step 3:

[1139] The server receives images uploaded to the cloud server and analyzes each image using an AI algorithm. The input is the uploaded image data, and the output is a diagnosis of the dental health condition (e.g., cavities, tartar, gum inflammation, etc.). Specifically, the server preprocesses the images using an image processing library (e.g., OpenCV) and runs a deep learning model to evaluate the dental condition.

[1140] Step 4:

[1141] The server extracts emotions from the user's facial expressions and tone of voice to recognize the user's emotional state based on the diagnosis results. The input is the diagnosis results and the user's facial expressions and voice data, and the output is a judgment of the user's emotional state (e.g., anxiety, relief, etc.). Specifically, the server uses an emotion recognition engine (e.g., EmotionRecognizer) to analyze the facial images and voice data.

[1142] Step 5:

[1143] The server generates optimal feedback for the user based on the diagnosis results and emotion recognition results. The input is the diagnosis results and emotion determination results, and the output is advice or a message to be provided to the user. Specifically, the server generates a positive message if the diagnosis results are good, and a comforting message if the user is very anxious, and sends it to the user.

[1144] Step 6:

[1145] If the diagnosis result indicates that a specialist diagnosis is necessary, the server obtains the user's location information. The input is the diagnosis result and the user's location information, and the output is a list of nearby medical institutions. Specifically, the server obtains the user's location using a location information API and searches for the nearest medical institution based on that location.

[1146] Step 7:

[1147] The server introduces nearby medical institutions to the user. The input is the location information of the user who is deemed to need a specialist diagnosis and a list of medical institutions, and the output is a notification message including the referral details. Specifically, the server sends information about appropriate medical institutions to the user's device and displays it as a notification.

[1148] These steps allow users to easily check their dental health status at home and receive appropriate advice and support according to their emotions.

[1149] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1150] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1151] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1152] [Fourth embodiment]

[1153] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1154] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1155] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1156] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1157] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1158] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1159] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1160] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1161] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1162] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1163] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1164] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1165] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1166] The present invention is a system that allows users to easily check their dental health at home and receive appropriate advice. This system acquires images of the user's teeth and is equipped with an AI algorithm that analyzes the acquired images to diagnose the dental health. Below, we will explain in detail the embodiments of the present invention.

[1167] Image acquisition

[1168] The user takes photos of their teeth and gums in their oral cavity using a smartphone or digital camera. At least five images are taken from different angles, particularly of the front teeth, back teeth, and gums. The app guides the user through the photo-taking process, making it easy for them to obtain the appropriate images. The captured images are then saved on the device.

[1169] Uploading an image

[1170] The device uploads the captured images to a cloud server via the Internet, which securely and quickly transmits the images to the server, which receives them and uses them for analysis.

[1171] Image Processing and Diagnosis

[1172] The server analyzes the received images using an AI algorithm. The AI ​​algorithm processes the tooth images and diagnoses the health of the teeth, including cavities, tartar, and gum inflammation. The analysis results are stored on the server and managed for each user, allowing users to check the health of their own teeth.

[1173] Providing diagnostic results

[1174] Based on the diagnostic results, the server evaluates the user's brushing method and generates recommendations for improvements. It also recommends the appropriate toothbrush replacement timing and type based on the diagnostic results. This information is sent to the device and notified to the user. The user can receive advice through the app and learn more effective brushing methods.

[1175] Specialist referrals

[1176] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information. Based on the obtained location information, the server searches for nearby dental clinics and lists appropriate dental clinics. This allows the user to quickly receive a specialist diagnosis even if a serious health problem is detected.

[1177] Specific examples

[1178] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses that there is early-stage decay in the front teeth. As a result, specific advice on brushing techniques (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specialized for preventing cavities) are provided. Furthermore, if the diagnosis determines that a specialist diagnosis is necessary, the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[1179] This allows users to easily check the health of their teeth at home and receive necessary advice and information on how to seek a diagnosis from a specialist. Using this system will greatly simplify dental health management for users and enable early detection and treatment.

[1180] The processing flow will be explained below.

[1181] Step 1:

[1182] The user launches the app and takes photos of their teeth and gums in their mouth. Following the in-app guide, they take at least five images from different angles, including the front teeth, back teeth, and gums. The images are then saved on the device.

[1183] Step 2:

[1184] The device uploads the captured images to a cloud server, where they are sent securely and quickly via the Internet.

[1185] Step 3:

[1186] The server receives the uploaded images and passes them to the image processing means, where they are processed by AI algorithms for analysis.

[1187] Step 4:

[1188] The server uses an AI algorithm to analyze the images and diagnose the health of the teeth. The analysis results include information on cavities, tartar, gum inflammation, etc. The diagnosis results are stored on the server and managed for each user.

[1189] Step 5:

[1190] The server evaluates the user's brushing method based on the diagnostic results, assesses the effectiveness of the current brushing method, and identifies areas for improvement.

[1191] Step 6:

[1192] The server generates information based on the diagnosis results that recommends the appropriate time and type of toothbrush to replace, and provides the user with information on effective brushing methods and recommended toothbrushes.

[1193] Step 7:

[1194] The server sends the generated advice information to the device, and the user receives an evaluation of their brushing method, areas for improvement, when to replace their toothbrush, and notifications of recommended toothbrushes through the app.

[1195] Step 8:

[1196] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information and uses the obtained location information to search for dental clinics near the user's location.

[1197] Step 9:

[1198] The server creates a list of suitable dental clinics from the search results and sends it to the device. The user receives information about nearby dental clinics via the app and can take next steps, such as making an appointment.

[1199] Through these steps, the system allows users to easily understand the state of their dental health and receive appropriate guidance. If necessary, it also provides information to promptly seek a diagnosis from a specialist.

[1200] Example 1

[1201] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1202] Conventional dental diagnostic systems require users to visit a dentist regularly, making it difficult to easily check the health of their teeth at home and receive appropriate advice. While there are applications for checking dental health, they lack the ability to respond quickly when a professional diagnosis is required. Furthermore, the lack of specific brushing methods or recommendations for appropriate toothbrushes based on the diagnosis results makes it difficult for users to efficiently perform oral care.

[1203] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1204] In this invention, the server includes means for acquiring tooth images taken by a user, means for saving the captured tooth images in a digital device, means for uploading the saved images to a cloud computing environment via the Internet, image processing means including a generative AI algorithm for analyzing the acquired images and diagnosing the dental health condition, means for providing an evaluation of the user's brushing method and suggestions for improvement based on the diagnosis results, means for recommending the appropriate type of toothbrush and replacement timing to the user based on the diagnosis results, and means for acquiring the user's location information and referring the user to a nearby specialist facility if the diagnosis results determine that a specialist diagnosis is necessary. This allows users to easily check their oral health at home and quickly obtain necessary advice and specialist information.

[1205] "User" refers to a person who uses the system to check their dental health and receive advice and diagnostic results.

[1206] "Digital devices" refers to electronic devices used to capture and record images, such as smartphones and digital cameras.

[1207] "Cloud computing environment" refers to remote server storage and processing power provided over the Internet.

[1208] "Generative AI algorithm" refers to a program that uses AI technology to analyze data and diagnose dental health.

[1209] "Image processing means" refers to the technical process for analyzing the acquired images and generating a diagnostic result.

[1210] "Means for evaluating brushing methods and providing points for improvement" refers to a system function that evaluates the user's tooth brushing method based on the diagnostic results and specifically suggests points for improvement.

[1211] "Means for recommending toothbrush types and replacement timing" refers to a system function that indicates to the user the appropriate toothbrush selection and replacement timing based on the diagnostic results.

[1212] "When it is determined that a specialist diagnosis is necessary" refers to a situation where the analysis results of the AI ​​algorithm indicate a serious oral problem and a specialist diagnosis is recommended.

[1213] "Means for obtaining location information and introducing nearby specialist facilities" refers to a system function that identifies the user's current geographic location and searches for and recommends specialist facilities such as dental clinics in the vicinity.

[1214] This invention is a system that allows users to easily check their dental health at home and receive appropriate advice. This system acquires images of the user's teeth and includes a generative AI algorithm that analyzes the acquired images to diagnose the dental health. Below, we will explain in detail the embodiments of the invention.

[1215] Image acquisition

[1216] The user uses a smartphone or digital camera (hereinafter referred to as "digital device") to take photos of their teeth and gums in their mouth. They launch the application and follow the in-app guide to take photos from the appropriate angle and position. At least five images should be taken, including front teeth, back teeth, gums, etc. The captured images are saved in the device's internal storage.

[1217] Uploading an image

[1218] The device uploads the captured images to a cloud computing environment (hereafter referred to as the cloud server) via the Internet. When uploading, the device checks for Internet connectivity and compresses and encrypts the image files. The device uses the upload API to send the images to the cloud server, and the server confirms receipt.

[1219] Image Processing and Diagnosis

[1220] The server analyzes the received images using a generative AI algorithm, which specifically includes the following processes:

[1221] 1. Decode the received image file and restore the original image.

[1222] 2. Pre-process the image to adjust the sharpness and contrast.

[1223] 3. An AI algorithm analyzes the images and diagnoses dental health conditions such as cavities, tartar, and gum inflammation. The results are stored in a database for each user.

[1224] Providing diagnostic results

[1225] The server generates a diagnosis based on the image analysis, which includes the following information:

[1226] 1. Detailed information such as whether or not you have cavities, their progression, whether or not you have tartar, and whether or not you have gum inflammation.

[1227] 2. Evaluate your brushing technique and suggest areas for improvement, specifically, advice on which areas to brush more carefully.

[1228] 3. Recommended types of toothbrushes and when to replace them.

[1229] This information is sent from the server to the terminal and notified to the user.

[1230] Specialist referrals

[1231] If the server determines that a diagnosis by a specialist is necessary based on the diagnostic results, it performs the following process:

[1232] 1. Use the device's location sensor to obtain the user's location information.

[1233] 2. Search for nearby dental clinics based on location information and list suitable dental clinics.

[1234] 3. The created list is sent to the terminal and notified to the user.

[1235] Specific examples

[1236] For example, if a user uses an app to take a picture of their teeth and upload it, the server would do the following:

[1237] 1. The user follows the in-app guide to take five intraoral images from different angles. The device stores these images in its internal storage.

[1238] 2. When you press a button in the app, the device will encrypt the captured image and upload it to the cloud server. A progress bar will be displayed until the transfer is complete.

[1239] 3. The server decodes the received image and converts it into a format for analysis. A generative AI algorithm scans the image and detects cavities, tartar, gum inflammation, etc. For example, an early stage of cavities was detected in a front tooth.

[1240] 4. Based on the analysis results, the server generates information on how to improve the user's brushing method and recommends toothbrushes. For example, it suggests "brushing the insides of your front teeth more carefully and using a toothbrush specifically designed to prevent cavities."

[1241] 5. The server obtains the user's location information and searches for nearby dental clinics. For example, it creates a list of "XX Hospital, △△ Dental Clinic" and sends it to the terminal.

[1242] Prompt Sentence Examples

[1243] Here is an example prompt:

[1244] Example prompt 1:

[1245] "Please analyze the images of my teeth and tell me if I have cavities. I can also learn the correct way to brush my teeth and recommend a toothbrush."

[1246] Example prompt 2:

[1247] "Based on my dental diagnosis, if I need to see a specialist, please tell me the nearest dental clinic based on my location."

[1248] In this way, users can easily check their oral health at home and quickly obtain necessary advice and information from specialists.

[1249] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1250] Step 1:

[1251] A user takes a photograph of the teeth and gums in their mouth using a digital device (smartphone or digital camera).

[1252] Input: A video of the user's mouth.

[1253] How it works: Users launch the app and follow the in-app guide to take at least five images from different angles, which provides detailed images of the front teeth, back teeth, and gums.

[1254] Output: Multiple image files of the teeth (JPEG, PNG, etc.).

[1255] Step 2:

[1256] The device saves the captured image file to its internal storage.

[1257] Input: The image file taken in step 1.

[1258] What it does: The device automatically saves the image in a specific folder for further processing, and adds a timestamp to the file name for easy identification.

[1259] Output: Image file saved on the device.

[1260] Step 3:

[1261] The device uploads the image file to the cloud server.

[1262] Input: The image file saved in step 2.

[1263] What it does: The device checks for an internet connection, compresses and encrypts the image file, then sends it to the cloud server via the upload API, where it confirms receipt. A progress bar is displayed in the app.

[1264] Output: Encrypted image files stored in the cloud server.

[1265] Step 4:

[1266] The server decodes the received image file and performs image preprocessing.

[1267] Input: The encrypted image file uploaded to the cloud server in step 3.

[1268] What it does: The server decodes and restores the image file, then performs preprocessing such as adjusting sharpness and contrast to improve the image quality and make it easier to analyze.

[1269] Output: Preprocessed tooth image.

[1270] Step 5:

[1271] The server uses a generative AI algorithm to analyze the images and diagnose the health of the teeth.

[1272] Input: Image files preprocessed in step 4.

[1273] How it works: Generative AI algorithms scan images to detect cavities, tartar, gum inflammation, etc. The algorithms are trained on large medical datasets and therefore produce highly accurate diagnoses.

[1274] Output: Dental health diagnosis (e.g., presence of cavities, tartar, and gum inflammation).

[1275] Step 6:

[1276] Based on the diagnostic results, the server evaluates the user's brushing method, recommends areas for improvement, and recommends the appropriate type of toothbrush and when to replace it.

[1277] Input: The diagnostic results obtained in step 5.

[1278] Specific operation: The server evaluates the user's brushing method based on the analysis results and suggests areas for improvement, such as brushing specific areas more carefully. It also generates information on toothbrushes and replacement times based on the user's situation.

[1279] Output: Brushing advice and toothbrush recommendations for the user.

[1280] Step 7:

[1281] The server acquires the user's location information, and if the diagnosis indicates that a specialist's diagnosis is necessary, the server will refer the user to a nearby specialist facility.

[1282] Input: The diagnostic results generated in step 6 and the user's location information.

[1283] Specific operation: The server uses the user's location sensor to obtain their current geographic location, searches for nearby dental clinics based on the location information, and creates a list of suitable dental clinics.

[1284] Output: A list of nearby dental clinics provided to the user.

[1285] In this way, the present invention provides a system that allows users to easily check their dental health at home and quickly obtain necessary advice and information from specialists.

[1286] (Application example 1)

[1287] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1288] Conventional methods for checking dental health required regular visits to a dental clinic to receive a diagnosis from a specialist, which was time-consuming and laborious. Furthermore, selecting the appropriate dental care products required specialized knowledge, making it difficult for average users. This led to neglecting dental health management and often overlooking preventable problems. Furthermore, there was a need for a system that could instantly recommend and purchase appropriate dental care products based on diagnosis results through an online shopping function.

[1289] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1290] In this invention, the server includes: a means for acquiring images of teeth taken by a user; an image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; a means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; a means for referring the user to a nearby dental clinic if the diagnosis results indicate that a specialist diagnosis is necessary; and a means for recommending appropriate dental care products through an online shopping function based on the diagnosis results and providing a purchase link. This allows users to easily check the health condition of their teeth from home and obtain appropriate dental care products based on the diagnosis results. Furthermore, if a specialist diagnosis is necessary, appropriate information can be obtained quickly, making it easier to access a dental clinic.

[1291] "User" refers to an individual who uses the system to take images of their teeth and check their health.

[1292] The "image acquisition means" refers to a means including a device such as a smartphone or a digital camera for acquiring images of teeth taken by a user.

[1293] "Image processing means" refers to means including an AI algorithm that analyzes the acquired images and diagnoses the health of the teeth, such as detecting cavities, tartar, and gum inflammation.

[1294] The "diagnosis result providing means" is a means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results, thereby enabling the user to learn appropriate tooth brushing methods.

[1295] The "specialist referral means" is a means for referring the user to a nearby dental clinic when the diagnosis result indicates that a specialist diagnosis is necessary. This means is performed based on the user's location information.

[1296] "Online Shopping Feature" refers to a means of recommending appropriate dental care products based on the diagnosis results and providing a purchase link, allowing the user to immediately purchase the recommended products.

[1297] "Dental care products" refers to toothbrushes, toothpastes, mouthwashes, etc. for preventing cavities and removing tartar that are recommended based on the diagnosis results.

[1298] "Diagnosis results" refer to the evaluation results of dental health analyzed by the AI ​​algorithm, which determine the presence or absence of cavities, tartar, gum inflammation, etc.

[1299] Overall system configuration

[1300] This system allows users to easily check their dental health at home and instantly purchase appropriate dental care products. The system mainly consists of the following elements: image acquisition means, image processing means, diagnostic result provision means, specialist referral means, and online shopping function.

[1301] Hardware and software configuration

[1302] User device: A device with a camera and internet connection, such as a smartphone or tablet. The user device takes pictures and uploads them to the server.

[1303] Server: Cloud server. A central server that integrates image processing, diagnostic result provision, specialist referral, and online shopping functions.

[1304] Image processing software: Python libraries such as OpenCV and Keras, which allow captured images to be pre-processed and the AI ​​algorithms to make the diagnosis.

[1305] Communication software: Data is sent and received between the user device and the cloud server using HTTP libraries such as Requests.

[1306] Data Flow and Processing

[1307] 1. Image Acquisition:

[1308] The user takes multiple images of the oral cavity using a user device such as a smartphone, taking pictures of different areas of the mouth, particularly the front teeth, back teeth, and gums, and uses the in-app guide to ensure clear images are obtained.

[1309] 2. Upload your image:

[1310] The user device uploads the captured images to a cloud server via the Internet, and the image data is securely transmitted using communication software.

[1311] 3. Image Analysis and Diagnosis:

[1312] The cloud server analyzes the received images using AI algorithms, preprocessing the image data using image processing software, and then using an AI model (built with Keras) to diagnose dental health conditions such as cavities, tartar, and gum inflammation.

[1313] 4. Providing diagnostic results and recommendations:

[1314] The server provides the user with an evaluation and recommendations on brushing methods based on the diagnostic results. It also recommends appropriate dental care products (e.g., anti-cavity toothbrushes) based on the diagnostic results and provides a link to purchase them through an online shopping function.

[1315] 5. Specialist referrals:

[1316] If the diagnosis indicates that a specialist consultation is required, the server obtains the user's location and searches for a nearby dental clinic to refer the patient to, using an appropriate geographical information API.

[1317] Specific examples

[1318] Example 1: A user uses an app to take and upload a picture of their front teeth. This image is sent to a cloud server and analyzed by an AI algorithm. The diagnosis is that the patient has "early stages of cavities," and the app recommends the user to use an "anti-cavity toothbrush" and "anti-cavity gourmet paste." If the app determines that a dentist's diagnosis is necessary, the app will refer the user to a nearby dental clinic based on the user's location.

[1319] Prompt Sentence Examples

[1320] "I used the app to take a photo of my front teeth and uploaded it. The diagnosis was 'early stage of tooth decay.' What toothbrush should I buy?"

[1321] This system allows users to easily check the health of their teeth from the comfort of their own home, instantly purchase the necessary dental care products, and quickly receive a diagnosis from a specialist.

[1322] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1323] Step 1:

[1324] Taking an image

[1325] The user uses the smartphone camera to take multiple images of the teeth and gums in the oral cavity. These images include different areas such as the front teeth, back teeth, and gums. The input here is raw image data captured through the smartphone camera, and the output is a JPEG image file saved on the user's device. Specifically, the user launches the camera app and follows the in-app guide to adjust the shooting location while taking the image.

[1326] Step 2:

[1327] Uploading an image

[1328] Images taken on the user's device are selected and uploaded to the cloud server through the application. In this process, the user selects an image and taps the "Upload" button. The input is the image file stored on the user's device, and the output is the image file received by the cloud server. Specifically, the application generates an HTTP request to send the selected image file to the server. This is done using communication software (e.g., the Requests library).

[1329] Step 3:

[1330] Image preprocessing

[1331] The server preprocesses the received images, using the OpenCV library to resize and convert colors (e.g., to RGB). The input is a JPEG image file received by the cloud server, and the output is image data converted into a format suitable for AI algorithms. Specifically, the image is resized to 256x256 pixels, color converted, and then the data is normalized.

[1332] Step 4:

[1333] Image Analysis and Diagnosis

[1334] The server inputs the preprocessed images into an AI algorithm for analysis. Here, an AI model built with Keras is used. The input is the preprocessed image data, and the output is the diagnosis result (e.g., whether or not there is tooth decay, tartar, or gum inflammation). Specifically, the AI ​​model analyzes the image data and calculates the confidence level for each class (e.g., tooth decay, tartar, gum inflammation, etc.).

[1335] Step 5:

[1336] Providing diagnostic results and recommendations

[1337] The server evaluates and recommends brushing methods to the user based on the diagnostic results. It also recommends appropriate dental care products (e.g., anti-cavity toothbrushes) based on the diagnostic results and provides a purchase link through an online shopping function. The input is the diagnostic results obtained from the AI ​​model, and the output is recommendations and purchase links sent to the user. Specifically, it analyzes the diagnostic results, generates appropriate recommendations and product links, and outputs them to the user's application.

[1338] Step 6:

[1339] Specialist referrals

[1340] If the diagnosis result indicates that a specialist diagnosis is required, the server obtains the user's location information, searches for nearby dental clinics, and refers the user to them. The input is the diagnosis result and the user's location information, and the output is information about nearby dental clinics that is notified to the user. Specifically, the server uses a geographic information API to search for the most suitable dental clinic based on the location information, and provides that information to the user.

[1341] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1342] The present invention is a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. The system is equipped with an AI algorithm that acquires images of the user's teeth, analyzes the acquired images, and diagnoses the dental health condition, as well as an emotion engine that recognizes the user's emotions.

[1343] Image acquisition

[1344] The user takes photos of their teeth and gums in their oral cavity using a smartphone or digital camera. At least five images are taken from different angles, particularly of the front teeth, back teeth, and gums. The app guides the user through the photo-taking process, making it easy for them to obtain the appropriate images. The captured images are then saved on the device.

[1345] Uploading an image

[1346] The device uploads the captured images to a cloud server via the Internet, which securely and quickly transmits the images to the server, which receives them and uses them for analysis.

[1347] Image Processing and Diagnosis

[1348] The server analyzes the received images using an AI algorithm. The AI ​​algorithm processes the tooth images and diagnoses the health of the teeth, including cavities, tartar, and gum inflammation. The analysis results are stored on the server and managed for each user, allowing users to check the health of their own teeth.

[1349] emotion recognition

[1350] Before providing a diagnosis result, the server analyzes the user's emotional state using an emotion engine. The emotion engine reads emotions from the user's facial expressions and tone of voice to determine their current state of mind. This allows the server to provide feedback according to the user's emotional state.

[1351] Providing diagnostic results

[1352] Based on the diagnosis results, the server evaluates the user's brushing method and generates recommendations for improvements. Furthermore, depending on the diagnosis results, it recommends the appropriate toothbrush replacement timing and type. Taking into account the analysis results of the emotion engine, it provides encouraging and comforting messages according to the user's feelings.

[1353] For example, if the diagnosis is good, send a positive message such as, "You're polishing really well! Keep it up!"; if the diagnosis is bad, send a comforting message such as, "You need to improve a little. Don't worry, we're here to support you!"

[1354] Specialist referrals

[1355] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information. Based on the obtained location information, the server searches for nearby dental clinics and lists appropriate dental clinics. This allows the user to quickly receive a specialist diagnosis even if a serious health problem is detected.

[1356] Specific examples

[1357] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses the presence of early-stage cavities in the front teeth. After analyzing the image, the server then uses its emotion engine to analyze the user's emotional state and determines that the user is feeling anxious. The server then provides specific brushing advice (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specifically designed to prevent cavities), while also sending a comforting message: "Don't worry. We'll improve this together over time." Furthermore, the diagnosis indicates the need for a specialist consultation, so the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[1358] This allows users to easily check the health status of their teeth at home and receive appropriate advice. Feedback is also provided by the emotion engine, providing information to promptly seek a diagnosis from a specialist if necessary. Using this system significantly simplifies dental health management for users, while also providing psychological support.

[1359] The processing flow will be explained below.

[1360] Step 1:

[1361] The user launches the app and takes photos of their teeth and gums in their mouth. Following the in-app guide, they take at least five images from different angles, including the front teeth, back teeth, and gums. The images are then saved on the device.

[1362] Step 2:

[1363] The device uploads the captured images to a cloud server, where they are sent securely and quickly via the Internet.

[1364] Step 3:

[1365] The server receives the uploaded images and passes them to the image processing means, where they are processed by AI algorithms for analysis.

[1366] Step 4:

[1367] The server uses an AI algorithm to analyze the images and diagnose the health of the teeth. The analysis results include information on cavities, tartar, gum inflammation, etc. The diagnosis results are stored on the server and managed for each user.

[1368] Step 5:

[1369] The server evaluates the user's brushing method based on the diagnostic results, assesses the effectiveness of the current brushing method, and identifies areas for improvement.

[1370] Step 6:

[1371] The server generates information based on the diagnosis results that recommends the appropriate time and type of toothbrush to replace, and provides the user with information on effective brushing methods and recommended toothbrushes.

[1372] Step 7:

[1373] The server uses an emotion engine to analyze the user's emotional state before providing a diagnosis, reading emotions from the user's facial expressions and tone of voice to determine their current state of mind.

[1374] Step 8:

[1375] The server adjusts the generated advice information according to the user's emotional state and sends it to the device. The user receives an evaluation of their brushing technique, areas for improvement, notification of when to replace their toothbrush, and notifications of recommended toothbrushes through the app. The app also provides encouraging and comforting messages that match the user's emotional state.

[1376] Step 9:

[1377] If the server determines that a specialist diagnosis is necessary based on the diagnosis results, it obtains the user's location information and uses the obtained location information to search for dental clinics near the user's location.

[1378] Step 10:

[1379] The server creates a list of suitable dental clinics from the search results and sends it to the device. The user receives information about nearby dental clinics via the app and can take next steps, such as making an appointment.

[1380] Through this series of steps, users can easily check their dental health at home, receive appropriate guidance and psychological support, and be provided with information to quickly seek a specialist diagnosis if necessary.

[1381] Example 2

[1382] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1383] Conventional dental examination systems make it difficult for users to easily check their dental health at home and receive appropriate advice based on the results. They also fail to provide feedback that takes into account the user's emotional state, and lack sufficient psychological support. Furthermore, it is difficult for users to quickly find an appropriate dental clinic when a specialist diagnosis is required.

[1384] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1385] In this invention, the server includes: means for acquiring images of teeth taken by a user; image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; means including an emotion engine for analyzing the user's emotional state and providing feedback according to the emotion; and means for referring the user to a nearby dental clinic if the diagnosis results indicate that a specialist diagnosis is necessary. This enables a user to easily check the dental health condition at home and receive a referral to a specialist along with emotionally sensitive feedback.

[1386] "Means for acquiring images of teeth taken by a user" refers to a mechanism that allows the system to receive image data of teeth taken by a user using a smartphone or digital camera and make it available for storage or processing.

[1387] "Image processing means including an AI algorithm that analyzes acquired images and diagnoses the health condition of teeth" refers to a combination of software and hardware that uses artificial intelligence technology to analyze received dental image data and determine health conditions such as cavities, tartar, and gum inflammation.

[1388] "Means of providing users with an evaluation and recommendations on brushing methods based on the diagnostic results" refers to a function that provides specific guidance to users on appropriate brushing methods, areas for improvement, the type of toothbrush they should use, etc., based on the results of a dental health diagnosis by an AI algorithm.

[1389] "Means including an emotion engine that analyzes the user's emotional state and provides feedback according to that emotion" refers to a combination of software and hardware that reads the user's current emotion by analyzing the user's facial expression and tone of voice, and generates feedback or messages appropriate to that emotion.

[1390] "Means for referring the user to a nearby dental clinic when the diagnostic results indicate that a specialist diagnosis is necessary" refers to a mechanism by which the system refers to the user's diagnostic results, and when it determines that a specialist diagnosis is necessary, searches for the nearest appropriate dental clinic using the user's geographical location information and provides a list or reservation information.

[1391] The present invention provides a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. This system is implemented using the following hardware and software.

[1392] Hardware Configuration

[1393] 1. Smartphone or digital camera: Used by the user to take images of teeth. Equipped with a camera function and internet connection.

[1394] 2. Cloud Server: Analyzes the received images, generates and stores diagnostic results. This server is equipped with advanced computing resources and storage.

[1395] 3. Device: A smartphone or tablet used by a user to take pictures, upload them, and receive feedback.

[1396] Software Configuration

[1397] 1. Dedicated application: An application that allows users to take images of their own teeth and upload them to a cloud server.

[1398] 2. AI Algorithm: An artificial intelligence program that runs on a cloud server and analyzes received dental images to diagnose health conditions such as cavities, tartar, and gum inflammation.

[1399] 3. Emotion Engine: Another AI program running on a cloud server that analyzes the user's emotions from their facial expressions and tone of voice and generates appropriate feedback.

[1400] Operation explanation

[1401] Users take pictures of their teeth and gums using a smartphone or digital camera with a dedicated application installed. At least five images are taken from different angles, including the front teeth, back teeth, and gums. A shooting guide is displayed in the app, making it easy for users to take the appropriate images. These images are temporarily saved on the device.

[1402] The device then uploads these images to a cloud server, where they are encrypted and securely transmitted. The server then passes the images to an AI algorithm, which analyzes them and diagnoses dental health conditions such as cavities, tartar, and gum inflammation. The results are stored in a database for each user, allowing them to view them later.

[1403] Before providing a diagnosis, the server analyzes the user's emotions using an emotion engine. The emotion engine determines the user's current emotional state from the facial expressions and tone of voice provided by the user through the camera. Based on this, the server generates messages and feedback according to the user's emotional state.

[1404] Finally, the server sends the diagnosis results along with an evaluation of the user's brushing technique, along with recommendations for improvement and suggestions for improvement. For example, it may recommend using fluoride toothpaste or adopting a specific brushing technique to prevent cavities. Depending on the results of the emotion engine, a comforting message such as "Don't worry. We'll work together to improve over time" may also be sent.

[1405] If a specialist consultation is required, the server obtains the user's location information and searches for and lists nearby dental clinics, allowing the user to quickly find the appropriate clinic and use the appointment link to receive a prompt diagnosis.

[1406] Examples of concrete examples and prompts

[1407] For example, suppose a user uses the app to take a picture of their teeth and upload it. The server uses AI to analyze the received image and diagnoses the presence of early-stage cavities in the front teeth. After analyzing the image, the server then uses its emotion engine to analyze the user's emotional state and determines that the user is feeling anxious. The server then provides specific brushing advice (e.g., which areas should be brushed more carefully) and a recommended toothbrush (e.g., a toothbrush specifically designed to prevent cavities), while also sending a comforting message: "Don't worry. We'll improve this together over time." Furthermore, the diagnosis indicates the need for a specialist consultation, so the server searches for nearby dental clinics based on the user's location and sends information about appropriate dental clinics to the device.

[1408] Example prompt (input to generative AI model):

[1409] Analyze the tooth images uploaded by the user and generate a diagnosis. Based on the analysis, provide specific brushing improvement suggestions and recommend suitable toothbrushes. Also, analyze the user's emotional state and generate messages to alleviate their concerns.

[1410] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1411] Step 1:

[1412] The user launches the dedicated application and takes pictures of their teeth using a smartphone or digital camera. The user follows the in-app guide to take at least five images from different angles, including the front teeth, back teeth, and gums. The guide displays instructions for taking appropriate images. The captured images are temporarily stored on the device.

[1413] Specific behavior:

[1414] 1. The user launches the application and taps the "Take Image" button.

[1415] 2. The device will start the camera and display a shooting guide.

[1416] 3. The user follows the guide to take images of each part of the body and taps "Save" to save them to the device.

[1417] Input: User actions and camera guide

[1418] Output: Saved tooth image data

[1419] Step 2:

[1420] The device uploads the saved images to the cloud server. The images are encrypted and sent securely. After the upload is complete, the device notifies the user.

[1421] Specific behavior:

[1422] 1. Select the image saved on your device and tap the "Upload" button.

[1423] 2. The device encrypts the image and sends it to a server over the Internet.

[1424] 3. The server receives the image, verifies the status, and notifies the device of a successful upload.

[1425] Input: Stored tooth image data, user command

[1426] Output: Image data uploaded to the cloud server

[1427] Step 3:

[1428] The server then passes the received image to an AI algorithm to begin analysis. The AI ​​algorithm analyzes the image and diagnoses health conditions such as cavities, tartar, and gum inflammation. The analysis results are then stored in a database.

[1429] Specific behavior:

[1430] 1. The server inputs the received image data into the AI ​​algorithm.

[1431] 2. AI algorithms analyze images and diagnose cavities, tartar, and gum inflammation.

[1432] 3. Diagnostic results are stored in a database for each user.

[1433] Input: Uploaded image data

[1434] Output: Save diagnostic results to database

[1435] Step 4:

[1436] The server uses an emotion engine to analyze the user's emotional state before providing a diagnosis. It determines emotions from the user's camera image and tone of voice and stores the results.

[1437] Specific behavior:

[1438] 1. The server passes the user's camera image and voice data to the emotion engine.

[1439] 2. The emotion engine analyzes this data and identifies the user's emotional state.

[1440] 3. The results of the sentiment analysis are generated and stored in a database for each user.

[1441] Input: Camera image and audio data

[1442] Output: Emotion analysis results

[1443] Step 5:

[1444] The server generates feedback based on the diagnosis and emotion analysis results, including recommendations for brushing method improvements and messages based on the user's emotional state. This feedback is then sent to the device.

[1445] Specific behavior:

[1446] 1. The server generates a feedback message based on the diagnosis and emotion analysis results.

[1447] 2. Simultaneously generate improvements and specific recommendations for brushing methods.

[1448] 3. Send feedback to the device and notify the user.

[1449] Input: Diagnosis results, emotion analysis results

[1450] Output: Feedback messages and brushing advice

[1451] Step 6:

[1452] If the diagnosis indicates that a specialist consultation is necessary, the server acquires the user's location information, searches for and lists nearby dental clinics, and sends the information to the user.

[1453] Specific behavior:

[1454] 1. The server obtains the user's location information and searches for nearby dental clinics.

[1455] 2. Create a list of suitable dental clinics and create a link to book appointments.

[1456] 3. This information is sent to the terminal and notified to the user.

[1457] Input: Diagnostic results, user location information

[1458] Output: A list of dental clinics and a link to book an appointment

[1459] (Application example 2)

[1460] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1461] In today's busy lifestyles, there is a need for a system that allows users to easily check their dental health at home and receive appropriate feedback and a prompt diagnosis. However, existing systems lack the functionality to provide feedback based on emotional state or recommend experts, making it difficult to provide effective advice while alleviating users' concerns.

[1462] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: means for acquiring images of teeth taken by the user; image processing means including an AI algorithm for analyzing the acquired images and diagnosing the dental health condition; means for providing the user with an evaluation and recommendations on brushing methods based on the diagnosis results; means for referring the user to a nearby medical institution if the diagnosis results indicate that a professional diagnosis is necessary; and means for recognizing the user's emotional state and providing feedback according to the user's emotion. This allows the user to easily check the health condition of their teeth at home and receive appropriate advice and support according to their emotion.

[1463] "User" refers to the person who uses the system, and in this case, refers to an individual who wants to check and manage their dental health status.

[1464] A "tooth image" is a photograph of the teeth and gums inside the oral cavity taken by the user.

[1465] The "means for acquiring" is a function for importing images of teeth taken by the user into the system.

[1466] "Image processing means" is a function for analyzing acquired tooth images using an AI algorithm.

[1467] "AI algorithm" refers to an analytical method that uses artificial intelligence, and refers to the function of analyzing images of teeth to diagnose their health condition.

[1468] "Diagnosis" refers to the evaluation of dental health based on the analysis results.

[1469] "Health status" refers to the state of the teeth and gums, and whether or not there are any symptoms such as cavities or inflammation.

[1470] "Brushing technique" refers to the method of brushing your teeth effectively, such as how you brush your teeth and the type of toothbrush you use.

[1471] "Evaluation" refers to determining whether the current brushing method is appropriate based on the diagnostic results.

[1472] "Recommendations" are specific advice or suggestions for improvement provided to the user based on the diagnostic results.

[1473] A "professional" is someone who has specialized knowledge and qualifications related to teeth and oral health, such as a dentist or dental hygienist.

[1474] "Diagnosis required" means that the system's analysis results indicate that the user's dental health condition requires a specialist examination.

[1475] "Nearby medical institution" refers to a medical facility such as a dental clinic that is located within easy access from the user's current location.

[1476] "Means for referral" is a function that guides the user to an appropriate medical institution based on the diagnosis results.

[1477] "Emotional state" refers to the user's psychological state or mood, which in this case is determined from facial expressions and tone of voice.

[1478] "Feedback" is a general term for comments and advice provided to users based on diagnostic and analytical results.

[1479] This invention is a system that allows users to easily check their dental health at home and receive appropriate advice and emotional feedback. This system supports users' dental health management by analyzing dental images taken by the user and providing appropriate advice and emotional feedback based on the diagnosis results.

[1480] The system includes the following means:

[1481] 1. Image acquisition method:

[1482] The user takes images of their teeth using a smartphone or digital camera. The images are taken from at least five angles: front, left side, right side, top, and bottom. This allows detailed image data to be obtained. The captured image data is saved on the user's device.

[1483] 2. Image upload method:

[1484] The user's device uploads the captured images to a cloud server via the Internet, where the image data is received securely and quickly.

[1485] 3. Imaging and diagnostic tools:

[1486] The cloud server analyzes the received images using an AI algorithm, which diagnoses the health of the teeth (e.g., cavities, tartar, gum inflammation, etc.) and stores the results on the server.

[1487] 4. Emotion recognition means:

[1488] Before providing a diagnosis, the server analyzes the user's emotional state using an emotion engine, which identifies emotions from the user's facial expressions and tone of voice to determine the user's current state of mind.

[1489] 5. Feedback can be provided by:

[1490] The server provides optimal feedback to the user based on the diagnosis and emotion recognition results. The feedback includes an evaluation of brushing methods, areas for improvement, and recommendations. It also recommends the type and timing of toothbrush replacement based on the diagnosis results. It also provides encouraging and comforting messages based on the emotion engine results.

[1491] 6. Specialist referral methods:

[1492] If the diagnosis results indicate that a specialist diagnosis is necessary, the server obtains the user's location information and searches for and refers the user to a nearby medical institution. This allows the user to receive a prompt, professional diagnosis even if a serious health problem is detected.

[1493] Hardware and software used

[1494] Hardware:

[1495] Smartphone or digital camera (for taking pictures)

[1496] Internet-connected device (image upload)

[1497] software:

[1498] OpenCV (image processing)

[1499] EmotionRecognizer

[1500] AI Diagnostic (Diagnosis by AI algorithm)

[1501] requests (REST API calls)

[1502] Specific examples

[1503] For example, a user can use the app to take pictures of their teeth from the front, left side, right side, top, and bottom, and upload them to a cloud server. The server then uses AI to analyze the received images and diagnoses the presence of early-stage cavities in the front teeth. The emotion engine then analyzes the user's emotional state and determines that the user is feeling anxious. The app then provides specific brushing advice and recommended toothbrushes, along with a comforting message: "Don't worry. We'll take the time to improve together." Furthermore, if the diagnosis indicates that a professional diagnosis is necessary, the app searches for nearby dental clinics based on the user's location and provides information on appropriate dental clinics.

[1504] Prompt Sentence Examples

[1505] A user takes photos of their teeth with their smartphone from the following angles: front, left side, right side, top, and bottom. Upload these images to a cloud server and use an AI algorithm to analyze the health of their teeth. Additionally, recognize emotions based on the user's facial expressions and voice and provide feedback based on the following criteria:

[1506] If a user is feeling anxious, provide comforting messages such as, "Don't worry. We'll improve this together over time."

[1507] If the diagnosis is positive, a positive message such as "You're polishing very well! Keep it up!" is provided.

[1508] If the diagnosis is negative, provide comforting messages such as, "You need to improve a little. It's okay, we're here to help!"

[1509] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1510] Step 1:

[1511] The user takes images of the teeth using a smartphone or digital camera. The images are taken from at least five angles: front, left side, right side, top, and bottom. The user follows the guide within the app to obtain the appropriate images. This is the input, and the images are saved in the smartphone's storage as the output. Specifically, the user launches the camera app and takes photos from each angle while following the guide display.

[1512] Step 2:

[1513] The user's device uploads the captured tooth images to the cloud server via the Internet. The input is the image stored on the device, and the output is the image stored on the cloud server. Specifically, the device sends the image file to the cloud server using a REST API.

[1514] Step 3:

[1515] The server receives images uploaded to the cloud server and analyzes each image using an AI algorithm. The input is the uploaded image data, and the output is a diagnosis of the dental health condition (e.g., cavities, tartar, gum inflammation, etc.). Specifically, the server preprocesses the images using an image processing library (e.g., OpenCV) and runs a deep learning model to evaluate the dental condition.

[1516] Step 4:

[1517] The server extracts emotions from the user's facial expressions and tone of voice to recognize the user's emotional state based on the diagnosis results. The input is the diagnosis results and the user's facial expressions and voice data, and the output is a judgment of the user's emotional state (e.g., anxiety, relief, etc.). Specifically, the server uses an emotion recognition engine (e.g., EmotionRecognizer) to analyze the facial images and voice data.

[1518] Step 5:

[1519] The server generates optimal feedback for the user based on the diagnosis results and emotion recognition results. The input is the diagnosis results and emotion determination results, and the output is advice or a message to be provided to the user. Specifically, the server generates a positive message if the diagnosis results are good, and a comforting message if the user is very anxious, and sends it to the user.

[1520] Step 6:

[1521] If the diagnosis result indicates that a specialist diagnosis is necessary, the server obtains the user's location information. The input is the diagnosis result and the user's location information, and the output is a list of nearby medical institutions. Specifically, the server obtains the user's location using a location information API and searches for the nearest medical institution based on that location.

[1522] Step 7:

[1523] The server introduces nearby medical institutions to the user. The input is the location information of the user who is deemed to need a specialist diagnosis and a list of medical institutions, and the output is a notification message including the referral details. Specifically, the server sends information about appropriate medical institutions to the user's device and displays it as a notification.

[1524] These steps allow users to easily check their dental health status at home and receive appropriate advice and support according to their emotions.

[1525] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1526] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1527] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1528] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1529] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1530] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1531] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1532] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1533] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1534] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1535] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1536] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1537] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1538] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1539] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1540] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1541] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1542] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1543] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1544] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1545] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1546] The following is further disclosed regarding the above embodiment.

[1547] (Claim 1)

[1548] means for acquiring user-taken images of teeth;

[1549] Image processing means including an AI algorithm that analyzes the acquired images and diagnoses the health condition of the teeth;

[1550] A means for providing the user with an evaluation and recommendations on brushing methods based on the diagnostic results;

[1551] The system includes a means for referring the user to a nearby dental clinic if the diagnostic results indicate that a specialist diagnosis is necessary.

[1552] (Claim 2)

[1553] The system according to claim 1, further comprising means for determining when to replace the toothbrush used by the user based on the diagnostic results and recommending an appropriate type of toothbrush.

[1554] (Claim 3)

[1555] The system according to claim 1, further comprising means for acquiring the user's location information and, if the diagnosis results indicate that a specialist diagnosis is required, searching for and referring the user to a nearby dental clinic based on the user's location information.

[1556] "Example 1"

[1557] (Claim 1)

[1558] means for acquiring user-taken images of teeth;

[1559] a means for storing the captured dental images in a digital device;

[1560] means for uploading the stored images to a cloud computing environment via the Internet;

[1561] Image processing means including a generating AI algorithm that analyzes the acquired image and diagnoses the dental health condition;

[1562] A means for evaluating the user's brushing method and providing points for improvement based on the diagnostic results;

[1563] Based on the diagnostic results, the system recommends the appropriate type of toothbrush and the timing of replacement to the user.

[1564] If the diagnosis results indicate that a specialist diagnosis is necessary, the system will acquire the user's location information and introduce them to a nearby specialist facility.

[1565] A system including:

[1566] (Claim 2)

[1567] 10. The system according to claim 1, further comprising means for determining when to replace the oral care appliance used by the user based on the diagnostic results and recommending an appropriate type of appliance.

[1568] (Claim 3)

[1569] The system according to claim 1, further comprising means for acquiring the user's location information, and if the diagnosis results indicate that a specialist diagnosis is necessary, searching for and introducing a nearby specialist facility based on the user's location information.

[1570] "Application Example 1"

[1571] (Claim 1)

[1572] means for acquiring user-taken images of teeth;

[1573] Image processing means including an AI algorithm that analyzes the acquired images and diagnoses the health condition of the teeth;

[1574] A means for providing the user with an evaluation and recommendations on brushing methods based on the diagnostic results;

[1575] If the diagnosis result indicates that a specialist diagnosis is necessary, a means for referring the user to a nearby dental clinic;

[1576] Based on the diagnostic results, we recommend appropriate dental care products through online shopping and provide a link to purchase them.

[1577] A system including:

[1578] (Claim 2)

[1579] The system according to claim 1, further comprising means for determining when to replace the toothbrush used by the user based on the diagnostic results and recommending an appropriate type of toothbrush.

[1580] (Claim 3)

[1581] The system according to claim 1, further comprising means for acquiring the user's location information and, if the diagnosis results indicate that a specialist diagnosis is required, searching for and referring the user to a nearby dental clinic based on the user's location information.

[1582] "Example 2: Combining Emotion Engines"

[1583] (Claim 1)

[1584] means for acquiring user-taken images of teeth;

[1585] Image processing means including an AI algorithm that analyzes the acquired images and diagnoses the health condition of the teeth;

[1586] A means for providing the user with an evaluation and recommendations on brushing methods based on the diagnostic results;

[1587] means for analyzing the emotional state of a user and providing feedback according to the emotional state;

[1588] The system includes a means for referring the user to a nearby dental clinic if the diagnostic results indicate that a specialist diagnosis is necessary.

[1589] (Claim 2)

[1590] The system according to claim 1, further comprising means for determining when to replace the toothbrush used by the user based on the diagnostic results and recommending an appropriate type of toothbrush.

[1591] (Claim 3)

[1592] The system according to claim 1, further comprising means for acquiring the user's location information and, if the diagnosis results indicate that a specialist diagnosis is required, searching for and referring the user to a nearby dental clinic based on the user's location information.

[1593] "Application example 2 when combining emotion engines"

[1594] (Claim 1)

[1595] means for acquiring user-taken images of teeth;

[1596] Image processing means including an AI algorithm that analyzes the acquired images and diagnoses the health condition of the teeth;

[1597] A means for providing the user with an evaluation and recommendations on brushing methods based on the diagnostic results;

[1598] A means for referring the user to a nearby medical institution when it is determined from the diagnostic results that a specialist diagnosis is necessary;

[1599] means for recognizing the emotional state of the user and providing feedback according to the emotional state;

[1600] A system including:

[1601] (Claim 2)

[1602] The system according to claim 1, further comprising means for determining when to replace the toothbrush used by the user based on the diagnostic results and recommending an appropriate type of toothbrush.

[1603] (Claim 3)

[1604] The system according to claim 1, further comprising means for acquiring the user's location information and, if the diagnosis result indicates that a specialist diagnosis is required, searching for and referring the user to a nearby medical institution based on the user's location information. [Explanation of symbols]

[1605] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for acquiring user-taken images of teeth; Image processing means including an AI algorithm that analyzes the acquired images and diagnoses the health condition of the teeth; A means for providing the user with an evaluation and recommendations on brushing methods based on the diagnostic results; The system includes a means for referring the user to a nearby dental clinic if the diagnostic results indicate that a specialist diagnosis is necessary.

2. The system according to claim 1, further comprising means for determining when to replace the toothbrush used by the user based on the diagnostic results and for recommending an appropriate type of toothbrush.

3. The system according to claim 1, further comprising means for acquiring the user's location information and, if the diagnosis results indicate that a specialist diagnosis is required, searching for and referring the user to a nearby dental clinic based on the user's location information.

Images