Microbial flora analysis device, microbial flora analysis method, program, and microbial flora analysis system

Abstract

This invention provides a microbiome analysis device, a microbiome analysis method, and a program that analyze the state of the oral microbiome and propose improvements to the user's oral environment. [Solution] The tongue image of the subject is input into an image learning model obtained by machine learning the relationship between tongue images taken from the tongue and the results of the analysis of the oral microbiome state, and a first predicted value of the oral microbiome state corresponding to the subject's tongue image is obtained. Based on the first predicted value, information about the subject's oral environment is generated.

Description

Technical Field

[0001] The present disclosure relates to a flora analysis apparatus, a flora analysis method, a program, and a flora analysis system.

Background Art

[0002] In the oral cavity, various bacteria form aggregates (flora), and this state is called the oral flora. It is becoming clear that the oral flora contains not only harmful bacteria (bad bacteria) related to oral diseases such as dental caries and periodontal disease, but also beneficial bacteria (good bacteria) for humans. Until now, the mainstream idea has been to deal with oral diseases by eliminating bacteria, but it is difficult to completely eliminate bacteria, and eliminating good bacteria may rather cause problems, or the problems may recur due to the re-colonization of bad bacteria. In recent years, the idea of aiming for the prevention of oral diseases, not just temporarily, by constructing a coexistence relationship (flora control) in which good bacteria are in balance rather than eliminating bacteria, is spreading.

[0003] Currently, there is no easy way or opportunity for ordinary people to know their own oral flora. In the research field, the next-generation sequencing technology that comprehensively analyzes bacterial DNA information in oral samples such as saliva and tongue coating can accurately identify the oral flora. However, the above technology requires implementation in a specialized institution, and it requires labor and time such as collecting oral samples and mailing them to a specialized institution, as well as high analysis costs.

[0004] The inventors of the present invention focused on technologies such as AI, and thought that if a technology for predicting the oral flora with a familiar device such as a smartphone via a network could be developed, people could easily, quickly, and at a low cost know their own oral flora, and by providing information on flora control, it would be possible to enhance the recognition of the oral flora and widely spread its importance among ordinary people.

[0005] Each individual's oral microbiome is unique, and care tailored to this microbiome is crucial for the prevention and treatment of oral diseases. Therefore, by helping individuals understand their own oral microbiome and providing information on optimal lifestyle habits (oral hygiene habits, dietary habits, etc.), we can contribute to creating a society with fewer oral diseases.

[0006] As a related technology, Patent Document 1 discloses a technology that obtains the results of a questionnaire consisting of multiple questions, including questions about the lifestyle habits of the organism to be predicted, and uses the obtained response results as input to predict the state of the intestinal flora of the organism to be predicted based on the correlation between the response results and test data indicating the state of the intestinal flora that is pre-associated with the questionnaire. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2023-73856 [Overview of the Initiative] [Problems that the invention aims to solve]

[0008] In relation to the technologies mentioned above, there was a need for a technology that could analyze the state of the oral microbiome and provide suggestions for improving the oral environment to the subjects of the examination.

[0009] The purpose of this disclosure is to provide a microbiome analysis device, a microbiome analysis method, a program, and a microbiome analysis system that solve the above-mentioned problems. [Means for solving the problem]

[0010] A microbiome analysis device according to one aspect of this disclosure includes: a first prediction means that inputs a tongue image of a subject to be examined into an image learning model obtained by machine learning the relationship between a tongue image taken of the tongue and the analysis results of the microbiome state in the oral cavity, and obtains a first predicted value of the microbiome state in the oral cavity corresponding to the tongue image of the subject; and an oral information generation means that generates information about the oral environment of the subject based on the first predicted value.

[0011] A microbiome analysis method according to one aspect of this disclosure involves inputting a tongue image of a subject into an image learning model obtained by machine learning the relationship between a tongue image taken from the tongue and the analysis results of the microbiome state in the oral cavity, obtaining a first predicted value of the microbiome state in the oral cavity corresponding to the tongue image of the subject, and generating information about the oral environment of the subject based on the first predicted value.

[0012] A program according to one aspect of this disclosure causes the computer of a microbiome analysis device to function as a first prediction means that inputs the tongue image of a subject into an image learning model obtained by machine learning the relationship between a tongue image taken of the tongue and the analysis results of the microbiome state in the oral cavity, and obtains a first predicted value of the microbiome state in the oral cavity corresponding to the tongue image of the subject; and an oral information generation means that generates information about the oral environment of the subject based on the first predicted value.

[0013] A microbiome analysis system according to one aspect of this disclosure inputs the tongue image of a subject to an image learning model obtained by machine learning the relationship between a tongue image taken from the tongue and the analysis results of the microbiome state in the oral cavity, obtains a first predicted value of the microbiome state in the oral cavity corresponding to the tongue image of the subject, and generates information about the oral environment of the subject based on the first predicted value. [Effects of the Invention]

[0014] According to the above embodiment, the state of the oral microbiome can be analyzed and suggestions for improving the oral environment can be made to the person being tested. [Brief explanation of the drawing]

[0015] [Figure 1] This figure shows an overview of a microbiome analysis system according to one aspect of this disclosure. [Figure 2] This is a hardware configuration diagram of a microbiome analysis device according to one aspect of this disclosure. [Figure 3] This is a functional block diagram of a microbiome analysis device according to one aspect of the present disclosure. [Figure 4]A diagram showing a questionnaire page according to one aspect of the present disclosure. [Figure 5] A first diagram showing guidance for taking a tongue image according to one aspect of the present disclosure. [Figure 6] A second diagram showing guidance for taking a tongue image according to one aspect of the present disclosure. [Figure 7] A diagram showing an overview of the processing of a flora analysis apparatus according to one aspect of the present disclosure. [Figure 8] A diagram showing a processing flow of a flora analysis apparatus according to one aspect of the present disclosure. [Figure 9] A diagram showing an example of a result display page according to one aspect of the present disclosure. [Figure 10] A functional block diagram of a flora analysis apparatus according to another aspect of the present disclosure. [Figure 11] A diagram showing a processing flow of a flora analysis apparatus according to another aspect of the present disclosure.

Mode for Carrying Out the Invention

[0016] Hereinafter, a flora analysis apparatus, a flora analysis method, and a program of the present disclosure will be described with reference to the drawings.

[0017] FIG. 1 is a diagram showing an overview of a flora analysis system according to one aspect of the present disclosure. The flora analysis system 100 includes at least the flora analysis apparatus 1. The flora analysis apparatus 1 is, for example, a cloud server. In FIG. 1, the terminal 2 is a terminal such as a smartphone, a tablet, or a personal computer terminal held by a user (a test subject or an operator). The terminal 2 is communicatively connected to the flora analysis apparatus 1 via a communication network.

[0018] FIG. 2 is a hardware configuration diagram of a flora analysis apparatus according to one aspect of the present disclosure. As shown in Figure 2, the microbiome analysis device 1 is a computer equipped with hardware such as a CPU (Central Processing Unit) 101, ROM (Read Only Memory) 102, RAM (Random Access Memory) 103, storage device 104, and communication module 105. Terminal 2 is also a computer equipped with the hardware configuration shown in Figure 2.

[0019] Figure 3 is a functional block diagram of a microbiome analysis device according to one aspect of this disclosure. The CPU 101 of the microbiome analysis device 1 executes a microbiome analysis program that is stored in advance. As a result, the microbiome analysis device 1 performs functions such as the control unit 11, questionnaire processing unit 12, shooting guidance unit 13, acquisition unit 14, learning unit 15, prediction unit 16, oral information generation unit 17, and output unit 18.

[0020] The control unit 11 controls other functional units. For the sake of explanation, the control unit 11 is assumed to include web server functionality and the ability to output web pages based on access from terminal 2. The questionnaire processing unit 12 works in conjunction with terminal 2 to process the questionnaire responses of the test subjects. The imaging guidance unit 13 works in conjunction with terminal 2 to provide guidance for taking images of the tongue. The acquisition unit 14 acquires information entered into the terminal 2 or the like. The learning unit 15 generates a learning model. The prediction unit 16 calculates predicted values ​​based on tongue images and questionnaire results. The oral information generation unit 17 generates information about the oral environment of the person being examined based on predicted values. The output unit 18 outputs output information, including the information generated by the oral information generation unit 17, to a predetermined output destination such as the terminal 2.

[0021] The microbiome analysis device 1 of this disclosure inputs the tongue image of the subject into an image learning model obtained by machine learning the relationship between a tongue image taken from the tongue and the analysis results of the microbiome state in the oral cavity, obtains a predicted value of the microbiome state in the oral cavity corresponding to the tongue image of the subject, and generates information about the oral environment of the subject based on the predicted value.

[0022] In this process, the microbiome analyzer 1 inputs the tongue image of the subject into an image learning model obtained by machine learning the relationship between a tongue image taken of the tongue and an analysis result showing the ratio of beneficial bacteria in the oral microbiome. It then obtains a first predicted value indicating the ratio of beneficial bacteria corresponding to the subject's tongue image, and generates information about the subject's oral beneficial bacteria level by referring to an index value such as this first predicted value or a value calculated based on this first predicted value, and a threshold optimized for generating the oral beneficial bacteria level.

[0023] Furthermore, the microbiome analysis device 1 inputs the questionnaire results of the person being tested into a questionnaire learning model obtained by machine learning the relationship between the questionnaire results for multiple habit items that affect the oral microbiome and the analysis results of the oral microbiome state, obtains a second predicted value of the oral microbiome state corresponding to the questionnaire results of the person being tested, and generates information on the oral beneficial bacteria level of the person being tested by referring to a third predicted value (index value) calculated based on the first and second predicted values ​​and a threshold optimized for generating the oral beneficial bacteria level.

[0024] More specifically, the microbiome analysis device 1 uses a deep learning model as an image learning model. The microbiome analysis device 1 inputs an image of the subject's tongue into the deep learning model, obtains a first predicted value indicating the ratio of beneficial bacteria corresponding to the subject's tongue image, and generates information about the oral level of beneficial bacteria, which indicates the oral environment, based on this first predicted value and a threshold optimized for generating oral beneficial bacteria levels. In this case as well, the microbiome analysis device 1 may use the first predicted value as an index value and generate information about the oral level of beneficial bacteria, which indicates the oral environment, based on this index value and the threshold.

[0025] More specifically, the microbiome analyzer 1 inputs a first predicted value representing the ratio of beneficial bacteria obtained from a deep learning model and a second predicted value representing the ratio of beneficial bacteria obtained from a questionnaire learning model into an ensemble model to calculate a third predicted value (an example of an indicator value). Based on this calculated third predicted value (an example of an indicator value) and a threshold optimized for generating oral beneficial bacteria levels, it generates information about the oral environment and the level of beneficial bacteria in the mouth.

[0026] For processing ensemble models, known methods can be used. In regression ensemble models, it is preferable to use the mean or weighted mean.

[0027] The microbiome analyzer 1 generates information regarding the level of beneficial bacteria in the mouth, including information indicating whether there are many or few beneficial bacteria, and presentation information that encourages improvement of the oral environment, obtained by inputting at least one of the indicator value and questionnaire results into a large-scale language model, and transmits the output results to terminal 2. Terminal 2 displays the output results received from the microbiome analyzer 1.

[0028] Terminal 2 stores the program for the microbiome check application. When Terminal 2 detects the execution of the microbiome check application program based on user (test subject or operator) actions, it displays page data and accesses the microbiome analysis device 1. Terminal 2 then works in conjunction with the microbiome analysis device 1 to perform processing as a user interface.

[0029] Figure 4 shows a questionnaire page according to one aspect of this disclosure. Terminal 2 accesses the microbiome analysis device 1 based on user input. The web server function of the microbiome analysis device 1 outputs page data to terminal 2, including a button indicating a request to start the microbiome analysis. When the user operates terminal 2 and presses the button indicating the request to start the microbiome analysis, the microbiome analysis device 1 performs questionnaire acquisition processing and tongue image acquisition processing.

[0030] In the questionnaire acquisition process, the web server function of the microbiome analysis device 1 sends a questionnaire page 41 (Figure 4), which is one form of page data, to terminal 2. The questionnaire page 41 contains multiple questionnaire items. For example, the questionnaire items include a questionnaire item 42 regarding the time of brushing teeth after breakfast, a questionnaire item 43 regarding the time of brushing teeth after dinner, a questionnaire item 44 to check whether the user pays attention to the nutritional balance of their meals, a questionnaire item 45 to check the frequency of consuming dishes containing leafy vegetables, a questionnaire item 46 to check the frequency of consuming commercially available beverages containing sugar, a questionnaire item 47 to check whether the user has a smoking habit, and a questionnaire item 48 to check whether the user is aware of the relationship between periodontal disease and smoking cessation.

[0031] Other survey items include questions about eating habits (number of meals per day, frequency of eating out, breakfast consumption, whether or not they have a habit of snacking, frequency of fermented food consumption, frequency of salty food consumption, frequency of vegetable consumption other than leafy greens, frequency of alcohol consumption, frequency of 100% vegetable / fruit juice consumption, whether or not they try to maintain a healthy weight), questions about exercise habits (frequency of exercise, duration of exercise per session, daily walking time, etc.), questions about work status / work style (whether or not they have irregular work hours, exercise time on the way to work, frequency of coming to work, walking time during commute, frequency of business trips, motivation to work, etc.), questions about sleep habits (sleep duration, whether or not they experience daytime sleepiness, etc.) The questionnaire may include items such as: questions about bedtime and wake-up time; questions about stress levels (questions about perceived stress levels); questions about oral hygiene habits (questions about frequency of dental visits, frequency of use of oral cleaning tools other than toothbrushes, number of times brushing per day, duration of brushing per session, etc.); questions about oral health (questions about the presence or absence of symptoms of oral disease, oral pain, appearance of teeth, bite, bad breath, degree of dry mouth, jaw pain, teeth grinding, oral function, etc.); questions about overall health (questions about medical history such as diabetes, heart disease, and osteoporosis); and questions about medication use (questions about antibiotic use).

[0032] Figure 5 is a first diagram showing instructions for taking a tongue image according to one aspect of the present disclosure. In the tongue image acquisition process, the web server function of the microbiome analysis device 1 sequentially sends multiple guidance pages for tongue image acquisition guidance to terminal 2 based on user operations. Figure 5 shows the first guidance page 51 and the second guidance page 52. The first guidance page 51 is page data that guides the user on how to take a tongue image, explaining that the tongue should be captured within a rectangular frame. The second guidance page 52 is page data that terminal 2 displays after the camera is activated, displaying the face and tongue of the person being examined and prompting the user to ensure that the tongue is largely captured within the rectangular frame. The user operates terminal 2 (e.g., pinching out of the image) and changes the relative position of terminal 2 to the face to ensure that the person being examined's tongue is largely captured within the rectangular frame. In Figure 5, of the two second guidance pages 52, the left one shows the face immediately after the camera is activated, and the right one shows the person being examined with their tongue largely captured.

[0033] Figure 6 is a second figure showing instructions for taking a tongue image according to one aspect of the present disclosure. In the tongue image acquisition process, terminal 2, in cooperation with the microbiome analysis device 1, displays the third guide page 53 and the fourth guide page 54 sequentially after the subject displays their tongue enlarged within a rectangle on the second guide page 52. For example, terminal 2 outputs the second guide page to terminal 2, automatically takes a picture after a predetermined countdown of seconds, and displays the third guide page 53, which includes the image acquired by that picture. The third guide page 53 displays indicators for determining whether the image of the tongue is OK or NG (whether there is overexposure, out of focus, etc.), as well as buttons to select whether to use this image or take another one. When terminal 2 detects that the button indicating to use this image has been pressed on the third guide page 53, it displays the fourth guide page 54. On the fourth guide page 54, terminal 2 displays a range for identifying the area at the base of the tongue to be used for machine learning in the image of the tongue. This area for identification may be a rectangular range. The user can move the rectangular range on the image. Terminal 2 moves the position of the rectangular range displayed on the image based on the user's drag operation of the rectangular range. Page 54 of the fourth guide displays instructions prompting the user to select a less shaded area near the base of the tongue. The user then aligns a rectangular area with the less shaded area near the base of the tongue and presses the button indicating to crop.

[0034] Figure 7 shows an overview of the processing of a microbiome analysis device according to one aspect of this disclosure. The acquisition unit 14 of the microbiome analysis device 1 acquires tongue images and questionnaire results using the same method as described above. The acquisition unit 14 also reads the analysis results of the state of specific bacteria (beneficial bacteria) contained in the saliva of the person being examined, corresponding to the tongue image (analysis results of the state of the oral microbiome), from a recording medium or the like based on user input. Specifically, the analysis results of the state of bacteria show the ratio of beneficial bacteria to the total number of bacteria present in the saliva, as analyzed by a next-generation sequencer, and results previously obtained from the microbiome analysis device 1 (first predicted value, second predicted value, third predicted value). Beneficial bacteria include, for example, nitrate-reducing bacteria and lactic acid bacteria. More specifically, examples of nitrate-reducing bacteria include Neisseria, Haemophilus, Russia, and Actinomyces, and examples of lactic acid bacteria include Lactobacillus reuteri.

[0035] The learning unit 15 stores tongue images of numerous subjects and the state of the oral microbiota contained in the saliva of those subjects. It then uses machine learning to analyze the relationship between the two and generates an image deep learning model. The machine learning method for generating the image deep learning model may utilize known techniques. As data used for machine learning, for example, data obtained from male and female subjects aged 3 to 80 years (with an approximately equal distribution of age and gender) can be used. The analysis of the state of the oral microbiota may be based on the ratio of beneficial bacteria obtained from the analysis of bacterial cells in the saliva of each subject using a next-generation sequencer. Beneficial bacteria include, for example, nitrate-reducing bacteria and lactic acid bacteria. More specifically, examples of nitrate-reducing bacteria include Neisseria, Haemophilus, Russia, and Actinomyces, and examples of lactic acid bacteria include Lactobacillus reuteri.

[0036] More specifically, the learning unit 15 generates an image-based deep learning regression model by machine learning the relationship between the tongue image (explanatory variable) and the analysis result (dependent variable) showing the ratio of beneficial bacteria in the total bacteria contained in the saliva. This image-based deep learning regression model is a machine learning model that outputs a predicted value of the proportion of beneficial bacteria with a predetermined accuracy.

[0037] Specifically, the learning unit 15 generates an image deep learning classification model by machine learning the relationship between the tongue image (explanatory variable) and the analysis result (target variable) showing the ratio of beneficial bacteria in the total number of bacteria contained in the saliva. This image deep learning classification model is also a machine learning model that outputs the ratio of beneficial bacteria with a predetermined accuracy.

[0038] Image deep learning models are a well-known technique that uses the architecture of convolutional neural networks (CNNs). Examples include ResNet, DenseNet, EfficientNet, MobileNet, and ShuffleNet, but the GC_EfficientNetV2_RW_T model is suitable for analyzing tongue images. This model combines the high efficiency of EfficientNetV2 with GCNet (Global Context Network) as a self-attention mechanism. This combination allows EfficientNetV2 to extract local features (subtle color differences, textures, shape changes, etc.) while GCNet comprehensively captures global features of the entire image. GCNet considers the context of the entire image and enhances the relationships between subtle patterns. As a result, it can effectively utilize subtle features that were difficult to capture with conventional models for tongue images, achieving highly accurate predictions.

[0039] In generating image deep learning models, regression models or classification models may be used. The results obtained from each model may be combined using an ensemble method (ensemble model) to calculate a first ensemble prediction value (index value). Evaluation metrics for regression models include mean absolute error (MAE) and mean absolute percentage error (MAPE). Evaluation metrics for classification models include cross-entropy error (CEE), accuracy, and AUC (Area Under Curve). Evaluation metrics for ensemble models include accuracy and AUC (Area Under Curve), but it is preferable to set them appropriately depending on the nature of the data and the desired output. When combining each model using an ensemble method, averaging and weighted averaging can be used as ensemble methods. For example, when using weighted averaging, the optimal weights that best improve the accuracy (correctness) of the beneficial bacteria ratio prediction value output from the ensemble model can be set using Bayesian optimization or grid search.

[0040] The learning unit 15 also generates a questionnaire learning model by machine learning the relationship between the questionnaire results of a large number of subjects and the analysis results of the oral microbiota status of those subjects. The method for generating the questionnaire learning model may utilize publicly known techniques. LightGBM can be used as a publicly known technique. LightGBM is an open-source machine learning algorithm that can be applied to processes such as classification and regression.

[0041] For example, the learning unit 15 generates a survey learning model by machine learning the relationship between the survey results converted into features (explanatory variables) and the analysis results (dependent variable) showing the ratio of beneficial bacteria in the total bacteria contained in saliva. This survey learning model is a machine learning model that outputs a predicted value of the proportion of beneficial bacteria with a predetermined accuracy.

[0042] In this disclosure, the questionnaire learning model calculates indicators such as feature importance and correlation coefficients (Pearson product-moment correlation coefficient, Spearman rank correlation coefficient) based on the features obtained by converting the questionnaire results into features (explanatory variables) and the analysis results showing the ratio of beneficial bacteria in the total bacteria contained in saliva (dependent variable). Based on these indicators, one or more questionnaire items considered promising for predicting the dependent variable were selected, and the prediction accuracy of the beneficial bacteria ratio was measured. If accuracy improved by adding the selected questionnaire items, the correlation between the explanatory variables and the dependent variable, and the direction that influences the predicted value of the beneficial bacteria ratio (how consistent the change in the value of the questionnaire item is with the change in the predicted value) were investigated, and the item was adopted only if there was a clear positive or negative direction and it was consistent with the prior knowledge. As an example, the model used is a "LightGBM (regression) model," the validation design is "Repeated-Kfold (5 divisions x 5 patterns)," and the evaluation metric can be the mean absolute error (MAE).

[0043] The ratio of beneficial bacteria (first prediction) output using an image-based deep learning model and the ratio of beneficial bacteria (second prediction) output using a questionnaire-based learning model may be combined using an ensemble method (ensemble model). Averaging or weighted averaging can be used as the ensemble method. When weighted averaging is used, the ratio of beneficial bacteria (third prediction; index value) and threshold output from the ensemble model can be optimized for high accuracy using Bayesian optimization or grid search.

[0044] The analysis results of the bacterial status of the subject obtained by the acquisition unit 14, specifically the ratio of beneficial bacteria to the total number of bacteria present in the saliva analyzed by a next-generation sequencer (dependent variable) and the tongue image of the subject (explanatory variable), can be used to generate an image learning model.

[0045] The analysis results of the bacterial status of the subjects of testing obtained by the acquisition unit 14, specifically the ratio of beneficial bacteria to the total number of bacteria present in the saliva analyzed by a next-generation sequencer (the target variable), and the results of the subject's questionnaire can be used to generate a questionnaire learning model.

[0046] Figure 8 shows the processing flow of a microbiome analysis device according to one aspect of this disclosure. Next, we will explain the processing flow of terminal 2 and microbiome analysis device 1 step by step. When terminal 2 detects the execution of the microbiome check application program based on user operation, it displays page data and accesses the microbiome analysis device 1 (step S101). The control unit 11 of the microbiome analysis device 1 detects the access from terminal 2 and performs login processing, etc. (step S102). Terminal 2 displays the questionnaire page 41 (step S103). Terminal 2 may display the questionnaire page 41 sent by the questionnaire processing unit 12 or web server function of the microbiome analysis device 1, or it may display the questionnaire page 41 stored as a function of the microbiome check application. The user enters answers to the multiple questionnaire items listed on the questionnaire page 41. Answers are entered by checking one of the multiple checkboxes provided for each item. After answering all items, the user presses the submit questionnaire button. Terminal 2 sends the questionnaire results, including identification information of the checked checkboxes for each item, to the microbiome analysis device 1 (step S104).

[0047] Terminal 2 then displays the first guide page 51 to instruct the user on how to take a picture of their tongue (step S105). Based on the user's press of the start button, Terminal 2 displays the second guide page 52, showing the user's face and tongue within the image display frame, and displaying explanatory text prompting the user to ensure the tongue is clearly visible within the rectangular area. After outputting the second guide page to Terminal 2, it performs a countdown for a predetermined number of seconds, and when the countdown is complete, it automatically starts taking a picture (step S106), and displays the third guide page 53 containing the image acquired by that picture (step S107). Based on the user's press of the button indicating that this is the image, Terminal 2 displays the fourth guide page 54 (step S108). If Terminal 2 detects the user's press of the button indicating that they want to take another picture on the third guide page, it takes another picture and displays that image on the third guide page 53. When terminal 2 detects that the user has aligned a rectangular area to a less shaded area near the base of the tongue using the fourth guidance page 54 and has pressed the button indicating cropping, it sends the cropped tongue image to the microbiome analysis device 1 (step S109). Terminal 2 may also send the questionnaire results sent in step S104 and the tongue image sent in step S109 together to the microbiome analysis device 1. Terminal 2 may display the first guidance pages 51 to the fourth guidance pages 54 that have been sent by the processing of the shooting guidance unit 13 or the web server function of the microbiome analysis device 1, or it may display the first guidance pages 51 to the fourth guidance pages 54 that are stored as a function of the microbiome check application.

[0048] When the questionnaire processing unit 12 obtains the questionnaire results from the terminal 2, it converts the questionnaire results into features (step S110). The questionnaire processing unit 12 outputs the features of the questionnaire results to the prediction unit 16. When the shooting guidance unit 13 obtains the tongue image from the terminal 2, it outputs the tongue image to the prediction unit 16. The shooting guidance unit 13 determines whether there is any overexposure or if the brightness is above a threshold based on the RGB values ​​and brightness of the tongue image, and may output the tongue image to the prediction unit 16 if there is no overexposure or if the brightness is within a predetermined range. The shooting guidance unit 13 determines whether there is any overexposure or if the brightness is above a threshold based on the RGB values ​​and brightness of the tongue image, and may request the terminal 2 to retake the image if there is overexposure or if the brightness is outside a predetermined range.

[0049] The prediction unit 16 inputs the features of the questionnaire results into the questionnaire learning model and calculates the predicted value X (second predicted value) of the beneficial bacteria ratio (step S111). The prediction unit 16 also inputs the tongue image into an image deep learning model and calculates a predicted value Y (first predicted value) for the ratio of beneficial bacteria (step S112).

[0050] The prediction unit 16 obtains the coefficient (weight) α of the predicted value X of the beneficial bacteria abundance ratio and the coefficient (weight) β of the predicted value Y of the beneficial bacteria abundance ratio from the storage unit. The prediction unit 16 calculates a third predicted value of the beneficial bacteria abundance ratio by performing a weighted average using these predicted values ​​of the beneficial bacteria abundance ratio and the corresponding coefficients (step S114). The prediction unit 16 outputs the third predicted value to the oral information generation unit 17.

[0051] The oral cavity information generation unit 17 compares a threshold value with a third predicted value of the beneficial bacteria abundance ratio. Based on the third predicted value of the beneficial bacteria abundance ratio and the threshold value, the oral cavity information generation unit 17 determines whether the beneficial bacteria abundance ratio is high or low (step S115). The threshold value can be set to a value with a high accuracy rate using grid research or Bayesian optimization.

[0052] The oral information generation unit 17 obtains the questionnaire results from the questionnaire processing unit 12 and the predicted values ​​from the prediction unit. The oral information generation unit 17 identifies the checked checkboxes and their respective predicted values ​​for each questionnaire item. The oral information generation unit 17 obtains text from a storage unit such as a database that contains "recommended lifestyle habits" to promote improvement of the oral environment, corresponding to the checkbox identification information and predicted values ​​for each questionnaire item (step S116). In addition to a storage unit such as a database, the oral information generation unit 17 may also input the questionnaire results obtained from the questionnaire processing unit 12 and the predicted values ​​obtained from the prediction unit into a known large-scale language model, and as a result, obtain text from the large-scale language model that contains "recommended lifestyle habits" to promote improvement of the oral environment.

[0053] The oral information generation unit 17 generates presentation information that includes the level of beneficial bacteria ratio (e.g., high, low) and text about "recommended lifestyle habits" (step S117). The oral information generation unit 17 outputs the presentation information to the output unit 18. The output unit 18 transmits the presentation information to the terminal 2 (step S118).

[0054] Terminal 2 receives the information to be presented. Terminal 2 displays a results page that includes an indicator showing the level of beneficial bacteria included in the information to be present, and text about "recommended habits" (step S119).

[0055] Figure 9 shows an example of a results display page according to one aspect of this disclosure. In Figure 9, the results page 91 on the left shows an example where the ratio of beneficial bacteria is below the threshold. In Figure 9, the results page 92 on the right shows an example where the ratio of beneficial bacteria is above the threshold. Results pages 91 and 92 each include a display showing whether the ratio of beneficial bacteria (third predicted value) is above or below the threshold, an image level displayed on indicator 83 which is divided into a high zone and a low zone at the threshold, and a display area 84 which displays text about "recommended lifestyle habits".

[0056] The "Recommended Lifestyle Habits" section on the results page displayed on terminal 2 may be information displayed for each survey item. For example, in the display area 84 for "recommended lifestyle habits" in response to survey item 42 regarding the time spent brushing teeth after breakfast, recommended information regarding morning tooth brushing habits may be displayed according to the length of time indicated by the response to that survey item. The same recommended information may be displayed regardless of the response. Furthermore, in the display area 84 for "recommended lifestyle habits" in response to survey item 43 regarding the time spent brushing teeth after dinner, recommended information regarding habits related to brushing teeth after dinner may be displayed according to the length of time indicated by the response to the survey item. The same recommended information may be displayed regardless of the response.

[0057] Furthermore, for the "recommended lifestyle habits" section of questionnaire item 44, which checks whether respondents pay attention to the nutritional balance of their meals, it is acceptable to display recommended information as habits for maintaining nutritional balance in meals, depending on the answer to that questionnaire item. In this manner, terminal 2 may obtain and display a results page from the microbiome analysis device 1 that shows matching or different recommended information for each survey item and answer.

[0058] Furthermore, questionnaire item 46, which asks about the frequency of consuming commercially available sugary beverages, and questionnaire item 47, which asks about the habit of smoking, are particularly important questions regarding the impact on dental health, and different recommended information may be displayed depending on the answers to these questionnaire items.

[0059] The aforementioned microbiome analysis results and recommendation information are stored in the memory device 104. This stored information can be used in subsequent tests to display improvements since the previous test by comparing it with new analysis results, and to generate the recommendation information for the current test.

[0060] In addition, the microbiome analysis device 1 may transmit the obtained microbiome status and questionnaire results of the subject to a specialist such as a dentist who visits the subject. The microbiome analysis device 1 may also receive the subject's visit results (professional care information and microbiome analysis information) transmitted from the dental clinic, store them in the storage device 104, and add them as training data. Furthermore, the microbiome analysis device 1 may add the oral environment information at the time of the visit, transmitted from a specialist such as a dentist, to the improvement suggestion generation process. For example, when the subject receives an examination at a dental clinic, the microbiome analysis device 1 may show the areas where there was a lot of plaque buildup and display suggestions for consciously brushing those areas.

[0061] According to the processing performed by the above-described microbiota analysis device 1, the state of the microbiota in the mouth can be analyzed, and the user can be provided with information on the oral environment of the person being examined and suggestions for improving the oral environment.

[0062] The processing of the microbiome analysis device 1 described above is an example of a process in which the tongue image of the subject is input into an image learning model obtained by machine learning the relationship between the tongue image taken from the tongue and the analysis results of the oral microbiome state, and a first predicted value of the oral microbiome state corresponding to the subject's tongue image is obtained. In another example, the microbiome analysis device 1 may calculate information indicating whether there are many or few bacteria other than beneficial bacteria as a state of bacteria using a similar process, and display a results page on terminal 2 that shows whether the ratio of non-beneficial bacteria is high or low.

[0063] Furthermore, the processing of the above-described microbiome analysis device 1 is an example of a process in which the tongue image of the subject is input into an image learning model obtained by machine learning the relationship between the tongue image taken from the tongue and the analysis results of the microbiome state in the oral cavity, a first predicted value of the microbiome state in the oral cavity corresponding to the subject's tongue image is obtained, and information on the level of the oral environment of the subject is generated based on that first predicted value. As another example, the microbiome analysis device 1 may determine the degree of beneficial bacteria ratio based on a predicted value and threshold identified using at least one of a predicted value Y of beneficial bacteria abundance ratio calculated using an image deep learning regression model and a predicted value Z of beneficial bacteria abundance ratio calculated using an image deep learning classification model, without using the features of the questionnaire results.

[0064] Furthermore, the processing of the microbiome analysis device 1 described above is an example of a process in which the survey results are input into a survey learning model obtained by machine learning the relationship between the survey results for multiple habit items that affect beneficial bacteria and the analysis results showing the ratio of beneficial bacteria, a second predicted value of the ratio of beneficial bacteria corresponding to the survey results is obtained, and information on the level of beneficial bacteria in the oral environment is generated based on a first predicted value showing the ratio of beneficial bacteria corresponding to the tongue image, a third predicted value calculated based on the second predicted value and a threshold.

[0065] The above example shows a case where the microbiome analysis device 1 has functions corresponding to each functional block shown in Figure 3. However, terminal 2 may perform functions corresponding to one or more of the functional blocks shown in Figure 3. In addition, one or more of the functions of the learning unit 15, prediction unit 16, and oral information generation unit 17 that generate each machine learning model may be included in an external server, and each machine learning model may be generated on the external server.

[0066] Figure 10 is a functional block diagram of a microbiome analysis device according to another aspect of this disclosure. Figure 11 shows the processing flow of a microbiome analysis device according to another aspect of this disclosure. The microbiome analysis device 1 may include at least a prediction means 21 and an oral information generation means 22. The prediction means 21 inputs the tongue image of the person being examined into an image learning model obtained by machine learning the relationship between the tongue image taken of the tongue and the results of the analysis of the state of the bacterial flora in the oral cavity, and obtains a predicted value of the state of the bacterial flora in the oral cavity corresponding to the tongue image of the person being examined (step S201). The oral information generation means 22 generates information about the oral environment of the subject of examination based on predicted values ​​(step S202).

[0067] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above.

[0068] (Note 1) A first prediction means inputs the tongue image of the subject into an image learning model obtained by machine learning the relationship between tongue images taken from the tongue and the results of the analysis of the oral microbiota state, and obtains a first predicted value of the oral microbiota state corresponding to the tongue image of the subject. Oral information generation means that generates information about the oral environment of the subject of examination based on the first predicted value, A microbiome analysis device equipped with the following features.

[0069] (Note 2) The first prediction means inputs the tongue image of the person being examined into the image learning model obtained by machine learning the relationship between a tongue image taken of the tongue and the analysis result showing the ratio of beneficial bacteria in the oral microbiota, and obtains the first predicted value showing the ratio of beneficial bacteria corresponding to the tongue image of the person being examined. The oral information generation means generates information regarding the level of beneficial bacteria in the mouth of the subject of examination, based on the first predicted value which indicates the ratio of beneficial bacteria in the subject of examination. Microbial flora analysis device as described in Appendix 1.

[0070] (Note 3) The system includes a questionnaire learning model obtained by machine learning the relationship between the questionnaire results for multiple habitual items affecting the oral microbiota and the analysis results of the oral microbiota state, a second prediction means that inputs the questionnaire results of the person being tested and obtains a second predicted value of the oral microbiota state corresponding to the questionnaire results of the person being tested, The oral information generation means generates information regarding the level of beneficial bacteria in the mouth of the subject of the examination by referring to an index value and a threshold calculated based on the first predicted value and the second predicted value. Microbial flora analysis device as described in Appendix 2.

[0071] (Note 4) The aforementioned index value is calculated by the average or weighted average of the first predicted value and the second predicted value, and the threshold value is a value that optimizes the generation of the oral beneficial bacteria level. Microbial flora analysis device as described in Appendix 3.

[0072] (Note 5) The oral information generation means generates presentation information that promotes improvement of the oral environment in accordance with at least one of the indicator value and the questionnaire results. Microbial flora analysis device as described in Appendix 3 or Appendix 4.

[0073] (Note 6) The oral information generation means inputs at least one of the index value and the questionnaire results into a large-scale language model to generate presentation information that promotes improvement of the oral environment. A microbiome analysis device as described in any one of the items in Appendix 3 to Appendix 5.

[0074] (Note 7) A microbial flora analysis device according to any one of the appendices 1 to 6, comprising a shooting guidance means for guiding the capture of the aforementioned tongue image.

[0075] (Note 8) The tongue image of the subject is input into an image learning model obtained by machine learning the relationship between tongue images taken from the tongue and the results of the analysis of the oral microbiome state, and a first predicted value of the oral microbiome state corresponding to the subject's tongue image is obtained. Based on the aforementioned first predicted value, information regarding the oral environment of the subject of the examination is generated. Bacterial flora analysis method.

[0076] (Note 9) The tongue image of the subject is input into the image learning model obtained by machine learning the relationship between the tongue image taken from the tongue and the analysis results showing the ratio of beneficial bacteria in the oral microbiota, and the first predicted value showing the ratio of beneficial bacteria corresponding to the tongue image of the subject is obtained. Based on the first predicted value indicating the ratio of beneficial bacteria in the subject of the test, information regarding the oral beneficial bacteria level of the subject of the test is generated. The method for analyzing the microbial flora, as described in Appendix 8.

[0077] (Note 10) The system includes a questionnaire learning model obtained by machine learning the relationship between the questionnaire results for multiple habitual items affecting the oral microbiota and the analysis results of the oral microbiota state, a second prediction means that inputs the questionnaire results of the person being tested and obtains a second predicted value of the oral microbiota state corresponding to the questionnaire results of the person being tested, Based on the first and second predicted values, the system generates information regarding the level of beneficial bacteria in the mouth of the subject of the test by referring to the index value and threshold calculated based on those values. The method for analyzing the microbial flora as described in Appendix 9.

[0078] (Note 11) The aforementioned index value is calculated by the average or weighted average of the first predicted value and the second predicted value, and the threshold value is a value that optimizes the generation of the oral beneficial bacteria level. The method for analyzing the microbial flora, as described in Appendix 10.

[0079] (Note 12) The system generates presentation information that promotes improvement of the oral environment, corresponding to at least one of the aforementioned indicator values ​​and the results of the aforementioned questionnaire. The method for analyzing the microbial flora as described in Appendix 10 or Appendix 11.

[0080] (Note 13) At least one of the aforementioned indicator values ​​and the survey results is input into a large-scale language model to generate presentation information that promotes improvement of the oral environment. A method for analyzing the microbial flora as described in any one of the appendices 10 to 12.

[0081] (Note 14) A method for analyzing the microbial flora according to any one of the appendices 8 to 13, comprising a shooting guidance means for guiding the shooting of the aforementioned tongue image.

[0082] (Note 15) The computer for the microbiome analysis device, A first prediction means inputs the tongue image of the subject into an image learning model obtained by machine learning the relationship between tongue images taken from the tongue and the results of the analysis of the oral microbiota state, and obtains a first predicted value of the oral microbiota state corresponding to the tongue image of the subject. Oral information generation means that generates information about the oral environment of the subject of the examination based on the first predicted value, A program that makes it function as such.

[0083] (Note 16) The first prediction means inputs the tongue image of the person being examined into the image learning model obtained by machine learning the relationship between a tongue image taken of the tongue and the analysis result showing the ratio of beneficial bacteria in the oral microbiota, and obtains the first predicted value showing the ratio of beneficial bacteria corresponding to the tongue image of the person being examined. The oral information generation means generates information regarding the level of beneficial bacteria in the mouth of the subject of examination, based on the first predicted value which indicates the ratio of beneficial bacteria in the subject of examination. The program described in Appendix 15.

[0084] (Note 17) The survey learning model, obtained by machine learning the relationship between the survey results on multiple habitual items affecting the oral microbiota and the analysis results of the oral microbiota state, is used as a second prediction means to input the survey results of the subject being tested and to obtain a second predicted value of the oral microbiota state corresponding to the subject's survey results. The oral information generation means generates information regarding the level of beneficial bacteria in the mouth of the subject of the examination by referring to an index value and a threshold calculated based on the first predicted value and the second predicted value. The program described in Appendix 16.

[0085] (Note 18) The aforementioned index value is calculated by the average or weighted average of the first predicted value and the second predicted value, and the threshold value is a value that optimizes the generation of the oral beneficial bacteria level. The program described in Appendix 17.

[0086] (Note 19) The oral information generation means generates presentation information that promotes improvement of the oral environment in accordance with at least one of the indicator value and the questionnaire results. The program described in Appendix 17 or Appendix 18.

[0087] (Note 20) The oral information generation means inputs at least one of the index value and the questionnaire results into a large-scale language model to generate presentation information that promotes improvement of the oral environment. The program described in any one of the items in Appendix 17 to Appendix 19.

[0088] (Note 21) A program described in any one of the appendices 15 to 20 that functions as a shooting guidance means for guiding the acquisition of the aforementioned tongue image.

[0089] (Note 21) The tongue image of the subject is input into an image learning model obtained by machine learning the relationship between tongue images taken from the tongue and the results of the analysis of the oral microbiome state, and a first predicted value of the oral microbiome state corresponding to the subject's tongue image is obtained. Based on the aforementioned first predicted value, information regarding the oral environment of the subject of the examination is generated. Microbial flora analysis system.

[0090] (Note 22) The tongue image of the subject is input into the image learning model obtained by machine learning the relationship between the tongue image taken from the tongue and the analysis results showing the ratio of beneficial bacteria in the oral microbiota, and the first predicted value showing the ratio of beneficial bacteria corresponding to the tongue image of the subject is obtained. Based on the first predicted value indicating the ratio of beneficial bacteria in the subject of the test, information regarding the oral beneficial bacteria level of the subject of the test is generated. The microbiome analysis system described in Appendix 21.

[0091] (Note 23) The system includes a questionnaire learning model obtained by machine learning the relationship between the questionnaire results for multiple habitual items affecting the oral microbiota and the analysis results of the oral microbiota state, a second prediction means that inputs the questionnaire results of the person being tested and obtains a second predicted value of the oral microbiota state corresponding to the questionnaire results of the person being tested, Based on the first and second predicted values, the system generates information regarding the level of beneficial bacteria in the mouth of the subject of the test by referring to the index value and threshold calculated based on those values. The microbiome analysis system described in Appendix 22.

[0092] (Note 24) The aforementioned index value is calculated by the average or weighted average of the first predicted value and the second predicted value, and the threshold value is a value that optimizes the generation of the oral beneficial bacteria level. The microbiome analysis system described in Appendix 23.

[0093] (Note 25) The system generates presentation information that promotes improvement of the oral environment, corresponding to at least one of the aforementioned indicator values ​​and the results of the aforementioned questionnaire. Microbial flora analysis system as described in Appendix 23 or Appendix 24.

[0094] (Note 26) At least one of the aforementioned indicator values ​​and the survey results is input into a large-scale language model to generate presentation information that promotes improvement of the oral environment. A microbiome analysis system as described in any one of the items in Appendix 23 to Appendix 25.

[0095] (Note 27) A microbiota analysis system according to any one of the appendices 21 to 26, comprising a shooting guidance means for guiding the shooting of the aforementioned tongue image. [Explanation of Symbols]

[0096] 1... Bacterial flora analysis device 2. Terminal 11. Control Unit 12. Questionnaire Processing Section 13. Filming Information Section 14...Acquisition part 15. Learning Department 16. Prediction unit (prediction means 21) 17...Oral information generation unit (oral information generation means 22) 18. Output section

Claims

1. A first prediction means inputs the tongue image of the subject into an image learning model obtained by machine learning the relationship between tongue images taken from the tongue and the results of the analysis of the oral microbiota state, and obtains a first predicted value of the oral microbiota state corresponding to the tongue image of the subject. Oral information generation means that generates information about the oral environment of the subject of examination based on the first predicted value, A microbiome analysis device equipped with the following features.

2. The first prediction means inputs the tongue image of the person being examined into the image learning model obtained by machine learning the relationship between a tongue image taken of the tongue and the analysis result showing the ratio of beneficial bacteria in the oral microbiota, and obtains the first predicted value showing the ratio of beneficial bacteria corresponding to the tongue image of the person being examined. The oral information generation means generates information regarding the level of beneficial bacteria in the mouth of the subject of examination based on the first predicted value indicating the ratio of beneficial bacteria in the subject of examination, or an index value calculated based on the first predicted value. The microbiome analysis device according to claim 1.

3. The system includes a questionnaire learning model obtained by machine learning the relationship between the questionnaire results for multiple habitual items affecting the oral microbiota and the analysis results of the oral microbiota state, a second prediction means that inputs the questionnaire results of the person being tested and obtains a second predicted value of the oral microbiota state corresponding to the questionnaire results of the person being tested, The oral information generation means generates information regarding the level of beneficial bacteria in the mouth of the subject of the examination by referring to an index value and a threshold calculated based on the first predicted value and the second predicted value. The microbiome analysis device according to claim 2.

4. The aforementioned index value is calculated by the average or weighted average of the first predicted value and the second predicted value, and the threshold value is a value that optimizes the generation of the oral beneficial bacteria level. The microbiome analysis device according to claim 3.

5. The oral information generation means generates presentation information that promotes improvement of the oral environment in accordance with at least one of the indicator value and the questionnaire results. The microbiome analysis device according to claim 4.

6. The oral information generation means inputs at least one of the index value and the questionnaire results into a large-scale language model to generate presentation information that promotes improvement of the oral environment. The microbiome analysis device according to claim 4.

7. A microbiota analyzer according to any one of claims 1 to 6, further comprising a shooting guidance means for guiding the shooting of the tongue image.

8. The tongue image of the subject is input into an image learning model obtained by machine learning the relationship between tongue images taken from the tongue and the results of the analysis of the oral microbiota state, and a first predicted value of the oral microbiota state corresponding to the subject's tongue image is obtained. Based on the aforementioned first predicted value, information regarding the oral environment of the subject of the examination is generated. Bacterial flora analysis method.

9. The computer for the microbiome analysis device, A first prediction means inputs the tongue image of the subject into an image learning model obtained by machine learning the relationship between tongue images taken from the tongue and the results of the analysis of the oral microbiota state, and obtains a first predicted value of the oral microbiota state corresponding to the tongue image of the subject. Oral information generation means that generates information about the oral environment of the subject of the examination based on the first predicted value, A program that makes it function as such.

10. The tongue image of the subject is input into an image learning model obtained by machine learning the relationship between tongue images taken from the tongue and the results of the analysis of the oral microbiota state, and a first predicted value of the oral microbiota state corresponding to the subject's tongue image is obtained. Based on the aforementioned first predicted value, information regarding the oral environment of the subject of the examination is generated. Microbial flora analysis system.

Images