Odor measurement device, odor measurement method, and odor measurement system

Abstract

To provide an odor measurement device, an odor measurement method, and an odor measurement system capable of measuring odor more easily and with higher accuracy of odor measurement, as compared with the case where the configuration of the present invention is not used.SOLUTION: An odor measurement device includes a server 21 that acquires a measurement value obtained by measuring a plurality of specific gas components in exhalation by a multi-gas sensor 11 and a captured tongue body image, and a processing unit 22 that receives the measurement value and the tongue body image and determines an odor level and an odor factor in the exhalation by using a learning model.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an odor measurement device, an odor measurement method, and an odor measurement system, and more particularly to an odor measurement device, an odor measurement method, and an odor measurement system that measure odors using an odor sensor and a tongue body image. [Background technology]

[0002] In recent years, for example, when people are concerned about bad breath, there is a demand for a means to identify the cause of bad breath.

[0003] Patent Document 1 discloses an odor measurement device. This odor measurement device has a cylindrical case and a sensor provided so as to leave a space of a predetermined volume between the case and the sensor, a probe inserted into or close to the test area to sample and measure odor, and a main body electrically connected to the sensor and having a calculation means, a display means, a power source, etc.

[0004] Patent Document 2 discloses a health condition determination device that analyzes the exhaled gases emitted from a pet's breath, enabling early detection of internal organ diseases in the pet and identification of diseased areas. This health condition determination device includes a feeding device that provides meals including water and food to the pet, an exhaled air sampling unit that is provided near the feeding device and samples the exhaled gases of the pet, an exhaled air component detection unit that detects components contained in the exhaled gas sampled by the exhaled air sampling unit, and a health condition determination unit that determines the health condition of the pet based on the detection results by the exhaled air component detection unit.

[0005] Patent Document 3 discloses a tongue body image processing method, which includes an imaging step of capturing an image including the tongue body of a subject to be used to determine the state of the oral cavity of the subject, and a resolution setting step of setting the resolution of a still image to be used to determine the state of the oral cavity of the subject, wherein the pixel size of the resolution set in the resolution setting step is larger than the diameter of the filiform papillae of the tongue body included in the image captured in the imaging step. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-184265 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-220046 [Patent Document 3] Japanese Patent Application Publication No. 2020-58710 Summary of the Invention [Problem to be solved by the invention]

[0007] Conventionally, measuring odors requires equipment such as gas chromatography, which is difficult for individuals to use and takes a long time to perform. Furthermore, specialized knowledge is required for operating the equipment and conducting the analysis. Furthermore, simpler methods, such as using a gas sensor or estimating odors from tongue images, have the problem of low accuracy in odor measurement. The present invention aims to provide an odor measurement device, an odor measurement method, and an odor measurement system that can measure odors more easily and with higher accuracy than when the configuration of the present invention is not used. [Means for solving the problem]

[0008] To solve this problem, the present invention provides an odor measurement device that includes an acquisition unit that acquires measurement values ​​of multiple specific gas components in exhaled breath using an odor sensor and a photographed image of the tongue, and a determination unit that receives the measurement values ​​and the tongue image and determines the odor level and odor factors in the exhaled breath using a learning model.In this case, it is possible to provide an odor measurement device that can measure odors more easily and with higher accuracy than when the configuration of the present invention is not used.

[0009] Here, for example, the measured value further includes a value measured by a water vapor sensor that measures water vapor contained in exhaled breath. In this case, even if the odor sensor cannot measure water vapor, it is possible to measure water vapor and obtain important data for measuring odors. Furthermore, for example, the acquisition unit acquires measurement values ​​for at least volatile sulfur compounds, volatile organic compounds, ethanol, hydrogen, volatile nitrogen compounds, and water vapor, which can measure the main odor components that cause bad breath. Furthermore, for example, the determination unit performs a first determination process to determine a tongue body state estimation result indicating the state of the tongue body from the tongue body image using a learning model, and a second determination process to determine an odor level and odor factors from the tongue body state estimation result and measurement values ​​using the learning model. In this case, the tongue body state can be used to determine odor factors. Furthermore, for example, the determination unit further determines the estimation basis, which is the basis for estimating the odor level and odor factors in the second determination process. In this case, the estimation basis can also be estimated using a learning model. For example, in the second determination process, the determination unit uses a learning model that further inputs previously determined odor levels and time-series changes in odor factors, which further increases the accuracy of odor measurement. Furthermore, the determination unit may correct the newly determined odor level and odor factor based on previously determined odor levels and odor factors, thereby further increasing the accuracy of odor measurement. Furthermore, for cases where it is difficult to distinguish based on measurement values ​​alone, the determination unit may also combine tongue body images and make a determination using a learning model. In this case, odors that can be measured by the odor sensor are determined using the measurement results from the odor sensor, and if using tongue body images together is more advantageous for odor measurement, the accuracy of odor measurement can be improved by using the tongue body images. Furthermore, for example, when volatile sulfur compounds are contained in the exhaled breath, the determination unit also combines the tongue body image and performs determination using a learning model, which increases the accuracy of odor measurement when volatile sulfur compounds are contained in the exhaled breath. Furthermore, for example, the determination unit extracts medical features from the tongue body image and uses normalized data, which prevents deterioration of estimation accuracy and makes it easier to provide a logical explanation for the basis of estimation. For example, if the medical feature is an image of the tongue body, images other than the tongue body can be excluded.

[0010] To solve this problem, the present invention provides an odor measurement method in which a processor executes a program stored in memory to acquire measurement values ​​of multiple specific gas components in exhaled breath and a photographed image of the tongue, and the measurement values ​​and the tongue image are used as inputs to determine the odor level and odor factors in the exhaled breath using a learning model.In this case, compared to a case in which the configuration of the present invention is not used, an odor measurement method can be provided that can measure odors more easily and with higher accuracy.

[0011] Furthermore, to solve the above problems, the present invention provides an odor measurement system including an odor sensor that detects odor components contained in exhaled breath, an imaging device that images the tongue, and an odor measurement device that measures odors based on measurements of multiple specific gas components in the exhaled breath by the odor sensor and images of the tongue body captured by the imaging device, wherein the odor measurement device includes an acquisition unit that acquires the measurements and the tongue body image, and a determination unit that receives the measurements and the tongue body image and determines the odor level and odor factors in the exhaled breath using a learning model. In this case, an odor measurement system can be provided that can measure odors more easily and with higher accuracy than when the configuration of the present invention is not used. [Effects of the Invention]

[0012] According to the present invention, it is possible to provide an odor measurement device, an odor measurement method, and an odor measurement system that can measure odors more easily and with higher accuracy than when the configuration of the present invention is not used. [Brief explanation of the drawings]

[0013] [Figure 1] 1(a) and 1(b) are diagrams showing an example of a schematic configuration of an odor measurement system according to an embodiment of the present invention. [Figure 2] 1(a) is a diagram showing a schematic operation of the odor measurement system shown in Fig. 1(a), and FIG. 1(b) is a diagram showing a schematic operation of the odor measurement system shown in Fig. 1(b). [Figure 3] FIG. 10 is a diagram showing another mode of the schematic operation of the odor measurement system. [Figure 4] 10(a) and 10(b) are diagrams showing guidance that is displayed on a display when imaging the body of the tongue. [Figure 5] FIG. 2 is a conceptual diagram showing the correlation between the intensities of a plurality of gas sensors provided in a multi-gas sensor. [Figure 6] 10(a) and 10(b) are diagrams showing examples of output results of process A. [Figure 7] FIG. 10 is a conceptual diagram showing the state of the tongue body. [Figure 8] 10(a) and 10(b) are diagrams showing examples of measurement intervals. [Figure 9] FIG. 2 is a functional block diagram showing the functional configuration of a control unit and a processing unit. [Figure 10] 10 is a flowchart illustrating the operation of the odor measurement system. [Figure 11] FIG. 11 is a conceptual diagram of the process corresponding to FIG. 10. DETAILED DESCRIPTION OF THE INVENTION

[0014] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

[0015] <Overall explanation of odor measurement system 1> 1(a) and 1(b) are diagrams showing an example of the schematic configuration of an odor measurement system 1 according to this embodiment. In the odor measurement system 1 shown in FIG. 1( a ), a client-side device 10 and a server-side device 20 are connected via the Internet 30 . The client-side device 10 is a device installed on the side of a user who wishes to measure odors. The client-side device 10 may be installed in a home where the user lives, a care facility where the user is admitted, or the like.

[0016] In the client-side device 10, a multi-gas sensor 11, an image sensor 12, a display 13, and a storage area 14 are connected to a control unit 15. The control unit 15 is connected to a client machine 16 via wire or wirelessly. The connection method is not particularly limited, and examples of wired connection methods that can be used include a wired local area network (LAN), a universal serial bus (USB), and RS-232C. Examples of wireless connection methods that can be used include Wi-Fi (Wireless Fidelity, registered trademark), Bluetooth (registered trademark), ZigBee (registered trademark), and UWB (Ultra-Wide Band).

[0017] The multi-gas sensor 11 is an example of an odor sensor that detects odor components contained in exhaled breath, measures the odors contained in exhaled breath, and outputs the measurement values. In this case, the multi-gas sensor 11 measures components contained in exhaled breath that are the source of odors such as bad breath. In this case, the multi-gas sensor 11 can detect multiple gases, and thereby measure multiple types of odor molecules that are contained in exhaled breath and are the source of odors. The multi-gas sensor 11 is not particularly limited as long as it can detect odors. For example, the multi-gas sensor 11 may be a semiconductor-type odor sensor that detects odors by utilizing the change in resistance that occurs when a MOS (Metal Oxide Semiconductor) comes into contact with odor molecules. Alternatively, the multi-gas sensor 11 may be a quartz crystal-type odor sensor that includes a quartz crystal oscillator and a sensitive membrane and detects the presence and amount of odor molecules by detecting the change in the resonant frequency of the quartz crystal oscillator when odor molecules are adsorbed and desorbed to the sensitive membrane. Alternatively, the multi-gas sensor may use a CMOS (Complementary Metal Oxide Semiconductor) or a piezoelectric element instead of a quartz crystal oscillator. Furthermore, the multi-gas sensor may be an odor sensor that utilizes the concentration of odor molecules in molecular nanowires and detects odor molecules using chemoresistance or a field effect transistor (FET), or an odor sensor that uses a biological membrane mimicking human olfactory receptors and captures the changes that occur when odor molecules are adsorbed to the biological membrane with a camera.

[0018] The image sensor 12 photographs the tongue body and outputs a tongue body image, which is an image of the tongue body. The image sensor 12 is a camera or the like, and can also be said to be an imaging device that photographs the tongue body. Note that the tongue body image is an image of the tongue body, but it is preferable that the image is taken in a state where the upper surface of the tongue body and the tongue edge are visible. Furthermore, of the upper surface of the tongue body, the anterior part needs to be visible when photographed, but the posterior part does not necessarily need to be visible when photographed.

[0019] Display 13 is a display device such as a liquid crystal display or an organic EL display. Display 13 displays odor measurement guidance, odor measurement results, odor measurement history, etc. Display 13 may be a touch panel that displays operation buttons to accept input from the user.

[0020] Memory area 14 stores apps and history data used when measuring odors. Memory area 14 is, for example, a storage device such as an HDD (Hard Disk Drive) or SSD (Solid State Drive). Memory area 14 stores measurement values ​​from multi-gas sensor 11, image data of the tongue body from image sensor 12 (hereinafter, this may be referred to as "tongue body image data"), odor measurement results, odor measurement history, etc.

[0021] The control unit 15 controls the multi-gas sensor 11, the image sensor 12, the display 13, and the storage area 14. The control unit 15 is, for example, a control unit for controlling these components.

[0022] The client machine 16 acquires breath measurement data as measurement values ​​of multiple specific gas components in the breath from the multi-gas sensor 11 via the control unit 15. The client machine 16 also acquires tongue body image data from the image sensor 12. The client machine 16 then transmits this data to the server-side device 20. The client machine 16 is, for example, a computer device such as a PC (Personal Computer), a smartphone, a tablet, a mobile phone, etc. The client machine 16 can also display the odor measurement results, the odor measurement history, etc.

[0023] The server-side device 20 analyzes and measures odors based on the breath measurement data and tongue body image data acquired by the client-side device 10. The server-side device 20 then transmits the measurement results to the client-side device 10. The server-side device 20 includes a server 21, a processing unit 22, and a storage device 23.

[0024] The server 21 is an example of an acquisition unit, and acquires measurement values ​​of multiple specific gas components in exhaled breath measured by the multi-gas sensor 11 and tongue body images captured by the image sensor 12. Here, the server 21 is a server computer, but may also be a PC, smartphone, tablet, mobile phone, etc.

[0025] The processing unit 22 is an example of a determination unit, and receives as input the measurement values ​​of multiple specific gas components in the exhaled breath obtained by the multi-gas sensor 11 and the tongue body image captured by the image sensor 12, and determines the odor level and odor factors in the exhaled breath using a learning model. In other words, the processing unit 22 determines the odor level and odor factors in the exhaled breath using a learning model based on the exhaled breath measurement data and tongue body image data. The detailed method of this determination will be described later. The server 21 and processing unit 22 function as an odor measuring device that measures odors based on the measurement values ​​of multiple specific gas components in the exhaled breath measured by the multi-gas sensor 11 and the image of the tongue taken by the image sensor 12.

[0026] The storage device 23 stores the breath measurement data and tongue body image data acquired from the client-side device 10, the learning model, the processing results by the processing unit 22, the odor measurement history, and the like. The server 21 and the processing unit 22 may be integrated into one computer device. The server 21, the processing unit 22, and the storage device 23 may be integrated into one computer device.

[0027] FIG. 1(b) is a diagram showing another example of the schematic configuration of the odor measurement system 1. Compared to the client-side device 10 described in FIG. 1(a), the illustrated odor measurement system 1 does not include a control unit 15, and the image sensor 12, display 13, and memory area 14 are integrated into a client machine 16. In this case, the client machine 16 is, for example, a mobile terminal such as a smartphone, tablet, or mobile phone, and is configured to incorporate the image sensor 12, display 13, and memory area 14. In this case, the image sensor 12, such as a camera, provided in the mobile terminal captures an image of the tongue. Various information is displayed on the display 13 provided in the mobile terminal. Furthermore, various information is stored in the memory area 14 provided in the mobile terminal. The multi-gas sensor 11 is connected to and controlled by the mobile terminal. The multi-gas sensor 11 and client machine 16 are connected via wire or wirelessly. The method of connection is not particularly limited.

[0028] FIG. 2(a) is a diagram showing an outline of the operation of the odor measurement system 1 shown in FIG. 1(a). First, the multi-gas sensor 11 of the client-side device 10 measures the breath of the user, and the control unit 15 acquires breath measurement data as measurement values ​​of a plurality of specific gas components in the breath (1A). Furthermore, the image sensor 12 captures an image of the user's tongue, and the control unit 15 acquires tongue body image data as output data (1B). Then, the control unit 15 transmits the breath measurement data and the tongue body image data to the client machine 16 (1C), and the client machine 16 further transmits these data to the server-side device 20 (1D). In the server-side device 20, the server 21 receives these data, and the processing unit 22 measures the odor (1E). The measurement results are stored in the storage device 23. The measurement results are then sent to the client machine 16 (1F). The client machine 16 displays the odor measurement results.

[0029] FIG. 2(b) is a diagram showing the general operation of the odor measurement system 1 shown in FIG. 1(b). First, the multi-gas sensor 11 of the client-side device 10 measures the user's exhaled breath, and the client machine 16 acquires exhaled breath measurement data as measurement values ​​of multiple specific gas components in the exhaled breath (2A). The client machine 16 also uses the image sensor 12 to photograph the tongue of the user and acquire tongue body image data (2B). Then, the client machine 16 transmits the breath measurement data and tongue body image data to the server side device 20 (2C). In the server-side device 20, the server 21 receives these data, and the processing unit 22 measures the odor (2D). The measurement results are stored in the storage device 23. The measurement results are then sent to the client machine 16 (2E). The client machine 16 displays the odor measurement results on the display 13.

[0030] FIG. 3 is a diagram showing another mode of the schematic operation of the odor measurement system 1. In FIG. In this case, first, the control unit 15 etc. acquires measurement data (breath measurement data, tongue body image data) acquired from a user illustrated as an individual user (3A). Next, the control unit 15 and the like transmit the measurement data to the server-side device 20 (3B). In the server-side device 20, the server 21 receives these data, and the processing unit 22 measures the odor (3C). The measurement results are stored in the storage device 23. The measurement results are then sent to the client-side device 10 (3D). The client-side device 10 displays the odor measurement results to the caregiver. In this case, the odor measurement results are displayed, for example, on a mobile terminal carried by the caregiver.

[0031] <Explanation of how to obtain breath measurement data and tongue image data> Next, a method for acquiring breath measurement data and tongue body image data will be described in detail. The breath measurement data is acquired by measuring the breath. Breath measurement is performed by blowing the breath onto the multi-gas sensor 11. There are six types of odors to be measured: volatile sulfur compounds (VSCs), volatile organic compounds (VOCs), ethanol, hydrogen, volatile nitrogen compounds (VNCs), and water vapor. Examples of volatile sulfur compounds (VSCs) include hydrogen sulfide, methyl mercaptan, and dimethyl sulfide. Examples of volatile organic compounds (VOCs) include acetone, acetaldehyde, toluene, and isoprene. Examples of volatile nitrogen compounds (VNCs) include ammonia, dimethylamine, trimethylamine, indole, and skatole. It is preferable that the multi-gas sensor 11 be capable of measuring at least five types of gases excluding water vapor (humidity), and as mentioned above, the sensor type will not be mentioned.

[0032] The multi-gas sensor 11 may use, for example, multiple types of gas sensors, and the types of gases that can be measured by each gas sensor may overlap. If the multi-gas sensor 11 cannot measure water vapor, an additional water vapor sensor must be provided to measure the water vapor contained in the exhaled breath. Water vapor (humidity) may be measured using either a resistive or capacitive method. As long as the gas sensors are positioned and shaped so that they can all come into contact with the exhaled breath when the exhaled breath is blown into the sensor, the sensor placement, duct shape, and the presence or absence of fans for intake and exhaust are not important. A temperature sensor may also be installed to determine whether the exhaled breath is being measured.

[0033] When measuring the exhaled breath, the user is guided by instructions displayed on the display 13, voice instructions, etc. to blow the exhaled breath towards the intake port of the multi-gas sensor 11, and the exhaled breath is measured.

[0034] As described above, tongue body image data is acquired by capturing images using an image sensor 12 such as a camera. The image sensor 12 may be, for example, a visible light (RGB) sensor. To encourage the user to capture the face during tongue body image capture, the image sensor 12 preferably has a resolution of at least VGA (640 × 480, or approximately 310,000 pixels), and more preferably 1 million pixels or more. For the image sensor 12 in this embodiment, the number of pixels is more important, while color reproducibility, high image quality, and high-speed capture performance are less important. Therefore, any type of RGB pixel arrangement (e.g., Bayer arrangement), output bit count, and charge readout method (e.g., CCD (Charge Coupled Device), CMOS (Complementary Metal Oxide Semiconductor)) may be used. Furthermore, any shutter may be used, including but not limited to a global shutter or a rolling shutter.

[0035] 4(a) and 4(b) are diagrams showing guidance displayed on the display 13 when imaging the body of the tongue. Here, as shown in FIG. 4(a), images continuously acquired by the image sensor 12 are displayed as live video on the display 13. Guide lines for the contours of the face and tongue are also displayed superimposed on the image. In FIG. 4(a), these guide lines are shown as dotted lines. Then, a message "First, please align your face position" is displayed on the display 13, and the face and tongue are guided to the shooting position (Step 1).

[0036] When the face position is aligned, the message "Next, please extend your lower hand to the guide position" is displayed on the display 13 as shown in Figure 4(b) (Step 2). The image when it is determined that the tongue is positioned at the guide position is saved as tongue body image data. Note that the guide procedure may guide the user to align the face and tongue positions simultaneously, or may guide the user to align the face position first and then the tongue position.

[0037] The order of breath measurement and tongue body image capture may be reversed. However, since it is more desirable to minimize the time difference between image capture and breath measurement, it is preferable to guide the breath measurement and tongue body image capture to be performed consecutively without any time gap. <Detailed Description of Processing Unit 22> Next, the operation of the processing unit 22 will be described in detail. The processing unit 22 receives the breath measurement data and tongue body image data as inputs, and uses AI (Artificial Intelligence) to estimate the breath odor level and odor factors, and displays the estimation results and the basis for the estimation on the display 13 or the client machine 16. The AI's judgment factors are the intensity and correlation of each of the multiple gas sensors provided in the multi-gas sensor 11, and the tongue body state estimation results.

[0038] FIG. 5 is a conceptual diagram showing the intensities of the multiple gas sensors provided in the multi-gas sensor 11 and the correlations therebetween. Here, the multi-gas sensor 11 is equipped with six types of gas sensors (sensors 1 to 6), and the intensity of each gas sensor is shown. The intensity of the measured values ​​by these gas sensors and which gas sensor reacts vary depending on the odor components. Therefore, there is a relationship between the intensity and correlation of each gas sensor and the odor components, and the odor components can be estimated by looking at the intensity and correlation of each gas sensor.

[0039] The processing unit 22 performs the following processing A and processing B to output the above results.

[0040] (Process A) Process A takes breath measurement data acquired by the multiple gas sensors provided in the multi-gas sensor 11, the tongue body state estimation results described in the next process B, and the user's previously acquired data as inputs (explanatory variables), and directly outputs the breath odor level, odor factors, and estimation basis factors (objective variables). To realize this process A, supervised learning is used to learn the weighting coefficients of a neural network (hereinafter abbreviated as NN).

[0041] In process A, the breath odor level is the degree of odor strength. The breath odor level is output in three levels, for example, elementary, middle, and high school. Note that the odor level does not have to be three levels, and the degree of odor strength can also be expressed numerically. The odor factor is the cause of the odor, and the system outputs the most reliable candidate from a list of pre-prepared candidates. These candidates are diseases or external factors that manifest as bad breath. Specific examples include periodontal disease, tooth decay, dehydration, dry mouth, constipation, Helicobacter pylori, diabetes, oral bacteria, eating and drinking alcohol, hangovers, smoking, and internal organ diseases. The estimation basis is the basis for estimating the odor factor, and two with the highest confidence are output from pre-prepared candidates, such as odor intensity, change in odor intensity, tongue body condition, and change in tongue body condition.

[0042] 6(a) and 6(b) are diagrams showing examples of output results of process A. Here, the breath odor level, odor factors, and estimation grounds are displayed on the display 13 and the client machine 16. Figure 6(a) shows that the breath odor level is "high" and the odor factor is "food and drink." It also shows that the most reliable inference factor is "odor intensity," followed by "odor change." Figure 6(b) shows that the breath odor level is "moderate" and the odor factor is "visceral disease." It also shows that the most important inference factors are "odor components" and the second most important inference factor is "tongue condition."

[0043] (Process B) Process B uses tongue body image data acquired by the image sensor 12 as input (explanatory variables) and outputs a tongue body condition estimation result (objective variable). To achieve this process B, supervised learning is used to learn weighting coefficients for a convolutional neural network (CNN). Process B is a preprocessing step for process A and includes extracting and normalizing medical features related to the tongue body condition from the tongue body image data. The normalized data is then used to obtain a tongue body condition estimation result. The medical features are images of the tongue body extracted based on the tongue body image data. In other words, the images captured by the image sensor 12 include not only the tongue body but also the lips, gums, teeth, etc. Furthermore, since the accuracy of the learning model decreases if images other than the tongue body are included in the learning process, it is desirable to train only parts related to the tongue body. This is the process of extracting medical features, and the images of the tongue body are extracted as features.

[0044] If Process A is trained on image data as is without extracting medical features, there is a possibility that medically insignificant areas will be weighted. Furthermore, normalization is a process that corrects for the effects of ambient light and imaging angles. Without normalization, the effects of ambient light and differences in imaging angles may adversely affect the estimation accuracy of Process A. Therefore, by extracting and normalizing medical features, it is possible to prevent a deterioration in estimation accuracy and to facilitate a logical explanation of the basis for estimation.

[0045] In process B, the tongue body condition estimation result is obtained by using AI with tongue body image data as input to quantify (or level) the tongue body condition (average and variation in tongue body color, degree of adhesion on the tongue surface, presence or absence of cracks, presence or absence of injury (stomatitis), degree of unevenness of the tongue outline). In addition, the higher-level factors are presented as the basis for the estimation.

[0046] FIG. 7 is a conceptual diagram showing the state of the tongue body. Here, the condition of the tongue body is measured for the anterior part (upper surface) of the tongue body, including (1) color average and variation, (2) the degree of adhesion (tongue coating), and the presence or absence of cracks. Also, the tongue edge is measured for (4) the presence or absence of injury (stomatitis), and (5) the degree of unevenness. In other words, five items (1) to (5) are measured.

[0047] In process A, the AI-based breath odor level and odor factors are estimated and corrected based on the user's previous measurement results. In other words, the processing unit 22 corrects the newly determined odor level and odor factors based on the previously determined odor level and odor factors. For this purpose, a time-elapsed database is created for each user. It is desirable to measure periodically, for example, after waking up, before and after meals, etc. As an example of estimation correction using time, if there is little change in the tongue body condition compared to previous measurements but a large change in the gas sensor output intensity, it is estimated that an external factor such as eating or drinking is to blame. Conversely, if there is little change in the gas sensor output intensity compared to previous measurements but a large change in the tongue body condition (especially the color distribution and unevenness of the tongue edge), there is concern about dehydration. The time-elapsed database is constructed in the control unit 15 or the processing unit 22. Note that the capacity of the storage area is limited, so, for example, when the storage area becomes full, the oldest data is deleted. In addition, an embodiment is conceivable in which data that is an inferred result that the user finds interesting or an unusual inferred result is protected from being automatically deleted.

[0048] 8(a) and 8(b) are diagrams showing examples of measurement intervals. 8(a) shows an example of measuring every day. In this case, measurements are taken at approximately the same time every day, and changes in the measurement results from day to day can be ascertained. Figure 8(b) shows an example of more frequent measurement intervals. In this case, measurements are taken before a meal, immediately after a meal, several tens of minutes after a meal, and several hours after a meal, allowing changes at each time interval to be ascertained.

[0049] FIG. 9 is a functional block diagram showing the functional configuration of the control unit 15 and the processing unit 22. Here, the control unit 15 includes a gas data acquisition unit 151, an image data acquisition unit 152, and an acquisition guide control unit 153. The processing unit 22 also includes an odor level and odor factor estimation unit 221 and a tongue body feature extraction unit 222. The gas data acquisition unit 151 acquires breath measurement data from the multi-gas sensor 11. The image data acquisition unit 152 acquires tongue body image data from the image sensor 12 . The acquisition guide control unit 153 displays on the display 13 guidance for measuring breath with the multi-gas sensor 11. The acquisition guide control unit 153 also displays on the display 13 guidance for photographing an image of the tongue with the image sensor 12. The odor level and odor factor estimation unit 221 performs the above process A. The tongue body feature extraction unit 222 performs the above process B.

[0050] Fig. 10 is a flowchart illustrating the operation of the odor measurement system 1. Fig. 11 is a conceptual diagram of the processing corresponding to Fig. 10. Note that the operation of the measurement system 1 in Fig. 1(a) and Fig. 2(a) will be illustrated here. First, the acquisition guide control unit 153 of the control unit 15 displays on the display 13 a guide for measuring breath with the multi-gas sensor 11 (step 101). The user follows the instructions displayed on the display 13 and blows their exhaled breath toward the intake port of the multi-gas sensor 11, which then measures the exhaled breath (step 102). Next, the acquisition guide control unit 153 displays on the display 13 a guide for capturing an image of the tongue body using the image sensor 12 (step 103). The user adjusts the position of the face and tongue according to the instructions displayed on the display 13, and the image sensor 12 captures an image of the tongue (step 104).

[0051] The control unit 15 transmits the breath measurement data acquired by the multi-gas sensor 11, the tongue body image data acquired by the image sensor 12, and the previously acquired data acquired from the memory area 14 to the server 21, and the server 21 receives these data (step 105). Next, the tongue body feature extraction unit 222 of the processing unit 22 performs the above-mentioned process B. That is, the tongue body feature extraction unit 222 receives the tongue body image data as an input and converts the tongue body state into a numerical value (or level) using AI (step 106). Furthermore, the odor level and odor factor estimation unit 221 performs the above-mentioned process A (step 107). That is, the odor level and odor factor estimation unit 221 receives the breath measurement data, the tongue body state estimation result, and the user's previously acquired data as inputs, and outputs the breath odor level, odor factors, and estimation basis factors using AI. The breath odor level, odor factors, and estimation basis factors output by the odor level and odor factor estimation unit 221 are displayed on the client machine 16 via the server 21 (step 108).

[0052] Here, it can be said that the processing unit 22 performs a first judgment process (in this case, process A) that uses a learning model to judge the tongue body state estimation result indicating the state of the tongue body from the tongue body image, and a second judgment process (in this case, process B) that uses the learning model to judge the odor level and odor factors from the tongue body state estimation result and the measurement values ​​of the multi-gas sensor 11 and the water vapor sensor. It can also be said that the processing unit 22 further determines the estimation basis that is the basis for estimating the odor level and odor factors in the second determination process (process B in this case). Furthermore, it can be said that the processing unit 22 uses a learning model that further inputs the previously determined odor levels and time-series changes in odor factors in the second determination process (process B in this case).

[0053] Furthermore, for cases that are difficult to distinguish based solely on the strength and correlation of the multi-gas sensor 11 and the water vapor sensor, a tongue image captured by the image sensor 12 can also be combined and judged using AI. For example, when volatile sulfur compounds are present in the breath, it is possible that the cause is a disease in the oral cavity or digestive system, or that the cause is food or drink. However, by combining the tongue image, this can be estimated. In this case, the processing unit 22 can also be said to combine the tongue image and make a judgment using a learning model for cases that are difficult to distinguish based solely on the measurement values ​​of the multi-gas sensor 11 and the water vapor sensor. It can also be said that the processing unit 22 combines the tongue image and makes a judgment using a learning model when volatile sulfur compounds are present in the breath.

[0054] <Explanation of effect> In the embodiments described above, it is possible to provide an odor measurement device and an odor measurement system that can measure odors more easily and with higher accuracy than when the configuration of the present invention is not used.

[0055] In other words, since no equipment such as gas chromatography is used, odors can be measured more easily. Also, measurement time can be shortened, allowing individuals to easily and objectively identify the cause of bad breath.

[0056] Furthermore, this method can measure odors with higher accuracy than when an odor sensor or tongue image is used alone. In other words, when using only a tongue image, the actual odor components are unknown, and it is not possible to estimate the influence of factors such as eating and drinking or digestive disorders. Furthermore, when estimating odors based on the redness of the tongue, as in conventional techniques, the target odor components are volatile sulfur compounds produced by the metabolism of oral bacteria (cavity-causing bacteria and periodontal disease bacteria). However, it is not possible to estimate the odor components and levels of odors such as malodors caused by digestive disorders (hydrogen sulfide and ammonia), alcohol odors (ethanol) caused by drinking, or rotten fruit odors (acetone) caused by diabetes.

[0057] On the other hand, factors that cause sulfur odor, which is one of the causes of bad breath, include oral bacteria, eating and drinking, internal organ diseases, and nasopharyngeal diseases.Furthermore, factors that cause ammonia odor include smoking and internal organ diseases.It is difficult to distinguish between these factors using an odor sensor alone.

[0058] In other words, this embodiment makes it possible to measure odors that cannot be measured using an odor sensor or tongue image alone. Furthermore, by using both an odor sensor and a tongue image and a learning model, it is possible to measure the odor level in exhaled breath with high accuracy and also present odor factors and estimated basis factors. Furthermore, by using previously acquired data, which is data from past odor measurements, it is possible to understand changes over time and correct odor factors and estimated basis factors.

[0059] In the embodiment described above, the processing A and processing B are performed by the processing unit 22, but these functions can also be performed by the client-side device 10. Furthermore, the processing A and processing B can also be performed by separate devices.

[0060] <Explanation of odor measurement method> The above-described processes performed by the server-side device 20 are realized by the cooperation of software and hardware resources. That is, a processor such as a CPU provided in the server-side device 20 loads into a main memory a program that realizes each function of the server-side device 20, executes the program, and realizes each of the above-described functions. Therefore, the processing performed by the server-side device 20 described above can be considered to be an odor measurement method in which a processor executes a program stored in memory to obtain measurement values ​​of multiple specific gas components in exhaled breath and a photographed image of the tongue using the multi-gas sensor 11, and then uses the measurement values ​​and the tongue image as inputs to determine the odor level and odor factors in the exhaled breath using a learning model. This makes it possible to provide an odor measurement method that can measure odors more easily and with higher accuracy than when the configuration of the present invention is not used.

[0061] The program running on the server-side device 20 can be considered to be a program for causing a computer to perform the following functions: acquire measurement values ​​of multiple specific gas components in exhaled breath using the multi-gas sensor 11 and a photographed image of the tongue; and use the measurement values ​​and the tongue image as inputs to determine the odor level and odor factors in the exhaled breath using a learning model. This allows the computer to more easily measure odors and achieve functions that increase the accuracy of odor measurement compared to when the configuration of the present invention is not used.

[0062] The program for realizing this embodiment can be provided not only by communication means but also by being stored on a recording medium such as a CD-ROM.

[0063] Although the present embodiment has been described above, the technical scope of the present invention is not limited to the scope described in the above embodiment. It is clear from the claims that various modifications and improvements to the above embodiment are also included in the technical scope of the present invention. [Explanation of symbols]

[0064] 1...Odor measurement system, 10...Client-side device, 11...Multi-gas sensor, 12...Image sensor, 13...Display, 14...Memory area, 15...Control unit, 20...Server-side device, 21...Server, 22...Processing unit, 23...Memory device, 151...Gas data acquisition unit, 152...Image data acquisition unit, 221...Odor level and odor factor estimation unit, 222...Tongue body feature extraction unit

Claims

1. an acquisition unit that acquires measurement values ​​obtained by measuring a plurality of specific gas components in exhaled breath using an odor sensor and an image of the tongue body; a determination unit that receives the measurement values ​​and the tongue body image as inputs and determines the odor level and odor factors in the exhaled breath using a learning model; An odor measuring device comprising:

2. The odor measuring device according to claim 1 , wherein the measured values ​​further include values ​​obtained by a water vapor sensor that measures water vapor contained in exhaled breath.

3. The odor measurement device according to claim 2 , wherein the acquisition unit acquires measurement values ​​of at least volatile sulfur compounds, volatile organic compounds, ethanol, hydrogen, volatile nitrogen compounds, and water vapor as measurement targets.

4. The odor measuring device of claim 1, wherein the determination unit performs a first determination process to determine a tongue body state estimation result indicating the state of the tongue body from the tongue body image using a learning model, and a second determination process to determine the odor level and the odor factors from the tongue body state estimation result and the measurement value using a learning model.

5. The odor measuring device according to claim 4 , wherein the determining unit further determines an estimation basis that is a basis for estimating the odor level and the odor factor in the second determination process.

6. The odor measuring device according to claim 5 , wherein the determining unit uses a learning model that further receives inputs of the previously determined odor level and time-series changes in the odor factors in the second determination process.

7. The odor measuring device according to claim 6 , wherein the determining unit corrects the newly determined odor level and the odor factor based on the previously determined odor level and the odor factor.

8. The odor measuring device according to claim 1 , wherein the determining unit also combines the tongue body image and uses the learning model to make a determination for cases that are difficult to identify using only the measurement value.

9. The odor measuring device according to claim 8 , wherein the determination unit also combines the tongue body image and makes a determination using the learning model when a volatile sulfur compound is contained in the exhaled breath.

10. The odor measuring device according to claim 1 , wherein the determining unit extracts medical features from the tongue body image and uses normalized data.

11. The odor measuring device according to claim 10 , wherein the medical feature is an image of the tongue body.

12. The processor executes the program stored in the memory. The odor sensor measures multiple specific gas components in the exhaled breath and acquires images of the tongue. The measurement values ​​and the tongue body image are input, and an odor level and odor factors in the exhaled breath are determined using a learning model. Odor measurement method.

13. an odor sensor that detects odor components contained in exhaled breath; an imaging device for imaging the tongue body; an odor measuring device that measures odors based on measurement values ​​obtained by measuring a plurality of specific gas components in the exhaled breath using the odor sensor and on tongue body images taken by the imaging device; Including, The odor measuring device is an acquisition unit that acquires the measurement values ​​and the tongue body image; a determination unit that receives the measurement values ​​and the tongue body image as inputs and determines the odor level and odor factors in the exhaled breath using a learning model; An odor measurement system comprising:

Images