Method of measuring stress condition based on tongue image, information processing apparatus, and information processing program

Abstract

To provide a technique capable of easily measuring a stress condition.SOLUTION: A method for measuring a stress condition based on a tongue image includes the steps of: acquiring a feature amount of a tongue of a measured person extracted from a captured image of the tongue of the measured person; and measuring the stress condition of the measured person from the feature amount of the tongue by using a leaned model having learned so as to output the stress condition of the measured person measured in advance when the feature amount of the tongue extracted from the captured image of the tongue of the measured person is input.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a method for measuring a stress state based on a tongue image, an information processing device, and an information processing program. [Background technology]

[0002] There are known methods for determining the stress state of a subject. For example, Patent Document 1 discloses a method for determining the stress state of a subject based on electroencephalographic signals acquired by attaching sensors to a plurality of different positions on the head of the subject. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-118908 Summary of the Invention [Problem to be solved by the invention]

[0004] The method described in Patent Document 1 requires obtaining brain potential signals from a sensor attached to the subject. However, such a method requires advanced techniques and knowledge, and is difficult for unskilled persons to carry out.

[0005] The present invention has been made in view of the above problems, and has an object to provide a technique that enables easy measurement of stress state. [Means for solving the problem]

[0006] One invention for achieving the above-mentioned object is a method for measuring a stress state based on a tongue image, comprising the steps of: acquiring tongue features of a subject extracted from an image of the tongue of the subject; and measuring the stress state of the subject from the tongue features using a trained model that has been trained to output a previously measured stress state of the subject when the tongue features extracted from an image of the tongue of the subject are input. Effect of the Invention

[0007] According to the present invention, a technique that enables a stress state to be easily measured can be provided. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram for explaining an overview of an information processing system 1 according to an embodiment. [Diagram 2] FIG. 2 illustrates an example of a hardware configuration of an information processing device 3 according to an embodiment. [Diagram 3] FIG. 2 is a diagram for explaining learning data used in the embodiment. [Figure 4] This is a diagram to explain the tip, sides and center of the tongue. [Diagram 5] FIG. 13 is a diagram illustrating feature amounts relating to shape. [Figure 6] FIG. 2 is a diagram illustrating functional blocks of an information processing device 3 according to an embodiment. [Figure 7] 1 is a flowchart illustrating a process executed by the information processing system 1 according to the embodiment. [Figure 8] FIG. 1 is a diagram illustrating validation by 3-fold cross validation. [Figure 9] FIG. 1 shows the results of 3-fold cross-validation. [Figure 10] FIG. 13 shows the measurement performance results when LGBM is used. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] == Implementation form == <<<Information Processing System 1>>> The information processing system 1 is a system that captures an image of the tongue of a person to be measured, and measures the stress state of the person to be measured based on the captured image of the tongue.

[0010] 1 is a diagram for explaining an overview of an information processing system 1 of this embodiment. The information processing system 1 includes an imaging device 2 and an information processing device 3. The imaging device 2 and the information processing device 3 are connected via a communication network NW such as the Internet. The information processing device 3 corresponds to a "computer."

[0011] <<Imaging device 2>> The imaging device 2 is a device for capturing an image P of the tongue of the subject. There are no particular limitations on the imaging device 2 as long as it can capture an image that allows observation of the state (color, shape, texture, etc.) of the tongue of the subject. The image P captured by the imaging device corresponds to a "captured image."

[0012] As shown in FIG. 1, an image P of the tongue captured by the imaging device 2 is transmitted to an information processing device 3 (to be described later) via a communication network NW.

[0013] As the imaging device 2, for example, a tongue image capturing system (TIAS: Tongue Image Analyzing System) manufactured by Takano Co., Ltd. can be used. The TIAS is a device that uses an integrating sphere to eliminate the influence of external light and can capture an image of the tongue by irradiating the tongue with a uniform amount of light.

[0014] <<Information processing device 3>> The information processing device 3 is a device that acquires an image P of the tongue captured by the imaging device 2 and measures the stress state of the subject based on the tongue image P. When the information processing device 3 measures the stress state of the subject, a trained model (described in detail later) generated in advance is used.

[0015] In the following, the hardware configuration of the information processing device 3, the trained model used by the information processing device 3, and the functional blocks of the information processing device 3 will be described in that order.

[0016] <Hardware configuration of information processing device 3> 2 is a diagram showing an example of a hardware configuration of the information processing device 3. The information processing device 3 includes a processor 301, a main storage device 302, an auxiliary storage device 303, an input device 304, an output device 305, and a communication device 306.

[0017] In addition, the information processing device 3 may be realized, in whole or in part, using virtual information processing resources provided using virtualization technology, process space isolation technology, etc., such as a virtual server provided by a cloud system.

[0018] In addition, all or part of the functions provided by the information processing device 3 may be realized by, for example, a service provided by a cloud system via an API (Application Programming Interface) or the like. In addition, the information processing system 1 may include a plurality of information processing devices 3 that are communicatively connected.

[0019] The processor 301 is configured using, for example, a Central Processing Unit (CPU), a Micro Processing Unit (MPU), a Graphics Processing Unit (GPU), a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Artificial Intelligence (AI) chip, etc.

[0020] The main memory device 302 is a device that stores programs and data, and is, for example, a read only memory (ROM), a random access memory (RAM), or a non-volatile memory (NVRAM (Non Volatile RAM)).

[0021] The auxiliary storage device 303 is, for example, an SSD (Solid State Drive), a hard disk drive, an optical storage device (CD (Compact Disc), DVD (Digital Versatile Disc), etc.), a storage system, an IC card, a read / write device for recording media such as an SD card or an optical recording medium, a memory area of ​​a cloud server, etc.

[0022] Programs and data can be read into the auxiliary storage device 303 via a recording medium reading device or a communication device 306. The programs and data stored (memorized) in the auxiliary storage device 303 are read into the main storage device 302 as needed.

[0023] The programs and data here include a program (corresponding to an "information processing program") for analyzing images of the tongue and extracting features (described below), a trained model for measuring the stress state of the person being measured, etc.

[0024] The input device 304 is an interface that accepts input from the outside, and is, for example, a keyboard, a mouse, a touch panel, a card reader, a pen-input tablet, a voice input device, or the like.

[0025] The output device 305 is an interface that outputs various information such as the process progress, the process results, etc. The output device 305 is, for example, a display device (liquid crystal monitor, LCD (Liquid Crystal Display), graphic card, etc.) that visualizes the various information described above, a device that converts the various information described above into voice (voice output device (speaker, etc.)), or a device that converts the various information described above into text (printer, etc.).

[0026] The input device 304 and the output device 305 constitute a user interface that receives information from the user and presents information.

[0027] The communication device 306 is a device that realizes communication with other devices. The communication device 306 is a wired or wireless communication interface that realizes communication with other devices via a communication network such as the Internet, and is, for example, a NIC (Network Interface Card), a wireless communication module, a USB module, etc. In this embodiment, the information processing device 3 can input and output information to and from the imaging device 2 via the communication device 306.

[0028] In the information processing device 3, for example, an operating system, a file system, a DBMS (DataBase Management System) (relational database, NoSQL, etc.), a KVS (Key-Value Store), etc. may be installed.

[0029] Each function of the information processing system 1, which will be described later, is realized by the processor 301 of the information processing device 3 reading and executing a program stored in the main storage device 302, or by hardware (FPGA, ASIC, AI chip, etc.) constituting the information processing system 1. The information processing system 1 stores the various information (data) described above, for example, as a table of a database or a file managed by a file system.

[0030] <Pre-trained model> The trained model is a machine learning model that has been trained to output the subject's pre-measured stress state when tongue features extracted from an image of the subject's tongue are input.

[0031] To create a trained model, training data is generated that includes multiple tongue images taken in advance and the stress state of the subjects corresponding to the tongue images (details will be described later). That is, to construct a trained model, multiple subjects are recruited and images of each of their tongues are obtained. Furthermore, the stress state of each of the multiple subjects is measured.

[0032] Below, we will first explain the details of the training data and the process of generating the training data, and then explain the construction of the trained model.

[0033] [Learn data details] 3 is a diagram for explaining the learning data 4 used when constructing the trained model of the present embodiment. The learning data 4 includes input data 4a and correct answer data 4b. FIG. 3 shows the input data 4a and the correct answer data 4b acquired from each of a plurality of subjects.

[0034] The input data 4a is data indicating features of the tongue of the subject, including features related to the shape, the color, and the texture (described in detail below).

[0035] The correct answer data 4b is data indicating the stress state of the subject. In this embodiment, the "stress state" is classified into two stages, "high" and "low." In the following, the "high" stress state may be referred to as "high stress" and the "low" stress state may be referred to as "low stress."

[0036] The stress state of the subject in the correct answer data 4b is not limited to two stages. The stress state of the subject may be indicated in multiple stages of three or more stages, or may be indicated by continuous numerical values ​​within a predetermined range.

[0037] [Learning data generation process] To generate the learning data, a number of subjects are recruited and a test is conducted to obtain images of the tongue and stress state of each of the subjects.

[0038] 3, the tongue features according to the present embodiment include features related to shape, color, and texture. These tongue features are extracted from an image of the tongue of the subject.

[0039] Furthermore, the color feature and the texture feature include feature amounts in each of the predetermined divided ranges of the tongue surface. In this embodiment, the color feature and the texture feature include feature amounts in each of the three divided ranges of the tongue surface (the tongue tip, the tongue side, and the tongue center). Note that the predetermined ranges include the tongue tip, the tongue side, and the tongue center in this embodiment, but are not limited thereto.

[0040] Fig. 4 is a diagram for explaining the tip, edge, and center of the surface of the tongue T. Fig. 4 shows an area divided using a method called the Chiu method (Chiu, C C. "A novel approach based on computerized image analysis for traditional Chinese medical diagnosis of the tongue." Computer methods and programs in biomedicine vol. 61, 2: 77-89).

[0041] According to the Chiu method, the surface area of ​​the tongue T is divided into the tongue tip, tongue border and tongue based on a predetermined fixed ratio shown in FIG.

[0042] The method of dividing the tongue surface is not limited to the Chiu method. Other examples include the extended Chiu method and the five-point method (Setsuo Ogasawara, "Introduction to Population Geography", Hara Shobo, 1999).

[0043] The procedure for extracting each feature amount will be described below in the order of feature amount related to shape, feature amount related to color, and feature amount related to texture.

[0044] Shape features The feature amount related to the shape is an amount indicating the feature of the shape of the tongue. In this embodiment, the feature amount related to the shape includes seven feature amounts, namely, the tongue area, the tongue length, the tongue width, the upper left coordinates (x and y coordinates), and the lower right coordinates (x and y coordinates).

[0045] Fig. 5 is a diagram for explaining shape-related features according to the present embodiment. In Fig. 5, the area S of the tongue, the length L of the tongue, the width W of the tongue, the upper left coordinate P1 (x1, y1), and the lower right coordinate P2 (x2, y2) are shown.

[0046] The x1 and y1 values ​​of the upper left coordinate P1 are the minimum x-coordinate and maximum y-coordinate of the area occupied by the tongue T in the tongue image, and the x2 and y2 values ​​of the lower right coordinate P2 are the maximum x-coordinate and minimum y-coordinate of the area occupied by the tongue T in the tongue image.

[0047] In this embodiment, the seven feature amounts described above are used as feature amounts related to shape, but the present invention is not limited to this. At least one of the seven feature amounts described above may be used as a feature amount related to shape.

[0048] Color-related features The color feature quantity is a quantity indicating the color feature of the tongue. In this embodiment, the color feature quantity is a statistic of colors in a plurality of pixels in a plurality of predetermined ranges of the tongue surface (in this embodiment, the tip, middle, and sides of the tongue).

[0049] Here, "color" refers to color coordinates expressed in a predetermined color space. In this embodiment, the CIEL*a*b* color space is used as the color space.

[0050] The CIEL*a*b* color space is a color space expressed by three color coordinates: lightness (L*), position between red and green (a*), and position between yellow and blue (b*).

[0051] In addition, the "statistics" include seven statistics: maximum, median, mean, minimum, variance, skewness, and kurtosis. However, the statistics are not limited to these, and at least one of these seven statistics may be used as the color statistics.

[0052] Therefore, in this embodiment, the color feature amount is composed of 63 (=3×3×7) statistics.

[0053] Of these 63 statistics, for example, the “maximum value” of “L*” (brightness) for the “tongue tip” is the maximum value of the L* (brightness) of all pixels located within the range of the tongue tip in the image of the subject's tongue.

[0054] Also, for example, the "average value" of "L*" (brightness) of the "tongue tip" is the average value of L* (brightness) of all pixels located within the range of the tongue tip in the image of the subject's tongue.

[0055] In the present embodiment, the color coordinates of all pixels located in the range of the tongue tip, middle, or edge are used when extracting color statistics, but this is not limited to this. It is also possible to use the color coordinates of some pixels located in the range of the tongue tip, middle, or edge. For example, a specific point on the tongue tip, middle, or edge may be set as a representative point, and the color coordinates of multiple pixels located within a range close to the representative point may be used.

[0056] Texture features The texture feature is an amount indicating the texture feature of the tongue surface. In this embodiment, the texture feature includes a feature based on a Gray Level Co-occurrence Matrix (GLCM) in each of a plurality of predetermined ranges of the tongue surface (in this embodiment, the tip, middle, and edge of the tongue surface).

[0057] The feature quantities based on the gray level co-occurrence matrix include six features: contrast, non-uniformity, uniformity, uniformity, energy, and correlation. Note that the feature quantities based on the gray level co-occurrence matrix are not limited to these, and at least one of these six feature quantities may be used as the feature quantity based on the gray level co-occurrence matrix.

[0058] Therefore, in this embodiment, the texture feature amount is made up of 18 (=3×6) statistics.

[0059] Of these 18 statistics, for example, the "contrast" for "tongue tip" is a contrast based on a gray-level co-occurrence matrix extracted from pixels located within the tongue tip area in the image of the subject's tongue.

[0060] The tongue features explained above are composed of a total of 88 features (7 + 63 + 18). Figure 3 omits some of these.

[0061] In this embodiment, three features of the tongue have been described: a feature related to shape, a feature related to color, and a feature related to texture. However, it is not necessary to include all three of these features; it is sufficient to include at least one of them.

[0062] Subject's stress state The stress state of the subject is measured based on a predetermined questionnaire. The predetermined questionnaire is created to measure the stress state. The predetermined questionnaire in this embodiment uses the "Simple Occupational Stress Questionnaire (simplified version, 23 items)" (hereinafter simply referred to as the "questionnaire") created by the Ministry of Health, Labor and Welfare.

[0063] According to the questionnaire, it is possible to measure the stress state of the subject based on the responses to the questionnaire from the subject and the points allocated to the questionnaire. This measurement method is hereinafter referred to as the "method using the points allocated to the questionnaire."

[0064] It is also possible to measure the stress state based on the subjects' responses to the questionnaire and a raw score conversion table prepared by the Ministry of Health, Labor and Welfare. This measurement method is hereinafter referred to as the "method using a raw score conversion table."

[0065] In this embodiment, the stress state of the subject is measured using a method that uses the points allocated to the questionnaire and a method that uses a raw score conversion table (details will be described later).

[0066] The subject's stress state is measured based on the subject's responses to a predetermined questionnaire.

[0067] The method of measuring the stress state of the subject is not limited to a questionnaire. For example, the stress state of the subject may be measured based on counseling for the subject, or may be measured using a medical device.

[0068] Using the method described above, tongue features can be obtained from each of multiple subjects and their stress levels can be measured, thereby obtaining specific numerical values ​​and stress levels ("high" or "low") for each subject in the learning data 4 shown in Figure 3.

[0069] [Building a trained model] The trained model is constructed by training a machine learning model with training data. As the machine learning model, a machine learning model based on supervised learning can be used.

[0070] There are no particular limitations on the machine learning model based on supervised learning, but examples of machine learning models that can be used include Random Forest (RF), Light Gradient Boost Machine (LGBM), and Extreme Gradient Boosting (XGB).

[0071] <Functional blocks of information processing device 3> 6 is a diagram illustrating functional blocks of the information processing device 3. The information processing device includes an image acquisition unit 311, a feature amount extraction unit 312, a feature amount acquisition unit 313, a measurement unit 314, and an output unit 315.

[0072] [Image Acquisition Unit 311] The image acquiring unit 311 acquires an image of the tongue of the person to be measured. The image of the tongue is an image captured by the above-mentioned imaging device 2. The image acquiring unit 311 acquires the image from the imaging device 2 via the communication device 306.

[0073] [Feature extraction unit 312] The feature extraction unit 312 extracts tongue features from the tongue image of the subject using a predetermined image analysis program. The tongue features according to this embodiment correspond to the input data 4a in the learning data 4 shown in Fig. 3. That is, the tongue features according to this embodiment are composed of 88 features as described above.

[0074] The feature extraction unit 312 performs image analysis on the tongue image of the subject and extracts a predetermined number of tongue features (88 in this embodiment). The method by which the feature extraction unit 312 extracts the tongue features is the same as the method used to extract the tongue features from the tongue image of the subject in generating the learning data 4 described above.

[0075] [Feature acquisition unit 313] The feature amount acquiring section 313 acquires the feature amount of the tongue of the subject extracted from the image of the tongue of the subject. In this embodiment, the feature amount acquiring section 313 acquires the feature amount of the tongue of the subject from the feature amount extracting section 312.

[0076] As will be described later, the image acquiring unit 311 and the feature extracting unit 312 described above do not necessarily have to be provided in the information processing device 3, and may be provided in another computer. In this case, the feature acquiring unit 313 acquires the tongue feature from the other computer via a communication network such as the Internet.

[0077] [Measurement section 314] The measurement unit 314 measures the stress state of the subject from the tongue feature amount using the trained model. The measurement unit 314 inputs the tongue feature amount acquired by the feature amount acquisition unit 313 into the trained model, and outputs the measurement result of the stress state of the subject.

[0078] [Output section 315] The output unit 315 outputs the stress state measurement results obtained by the measurement unit 314 via the output device 305 (FIG. 2) of the information processing device 3. The output device 305 shows the measurement results to the person being measured by displaying the measurement results on a display device such as a liquid crystal monitor or by vocalizing the measurement results using an audio output device such as a speaker.

[0079] Although the information processing device 3 of the present embodiment has been described above, this is merely an example and the present invention is not limited to this.

[0080] For example, in this embodiment, the information processing device 3 is provided with a feature extraction unit 312 for extracting tongue features from an image of the tongue of the subject. Meanwhile, the imaging device 2 may be provided with a configuration equivalent to the feature extraction unit 312. In this case, the feature acquisition unit 313 acquires the tongue features from the imaging device 2. Alternatively, a configuration equivalent to the feature extraction unit 312 may be provided in another device.

[0081] The information processing system 1 of the present embodiment may be provided in a mobile terminal such as a smartphone, a tablet terminal, or a smartwatch. In this case, an application program for realizing the functions of the information processing device 3 of the present embodiment is installed in the mobile terminal. As the imaging device 2, a camera built into the mobile terminal may be used.

[0082] When a user of a mobile device activates the camera of the mobile device and takes a picture of their tongue, the mobile device acquires an image of the tongue. The mobile device then extracts features using the image of the tongue and measures the stress state using a trained model.

[0083] The mobile device can present the stress state measurement results to the user by displaying the measurement results on a display.

[0084] <<Processing Executed by Information Processing System 1>> The process from when the information processing system 1 captures an image of the tongue of the subject to when it outputs the stress state of the subject will be described with reference to a flowchart. Fig. 7 is a flowchart explaining the process executed by the information processing system 1. In this example, it is assumed that a trained model using the above-mentioned training data has been constructed in advance.

[0085] First, in step S101, the imaging device 2 captures an image of the tongue of the subject.

[0086] Next, in step S102, the image acquisition unit 311 of the information processing device 3 acquires the image captured in step S101.

[0087] Next, in step S103, the feature extraction unit 312 performs image analysis on the image acquired in step S102, and extracts features of the tongue.

[0088] Next, in step S104, the feature amount acquiring unit 313 acquires the feature amount of the tongue extracted in step S103.

[0089] Next, in step S105, the measurement unit 314 measures the stress state of the subject based on the tongue feature amount acquired in step S104. At this time, the measurement unit 314 inputs the tongue feature amount to the trained model, and sets the output result of the trained model as the measurement result.

[0090] Finally, in step S106, the output section 315 displays the measurement result in step S105 to the person being measured via the output device 305.

[0091] ==Summary== The method for measuring a stress state based on a tongue image of the embodiment described above includes the steps of acquiring tongue features of the subject extracted from an image of the tongue of the subject, and measuring the stress state of the subject from the tongue features using a trained model that has been trained to output a previously measured stress state of the subject when the tongue features extracted from the image of the tongue of the subject are input.

[0092] According to such a method, the stress state of the person being measured can be measured from an image of the tongue of the person being measured captured by the imaging device 2, using a trained model that uses the correlation between the tongue feature amount and the stress state. Also, since the person being measured only needs to capture an image of the tongue with the imaging device 2, the method is simple and less burdensome. That is, according to the information processing method of the embodiment, the stress state of the person being measured can be measured by a simple method.

[0093] In the method for measuring a stress state based on a tongue image according to the embodiment, the feature amount of the tongue of the subject includes at least one of a feature amount related to shape, a feature amount related to color, and a feature amount related to texture. According to this method, the feature amount of the tongue of the subject can be extracted with high accuracy. Therefore, the accuracy of the measurement result of the stress state is improved.

[0094] In the method for measuring a stress state based on a tongue image according to the embodiment, the shape-related feature amount includes at least one of the tongue area, tongue length, and tongue width. According to this method, the shape-related feature amount can be extracted with high accuracy. Therefore, the accuracy of the stress state measurement result is further improved.

[0095] In the method for measuring a stress state based on a tongue image of the embodiment, the color feature amount is a color statistic of a plurality of pixels at a predetermined location on the tongue surface, and the color is a color coordinate expressed in a predetermined color space. According to this method, the color feature amount can be extracted with high accuracy. Therefore, the accuracy of the stress state measurement result is further improved.

[0096] In the method for measuring a stress state based on a tongue image according to the embodiment, the texture features include features based on a gray level co-occurrence matrix at a predetermined location on the tongue surface. This method allows the texture features to be extracted with high accuracy, thereby further improving the accuracy of the stress state measurement result.

[0097] In the method for measuring a stress state based on a tongue image according to the embodiment, the trained model is a machine learning model based on any one of Random Forest, LGBM, and XGB. According to such a method, it is possible to obtain a measurement result of a stress state based on a tongue image with higher accuracy than a machine learning model using other algorithms.

[0098] The method for measuring a stress state based on a tongue image of the embodiment further includes a step of acquiring an image of the tongue of the subject, and a step of extracting a feature amount of the tongue from the image of the tongue of the subject. According to this method, the process from acquiring the image of the tongue to measuring the stress state can be executed by a single information processing device 3.

[0099] In addition, the information processing device 3 of the embodiment is equipped with a feature acquisition unit 313 that acquires features of the tongue of the subject extracted from an image of the tongue, and a measurement unit 314 that measures the stress state of the subject from the tongue features using a trained model that has been trained to output a previously measured stress state of the subject when the tongue features extracted from an image of the tongue of the subject are input.

[0100] According to such a configuration, the stress state of the person being measured can be measured from an image of the tongue of the person being measured captured by the imaging device 2, using a trained model that uses the correlation between the tongue feature amount and the stress state. Also, since the person being measured only needs to capture an image of the tongue with the imaging device 2, the method is simple and less burdensome. That is, according to the information processing method of the embodiment, the stress state of the person being measured can be measured by a simple method.

[0101] In addition, the information processing program of the embodiment causes a computer to perform the following processes: acquiring tongue features of the subject extracted from an image of the tongue; and measuring the stress state of the subject from the tongue features using a trained model that has been trained to output a previously measured stress state of the subject when the tongue features extracted from the image of the tongue of the subject are input.

[0102] According to such a program, the stress state of the person being measured can be measured from an image of the tongue of the person being measured captured by the imaging device 2, using a trained model that uses the correlation between the tongue feature amount and the stress state. In addition, since the person being measured only needs to capture an image of the tongue with the imaging device 2, the process is simple and less burdensome. In other words, according to the information processing program of the embodiment, the stress state of the person being measured can be measured by a simple method.

[0103] The above-described embodiment is presented as an example of the invention, and is not intended to limit the scope of the invention. The above configuration may be omitted, replaced, or modified in various ways without departing from the spirit of the invention. The above-mentioned embodiment and its modifications are included within the scope and spirit of the invention, and are therefore not to be construed as limiting the present invention. This invention is included in the scope of the claims and their equivalents. EXAMPLES

[0104] This paper describes a stress state measurement test (hereinafter referred to as "this test") conducted on multiple subjects to generate training data. It also describes the results of verifying the validity of a trained model constructed from training data generated based on this test.

[0105] [Stress state measurement test] ·subject In this study, 98 men and women aged 20 to 60 were recruited and tested.

[0106] Then, for each subject, data was obtained twice on each of the first, third, and fifth days (i.e., six data for one subject). As a result, data for 570 cases was obtained, excluding error data.

[0107] Here, the 570 examples of data constitute the learning data, and are the tongue features and stress state measurements of each of the 98 subjects.

[0108] Acquiring tongue features First, in this test, images of the tongue of each subject were taken to obtain the tongue features. The tongue features are used as input data for the learning data. TIAS was used to take the tongue images.

[0109] The tongue features of each subject were then extracted by analyzing the images of the tongue, and were classified into 88 categories.

[0110] -Measurement of stress state Next, in the main test, the stress state of the subjects was measured. The stress state data was used as the correct answer data for the learning data.

[0111] In this study, the subjects' stress state was measured using the questionnaire and raw score conversion table mentioned above. The questionnaire included 23 questions, including work-related items (6 items), mental and physical stress responses (11 items), and environment / surroundings (6 items).

[0112] Each of the 23 questions in the questionnaire has four answer options according to the question (for example, question 1 has four options: "Yes," "Somewhat yes," "Somewhat different," and "Different," and question 7 has four options: "Seldom," "Sometimes," "Often," and "Almost always"). Each of the four options is assigned a score ranging from 1 to 4 points (for example, in question 1, "Yes" is assigned 1 point, "Somewhat yes" is assigned 2 points, "Somewhat different" is assigned 3 points, and "Different" is assigned 4 points).

[0113] On the other hand, the raw score conversion table is a table for calculating scores for multiple scales based on the answers to the questionnaire. Here, the multiple scales are a total of nine scales: two scales calculated from the causes of stress, five scales calculated from the mental and physical stress reactions, and two scales calculated from the environmental (work) stress factors.

[0114] After taking images of their tongues on the first, third, and fifth days of the study, the subjects answered 23 questions included in the survey. The subjects' stress state was then measured using the following method defined by the Ministry of Health, Labor, and Welfare.

[0115] First, the stress state of the subjects was measured using the method using the points on the questionnaire. At this time, if either a or b below was applicable, the stress state was determined to be "high" using the method using the points on the questionnaire. In all other cases, it was determined to be "low." A: A total of 31 points or more for 11 items related to mental and physical stress responses b: The total of the six items related to the environment and surroundings is 39 points or more, and a is 23 points or more

[0116] Next, the stress state was measured using a method that uses a raw score conversion table. At this time, if either c or d below was applicable, the stress state was determined to be "high" using the method that uses the raw score conversion table. In other cases, it was determined to be "low." c: The total of five scales calculated from the mental and physical stress response is 19 points or more. d: The sum of two scales calculated from environmental (work) stress factors is 16 points or more and c is 14 points or more

[0117] Finally, if the stress state was "high" either in the method using the questionnaire points or the method using the raw score conversion table, the final measurement result of the subject's stress state was deemed to be high stress. Otherwise, it was deemed to be low stress.

[0118] As a result, of the 570 cases mentioned above, 46 were high stress and 524 were low stress.

[0119] Through the above-mentioned tests, learning data was generated for each of the 570 cases, including the tongue features as input data and the stress state as correct answer data.

[0120] [Verification of trained model] A trained model was constructed by training the machine learning model with the training data generated in this test. In constructing the trained model, in this embodiment, the validity of the measurement results by the trained model was verified by 3-fold cross-validation.

[0121] Figure 8 is a diagram explaining the validation by 3-fold cross validation, and shows the number of samples used in the 3-fold cross validation. As shown in this figure, 46 cases measured as high stress were first used in the 3-fold cross validation. Furthermore, 46 cases were extracted from 524 cases measured as low stress so that the number was the same as the number of high stress cases, and were used for validation.

[0122] For each of the high-stress and low-stress cases, 36 cases were used as training data and 10 cases were used as test data.

[0123] Then, precision and recall were obtained in the three-fold cross-validation, and a precision-recall curve (PR curve) was drawn by plotting recall on the horizontal axis and precision on the vertical axis of a two-dimensional orthogonal coordinate system. The evaluation index of the three-fold cross-validation was the average precision, which indicates the area of ​​the PR curve.

[0124] The three-fold cross-validation described above was performed on three machine learning models: Random Forest (RF), Light Gradient Boost Machine (LGBM), and Extreme Gradient Boosting (XGB).

[0125] Figure 9 shows the results of the three-fold cross-validation, showing the average precision and its standard deviation obtained for each of the three machine learning models mentioned above.

[0126] The average precision rate of all three machine learning models was 0.77 or higher, which indicates that good results were obtained. In particular, the average precision rate of LGBM was 0.807, which was the best result.

[0127] Figure 10 shows the results of prediction performance when using LGBM. The prediction performance here refers to the accuracy rate, precision rate, recall rate, and F1 score. It can be seen that good results were obtained in all of these prediction performance items.

[0128] From the above results, it became clear that the trained model is preferably a machine learning model based on either Random Forest, LGBM, or XGB. Furthermore, it became clear that among these machine learning models, the machine learning model based on LGBM produced the best results. [Explanation of symbols]

[0129] Information Processing System 1 Data processing device 3 Processor 301 Main memory 302 Auxiliary storage device 303 Input Device 304 Output Device 305 Communication Equipment 306 Image acquisition unit 311 Feature extraction unit 312 Feature acquisition unit 313 Measuring part 314 Output section 315 Imaging device 2 Training data 4 Input data 4a Correct data 4b

Claims

1. acquiring a feature amount of the tongue of the subject extracted from a captured image of the tongue of the subject; measuring the stress state of the subject from the tongue feature quantity using a trained model that has been trained to output a previously measured stress state of the subject when the tongue feature quantity extracted from an image of the tongue of the subject is input; Including, A method for measuring stress state based on tongue images.

2. The feature amount of the subject's tongue includes at least one of a feature amount related to shape, a feature amount related to color, and a feature amount related to texture. The method for measuring a stress state based on a tongue image according to claim 1.

3. The feature amount relating to the shape includes at least one of a tongue area, a tongue length, and a tongue width. The method for measuring a stress state based on a tongue image according to claim 2.

4. The color feature amount is a color statistic amount for a plurality of pixels at a predetermined location on the tongue surface, The color is a color coordinate expressed in a predetermined color space. The method for measuring a stress state based on a tongue image according to claim 2.

5. the texture features include features based on a gray level co-occurrence matrix at a predetermined location on the tongue surface; The method for measuring a stress state based on a tongue image according to claim 2.

6. The trained model is a machine learning model based on any one of random forest, LGBM, and XGB. The method for measuring a stress state based on a tongue image according to claim 1.

7. acquiring an image of the tongue of the subject; extracting a feature amount of the tongue from a captured image of the tongue of the subject; Further comprising: A method for measuring a stress state based on a tongue image according to any one of claims 1 to 6.

8. a feature amount acquiring unit that acquires a feature amount of the tongue of the subject extracted from a captured image of the tongue of the subject; a measurement unit that measures the stress state of the subject from the tongue feature amount by using a trained model that has been trained to output a previously measured stress state of the subject when the tongue feature amount extracted from an image of the tongue of the subject is input; Equipped with Information processing device.

9. On the computer, A process of acquiring a feature amount of the tongue of the subject extracted from a captured image of the tongue of the subject; A process of measuring the stress state of the subject from the tongue feature amount using a trained model that has been trained to output a previously measured stress state of the subject when the tongue feature amount extracted from an image of the tongue of the subject is input; and An information processing program that executes the above.

Images