Device, method, and program

Abstract

This device comprises: an acquiring unit that acquires a spectroscopic image generated by causing an imaging signal obtained by imaging an object to spectrally diffract; a feature calculating unit that calculates a feature representing a texture on the basis of the spectroscopic image; and a predicting unit that inputs, into a trained model that has been trained using training data including the feature calculated for objects for learning and data representing the state of a microorganism included in the objects for learning, the feature calculated for an object for prediction, to predict data representing the state of a microorganism contained in the object for prediction.

Description

Apparatus, method and program

[0001] The present disclosure relates to an apparatus, a method, and a program.

[0002] As a method for investigating the intestinal bacteria of livestock, for example, a method is known in which the ileal contents or feces of the livestock are collected and the bacterial flora is analyzed. Generally, analysis by a culture method or 16S rRNA analysis is used for analyzing the bacterial flora.

[0003] On the other hand, a method for analyzing microbial flora, including bacterial flora, has been proposed that uses a reflectance spectrum obtained by irradiating a target object with light of a predetermined wavelength. This analysis method allows the types of microorganisms contained in the target object to be analyzed at low cost.

[0004] JP 2013-235014 A

[0005] However, the above-mentioned method using the reflectance spectrum analyzes the reflection intensity at each wavelength of light reflected from the object, and the analysis results depend on the reflection characteristics of various microorganisms. Therefore, for example, in the case of a microbial flora in which microorganisms with different reflection characteristics are mixed in the object, it is difficult to accurately analyze the state of various microorganisms (e.g., content ratio, etc.) using the above-mentioned method.

[0006] The present disclosure aims to improve the analytical accuracy when analyzing the state of various microorganisms in a microbiome.

[0007] According to one aspect, the device includes an acquisition unit that acquires a spectroscopic image generated by spectroscopically analyzing an image signal obtained by capturing an object; a feature calculation unit that calculates features representing a texture based on the spectroscopic image; and a prediction unit that predicts data representing the state of microorganisms contained in an object to be predicted by inputting the features calculated for the object to be predicted into a trained model that has been trained using training data including the features calculated for the object to be trained and data representing the state of microorganisms contained in the object to be trained.

[0008] According to the present disclosure, it is possible to improve the analytical accuracy when analyzing the state of various microorganisms in a microbiome.

[0009] 1 is a diagram showing an example of the system configuration of a microbiome analysis system in the learning phase. FIG. 2 is a diagram showing an example of the hardware configuration of a microbiome analysis device. FIG. 3 is a diagram showing an example of a hyperspectral image. FIG. 4 is a diagram showing an example of microbiome data. FIG. 5 is a diagram showing an example of the functional configuration of a microbiome analysis device in the learning phase. FIG. 6 is a diagram showing an example of processing by an image feature calculation unit. FIG. 7 is a diagram showing an example of processing by a learning data generation unit. FIG. 8 is a diagram showing an example of processing by a learning unit. FIG. 9 is an example of a flowchart showing the flow of learning processing by a microbiome analysis system. FIG. 10 is a diagram showing an example of the system configuration of a microbiome analysis system in the prediction phase. FIG. 11 is a diagram showing an example of the functional configuration of a microbiome analysis device in the prediction phase. FIG. 12 is a diagram showing an example of processing by a prediction unit. FIG. 13 is a diagram showing an example of processing by an output unit. FIG. 14 is an example of a flowchart showing the flow of prediction processing by a microbiome analysis system. FIG. 15 is a diagram showing an example of verification of the prediction accuracy of a trained model that predicts the content ratios of various microorganisms. FIG. 16 is a diagram showing an example of verification of the prediction accuracy of a trained model that predicts the type of microorganism with the highest content ratio.

[0010] Hereinafter, each embodiment will be described with reference to the accompanying drawings. In this specification and drawings, components having substantially the same functional configurations are designated by the same reference numerals, and redundant description will be omitted.

[0011] [First embodiment] <System configuration in the learning phase of the microbiome analysis system> First, the system configuration of a microbiome analysis system that analyzes microbiomes will be described. In the first embodiment, the microbiome analysis system has different system configurations for the learning phase, in which learning processing is performed, and the prediction phase, in which prediction processing is performed. Here, the system configuration of the microbiome analysis system in the learning phase will be described.

[0012] In the first embodiment, the microbiota analyzed by the microbiota analysis system is the bacterial flora contained in the ileal contents or feces of livestock. Therefore, in the first embodiment, the "object" includes livestock, and the "target sample" (target object) refers to the ileal contents or feces.

[0013] However, the "object" is not limited to livestock but may be other living organisms. Alternatively, in a scenario where the microbiome contained in soil is analyzed, the "object" may be a non-living organism such as soil. Furthermore, the "target sample" is not limited to ileal contents or feces but may be other intestinal contents or excrement, and if the object is soil, it may be a portion of the soil collected from the soil. In other words, the "target sample" is a biologically or environmentally derived sample containing a single or multiple types of microorganisms.

[0014] Fig. 1 shows an example of the system configuration of a microbiome analysis system in the learning phase. As shown in Fig. 1, the microbiome analysis system 100 in the learning phase comprises a near-infrared light output device 110, a hyperspectral camera 120, a next-generation sequencer 130, and a microbiome analysis device 140, which is the "device" according to the first embodiment.

[0015] The near-infrared light output device 110 is a device that emits near-infrared light (light with a wavelength of 800 nm to 2500 nm). The near-infrared light emitted from the near-infrared light output device 110 is irradiated onto a learning target sample 102 collected from an object 101.

[0016] The hyperspectral camera 120 captures the light reflected from the learning target sample 102 and disperses the captured signal into individual wavelengths to generate spectral images (hereinafter referred to as "hyperspectral images") that are captured images for each wavelength. The hyperspectral camera 120 transmits the hyperspectral images for each wavelength to the microbiome analyzer 140.

[0017] The next-generation sequencer 130 is an instrument that amplifies and analyzes the 16S rRNA genes of microorganisms using PCR (Polymerase Chain Reaction) to identify the types and distribution of microorganisms. By analyzing a training target sample 102 collected from an object 101, the next-generation sequencer 130 generates microbiome data for the training target sample 102. The microbiome data generated by the next-generation sequencer 130 represents the state of the microorganisms contained in the training target sample 102. In the first embodiment, the data representing the state of the microorganisms contained in the training target sample 102 includes information such as: the types of microorganisms contained in the training target sample 102; the content ratio representing the ratio of the amount of each type of microorganism contained in the training target sample 102 to the total amount of all types of microorganisms; and the type of microorganism with the highest content ratio among the microorganisms contained in the training target sample 102 (i.e., the microorganism most abundant in the training target sample 102). The next-generation sequencer 130 transmits the generated microbiome data to the microorganism analyzer 140.

[0018] In the learning phase, the microbiome analyzer 140 acquires hyperspectral images of the learning target sample 102 from the hyperspectral camera 120. The microbiome analyzer 140 also acquires microbiome data of the learning target sample 102 from the next-generation sequencer 130.

[0019] The microbiome analyzer 140 generates training data based on the acquired hyperspectral image and microbiome data, and uses the generated training data to train a machine learning model (hereinafter simply referred to as "model") to generate a trained model. The trained models generated by the microbiome analyzer 140 are trained models that predict data representing the state of microorganisms contained in a target sample for prediction. In the first embodiment, trained models that predict data representing the state of microorganisms contained in a target sample for prediction include: a trained model that predicts the content ratios of various microorganisms contained in the target sample for prediction; and a trained model that predicts the type of microorganism with the highest content ratio among the microorganisms contained in the target sample for prediction.

[0020] <Hardware configuration of microbiome analyzer> Next, the hardware configuration of the microbiome analyzer 140 will be described. Fig. 2 is a diagram showing an example of the hardware configuration of a microbiome analyzer. As shown in Fig. 2, the microbiome analyzer 140 has a processor 201, memory 202, an auxiliary storage device 203, a connection device 204, a communication device 205, and a drive device 206. Note that the pieces of hardware included in the microbiome analyzer 140 are connected to each other via a bus 207.

[0021] The processor 201 has various arithmetic devices such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), etc. The processor 201 reads various programs (for example, the microbiome analysis program, which is the "program" according to the first embodiment) into the memory 202 and executes them.

[0022] The memory 202 has a main storage device such as a read-only memory (ROM) or a random access memory (RAM). The processor 201 and the memory 202 form a so-called computer, and the processor 201 executes various programs read onto the memory 202, thereby enabling the computer to realize various functions.

[0023] The auxiliary storage device 203 stores various programs and various information used when the various programs are executed by the processor 201. An image storage unit 511, a data storage unit 512, a training data storage unit 513, and a trained model storage unit 514, which will be described later, are realized in the auxiliary storage device 203.

[0024] The connection device 204 is a connection device that connects to external devices (such as an operation device 211 and a display device 212).

[0025] The communication device 205 is a communication device for transmitting and receiving various information between the hyperspectral camera 120 and the next-generation sequencer 130 .

[0026] The drive device 206 is a device for loading the recording medium 213. The recording medium 213 here includes media that record information optically, electrically, or magnetically, such as CD-ROMs, flexible disks, magneto-optical disks, etc. The recording medium 213 may also include semiconductor memories that record information electrically, such as ROMs, flash memories, etc.

[0027] The various programs to be installed in the auxiliary storage device 203 are installed, for example, by setting the distributed recording medium 213 in the drive device 206 and reading the various programs recorded on the recording medium 213 by the drive device 206. Alternatively, the various programs to be installed in the auxiliary storage device 203 may be installed by being downloaded from a network (not shown) via the communication device 205.

[0028] <Specific Example of Hyperspectral Image> Next, a description will be given of a specific example of a hyperspectral image generated by the hyperspectral camera 120. Fig. 3 is a diagram showing an example of a hyperspectral image.

[0029] 3, the horizontal axis represents wavelength. Also, in FIG. 3, reference numerals 300_1, 300_2, 300_3, . . . , and 300_81 represent hyperspectral images of the respective wavelengths generated by the hyperspectral camera 120.

[0030] In the example of Fig. 3, reference numeral 300_1 represents a hyperspectral image with a wavelength of 900 [nm], reference numeral 300_2 represents a hyperspectral image with a wavelength of 910 [nm], reference numeral 300_3 represents a hyperspectral image with a wavelength of 920 [nm], and reference numeral 300_81 represents a hyperspectral image with a wavelength of 1700 [nm].

[0031] In this way, the hyperspectral image generated by the hyperspectral camera 120 is assumed to have the following: Wavelength range: 900 nm to 1700 nm; Wavelength increment: 10 nm; Number of images: 81. However, the hyperspectral image generated by the hyperspectral camera 120 is not limited to the above wavelength range, and may have a different wavelength range. Furthermore, the hyperspectral image generated by the hyperspectral camera 120 is not limited to the above wavelength increment, and may have a different increment.

[0032] In FIG. 3, each pixel value of the hyperspectral image for each wavelength, indicated by reference numerals 300_1, 300_2, 300_3, . . . 300_81, etc., correlates with the reflection intensity of light of each wavelength at each position on the target sample 102.

[0033] Therefore, for example, if microorganisms having predetermined reflection characteristics that reflect light of a specific wavelength are distributed within the target sample 102, the hyperspectral image of the specific wavelength will be a textured image. Specifically, the hyperspectral image of the specific wavelength will be a textured image in which shading appears at each position within the image that corresponds to each position of the microorganism within the target sample 102. Texture refers to a pattern expressed by some kind of regular, fine change in density within the image.

[0034] The microorganisms referred to here may be the microorganisms themselves, or may be metabolic products (substances related to the microorganisms) produced by decomposition by the microorganisms. In other words, the data representing the state of the microorganisms contained in the target sample 102 includes data representing the state of the microorganisms and substances related to the microorganisms.

[0035] <Specific Example of Microbiota Data> Next, a description will be given of a specific example of the microbiota data generated by the next-generation sequencer 130. Fig. 4 is a diagram showing an example of the microbiota data.

[0036] As shown in FIG. 4, the microbiome data 400 includes information items such as "target sample," "microorganism species," "content ratio," and "high-content microorganism species."

[0037] The “target sample” stores an identifier (in the example of FIG. 4, “sample 1”) for identifying the learning target sample (e.g., target sample 102) used to generate the microbiome data 400.

[0038] “Microorganism species” stores the type of microorganism (in the example of Figure 4, “Microorganism A,” “Microorganism B,” “Microorganism C,” etc.) contained in the learning target sample (e.g., target sample 102) used to generate the microbiome data 400.

[0039] The “content ratio” stores the content ratio (in the example of Figure 4, “a%, “b%, “c%, ...”, etc.) of various microorganisms contained in the learning target sample (e.g., target sample 102) used to generate the microbiome data 400.

[0040] The "highly abundant microbial species" stores the type of microorganism with the highest content ratio ("microorganism F" in the example of Figure 4) among the microorganisms contained in the learning target sample (e.g., target sample 102) used to generate the microbiome data 400.

[0041] <Functional configuration of the microbiome analyzer> Next, the functional configuration of the microbiome analyzer in the learning phase will be described. Fig. 5 is a diagram showing an example of the functional configuration of a microbiome analyzer in the learning phase. As described above, a microbiome analysis program is installed in the microbiome analyzer 140, and by executing this microbiome analysis program in the learning phase, the microbiome analyzer 140 functions as: a hyperspectral image acquisition unit 501, an image feature calculation unit 502, a microbiome data acquisition unit 503, a learning data generation unit 504, and a learning unit 505.

[0042] The hyperspectral image acquisition unit 501 acquires hyperspectral images of each wavelength from the hyperspectral camera 120 and stores them in the image storage unit 511 .

[0043] The image feature amount calculation unit 502 reads out the hyperspectral images of each wavelength stored in the image storage unit 511, calculates the image feature amount for each wavelength, and notifies the learning data generation unit 504 of the calculated image feature amount.

[0044] The image feature amount calculated by the image feature amount calculation unit 502 is a feature amount representing the texture in the hyperspectral image of each wavelength. The feature amount representing the texture refers to a quantitative representation of the properties of the texture.

[0045] By calculating the feature amount representing the texture in this way, the image feature amount calculation unit 502 can grasp the distribution of microorganisms contained in the target sample for each wavelength. Therefore, compared to the conventional method of simply analyzing the reflection intensity at each wavelength (method using the reflection spectrum), the image feature amount calculation unit 502 can calculate the feature amount that accurately represents the amount of microorganisms contained in the target sample.

[0046] As a result, by using the features representing the texture, it becomes possible to accurately analyze the state of various microorganisms (e.g., content ratio, etc.) even when microorganisms with different reflection characteristics are mixed within the target sample.

[0047] The feature representing the texture may include any feature, such as a feature (statistic) based on a histogram representing the frequency distribution of pixel values ​​of each pixel, a difference statistic, a density co-occurrence matrix, or a Fourier feature. In the first embodiment, the feature based on a histogram is used.

[0048] The microbiome data acquisition unit 503 acquires the microbiome data 400 from the next-generation sequencer 130 and stores it in the data storage unit 512.

[0049] When the learning data generation unit 504 is notified of the image features from the image feature calculation unit 502, it reads the corresponding microbiome data 400 from the data storage unit 512 and generates learning data. The learning data generated by the learning data generation unit 504 uses as input data all or part of the image features of the learning target sample 102 notified by the image feature calculation unit 502. Of the microbiome data 400 read from the data storage unit 512, data that represents the state of the microorganisms contained in the learning target sample 102 is used as correct answer data.

[0050] Specifically, in the first embodiment, the learning data generation unit 504 generates at least two types of learning data: - learning data in which the correct answer data is the content ratio of various microorganisms contained in the learning target sample 102; and - learning data in which the correct answer data is the type of microorganism with the highest content ratio among the microorganisms contained in the learning target sample 102.

[0051] The learning data generation unit 504 stores the two types of learning data that it has generated in the learning data storage unit 513. The learning data storage unit 513 stores the learning data generated by the learning data generation unit 504 for a plurality of target samples for learning.

[0052] The learning unit 505 learns a model using the learning data stored in the learning data storage unit 513, thereby generating a trained model that predicts data representing the state of microorganisms contained in the target sample for prediction, and stores the trained model in the trained model storage unit 514.

[0053] The trained models generated by the training unit 505 include: - a trained model trained using training data in which the content ratios of various microorganisms contained in the training target sample 102 are set as correct data (a trained model that predicts the content ratios of various microorganisms in a prediction target sample), - a trained model trained using training data in which the type of microorganism with the highest content ratio among the microorganisms contained in the training target sample 102 is set as correct data (a trained model that predicts the type of microorganism with the highest content ratio in a prediction target sample). Note that the number of trained models that predict the content ratios of various microorganisms is generated according to the number of types of microorganisms.

[0054] <Specific examples of processing by each functional unit of the microbiome analysis device> Next, specific examples of processing by each functional unit (here, the image feature calculation unit 502, the learning data generation unit 504, and the learning unit 505) in the learning phase of the microbiome analysis device 140 will be described.

[0055] (1) Specific Example of Processing by Image Feature Amount Calculation Unit 502 First, a specific example of processing by the image feature amount calculation unit 502 will be described. Fig. 6 is a diagram showing an example of processing by the image feature amount calculation unit. As shown in Fig. 6, the image feature amount calculation unit 502 further includes a normalization unit 610, a histogram generation unit 620, and a feature amount calculation unit 630.

[0056] The normalization unit 610 performs normalization processing on the hyperspectral images of each wavelength (reference numeral 300_1, reference numeral 300_2, ..., reference numeral 300_81) read out from the image storage unit 511. Specifically, the normalization unit 610 performs normalization processing by dividing the pixel value of each pixel included in the hyperspectral image of each wavelength (hyperspectral images excluding the hyperspectral image of the specific wavelength) by the pixel value of a specific pixel included in the hyperspectral image of the specific wavelength.

[0057] The pixel value of a specific pixel included in a hyperspectral image of a specific wavelength is, for example, the pixel value of a pixel in a water region included in a hyperspectral image of a wavelength of 1440 [nm]. By performing normalization in this way, it is possible to reduce the influence of variations in reflection intensity between hyperspectral images of each wavelength.

[0058] The hyperspectral images for each wavelength normalized by the normalization unit 610 are notified to the histogram generation unit 620 .

[0059] To calculate features representing texture, the histogram generation unit 620 generates histograms (frequency distributions of pixel values ​​of each pixel) of the normalized hyperspectral image for each wavelength and notifies the feature calculation unit 630. In Fig. 6, reference numeral 600_1 represents the histogram of the normalized hyperspectral image (reference numeral 300_1), reference numeral 600_2 represents the histogram of the normalized hyperspectral image (reference numeral 300_2), and reference numeral 600_81 represents the histogram of the normalized hyperspectral image (reference numeral 300_81).

[0060] When the histogram is notified by the histogram generation unit 620, the feature calculation unit 630 calculates a feature based on the histogram as a feature representing the texture. Specifically, the feature calculation unit 630 calculates, as the feature based on the histogram, for example: mean value, standard deviation, variance, minimum value, maximum value, median, first quartile, third quartile, mode, kurtosis, and skewness.

[0061] The feature calculation unit 630 calculates feature amounts based on histograms for each of the normalized hyperspectral images for each wavelength. Therefore, if the number of images (number of wavelengths) in the hyperspectral image is 81 and the number of feature amounts based on each histogram is 11, the image feature amount group 602 notified to the training data generation unit 504 by the feature amount calculation unit 630 will include 891 image feature amounts.

[0062] (2) Specific Example of Processing by the Training Data Generation Unit 504 Next, a specific example of processing by the training data generation unit 504 will be described. Fig. 7 is a diagram showing an example of processing by the training data generation unit. As shown in Fig. 7, the training data generation unit 504 further includes an image feature narrowing unit 710 and a combining unit 720.

[0063] When the image feature narrowing-down unit 710 is notified of the image feature group 602 from the feature calculation unit 630, it selects a predetermined image feature from the multiple image feature included in the notified image feature group 602 and notifies the combination unit 720. The image feature selected by the image feature narrowing-down unit 710 is an image feature that is useful for predicting data representing the state of microorganisms contained in the learning target sample 102, and is assumed to be determined in advance.

[0064] The image features selected by the image feature narrowing unit 710 are determined in advance based on the best verification results obtained by repeating learning and verification using various combinations of image features. The selected image features may be different for each piece of training data generated by the combining unit 720, or may be common regardless of the training data. For example, the image features selected when generating training data for predicting the content ratios of various microorganisms may be the same as or different from the image features selected when generating training data for predicting the type of microorganism with the highest content ratio. Furthermore, the image features selected when generating training data for predicting the content ratios of various microorganisms may be different for each type of microorganism, or may be the same.

[0065] When the combining unit 720 is notified of the refined image features by the image feature narrowing unit 710, the combining unit 720 reads the corresponding microbiome data 400 from the data storage unit 512 and generates learning data.

[0066] 7, reference numeral 730_1 denotes training data used when generating a trained model that predicts the content ratio of "microorganism A." Similarly, reference numerals 730_2 and 730_3 denote training data used when generating trained models that predict the content ratios of "microorganism B" and "microorganism C," respectively.

[0067] Reference numeral 730_1 denotes training data generated for the training target sample 102 identified by “sample 1”, and includes: an image feature (λ ) that is a refined image feature notified by the image feature refinement unit 710; １ ) to image feature (λ ８０ ) as input data, and the microbial species = "microorganism A" and content ratio = "a%" contained in the microbiome data 400 read from the data storage unit 512 as correct answer data.

[0068] In addition, in the reference numeral 730_1, input data=image feature amount (λ １ ) to image feature (λ 80 ) indicates that five wavelengths have been selected by the image feature narrowing unit 710. As described above, each wavelength includes 11 image features, so in the example of reference numeral 730_1, the input data includes 55 image features.

[0069] 7, reference numeral 740 denotes training data used when generating a trained model that predicts the type of microorganism with the highest content ratio. Reference numeral 740 denotes training data generated for the training target sample 102 identified by "sample 1," and includes: image feature (λ １ ) to image feature (λ ８０ ) as input data, and the highly abundant microbial species = "Microorganism F" included in the microbiota data 400 read from the data storage unit 512 is the correct answer data.

[0070] In addition, in the reference numeral 730_1, input data=image feature amount (λ １ ) to image feature (λ 80) indicates that five wavelengths have been selected by the image feature narrowing unit 710. As described above, each wavelength includes 11 image features, so in the example of reference numeral 740, the input data includes 55 image features.

[0071] (3) Specific Example of Processing by the Learning Unit 505 Next, a specific example of processing by the learning unit 505 will be described. Fig. 8 is a diagram showing an example of processing by the learning unit. As shown in Fig. 8, the learning unit 505 further includes a model A 801, a model B 802, a model C 803, ..., a model α 821, and a comparison and modification unit 811, a comparison and modification unit 812, a comparison and modification unit 813, ..., a comparison and modification unit 831.

[0072] Model A801 ​​is a model for predicting the content ratio of "microorganism A." Model A801 ​​outputs output data when "input data" of learning data indicated by reference numeral 730_1 is input.

[0073] The comparison and modification unit 811 compares the output data output from the model A 801 with the "correct answer data" of the learning data indicated by the reference numeral 730_1, and updates the model parameters of the model A 801 based on the error. In this way, learning is performed on the model A 801, and a learned model A is generated.

[0074] Similarly, model B802 is a model for predicting the content ratio of "microorganism B." Model B802 outputs output data when "input data" of the learning data indicated by reference numeral 730_2 is input.

[0075] The comparison and modification unit 812 compares the output data output from the model B 802 with the "correct answer data" of the learning data denoted by reference numeral 730_2, and updates the model parameters of the model B 802 based on the error. In this way, learning is performed on the model B 802, and a learned model B is generated.

[0076] Similarly, model C803 is a model for predicting the content ratio of "microorganism C." Model C803 outputs output data when "input data" of the learning data indicated by reference numeral 730_3 is input.

[0077] The comparison and modification unit 813 compares the output data output from the model C 803 with the "correct answer data" of the learning data indicated by the reference numeral 730_3, and based on the error, updates the model parameters of the model C 803. In this way, learning is performed on the model C 803, and a learned model C is generated.

[0078] The model α821 is a model for predicting the type of microorganism with the highest content ratio. The model α821 outputs output data when “input data” of the learning data indicated by the reference numeral 740 is input.

[0079] The comparison and modification unit 831 compares the output data output from the model α 821 with the "correct answer data" of the learning data indicated by the reference numeral 740, and updates the model parameters of the model α 821 based on the error. In this way, learning is performed on the model α 821, and a learned model α is generated.

[0080] As is clear from the above description, trained model A 801, trained model B 802, trained model C 803, ..., trained model α 821 are generated by machine learning. The machine learning here includes any machine learning such as linear regression, logistic regression, naive Bayes, decision tree, random forest, boosting, LASSO, Ridge, ElasticNet, neural network (deep learning, CNN, Transformer), etc.

[0081] <Learning Process Flow> Next, we will explain the flow of the learning process performed by the microbiome analysis system 100. Fig. 9 is an example of a flowchart showing the flow of the learning process performed by the microbiome analysis system.

[0082] In step S901, the microbiome analysis system 100 acquires a learning target sample 102 collected from an object 101.

[0083] In step S902, the microbiome analysis system 100 irradiates the target sample 102 with near-infrared light and captures the reflected light using the hyperspectral camera 120 to generate hyperspectral images for each wavelength.

[0084] In step S903, the microbiome analysis system 100 performs normalization processing on the acquired hyperspectral images for each wavelength.

[0085] In step S904, the microbiome analysis system 100 calculates image features for the normalized hyperspectral images at each wavelength.

[0086] In step S905, the microbiome analysis system 100 narrows down the image features by selecting predetermined image features from the calculated image features.

[0087] Furthermore, in step S906, the microbiome analysis system 100 uses the next-generation sequencer 130 to generate microbiome data 400 for the learning target sample 102.

[0088] In step S907, the microbiome analysis system 100 acquires the narrowed-down image features as input data, and acquires data representing the state of the microorganisms included in the microbiome data 400 as correct answer data.

[0089] In step S911, the microbiome analysis system 100 generates learning data based on the acquired input data and the acquired correct answer data (here, the content ratio of the target microorganism contained in the learning target sample 102).

[0090] In step S912, the microbiome analysis system 100 uses the generated training data to generate a trained model for predicting the content ratio of the target microorganism contained in the target sample for prediction.

[0091] In step S913, the microbiome analysis system 100 determines whether or not a trained model for predicting the content ratio of the target microorganism contained in the target sample for prediction has been generated for all types of microorganism.

[0092] In step S913, if it is determined that there is a microorganism for which a trained model has not been generated (if NO in step S913), the process returns to step S911.

[0093] Meanwhile, in step S921, the microbiome analysis system 100 generates learning data based on the acquired input data and the acquired correct answer data (here, the type of microorganism with the highest content ratio among the microorganisms contained in the learning target sample 102).

[0094] In step S922, the microbiome analysis system 100 uses the generated training data to generate a trained model that predicts the type of microorganism that is most abundant among the microorganisms contained in the target sample for prediction.

[0095] If it is determined in step S913 that trained models have been generated for all types of microorganisms, and if the generation of trained models is completed in step S922, the microbiome analysis system 100 terminates the learning process.

[0096] <System configuration of the microbiome analysis system in the prediction phase> Next, the system configuration of the microbiome analysis system in the prediction phase will be described. Fig. 10 is a diagram showing an example of the system configuration of the microbiome analysis system in the prediction phase. As shown in Fig. 10, the microbiome analysis system 1000 in the prediction phase comprises a near-infrared light output device 110, a hyperspectral camera 120, and a microbiome analysis device 140, which is the "device" according to the first embodiment.

[0097] The near-infrared light output device 110 is a device that emits near-infrared light (light with a wavelength of 800 nm to 2500 nm). The near-infrared light emitted from the near-infrared light output device 110 is irradiated onto a target sample 1002 for prediction that has been collected from an object 1001.

[0098] The hyperspectral camera 120 captures the light reflected from the prediction target sample 1002 and disperses the signal obtained by capturing the image (captured signal) into individual wavelengths to generate a hyperspectral image, which is an image captured for each wavelength. The hyperspectral camera 120 transmits the hyperspectral images for each wavelength to the microbiome analyzer 140.

[0099] In the prediction phase, the microbiome analysis device 140 acquires a hyperspectral image of the prediction target sample 1002 from the hyperspectral camera 120. The microbiome analysis device 140 uses the trained model generated in the learning phase to predict data representing the state of microorganisms contained in the prediction target sample from the acquired hyperspectral image. As described above, the trained models generated in the learning phase include: a trained model that predicts the content ratios of various microorganisms contained in the prediction target sample 1002, and a trained model that predicts the type of microorganism that is most abundant among the microorganisms contained in the prediction target sample 1002. Therefore, the microbiome analysis device 140 outputs the content ratios of various microorganisms contained in the prediction target sample 1002 and the type of microorganism that is most abundant among the microorganisms contained in the prediction target sample 1002 as prediction results.

[0100] <Functional configuration of the microbiome analyzer> Next, the functional configuration of the microbiome analyzer in the prediction phase will be described. Fig. 11 is a diagram showing an example of the functional configuration of a microbiome analyzer in the prediction phase. As described above, a microbiome analysis program is installed in the microbiome analyzer 140, and by executing this microbiome analysis program in the prediction phase, the microbiome analyzer 140 functions as: a hyperspectral image acquisition unit 501, an image feature calculation unit 502, an image feature narrowing down unit 701, a prediction unit 1101, and an output unit 1102.

[0101] Of these, the hyperspectral image acquisition unit 501 and the image feature amount calculation unit 502 have already been described using Fig. 5, and therefore their description will be omitted here. Also, the image feature amount narrowing down unit 701 has already been described using Fig. 7, and therefore their description will be omitted here.

[0102] The prediction unit 1101 reads out the trained model stored in the trained model storage unit 514, and inputs the narrowed-down image features notified by the image feature narrowing unit 701. As a result, the trained model predicts data representing the state of microorganisms contained in the target sample 1002 for prediction.

[0103] As described above, the trained models include: a trained model that predicts the content ratios of various microorganisms contained in the target sample for prediction 1002, and a trained model that predicts the type of microorganism that is contained most frequently among the microorganisms contained in the target sample for prediction 1002. For this reason, the prediction unit 1101 notifies the output unit 1102 of the content ratios of various microorganisms contained in the target sample for prediction 1002 and the type of microorganism that is contained most frequently among the microorganisms contained in the target sample for prediction 1002.

[0104] The output unit 1102 outputs the content ratio of various microorganisms contained in the target sample 1002 for prediction, as notified by the prediction unit 1101, and the type of microorganism with the highest content ratio among the microorganisms contained in the target sample 1002 for prediction, as prediction results.

[0105] <Specific example of processing by each functional unit of the microbiome analyzer> Next, a specific example of processing by each functional unit (here, the prediction unit 1101 and the output unit 1102) in the prediction phase of the microbiome analyzer 140 will be described.

[0106] (1) Specific Example of Processing by Prediction Unit 1101 First, a specific example of processing by the prediction unit 1101 will be described. Fig. 12 is a diagram showing an example of processing by the prediction unit. As shown in Fig. 12, the prediction unit 1101 has trained models read out from the trained model storage unit 514. The example in Fig. 12 shows how trained models A 1201, trained model B 1202, trained model C 1203, ..., and trained model α 1221 have been read out.

[0107] In FIG. 12, reference numeral 1200 indicates the image feature (λ) that is input from the image feature narrowing unit 701 to the target sample 1002 for prediction identified by “sample X.” １ ) to image feature (λ 80 ) is notified.

[0108] The trained model A1201 is a trained model that predicts the content ratio of "microorganism A." When the "input data" of the reference numeral 1200 is input, the trained model A1201 predicts the content ratio of microorganism A contained in the target sample for prediction 1002 identified by "sample X" = "a Ｘ %” is predicted.

[0109] Similarly, the trained model B1202 is a trained model that predicts the content ratio of "microorganism B." When the "input data" of the reference numeral 1200 is input, the trained model B1202 predicts the content ratio of microorganism B contained in the target sample for prediction 1002 identified by "sample X" = "b Ｘ %” is predicted.

[0110] Similarly, the trained model C1203 is a trained model that predicts the content ratio of "microorganism C." When the "input data" of the reference numeral 1200 is input, the trained model C1203 predicts the content ratio of microorganism C contained in the target sample 1002 for prediction identified by "sample X" = "c Ｘ %” is predicted.

[0111] Trained model α1221 is a trained model that predicts the type of microorganism with the highest content ratio. When the “input data” 1200 is input, trained model α1221 outputs the type of microorganism with the highest content ratio = “microorganism G” among the microorganisms contained in the target sample for prediction 1002 identified by “sample X.”

[0112] (2) Specific Example of Processing by Output Unit 1102 Next, a specific example of processing by the output unit 1102 will be described. Fig. 13 is a diagram showing an example of processing by the output unit.

[0113] As shown in FIG. 13, the prediction result 1300 includes information items such as "target sample," "microorganism species," "content ratio," and "high content microorganism species."

[0114] The "target sample" stores an identifier ("sample X" in the example of FIG. 13) for identifying the target sample for prediction (for example, target sample 1002).

[0115] "Microorganism species" stores the type of microorganism (in the example of Figure 13, "Microorganism A," "Microorganism B," "Microorganism C," etc.) contained in the target sample for prediction (e.g., target sample 1002).

[0116] The "content ratio" indicates the content ratio of various microorganisms contained in the target sample for prediction (for example, target sample 1002) (in the example of FIG. 13, "a Ｘ %, "b Ｘ %, "c Ｘ %", ..., etc.) are stored.

[0117] The "high content microbial species" stores the type of microorganism with the highest content ratio ("microorganism G" in the example of Figure 13) among the microorganisms contained in the target sample for prediction (e.g., target sample 1002).

[0118] <Flow of prediction processing> Next, we will explain the flow of prediction processing by the microbiome analysis system 1000. Fig. 14 is an example of a flowchart showing the flow of prediction processing by the microbiome analysis system.

[0119] In step S1401, the microbiome analysis system 1000 acquires a target sample 1002 for prediction collected from an object 1001.

[0120] In step S1402, the microbiome analysis system 1000 irradiates the target sample 1002 with near-infrared light and captures the reflected light using the hyperspectral camera 120 to generate hyperspectral images for each wavelength.

[0121] In step S1403, the microbiome analysis system 1000 performs normalization processing on the acquired hyperspectral images for each wavelength.

[0122] In step S1404, the microbiome analysis system 1000 calculates image features for the normalized hyperspectral images at each wavelength.

[0123] In step S1405, the microbiome analysis system 1000 narrows down the image features by selecting predetermined image features from the calculated image features.

[0124] In step S1406, the microbiome analysis system 1000 inputs the narrowed-down image features into each trained model that predicts the content ratio of various microorganisms contained in the target sample 1002 for prediction, and predicts the content ratio of various microorganisms.

[0125] In step S1407, the microbiome analysis system 1000 inputs the narrowed-down image features into a trained model that predicts the type of microorganism that is most abundant among the microorganisms contained in the target sample for prediction. In this way, the microbiome analysis system 1000 predicts the type of microorganism that is most abundant.

[0126] In step S1408, the microbiome analysis system 1000 outputs the prediction results.

[0127] In step S1409, the microbiome analysis system 1000 determines whether or not there are other target samples for prediction. If it is determined in step S1409 that there are other target samples for prediction (YES in step S1409), the process returns to step S1401.

[0128] On the other hand, if it is determined in step S1409 that there are no other target samples for prediction (NO in step S1409), the prediction process ends.

[0129] <Example of verification of prediction accuracy of trained model> Next, the prediction accuracy of the trained model possessed by the prediction unit 1101 of the microbiome analyzer 140 will be described.

[0130] (1) Trained model for predicting the content ratio of various microorganisms Figure 15 is a diagram showing an example of verification of the prediction accuracy of a trained model for predicting the content ratio of various microorganisms.

[0131] Of these, graph 1510 shows an example of verification of the prediction accuracy of the trained model that predicts the lactobacillus content ratio. In graph 1510, the horizontal axis shows the lactobacillus content ratio (actual value %) obtained for each test sample by analysis using the next-generation sequencer 130. In addition, in graph 1510, the vertical axis shows the lactobacillus content ratio (predicted value %) predicted for each test sample using the trained model that predicts the lactobacillus content ratio.

[0132] Each point in the graph 1510 is plotted at a position corresponding to the actual measured value and predicted value of each target sample. A straight line 1511 in the graph 1510 indicates a linear regression equation with the actual measured value as the explanatory variable and the predicted value as the response variable. According to the graph 1510, the coefficient of determination R ２ = 0.7971, which indicated that the prediction accuracy of the trained model for predicting the Lactobacillus content ratio was good.

[0133] Graph 1520 shows an example of verification of the prediction accuracy of a trained model that predicts the content ratio of Streptococcus (Streptococcus). In graph 1520, the horizontal axis shows the content ratio (actual value %) of Streptococcus obtained by analyzing each target sample under verification using the next-generation sequencer 130. In addition, in graph 1520, the vertical axis shows the content ratio (predicted value %) of Streptococcus predicted for each target sample under verification using the trained model that predicts the content ratio of Streptococcus.

[0134] Each point in the graph 1520 is plotted at a position corresponding to the actual measured value and predicted value of each target sample. A straight line 1521 in the graph 1520 indicates a linear regression equation in which the actual measured value is the explanatory variable and the predicted value is the objective variable. According to the graph 1520, the coefficient of determination R ２ = 0.7616, which indicated that the prediction accuracy of the trained model for predicting the Streptococcus content ratio was good.

[0135] Graph 1530 shows an example of verification of the prediction accuracy of a trained model that predicts the Clostridium content ratio. In graph 1530, the horizontal axis shows the Clostridium content ratio (actual measured value %) obtained for each target sample being verified by analysis using the next-generation sequencer 130. In addition, in graph 1530, the vertical axis shows the Clostridium content ratio (predicted value %) predicted for each target sample being verified using the trained model that predicts the Clostridium content ratio.

[0136] Each point in the graph 1530 is plotted at a position corresponding to the actual measurement value and predicted value of each target sample. A straight line 1531 in the graph 1530 indicates a linear regression equation with the actual measurement value as the explanatory variable and the predicted value as the response variable. According to the graph 1530, the coefficient of determination R ２ = 0.7584, which indicated that the prediction accuracy of the trained model for predicting the Clostridium content ratio was good.

[0137] Graph 1540 shows an example of verification of the prediction accuracy of a trained model that predicts the content ratio of Escherichia (E. coli). In graph 1540, the horizontal axis shows the content ratio of Escherichia (actual measured value %) obtained for each target sample being verified by analyzing it using the next-generation sequencer 130. In addition, in graph 1540, the vertical axis shows the content ratio of Escherichia (predicted value %) predicted for each target sample being verified using the trained model that predicts the content ratio of Escherichia.

[0138] Each point in the graph 1540 is plotted at a position corresponding to the actual measured value and predicted value of each target sample. A straight line 1541 in the graph 1540 indicates a linear regression equation with the actual measured value as the explanatory variable and the predicted value as the objective variable. According to the graph 1540, the coefficient of determination R ２ = 0.9154, which indicated that the prediction accuracy of the trained model for predicting the Escherichia content ratio was good.

[0139] (2) Trained Model for Predicting the Most Prevalent Microorganism Type Figure 16 is a diagram showing an example of verification of the prediction accuracy of a trained model for predicting the most prevalent microorganism type. In table 1600, each bacterial species arranged horizontally indicates the type of bacterial species with the highest prevalence (predicted value) for each target sample being verified, predicted using a trained model that predicts the most prevalent microorganism type. In table 1600, each bacterial species arranged vertically indicates the most prevalent microorganism type (actual value) obtained by analyzing each target sample being verified using the next-generation sequencer 130.

[0140] Row 1601 in table 1600 shows that 18 test samples were analyzed to have the highest lactobacillus content as measured. The trained model for predicting the type of microorganism with the highest content was able to predict that lactobacillus was the microorganism with the highest content for 17 of these test samples.

[0141] Row 1602 in table 1600 shows that 17 target samples were analyzed for validation and were found to contain Streptococcus (Streptococcus) at the highest concentration as an actual measurement. The trained model for predicting the type of microorganism with the highest concentration was able to predict that Streptococcus was the microorganism with the highest concentration for 16 of these target samples.

[0142] Row 1603 in table 1600 shows that 12 target samples were analyzed for validation and were found to have the highest Clostridium content as an actual measurement. The trained model for predicting the type of microorganism with the highest content was able to predict that Clostridium was the microorganism with the highest content in 11 of these target samples.

[0143] Furthermore, row 1604 in table 1600 shows that 18 target samples were obtained for validation analysis in which Escherichia (E. coli) was found to be the most abundant in the actual measured values. The trained model for predicting the type of microorganism with the highest abundance was able to predict that Escherichia was the microorganism with the highest abundance in 18 of these target samples.

[0144] As a result, it was verified that the accuracy rate of the trained model for predicting the type of microorganism with the highest content ratio was 95.4% (= 62 / 65 × 100).

[0145] <Summary> As is clear from the above explanation, in the learning phase, the microbiome analysis device 140, which is the device according to the first embodiment: - acquires a hyperspectral image generated by spectrally analyzing a signal (image signal) obtained by capturing an image of the learning target sample 102. - calculates features representing texture based on the hyperspectral image. - generates a trained model by learning using training data including the image features calculated for the learning target sample 102 and data representing the state of the microorganisms contained in the learning target sample 102.

[0146] Furthermore, in the prediction phase, the microbiome analysis device 140, which is the device according to the first embodiment, predicts data representing the state of microorganisms contained in the target sample 1002 for prediction by inputting features representing texture calculated for the target sample 1002 for prediction into the trained model.

[0147] In this way, the microbiome analyzer 140 according to the first embodiment calculates features that represent texture based on hyperspectral images, thereby capturing the distribution of microorganisms contained in a target sample for each wavelength. Therefore, according to the first embodiment, it is possible to calculate features that accurately represent the amount of microorganisms contained in a target sample, compared to conventional methods that simply analyze the reflection intensity at each wavelength (methods that use the reflectance spectrum).

[0148] As a result, according to the first embodiment, even if microorganisms with different reflection characteristics are mixed in the target sample for prediction, it is possible to accurately predict data representing the state of various microorganisms.

[0149] In other words, the microbiome analyzer 140 according to the first embodiment can improve the accuracy of analysis when analyzing the states of various microorganisms in a microbiome.

[0150] Second Embodiment In the first embodiment, the near-infrared light emitted from the near-infrared light output device 110 is irradiated onto a target sample, and the hyperspectral camera 120 captures the light reflected from the target sample. However, the light captured by the hyperspectral camera 120 is not limited to the light reflected from the target sample. For example, the hyperspectral camera 120 may capture the light transmitted through the target sample.

[0151] In the first embodiment, the microbiome analysis system is configured to include a near-infrared light output device and irradiate the target sample with near-infrared light. However, the light irradiated onto the target sample is not limited to near-infrared light, and light in other wavelength ranges may also be irradiated.

[0152] Furthermore, in the first embodiment described above, the microbiome analysis system is configured to include a hyperspectral camera 120 and generate hyperspectral images. However, the configuration of the microbiome analysis system is not limited to this, and instead of the hyperspectral camera 120, for example, a near-infrared camera and a spectral filter may be provided to generate spectral images by spectrally dividing the captured signal.

[0153] In the first embodiment, the pixel value of a pixel in a water region included in a hyperspectral image of wavelength 1440 nm is used as the pixel value of the specific pixel when normalizing by the normalization unit 610. However, the pixel value of the specific pixel used when normalizing by the normalization unit 610 is not limited to this, and the pixel value of another pixel included in a hyperspectral image of another wavelength may be used.

[0154] In the first embodiment, the feature representing the texture includes an arbitrary feature, and specific examples include a feature (statistic) based on a histogram representing the frequency distribution of pixel values ​​of each pixel, a difference statistic, a density co-occurrence matrix, a Fourier feature, etc. However, the arbitrary feature referred to here is not limited to a feature calculated using a classical image processing technique, and may be, for example, a variable calculated using a deep learning technique such as CNN or Vision Transformer.

[0155] Furthermore, in the first embodiment, as a specific example of the processing of the image feature amount calculation unit 502, a case has been described in which the image feature amount calculation unit 502 performs processing in the order of normalization, histogram generation, and feature amount calculation. However, the order of processing performed by the image feature amount calculation unit 502 is not limited to this, and the image feature amount calculation unit 502 may perform processing in the order of histogram generation, feature amount calculation, and normalization, for example. However, regardless of the processing order, the image feature amount calculation unit 502 calculates image feature amounts that have been normalized.

[0156] Furthermore, in the first embodiment, as an example of the normalization process in the image feature amount calculation unit 502, a method of dividing the pixel value of each pixel included in the hyperspectral image of each wavelength (hyperspectral image excluding the hyperspectral image of the specific wavelength) by the pixel value of a specific pixel included in the hyperspectral image of the specific wavelength was given, but the normalization process method is not limited to this. For example, normalization may be performed by subtraction instead of division, or by performing other calculations. Note that the normalization process in the image feature amount calculation unit 502 is not an essential component, and the image feature amount calculation unit 502 does not need to include the normalization unit 610.

[0157] The present invention is not limited to the configurations described in the above embodiments, but may be combined with other elements, etc. These aspects can be changed without departing from the spirit of the present invention, and can be appropriately determined depending on the application form.

[0158] 100: Microbiota analysis system 110: Near-infrared light output device 120: Hyperspectral camera 130: Next-generation sequencer 140: Microbiota analysis device 400: Microbiota data 501: Hyperspectral image acquisition unit 502: Image feature calculation unit 503: Microbiota data acquisition unit 504: Learning data generation unit 505: Learning unit 610: Normalization unit 620: Histogram generation unit 630: Feature calculation unit 710: Image feature narrowing unit 720: Combination unit 1101: Prediction unit 1102: Output unit

Claims

1. An apparatus having: an acquisition unit that acquires a spectroscopic image generated by spectroscopically analyzing an image signal obtained by capturing an object; a feature calculation unit that calculates features representing texture based on the spectroscopic image; and a prediction unit that predicts data representing the state of microorganisms contained in an object to be predicted by inputting the features calculated for the object to be predicted into a trained model that has been trained using training data including the features calculated for the object to be learned and data representing the state of microorganisms contained in the object to be learned.

2. The device according to claim 1, wherein the imaging signal is a signal obtained by irradiating an object with light in a predetermined wavelength range.

3. The device according to claim 1, wherein some of the feature amounts of some of the spectroscopic images are selected from the plurality of feature amounts of each of the plurality of spectroscopic images calculated by the feature amount calculation unit and used for the learning or the prediction.

4. The device according to claim 1, wherein the feature amount includes statistics of pixel values ​​of pixels included in the spectroscopic image.

5. The device according to claim 1, wherein the feature calculation unit calculates the feature after normalization processing.

6. The device according to claim 5, wherein the normalization process is a process of using pixel values ​​of a spectroscopic image of a specific wavelength among the spectroscopic images acquired by the acquisition unit to perform an operation on pixel values ​​of the spectroscopic images excluding the spectroscopic image of the specific wavelength.

7. The apparatus according to claim 1, wherein the object is a biological or environmental sample containing a single or multiple species of microorganisms.

8. The device according to claim 1, wherein the data representing the state of the microorganisms includes any one of the type, amount, and content ratio of the microorganisms contained in the object.

9. The device described in claim 8, wherein the data representing the state of microorganisms is the content ratio of each type of microorganism, which represents the ratio of the amount of each type of microorganism contained in the target object to the total amount of all types of microorganisms, and the prediction unit predicts the content ratio of each type of microorganism contained in the target object for prediction using a number of trained models corresponding to the number of types of microorganisms.

10. The device described in claim 8, wherein the data representing the state of microorganisms is data indicating the type of microorganism that is most abundant in the target object, and the prediction unit predicts the type of microorganism that is most abundant in the target object using a trained model that predicts the type of microorganism that is most abundant in the target object.

11. A method performed by a computer, comprising the steps of: acquiring a spectroscopic image generated by spectrally analyzing an image signal obtained by photographing an object; calculating features representing texture based on the spectroscopic image; and predicting data representing the state of microorganisms contained in an object to be predicted by inputting the features calculated for the object to be predicted into a trained model trained using training data including the features calculated for the object to be trained and data representing the state of microorganisms contained in the object to be predicted.

12. A program for causing a computer to execute the following steps: acquiring a spectroscopic image generated by spectrally analyzing an image signal obtained by photographing an object; calculating features representing texture based on the spectroscopic image; and predicting data representing the state of microorganisms contained in an object to be predicted by inputting the features calculated for the object to be predicted into a trained model trained using training data including the features calculated for the object to be learned and data representing the state of microorganisms contained in the object to be learned.

Images