Analysis method and determination system

Abstract

Provided is an analysis method comprising: a first step of acquiring a feature amount related to a plurality of values each corresponding to a wavelength from image data of a blood vessel of an animal which is a determination target, the data being generated by imaging a body surface shallow part (vein) of the animal by using a multispectral camera; and a second step of estimating a concentration of a blood component included in the blood in the blood vessel on the basis of the feature amount.

Description

Analysis method and judgment system

[0001] The present invention relates to an analytical method and a determination system. This application claims priority to Japanese Patent Application No. 2023-134494, filed on August 22, 2023, the contents of which are incorporated herein by reference.

[0002] Blood tests for cattle are called metabolic profile tests. Metabolic profile tests are a nutritional monitoring method for cattle herds that mainly involves blood biochemistry tests. By conducting metabolic profile tests, it is expected that the cows' milk yield and milk components will be improved, perinatal diseases such as ketosis and milk fever will be prevented, and reproductive performance will be improved. Metabolic activities evaluated by metabolic profile tests include, for example, energy metabolism, protein metabolism, lipid metabolism, liver damage, minerals, and body condition score.

[0003] A specific method of metabolic profile testing is known, for example, by using a clinical chemistry analyzer to analyze components related to the metabolic activity contained in blood collected from cows (see, for example, Non-Patent Document 1).

[0004] Takayuki Watanabe, "Metabolic Profile Test," Livestock Technology, February 2020 issue

[0005] However, not only is blood sampling a significant burden on the cow, but it is also only permitted to be performed by veterinarians, making it difficult to popularize metabolic profile testing.

[0006] Additionally, metabolic profile tests require expensive clinical chemistry analyzers to analyze blood samples, which typically must be sent to a medical institution or research institute, resulting in long wait times for test results.

[0007] As such, metabolic profile tests have the problem that they injure the cow's body when taking blood, and because it is not possible to determine the cow's health condition on the spot, the cow's health condition may change while the analysis is being performed.

[0008] The present invention has been made in consideration of the above circumstances, and aims to provide an analytical method and a determination system that can non-invasively grasp the health condition of the animal being evaluated on the spot.

[0009] The present invention has the following aspects. [1] An analytical method comprising: a first step of acquiring feature quantities relating to a plurality of wavelength-dependent values ​​from image data of the blood vessels of a target animal, generated by photographing the superficial body surface (veins) of the target animal with a multispectral camera; and a second step of estimating the concentrations of blood components contained in the blood vessels based on the feature quantities. [2] The analytical method described in [1], wherein the plurality of wavelength-dependent values ​​are values ​​relating to the absorbance of the blood in the blood vessels. [3] The analytical method described in [1] or [2], wherein the first step acquires at least n (n≧228) values ​​in the wavelength range of 900 nm to 1694 nm, and the second step classifies the concentrations of the blood components based on feature quantities relating to a plurality of values ​​including the n values. [4] The analytical method described in any of [1] to [3], wherein the image data of the blood vessels is image data of the underside of the tail (median tail vein) of the target animal. [5] A determination system comprising: an imaging device that photographs the body surface of an animal to be determined; a learning device that acquires feature quantities relating to multiple values ​​according to wavelength from image data of the animal's blood vessels generated by the imaging device and learns the feature quantities; and a determination device that determines the concentration of blood components contained in the blood in the blood vessels based on the feature quantities.

[0010] According to the present invention, it is possible to provide an analysis method and a determination system that can non-invasively and on-site grasp the health condition of an animal to be evaluated.

[0011] FIG. 1 is a diagram showing the flow of an analysis method according to one embodiment of the present invention. FIG. 2 is a diagram explaining an example of classifying the concentrations of blood components according to the concentrations of the blood components in an analysis method according to one embodiment of the present invention. FIG. 3 is a diagram explaining an example of classifying the concentrations of blood components according to the concentrations of the blood components in an analysis method according to one embodiment of the present invention. FIG. 4 is a diagram explaining an example of classifying the concentrations of blood components according to the concentrations of the blood components in an analysis method according to one embodiment of the present invention. FIG. 5 is a schematic block diagram showing a specific example of the functional configuration of a learning device of the determination system according to one embodiment of the present invention. FIG. 6 is a flowchart showing a specific example of processing of the learning device of the determination system according to one embodiment of the present invention. FIG. 7 is a schematic block diagram showing a specific example of the functional configuration of a determination device of the determination system according to one embodiment of the present invention. FIG. 8 is a flowchart showing a specific example of processing of the determination device of the determination system according to one embodiment of the present invention. FIG. 9 is a diagram showing an outline of an example of the hardware configuration of an information processing device 90 applied to the present invention. FIG. 10 is a diagram showing an absorption spectrum obtained by acquiring 46 absorbances related to bovine blood according to wavelengths in the range of 900 nm to 1694 nm from image data of bovine blood vessels generated by a multispectral camera in Example 1. FIG. 11 is a diagram showing an overview of learning and verification of a classification model in Example 2.

[0012] The analytical method and determination system of the present invention will be described below. Note that the present embodiment is specifically described to provide a better understanding of the gist of the invention, and does not limit the present invention unless otherwise specified.

[0013] [Analysis Method] An analysis method according to one embodiment of the present invention includes a first step of acquiring feature quantities relating to a plurality of values ​​corresponding to wavelength from image data of the blood vessels of an animal to be evaluated, which image data is generated by photographing the shallow portion (veins) of the body surface of the animal with a multispectral camera, and a second step of estimating the concentrations of blood components contained in the blood in the blood vessels based on the feature quantities. The multispectral camera used in this embodiment is a device that captures images with a spectral resolution higher than that of a camera that captures at least RGB. A hyperspectral camera may be used as the multispectral camera.

[0014] FIG. 1 is a diagram showing the flow of an analysis method according to one embodiment of the present invention.

[0015] "First Step" In the first step, first, the superficial body surface (veins) of the animal to be assessed is photographed with a multispectral camera, and image data of the blood vessels of the animal is generated (S1).

[0016] The animals to be determined by the analysis method of this embodiment are not particularly limited, but examples include livestock such as cows, pigs, horses, dogs, and chickens, as well as other animals kept by humans on ranches, zoos, etc.

[0017] The part of the animal to be assessed that is photographed with the multispectral camera is not particularly limited, but is preferably a hairless part such as the inside of the animal's ear or the base of the tail, where blood vessels can be visually observed from the body surface. When photographing a hairy part, the hair at that part is shaved so that blood vessels can be visually observed from the body surface. When the animal to be assessed is a cow, the base of the tail is preferred, since the base of a cow's tail does not have hair. The underside of the base of the tail may also be used.

[0018] A multispectral camera is a spectroscopic camera that captures images at multiple wavelengths. Image data captured by a multispectral camera includes information for each wavelength in addition to two-dimensional planar data in the x and y directions.

[0019] In the first step, feature quantities relating to a plurality of values ​​corresponding to wavelength are acquired from image data of the animal's blood vessels generated by a multispectral camera. Examples of the plurality of values ​​corresponding to wavelength include values ​​relating to the absorbance of the blood in the blood vessels (absorption spectra). Among these, values ​​relating to the absorbance of the blood in the blood vessels are preferred because different components contained in the blood in the blood vessels exhibit different absorption spectra.

[0020] In the first step, it is preferable to obtain at least n (n≧228) values ​​according to wavelengths in the wavelength range of 900 nm or more and 1694 nm or less.

[0021] Among the components contained in blood vessels, examples of components for which feature quantities related to multiple values ​​according to wavelength can be obtained from image data generated by a multispectral camera include glucose (for evaluating energy metabolism), serum urea nitrogen (for evaluating protein metabolism), total cholesterol (for evaluating lipid metabolism), calcium (for diagnosing minerals), and vitamin A. These components may be analyzed singly or in combination with two or more. Since the feature quantities (e.g., absorption spectra) of each component exhibit unique characteristics, feature quantities related to each component can be extracted even when multiple components are analyzed simultaneously.

[0022] "Second Step" In the second step, the concentrations of blood components contained in the blood in the blood vessel are estimated based on the feature amounts (for example, absorption spectra) obtained in the first step (S2).

[0023] To estimate the concentrations of the blood components, blood is collected from a blood vessel in advance, and feature quantities related to multiple values ​​corresponding to wavelength are obtained from the actual measured values ​​of the components and image data generated by a multispectral camera, and a database of the feature quantities is created.The database is compared with feature quantities related to multiple values ​​corresponding to wavelength obtained from the image data of the animal's blood vessels, and the concentrations of the blood components contained in the blood in the blood vessels are estimated.

[0024] Alternatively, to estimate the concentrations of the blood components, a database of the feature quantities is created by acquiring the actual measured values ​​of the components and feature quantities related to multiple values ​​corresponding to wavelengths obtained from the image data of the blood vessels of the animal, and comparing the database with the feature quantities related to multiple values ​​corresponding to wavelengths obtained from the image data of the blood vessels of the animal to estimate the concentrations of the blood components contained in the blood in the blood vessels.

[0025] Since absorption spectra are easily affected by optical interference, it is preferable to use a five-point moving average method as a method for estimating the concentrations of the blood components in order to remove noise. In the analysis method of this embodiment, the five-point moving average method selects any five points in the wavelength range for measuring the absorbance of blood in the blood vessels, and averages the absorbance within these wavelength ranges.

[0026] In the second step, it is preferable to classify the concentrations of the blood components based on feature quantities relating to a plurality of values ​​including the n values ​​acquired in the first step.

[0027] Here, with reference to FIGS. 2 to 4, an example of classifying blood component concentrations according to the blood component concentrations will be described. As shown in FIG. 2, in order to remove noise contained in the absorption spectrum data obtained in the first step, a moving average process of the absorption spectrum is performed in the input layer 101. Next, various feature quantities (specific frequency components) contained in the absorption spectrum are extracted using 64 types of filters in the convolutional layer 102. Note that the convolutional layer 102 has 64 types of filters for 228 spectral data. In the convolutional layer 102, an m×1 filter (m is a positive integer) is multiplied by the 228×1 spectral data, and an operation of adding and shifting the data horizontally is performed. For example, an 8×1 filter is multiplied by the 228×1 spectral data, and an operation of adding and shifting the data horizontally is performed. The obtained feature quantities (specific frequency components) are subjected to Max Pooling process (1), reducing the number of data by half. In Max Pooling process (1), the representative value of a region in the feature map is extracted. The maximum value in the 2x1 feature map is found, and the number of data is halved by sliding it.

[0028] Next, as shown in FIG. 3 , the convolutional layer 103 extracts features from the data halved by the Max Pooling process (1) using 64 types of filters. In the convolutional layer 103, 114 represents the spectral data halved from 228 by the Max Pooling process (1). The 64 in the convolutional layer 103 indicates that there are 64 types of filters, as in the convolutional layer 102. In the convolutional layer 103, an m×1 filter (m is a positive integer) is multiplied by the 114×1 spectral data, and the data is shifted horizontally while being added. For example, an 8×1 filter is multiplied by the 114×1 spectral data, and the data is shifted horizontally while being added. The resulting features (specific frequency components) are then subjected to Max Pooling process (2), further reducing the number of data by half. In Max Pooling process (2), the representative value of a region of the feature map is extracted. The maximum value in the 2x1 feature map is found, and the number of data is halved by sliding it.

[0029] Finally, as shown in FIG. 4, the convolutional layer 104 uses 32 types of filters to extract various features (specific frequency components) contained in the absorption spectrum. Note that in the convolutional layer 104, 57×32 indicates that there are half (57) of the 114 spectral data in the convolutional layer 103 and 32 types of filters. In FIG. 4, 1824×1 indicates that the results of 57×32 are arranged horizontally. The results of dividing the blood component concentrations into six classes are output as the output layer 105.

[0030] According to the analysis method of this embodiment, since it includes the first step and the second step, it is possible to grasp the health condition of the animal to be evaluated non-invasively and on the spot.

[0031] According to the analysis method of this embodiment, by photographing the superficial body surface (veins) of the animal to be evaluated with a multispectral camera, it is possible to directly photograph the thick midline tail vein located shallowly under the skin.

[0032] According to the analysis method of this embodiment, in the second step, the concentrations of blood components are classified according to the concentrations of the blood components, thereby making it possible to determine whether the component values ​​of the animal being evaluated deviate from the standard values.

[0033] According to the analysis method of this embodiment, the image data of the blood vessels is image data of the underside of the tail (median tail vein) of the animal being evaluated, so that the thick median tail vein located shallowly under the skin can be directly photographed.

[0034] [Determination System] Figure 5 is a schematic block diagram showing a determination system according to one embodiment of the present invention. The determination system 10 of this embodiment includes a learning device 20, a determination device 30, and a display device 5. An imaging device 40 that captures multispectral images is connected to the determination device 30. The imaging device 40 captures images with a spectral resolution higher than that of a camera that captures at least RGB. The imaging device 40 is, for example, a multispectral camera or a hyperspectral camera. The imaging device 40 is a camera that captures images of the body surface of the animal to be determined. Examples of the imaging device 40 include the near-infrared spectrometer "S-G1" manufactured by Nanoseed Co., Ltd. and the multi-hyperspectral camera XIMEA-SNAPSHOT manufactured by XIMEA.

[0035] The learning device 20 and the determination device 30 may be communicably connected via a network. The network may be a network using wireless communication or a network using wired communication. The network may be configured using, for example, the Internet or a local area network (LAN). The network may also be configured by combining multiple networks.

[0036] 6 is a schematic block diagram showing a specific example of the functional configuration of the learning device 20. The learning device 20 is configured using an information processing device such as a personal computer or a server device. The learning device 20 includes a communication unit 21, a storage unit 22, and a control unit 23.

[0037] The communication unit 21 is a communication device. For example, the communication unit 21 may be configured as a network interface. The communication unit 21 communicates data with other devices via a network in accordance with the control of the control unit 23. The communication unit 21 may be a device that performs wireless communication or a device that performs wired communication.

[0038] The storage unit 22 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 22 stores data used by the control unit 23. The storage unit 22 may function as, for example, a training data storage unit 221 and a feature extraction model storage unit 222.

[0039] The learning data storage unit 221 stores learning data used in the learning process executed by the learning device 20. The learning data includes data of multiple multispectral images obtained by capturing images of multiple learning target animals. The learning target animals may be animals of the same species as the animal to be determined, or animals of a different species, as long as the individual is different from the animal to be determined. Furthermore, multiple species of animals may be used as learning target animals, or animals of a single species may be used. Note that if the learning target animals and the animal to be determined are animals of similar species, the accuracy of determination may be higher. The learning data also includes label information indicating the correct information (concentration of blood components) that should be obtained as the determination result for the object to be determined captured in each multispectral image. A specific example of a blood component is glucose.

[0040] The feature extraction model storage unit 222 stores a trained model obtained by a training process using the training data stored in the training data storage unit 221.

[0041] The control unit 23 is configured using a processor such as a CPU and a memory. The control unit 23 functions as an information control unit 231 and a learning control unit 232 by the processor executing a program. Note that all or part of the functions of the control unit 23 may be realized using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and a semiconductor storage device (e.g., an SSD), as well as storage devices such as a hard disk or semiconductor storage device built into a computer system. The above program may be transmitted via a telecommunications line.

[0042] The information control unit 231 controls the input and output of information. For example, the information control unit 231 acquires learning data from other devices (information processing devices or storage media) and records the learning data in the learning data storage unit 221. For example, the information control unit 231 transmits the learned model stored in the feature extraction model storage unit 222 to another device (for example, the determination device 30).

[0043] The learning control unit 232 executes a learning process using the learning data. Specific examples of such learning processes include supervised learning for classification, such as support vector machines, random forests, and neural networks. The learning control unit 232 generates a trained model for outputting the concentrations of blood components based on the input multispectral image, for example, by performing supervised learning. The learning control unit 232 records the generated trained model in the feature extraction model storage unit 222. The trained model obtained by the learning control unit 232 may be transmitted to the determination device 30 and recorded in the trained model storage unit 321 of the determination device 30.

[0044] 7 is a flowchart showing a specific example of processing by the learning device 20. First, the information control unit 231 acquires learning data (step S101). The learning data may be, for example, input by a user, acquired via communication from another information device, or acquired from a recording medium connected to the learning device 20. The learning control unit 232 executes a learning process using the learning data and records a trained model in the feature extraction model storage unit 222 (step S102).

[0045] 8 is a schematic block diagram showing a specific example of the functional configuration of the determination device 30. The determination device 30 is configured using an information processing device such as a smartphone, a tablet, a personal computer, a dedicated device, etc. The determination device 30 includes an image input unit 31, a storage unit 32, a control unit 33, and an output unit 34.

[0046] The image input unit 31 accepts multispectral image data input to the determination device 30. The image input unit 31 accepts input of multispectral image data captured by the imaging device 40. There are no limitations on how the image input unit 31 accepts input of multispectral image data from the imaging device 40. The image input unit 31 may be connected to the imaging device 40 via, for example, a USB (Universal Serial Bus) or another communication cable, and may accept input of image data via the communication cable. The image input unit 31 may be communicably connected to the imaging device 40 via wireless communication such as Bluetooth (registered trademark), and may accept input of image data.

[0047] The storage unit 32 is configured using a storage device such as a magnetic hard disk drive or a semiconductor storage device. The storage unit 32 stores data used by the control unit 33. The storage unit 32 may function as a trained model storage unit 321, for example.

[0048] The trained model storage unit 321 stores a trained model to be used when the determination unit 332 performs determination processing. The trained model is generated in advance by the learning device 20.

[0049] The control unit 33 is configured using a processor such as a CPU and a memory. The control unit 33 functions as an information control unit 331 and a determination unit 332 by the processor executing a program. Note that all or part of the functions of the control unit 33 may be realized using hardware such as an ASIC, PLD, or FPGA. The above program may be recorded on a computer-readable recording medium. Examples of computer-readable recording media include portable media such as a flexible disk, a magneto-optical disk, a ROM, a CD-ROM, and a semiconductor storage device (e.g., an SSD), as well as storage devices such as a hard disk or semiconductor storage device built into a computer system. The above program may be transmitted via a telecommunications line.

[0050] The information control unit 331 acquires a trained model from another device such as the learning device 20. The information control unit 331 acquires multispectral image data input from the image input unit 31. The information control unit 331 outputs information indicating the determination result obtained by the determination unit 332 to the outside via the output unit 34. For example, the information control unit 331 may transmit the information indicating the determination result to another device, or may output the information indicating the determination result from the output unit 34 via a display or speaker.

[0051] The determination unit 332 performs a determination process using the trained model stored in the trained model storage unit 321 and the multispectral image input from the image input unit 31. The determination process determines the concentrations of blood components of the target animal for which the multispectral image was captured. Specifically, the determination unit 332 acquires feature quantities related to multiple values ​​corresponding to wavelengths from the image data of the animal's blood vessels generated by the imaging device 40. As described above, for example, the feature quantities are the absorption spectra of blood components contained in the blood in the upper blood vessels.

[0052] The output unit 34 outputs a signal to an output device such as a display, a speaker, etc. For example, the output unit 34 may output an image signal indicating the determination result or an audio signal indicating the determination result.

[0053] 9 is a flowchart showing a specific example of the processing of the determination device 30. First, the information control unit 331 acquires a multispectral image from the imaging device 40 (step S201). The determination unit 332 performs a determination process using the multispectral image (step S202). The determination unit 332 outputs information indicating the determination result from the output unit 34 to the output device (step S203).

[0054] FIG. 10 is a diagram illustrating an outline of an example hardware configuration of an information processing device 90 applied to this embodiment. The information processing device 90 includes a processor 91, a main storage device 92, a communication interface 93, an auxiliary storage device 94, an input / output interface 95, and an internal bus 96. The processor 91, the main storage device 92, the communication interface 93, the auxiliary storage device 94, and the input / output interface 95 are communicably connected to each other via the internal bus 96. The information processing device 90 may be applied to, for example, the learning device 20 and the determination device 30. In this case, for example, the communication unit 21 and the output unit 34 may be configured using the communication interface 93. For example, the image input unit 31 and the output unit 34 may be configured using the input / output interface 95. For example, the memory unit 22 and the memory unit 32 may be configured using the auxiliary storage device 94. Furthermore, the control unit 23 and the control unit 33 may be configured using the processor 91 and the main storage device 92.

[0055] A specific example of a device to which the output unit 34 outputs is a display device 5. The display device 5 displays the determination result obtained by the determination device 30. Examples of the display device 5 include a personal computer monitor, a liquid crystal display, an organic electroluminescence (EL) display, and a mobile terminal such as a smartphone or a tablet computer. The display device 5 is connected to the determination device 30 via a wireless LAN (Local Area Network) or a wired LAN. If a mobile terminal is used as the display device 5, the analyst can check the estimation result at his or her fingertips. Furthermore, if the display device 5 is equipped with an app for displaying the estimation result, the health condition of the animal being measured can be determined based on the estimation result and the determination result can be displayed. The determination result of the animal's health condition is displayed as, for example, "best," "good," or "poor." The determination device 30 and the display device 5 may be configured as an integrated device.

[0056] According to the determination system 10 of this embodiment, since it is equipped with an imaging device 40 (e.g., a multispectral camera) and a determination device 30, it is possible to grasp the health condition of the animal to be determined non-invasively and on the spot.

[0057] The present invention will be explained in more detail below with reference to examples, but the present invention is not limited to the following examples.

[0058] Example 1: The surface of the base of a cow's tail was photographed with a multispectral camera to generate image data of the cow's blood vessels. The multispectral camera used was a near-infrared spectrometer "S-G1" manufactured by Nanoseed Co., Ltd. Next, 46 absorbance values ​​for the cow's blood corresponding to wavelengths between 900 nm and 1694 nm were obtained from the image data of the cow's blood vessels generated by the multispectral camera, yielding the absorption spectrum shown in Figure 11. The glucose concentration in the cow's blood was estimated based on the obtained absorption spectrum. Table 1 shows an example in which the range of actual measured blood glucose concentrations and the number of samples were classified when blood was collected. Table 2 shows the classification accuracy of the glucose concentration in the cow's blood estimated for each class using the analytical method of the present invention.

[0059]

[0060]

[0061] From the results shown in Tables 1 and 2, the classification accuracy of glucose concentration was 97.05%. Therefore, it was confirmed that the analytical method of the present invention is an effective method for estimating the concentrations of components such as glucose contained in animal blood.

[0062] Example 2: The surface of the base of a cow's tail was photographed with a multispectral camera to generate image data of the cow's blood vessels. The multispectral camera used was a near-infrared spectrometer "S-G1" manufactured by Nanoseed Co., Ltd. Next, 46 absorbance values ​​related to the cow's blood corresponding to wavelengths between 900 nm and 1694 nm were acquired from the image data of the cow's blood vessels generated by the multispectral camera, to obtain an absorption spectrum. The obtained absorption spectrum data was divided into four training data (Train) and one test data (Test), as shown in Figure 12, and training and verification of a classification model were performed. The training data and test data were changed and repeated five times, and the average score was used as the evaluation value of the classification model. A total of 303 absorption spectra were measured for 22 Japanese Black cows over an eight-month period. The training data was divided into 242 sets and 61 sets of test data so that the ratio of the number of data for each label was the same, and the classification model was trained and verified. The results are shown in Table 3.

[0063]

[0064] 10 Determination system 20 Learning device 30 Determination device 40 Imaging device 5 Display device

Claims

1. An analysis method comprising: a first step of acquiring feature quantities relating to a plurality of values ​​corresponding to wavelengths from image data of the blood vessels of an animal to be evaluated, which is generated by photographing the superficial part of the body surface (veins) of the animal with a multispectral camera; and a second step of estimating the concentration of blood components contained in the blood in the blood vessels based on the feature quantities.

2. The analytical method according to claim 1, wherein the plurality of values ​​according to wavelength are values ​​related to the absorbance of blood in the blood vessel.

3. The analytical method according to claim 1, wherein in the first step, at least n (n≧228) of the values ​​in the wavelength range of 900 nm or more and 1694 nm or less are obtained, and in the second step, the concentrations of the blood components are classified into classes according to the concentrations of the blood components based on feature quantities relating to a plurality of values ​​including the n values.

4. The analysis method according to claim 1, wherein the blood vessel image data is image data of the underside of the tail (median tail vein) of the animal to be assessed.

5. A determination system comprising: an imaging device that photographs the body surface of an animal to be determined; a learning device that acquires feature amounts relating to a plurality of values ​​according to wavelength from image data of the animal's blood vessels generated by the imaging device and learns the feature amounts; and a determination device that estimates the concentration of blood components contained in the blood in the blood vessels based on the feature amounts.

Images