Information processing device, method, and program

Abstract

To provide a technology that does not reduce the accuracy of analytical processing in the analysis using a machine learning model based on spectral information.SOLUTION: An information processing device includes: a spectral information acquisition unit that acquires spectral information of an object from a multispectral image of the object; a learning processing unit that receives the spectral information of the object as an input and generates an estimation model that outputs component amounts of components in the object; an application processing unit that estimates the component amounts of the components in the object using the estimation model; and an output unit that outputs the component amounts estimated by the application processing unit. The application processing unit inputs, as input data, spectral information acquired from a second multispectral image, which is a multispectral image with lower spatial resolution than a first multispectral image, which is a multispectral image from which spectral information used as learning data when generating the estimation model was acquired, into the estimation model to estimate the component amounts.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a method, and a program. [Background technology]

[0002] In the technology described in Patent Document 1, the growth status of crops is analyzed using spectral information extracted from a spectral image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2006-250827 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, analysis using spectral information can be performed using a machine learning model. Generally, the resolution of the training data and the target data used in inference are consistent. The inventors discovered that, depending on the application of the analysis process, the accuracy of the analysis process decreases when the distribution of features in the feature space used in learning is close to the distribution of features in the feature space used in inference. Therefore, a technology that does not decrease the accuracy of the analysis process has been desired. [Means for solving the problem]

[0005] The present disclosure can be realized in the following forms.

[0006] According to a first aspect of the present disclosure, there is provided an information processing device. The information processing device includes: a spectral information acquisition unit that acquires spectral information of an object from a multispectral image of the object; a learning processing unit that receives the spectral information of the object as an input and generates an estimation model that outputs component amounts of the components of the object; an application processing unit that estimates the component amounts of the object using the estimation model; and an output unit that outputs the component amounts estimated by the application processing unit. The application processing unit inputs, as input data to the estimation model, spectral information acquired from a second multispectral image, which is a multispectral image with a lower positional resolution than a first multispectral image, which is a multispectral image from which the spectral information used as learning data when generating the estimation model was acquired, to the estimation model, thereby estimating the component amounts. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram illustrating a schematic configuration of an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 10 is an explanatory diagram showing an example of a spectrum. [Figure 3] 10 is a flowchart showing a processing procedure for generating an estimation model. [Figure 4] FIG. 2 is an explanatory diagram of an example of a stand for fixing a camera. [Figure 5] FIG. 10 is an explanatory diagram for specifying the position and size of a target area. [Figure 6] 10 is a flowchart showing a processing procedure for applying an estimation model. [Figure 7] FIG. 10 is an explanatory diagram of a reception image. [Figure 8] FIG. 10 is an explanatory diagram of a completed image. DETAILED DESCRIPTION OF THE INVENTION

[0008] A. First embodiment: 1 is a block diagram showing a schematic configuration of an information processing system 1 according to this embodiment. The information processing system 1 is used to calibrate the amounts of components of an object. The information processing system 1 includes an information processing device 100 and a camera 300.

[0009] A multispectral image is acquired by capturing an image of an object using camera 300. The object may be, for example, processed foods such as processed agricultural products and processed seafood products, fertilizer, animal feed, pet food, vegetables, pesticides, or soil. The object may be solid or liquid.

[0010] The camera 300 is a sensor capable of detecting information of two or more wavelength bands. For example, an RGB camera, a multispectral camera, or a hyperspectral camera can be used as the camera 300. A multispectral image is an image that records light in two or more wavelength ranges. The multispectral image acquired by the camera 300 is an image that records light of a specific wavelength reflected by an object. The specific wavelength is a wavelength band that the camera 300 can detect. In this embodiment, the multispectral camera that serves as the camera 300 captures an image of a vegetable VG, which is the object.

[0011] The information processing device 100 estimates the component amounts of components of an object using a multispectral image acquired by a camera 300. The information processing device 100 includes a memory 110, an interface circuit 120, an input device 130, a display device 140, and a processor 150. The display device 140 is also referred to as a "display unit." The information processing device 100 is also referred to as a "computer."

[0012] The memory 110 stores various programs and various data used for various processes executed by the information processing device 100. The memory 110 stores a program P1 and data representing an estimation model M1, which is a machine learning model described below.

[0013] An input device 130 and a display device 140 are connected to the interface circuit 120. The input device 130 is, for example, a keyboard or a mouse. The display device 140 is, for example, a liquid crystal display or an organic EL (Electro Luminescence) display.

[0014] The processor 150 realizes various functions by executing programs stored in the memory 110. By executing the program P1 stored in the memory 110, the processor 150 functions as a multispectral image acquisition unit 210, a spectrum extraction unit 220, a learning processing unit 230, an application processing unit 240, and an output unit 250.

[0015] The multispectral image acquisition unit 210 acquires a multispectral image captured by the camera 300 .

[0016] The spectrum extraction unit 220 acquires spectral information of the object from the multispectral image. The spectral information can be light intensity, spectral reflectance, or the time integral of light intensity (amount of light). The light intensity represents the intensity of light received by the image sensor of the camera 300. The time integral of light intensity represents the amount of light received by the image sensor of the camera 300 per unit time. The luminance of each pixel in the multispectral image represents the intensity of light received by the image sensor of the camera 300 corresponding to that pixel. If the multispectral image is an RGB image, the luminance is calculated from the three RGB pixel values ​​of each pixel using a known conversion formula. The spectral reflectance is obtained by dividing the spectrum representing the luminance for each wavelength by a reference spectrum. For example, the spectrum representing the spectral reflectance of a calibrated standard white board can be used as the reference spectrum. The spectrum extraction unit 220 is also referred to as a "spectral information acquisition unit."

[0017] FIG. 2 is an explanatory diagram showing an example of a spectrum as spectral information. When the spectral information indicates light intensity, the horizontal axis represents wavelength, and the vertical axis represents light intensity. When the spectral information indicates reflectance, the vertical axis represents reflectance. In this specification, spectrum refers to a spectral spectrum. The spectrum extraction unit 220 is also referred to as a "spectral information acquisition unit."

[0018] In this embodiment, the spectrum extraction unit 220 receives a specification from the user regarding a spectrum acquisition method, and acquires a spectrum from a multispectral image as spectroscopic information using the specified acquisition method. The spectrum extraction unit 220 is also referred to as a “reception unit.”

[0019] The learning processing unit 230 generates an estimation model M1, which is a machine learning model, through machine learning using the learning data. The estimation model M1 is a machine learning model that receives a spectrum indicating spectroscopic information as input and outputs the component amounts of the components of the target object.

[0020] The application processing unit 240 estimates the component amounts of the components of the object by inputting the spectrum to the estimation model M1.

[0021] 3 is a flowchart showing the processing steps of the generation processing of the estimation model M1. The processing shown in FIG. 3 is started when the processor 150 receives an operation instruction from the administrator via the input device 130, for example.

[0022] In step S101, a plurality of multispectral images IM1 for learning are acquired. The multispectral images IM1 are acquired by using the camera 300 to capture images of vegetables of the same type as the vegetable VG whose component amounts are to be estimated. For example, images of a plurality of vegetables of the same type as the target vegetable VG are captured to acquire a plurality of multispectral images IM1. The multispectral image IM1 is also referred to as a "first multispectral image." The processing of step S101 shown in FIG. 3 is executed by the processor 150 functioning as the multispectral image acquisition unit 210.

[0023] In each image capture to acquire multiple multispectral images IM1, the distance between camera 300 and the subject is adjusted so that the distance (shooting distance) between camera 300 and the subject, a vegetable of the same type as the subject vegetable VG, is the same. For example, the distance between camera 300 and the subject is adjusted as follows.

[0024] Fig. 4 is an explanatory diagram of an example of a stand for fixing camera 300. In Fig. 4, stand ST includes a support pole PL, a base part BS to which support pole PL is fixed, a tray TL fixed to base part BS, and a fixing part FX for fixing camera 300 to the tip of support pole PL. By having camera 300 capture an image of a subject placed on tray TL, the distance between camera 300 and the subject can be kept constant.

[0025] In step S102, a specification of a spectrum acquisition method for the multispectral image IM1 is accepted. For example, the processor 150 displays an image for accepting the specification of the spectrum acquisition method on the display device 140. The processor 150 accepts the specification value of the acquisition method input by the administrator using the input device 130.

[0026] Specification of the spectrum acquisition method includes the following: (i) Selection of preprocessing steps for multispectral images before spectral extraction (ii) The location and size of the target area from which the spectrum is to be extracted within the multispectral image. (iii) Selection of correction procedures for extracted spectra

[0027] As options for pre-processing of the multispectral image (i) above, for example, contrast correction, histogram equalization that flattens the brightness histogram overall, and blurring are presented to the administrator.

[0028] FIG. 5 is an explanatory diagram for specifying the position and size of a target region (partial region). To receive the specification of the position and size of the target region in (ii) above, for example, a multispectral image IM1 is displayed on the display device 140. The administrator uses the input device 130 to specify the position and size of the target region within the multispectral image IM1. Note that multiple target regions may be selected in one multispectral image IM1. In FIG. 5, the area enclosed by a dashed rectangular frame represents the specified target region. A spectrum is extracted for each specified target region.

[0029] As options for the correction process for the extracted spectrum in (iii) above, for example, baseline correction and second-order derivative processing are presented to the administrator. The correction process for the spectrum is also called "processing to maintain the consistency of the spectroscopic information." The process of step S102 is executed by the processor 150 functioning as the spectrum extraction unit 220.

[0030] As shown in FIG. 3, in step S103, a spectrum is acquired from each region of interest in each multispectral image IM1.

[0031] For example, if five target regions are selected by the administrator in step S102, five spectra are acquired. If no target region is specified in step S102, a preset default region is used as the target region. The default region is, for example, a region of a certain size centered on the central coordinates of the multispectral image IM1.

[0032] In step S102, if a designation of pre-processing for the multispectral image IM1 is accepted from the administrator, the designated pre-processing is first performed on the multispectral image IM1. Then, a spectrum is acquired from the target region of the multispectral image IM1 on which the pre-processing has been performed. Note that, if no pre-processing for the multispectral image IM1 is designated in step S102, a default pre-processing may be performed on the multispectral image IM1, or the execution of the pre-processing may be omitted.

[0033] Furthermore, in step S102, if a specification of a correction process for the spectrum is accepted from the administrator, a spectrum is acquired from the multispectral image IM1 for which preprocessing has been performed or preprocessing has been omitted. Then, the specified correction process is performed on the acquired spectrum. Note that, if a correction process for the spectrum is not specified in step S102, a default correction process may be performed on the spectrum, or the execution of the correction process may be omitted. The process of step S103 is executed by the processor 150 functioning as the spectrum extraction unit 220.

[0034] In step S104, a target value is set for each spectrum extracted from the multispectral image IM1 (see FIG. 2). The process of step S104 is executed by the processor 150 functioning as the learning processing unit 230.

[0035] The target values ​​represent the amounts of the components of the vegetables VG. Examples of the components of the vegetables VG include protein, carbohydrates, potassium, and calcium. For example, the amount of each component is analyzed by an analytical device for each object in each multispectral image IM1 from which the spectrum was extracted. The set of component amounts of each component output by the analytical device is used as the target values. In this embodiment, multiple spectra and target values ​​associated with each spectrum are used as training data.

[0036] Furthermore, each spectrum extracted from each of a plurality of target regions in the same multispectral image IM1 is associated with a set of component amounts of each component of the target object in that multispectral image IM1 as target values.

[0037] In step S105, an estimation model M1 is generated using training data including a plurality of spectra and target values ​​associated with each spectrum. The processing of step S105 is executed by the processor 150 functioning as the training processing unit 230. For example, the estimation model M1 is generated using a machine learning algorithm such as partial least squares regression (PLS regression), linear regression, or SVM (Support Vector Machine).

[0038] The estimation model M1 is a regression model that receives spectral information extracted from a multispectral image as input and estimates the component amounts of components in an object. In this embodiment, the spectral information input to the estimation model M1 is a spectrum (see FIG. 2). Data representing the generated estimation model M1 is stored in the memory 110. Thereafter, the processing shown in FIG. 3 ends. The above is the processing related to the generation of the estimation model M1.

[0039] As described above, in step S102, a specification of a method for acquiring spectral information (spectrum) is accepted. Furthermore, the administrator can specify, as the method for acquiring spectral information, the region in the multispectral image IM1 from which the spectral information is to be acquired, preprocessing, etc. Therefore, the estimation model M1 is generated using the spectral information acquired by the method desired by the administrator as training data.

[0040] Fig. 6 is a flowchart showing the processing steps of the application process of the estimation model M1. The processing shown in Fig. 6 starts, for example, when the processor 150 receives an operation instruction from the user via the input device 130. The user may be the same person as the administrator who performed the operation to generate the estimation model M1, or may be a different person. If the user is a person other than the administrator, it is desirable that the administrator explain to the user, for example, how to specify the spectrum acquisition method, etc., prior to executing the application process.

[0041] In step S201, an original multispectral image IM0 is acquired. The original multispectral image IM0 is acquired by capturing an image of a vegetable VG, the subject of which component amounts are to be estimated, using a camera 300. The distance (shooting distance) between the camera 300 and the subject vegetable VG is adjusted so that the distance becomes the same as the shooting distance when each multispectral image IM1 was acquired. The processing of step S201 is executed by the processor 150, which functions as the multispectral image acquisition unit 210.

[0042] In step S202, blurring is performed on the acquired original multispectral image IM0, thereby reducing the positional resolution of the original multispectral image IM0. For example, blurring is performed using a Gaussian filter. Using a Gaussian filter makes it possible to make the acquired multispectral image as close to the original image as possible. The blurring adds weighted values ​​of surrounding pixels to the pixel value of the pixel of interest. Furthermore, the shooting distance when the original multispectral image IM0 was acquired is the same as the shooting distance when each multispectral image IM1 was acquired. The multispectral image obtained by blurring is referred to as multispectral image IM2. Bluring can easily generate a multispectral image IM2 with a positional resolution lower than that of the training multispectral image IM1. In this specification, positional resolution refers to the ability to distinguish differences in position in real space corresponding to a region of interest using image information in that region. The unit of the region of interest is, for example, one pixel. The processing of step S202 is performed by the processor 150 functioning as the multispectral image acquisition unit 210. The original multispectral image IM0 before blurring is also referred to as the "third multispectral image." The multispectral image IM2 obtained by blurring is also referred to as the "second multispectral image." The blurring is also referred to as "filtering." The multispectral image acquisition unit 210 is also referred to as the "spectral image generation unit."

[0043] In step S203, a specification of a spectrum acquisition method for the multispectral image IM2 is accepted. For example, the processor 150 displays an image for accepting the specification of the spectrum acquisition method on the display device 140. The processor 150 accepts a specification value for the acquisition method input by the user using the input device 130. The specification of the spectrum acquisition method is similar to the specification specified in the process of generating the estimation model M1 as follows: (i) Selection of preprocessing steps for multispectral images before spectral extraction (ii) The location and size of the target area from which the spectrum is to be extracted within the multispectral image. (iii) Selection of correction procedures for extracted spectra

[0044] Here, the specification of the spectrum acquisition method in step S203 is preferably the same as or similar to the content specified for the spectrum acquisition method when generating the estimation model M1 (see step S102 in FIG. 3). Therefore, on the screen for accepting the specification of the spectrum acquisition method, the specification value specified for the spectrum acquisition method when generating the estimation model M1 is set as the initial setting. Furthermore, to provide the user with explanatory information on how to specify the spectrum acquisition method, an image showing the explanatory information may be displayed on the display device 140. The processing of step S203 is executed by the processor 150 functioning as the spectrum extraction unit 220.

[0045] In step S204, a spectrum is acquired from each target region of the multispectral image IM2. For example, if five target regions are designated by the user in step S203, five spectra are acquired. Note that if no target region is designated in step S203, a preset default region is used as the target region.

[0046] In step S203, if a user specifies preprocessing for the multispectral image IM2, the specified preprocessing is first performed on the multispectral image IM2. Then, a spectrum is acquired from the target region of the preprocessed multispectral image IM2. If no preprocessing for the multispectral image IM2 is specified in step S203, default preprocessing may be performed on the multispectral image IM2, or preprocessing may be omitted.

[0047] Furthermore, in step S203, if a specification of a correction process for the spectrum is accepted from the user, a spectrum is acquired from the multispectral image IM2 for which preprocessing has been performed or preprocessing has been omitted. The specified correction process is then performed on the acquired spectrum. Note that, if a correction process for the spectrum is not specified in step S203, a default correction process may be performed on the spectrum acquired from the multispectral image IM2, or the execution of the correction process may be omitted. The process of step S204 is performed by the processor 150 functioning as the spectrum extraction unit 220.

[0048] In step S205, the spectrum acquired from the multispectral image IM2 is input to the estimation model M1 to estimate the component amounts of the components of the vegetables VG. The process of step S204 is executed by the processor 150 functioning as the application processing unit 240.

[0049] In step S206, the component amount of each component of the vegetables VG indicated by the output value of the estimation model M1 is output as an estimation result to the memory 110. Furthermore, the estimated component amount of each component may be displayed on the display device 140 together with the multispectral image IM2 and the spectrum acquired from the multispectral image IM2. Furthermore, the original multispectral image IM0 may be displayed on the display device 140. The processing of step S206 is executed by the processor 150 functioning as the application processing unit 240.

[0050] In step S207, it is determined whether or not to continue the processing. For example, when processor 150 receives an operation instruction from the user via input device 130, the processing ends. If the processing continues (step S207; YES), the processing of step S201 is executed again. If the processing ends (step S207; NO), the processing shown in FIG. 6 ends. The above is the processing related to the application of estimation model M1.

[0051] As described above, in step S203, a specification of the method for acquiring spectral information (spectrum) is accepted. Furthermore, the user can specify the area within the multispectral image IM1 from which spectral information is acquired, preprocessing, and the like, as the method for acquiring spectral information. The user can obtain an estimated value of the component amount of the component of the object based on the spectral information acquired by the desired method. Furthermore, calibrating the component amount of the component of the object using analytical equipment can require a lot of time and complicated work. In this embodiment, after performing a blurring process on the original multispectral image IM0 capturing the object, calibration can be performed using the estimation model M1. Therefore, it is possible to reduce time and effort compared to using analytical equipment. Furthermore, calibration using the estimation model M1 of this embodiment does not require destroying the object.

[0052] Next, the reason why the configuration according to the embodiment can suppress a decrease in the accuracy of the estimation result will be explained.

[0053] For example, when estimating the component amounts of an object, the positional resolution of the multispectral image IM1 from which the spectra used as training data during training were extracted is matched with the positional resolution of the multispectral image IM2 from which the spectra used when applying the estimation model M1 were extracted. As a result, it is expected that the distribution range in feature space of the features of the training data during training will be close to the distribution range in feature space of the features of the input data input to the estimation model M1 when applying the estimation model M1. Here, the spectra extracted from the multispectral image IM1 correspond to the features of the training data. The spectra extracted from the multispectral image IM2 correspond to the features of the input data. Furthermore, the distribution range in feature space of the features of the training data during training being close to the distribution range in feature space of the features of the input data input to the estimation model M1 when applying the estimation model M1 means that, when the features of the training data during training and the features of the input data when applied are plotted in feature space, the position and size of the distribution range of the features of the training data during training are close to the position and size of the distribution range of the features of the input data when applied.

[0054] When the position resolution is the same, there is a possibility that part of the distribution range in the feature space of the features of the input data at the time of application may extend beyond the distribution range in the feature space of the features of the training data at the time of learning. The range of the features of the input data at the time of application that extends beyond the distribution range in the feature space of the features of the training data at the time of learning becomes an extrapolation range, and the accuracy of calibration by the estimation model M1 decreases.

[0055] In this embodiment, when estimating the component amounts of the components of an object, a spectrum extracted from a multispectral image IM2 having a lower positional resolution than the positional resolution of the multispectral image IM1 from which the spectrum used as training data during learning was extracted is input to an estimation model M1 to estimate the component amounts of the object.

[0056] As a result, the distribution range in feature space of the features of the input data input to the estimation model M1 when the estimation model M1 is applied becomes narrower than the distribution range in feature space of the features of the training data during training. Therefore, it is more likely that the distribution range in feature space of the features of the input data input to the estimation model M1 when the estimation model M1 is applied will be included in the distribution range in feature space of the features of the training data during training. Therefore, as described above, it is possible to reduce the possibility of the problem occurring in which part of the distribution range in feature space of the features of the input data during application exceeds the distribution range in feature space of the features of the training data during training. As a result, it is thought that it is possible to reduce a decrease in the accuracy of estimation using the estimation model M1.

[0057] B. Second embodiment: In the first embodiment described above, an example has been described in which the information processing system 1 is used to calibrate the component amounts of components in an object. However, the use of the information processing system 1 is not limited to this. In this embodiment, the information processing system 1 is used to classify objects. The following description will focus on configurations that differ from the first embodiment, and descriptions of configurations that are similar to the first embodiment will be omitted.

[0058] In this embodiment, the objects captured by the camera 300 are grains such as food and feed. Specifically, these are rice, wheat, corn, etc. Hay and rice straw used as feed are also included as objects.

[0059] The process for generating the estimation model M1 differs from the first embodiment in the following respects: In the first embodiment, when preparing the learning data, the camera 300 photographs a vegetable of the same type as the vegetable VG whose component amounts are to be estimated, thereby acquiring a multispectral image IM1. However, in the present embodiment, when preparing the learning data, the camera 300 photographs a different type of object, thereby acquiring a multispectral image IM1.

[0060] The target values ​​set for each spectrum extracted from the multiple multispectral images IM1 are the types of objects, for example, "rice," "wheat," "corn," "hay," and "rice straw" are set as target values ​​for each spectrum extracted from the multiple multispectral images IM1.

[0061] The estimation model M1 generated using the training data prepared in this manner takes a spectrum indicating spectroscopic information as input and outputs the type of object. The estimation model M1 generated using the training data prepared in this manner takes a spectrum indicating spectroscopic information as input and outputs the type of object. The estimation model M1 is generated using a machine learning algorithm such as k-nearest neighbor, decision tree, or random forest.

[0062] The application processing unit 240 inputs the spectrum into the estimation model M1 to estimate the type of the object.

[0063] In the process of applying the estimation model M1, a multispectral image having a positional resolution lower than that of the multispectral image IM1 for training is prepared, as in the first embodiment.

[0064] C. Other Embodiments: C1. Alternative Embodiment 1: In the first embodiment described above, an example has been described in which a multispectral image IM2 is acquired by blurring processing. However, the method of acquiring a multispectral image IM2 having a positional resolution lower than that of the training multispectral image IM1 is not limited to this. The multispectral image IM2 can be acquired using another camera having an image resolution lower than that of the camera 300 that acquires the multispectral image IM1. In this case, the camera 300 is also referred to as the "first imaging device." The other camera is also referred to as the "second imaging device." Hereinafter, image resolution will simply be referred to as resolution.

[0065] Furthermore, before acquiring the multispectral image IM2, the user may operate the setting menu of the camera 300 to change the resolution setting of the camera 300 to be lower than the resolution D1 of the training multispectral image IM1. The object is then imaged by the camera 300. As a result, an image having a resolution D2 lower than the resolution D1 of the training multispectral image IM1 is acquired as the multispectral image IM2. In this way, the multispectral image IM2 can be easily acquired. The camera 300 whose resolution is set to resolution D1 is also referred to as the "first imaging device." The camera 300 whose resolution is set to resolution D2 is also referred to as the "second imaging device."

[0066] Alternatively, the user may operate the camera 300 so as to intentionally defocus the image. In this case, an out-of-focus image is acquired as the multispectral image IM2. Alternatively, a filter that diffuses light may be attached to the lens of the camera 300.

[0067] The extent to which the resolution D2 of the multispectral image IM2 should be reduced relative to the resolution D1 of the multispectral image IM1 can be determined as follows: First, multiple multispectral images are prepared. The multiple multispectral images include multiple images with different resolutions. Furthermore, the multiple multispectral images include multiple images with the same resolution. Multiple spectra are extracted from each of the multiple multispectral images. Similar to the training data of the first embodiment, a set of component amounts for each component is associated as a target value with the multiple spectra. In this manner, validation data including multiple spectra and the associated target values ​​is prepared. One spectrum included in the validation data is input to the trained estimation model M1. The calibration accuracy of the estimation model M1 is evaluated using the output of the estimation model M1 and the target value associated with the input spectrum. Based on the results of evaluating the calibration accuracy of the estimation model M1 using each of the multiple spectra included in the validation data, the resolution D2 of the multispectral image IM2 that maintains the desired accuracy is determined. In the first embodiment, the extent to which the positional resolution of the multispectral image IM2 is to be reduced relative to the positional resolution of the multispectral image IM1 can also be determined using a method similar to that described above.

[0068] C2. Alternative Embodiment 2: When applying the estimation model M1, the user may be able to specify the degree of blurring for the original multispectral image IM0. Specifically, the user may specify the degree of blurring before the process of acquiring the multispectral image IM2 in the application of the estimation model M1 (see step S202 in FIG. 6).

[0069] FIG. 7 is an explanatory diagram of a received image SS1. FIG. 8 is an explanatory diagram of a completed image SS2. Before acquiring a multispectral image IM2 in the application process of the estimation model M1 (see step S202 in FIG. 6), the processor 150 displays the received image SS1 shown in FIG. 7 on the display device 140. The received image SS1 includes the original multispectral image IM0 acquired by the camera 300, an input field IF1 for inputting a specified value for the filter size of the blurring process, and a confirm button BT1. Displaying the received image SS1 on the display device 140 allows the user to visually recognize the multispectral image IM2. The user also inputs the desired filter size value into the input field IF1 using the input device 130. The filter size indicates the vertical and horizontal dimensions of the kernel for the filtering process. The larger the filter size, the stronger the blurring. For example, the user can select "3," "5," "7," or "9" in the input field IF1. For example, to instruct filtering using a 5x5 kernel, the user selects "5" in the input field IF1. In this embodiment, a predetermined value is used as the standard deviation σ of the Gaussian function.

[0070] When the "OK" button BT1 is pressed, the processor 150 performs filtering on the multispectral image IM2 using a kernel of the specified size. For example, if "5" is selected in the input field IF1 for the received image SS1, filtering is performed using a 5x5 kernel. The multispectral image IM2 generated by filtering the original multispectral image IM0 is stored in the memory 110. Furthermore, the processor 150 causes the display device 140 to display the completed image SS2 shown in FIG. 8.

[0071] The completed image SS2 includes the original multispectral image IM0 acquired by the camera 300, the multispectral image IM2, and a "Back" button BT2. By displaying the completed image SS2 on the display device 140, the user can view the multispectral image IM2 acquired by blurring the original multispectral image IM0, and the original multispectral image IM0 before the blurring process was performed. Furthermore, when the "Back" button BT2 is pressed, the processor 150 causes the display device 140 to display the acceptance image SS1 shown in FIG. 7. This allows the user to re-specify the desired filter size.

[0072] In this way, the user can specify the degree of blurring of the multispectral image IM2 to be input to the estimation model M1.

[0073] C3. Alternative Embodiment 3: Furthermore, during application processing, a process of extracting spectral features may be performed before inputting the spectrum to the estimation model M1. Principal component analysis (PCA), independent component analysis (ICA), normalization, etc. may be used as the process of extracting spectral features.

[0074] C4. Alternative Embodiment 4: In the first embodiment, an example has been described in which a user specifies the position and size of a target region in a multispectral image from which a spectrum is to be extracted, both when generating the estimation model M1 and when applying the estimation model M1. Before the user specifies the position and size of the target region, a region in the multispectral image may be divided using a segmentation algorithm. The segmentation algorithm divides the multispectral image into regions of interest and other regions. This allows the user to specify the target region more easily.

[0075] C5. Alternative Embodiment 5: In the first embodiment, the multispectral image acquisition unit 210 acquires a multispectral image captured by the camera 300. Alternatively, the multispectral image acquisition unit 210 may read a multispectral image previously stored in the memory 110. Alternatively, the multispectral image acquisition unit 210 may read list data representing a spectrum previously stored in the memory 110. The list data representing a spectrum is an array (list) representing light intensity, spectral reflectance, or time integral of light intensity for each wavelength. In the multispectral images IM1 and IM2, the background of the object is included in the image, but the list data includes only the intensity of light reflected by the object. The spectrum extraction unit 220 can generate a spectrum by plotting the values ​​of the list data representing the spectrum. When the user specifies the correction process for the extracted spectrum (iii) above, the spectrum extraction unit 220 executes the specified correction process for the spectrum. The corrected spectrum is input to the estimation model M1.

[0076] C6. Alternative Embodiment 6: In the first and second embodiments, an example has been described in which a Gaussian filter is used for the blurring process, but the present invention is not limited to this. An averaging filter can also be used as the filter for the blurring process.

[0077] C7. Alternative Embodiment 7: In the second embodiment, an example has been described in which the user selects a filter size to specify the strength of blur. The user may also be able to specify at least one of the filter size and σ of the Gaussian function.

[0078] D. Other forms: The present disclosure is not limited to the above-described embodiments and can be realized in various configurations without departing from the spirit thereof. For example, the technical features in the embodiments corresponding to the technical features in each aspect described in the Summary of the Invention section can be appropriately replaced or combined to solve some or all of the above-described problems or achieve some or all of the above-described effects. Furthermore, if a technical feature is not described as essential in this specification, it can be deleted as appropriate.

[0079] (1) According to a first aspect of the present disclosure, there is provided an information processing device. The information processing device includes: a spectral information acquisition unit that acquires spectral information of an object from a multispectral image of the object; a learning processing unit that receives the spectral information of the object as an input and generates an estimation model that outputs component amounts of the object; an application processing unit that estimates the component amounts of the object using the estimation model; and an output unit that outputs the component amounts estimated by the application processing unit. The application processing unit inputs, as input data to the estimation model, spectral information acquired from a second multispectral image, which is a multispectral image with a lower positional resolution than a first multispectral image, which is a multispectral image from which the spectral information used as learning data when generating the estimation model was acquired, to the estimation model, thereby estimating the component amounts. According to the above aspect, when estimating the component amounts of components of an object, spectral information acquired from a second multispectral image having a lower positional resolution than the first multispectral image from which spectroscopic information used as training data during training was acquired is input to an estimation model to estimate the component amounts. As a result, the distribution range in feature space of the features of the input data input to the estimation model when the estimation model is applied is narrower than the distribution range in feature space of the features of the training data during training. Therefore, it is more likely that the distribution range in feature space of the features of the input data input to the estimation model when the estimation model is applied will be included in the distribution range in feature space of the features of the training data during training. As a result, it is believed that a decrease in the accuracy of estimation using the estimation model can be suppressed. (2) The information processing device of the above aspect may further include a receiving unit that receives, from a user, a designation of a method for acquiring the spectral information for the second multispectral image, and the spectral information acquiring unit may acquire the spectral information from the second multispectral image as the input data using the acquisition method accepted by the receiving unit from the user. According to the above aspect, the user can obtain an estimated value of the component amount of the component of the object based on the spectroscopic information acquired by a desired method. (3) In the information processing device of the above aspect, the receiving unit may receive from the user a designation of the acquisition method of the spectral information for the first multispectral image, and the spectral information acquisition unit may acquire the spectral information from the first multispectral image as the training data using the acquisition method accepted by the receiving unit from the user. According to the above aspect, the user can obtain an estimation model generated using spectral information acquired by a desired method as training data. (4) In the information processing device of the above aspect, the receiving unit may receive, as the acquisition method, designation of at least one of one or more partial areas indicating areas within the multispectral image from which the spectral information is to be acquired, pre-processing to be performed on the multispectral image before the process of acquiring the spectral information, and processing to maintain consistency of the spectral information. According to the above aspect, the user can specify the region for acquiring the spectral information, pre-processing, etc. as the method for acquiring the spectral information. (5) The information processing device of the above aspect may further include a spectral image generation unit that generates the second multispectral image by performing a filter process on a third multispectral image, which is the multispectral image having the same positional resolution as the first multispectral image. According to the above aspect, the second multispectral image can be easily generated using the first multispectral image. (6) In the information processing device of the above aspect, the second multispectral image may be acquired by capturing an image of the object using a second imaging device having a lower resolution than the resolution of the first imaging device that acquired the first multispectral image. According to the above aspect, the second multispectral image can be easily acquired. (7) The information processing device of the above aspect further includes a display unit that displays the multispectral image. The accepting unit causes the display unit to display the second multispectral image and accepts from the user a designation of a positional resolution of the second multispectral image to be input to the estimation model. The application processing unit changes the positional resolution of the second multispectral image to the positional resolution designated by the user, causes the display unit to display the second multispectral image with the changed positional resolution, acquires the spectroscopic information from the second multispectral image with the changed positional resolution, and inputs the acquired spectroscopic information to the estimation model as the input data to estimate the component amounts. According to the above aspect, the user can specify a desired value as the position resolution of the second multispectral image for obtaining the spectral information to be input to the estimation model. (8) According to a second aspect of the present disclosure, there is provided a method, comprising the steps of: acquiring spectral information of an object from a multispectral image capturing the object; generating an estimation model that receives the spectral information of the object as an input and outputs amounts of components of the object; estimating the amounts of the components of the object using the estimation model, wherein the spectral information acquired from a second multispectral image, which is a multispectral image with a lower positional resolution than a first multispectral image, which is a multispectral image from which the spectral information used as learning data when generating the estimation model was acquired, is input to the estimation model to estimate the amounts of the components; and outputting the estimated amounts of the components. According to the above aspect, when estimating the component amounts of components of an object, spectral information acquired from a second multispectral image having a lower positional resolution than the first multispectral image from which spectroscopic information used as training data during training was acquired is input to an estimation model to estimate the component amounts. As a result, the distribution range in feature space of the features of the input data input to the estimation model when the estimation model is applied is narrower than the distribution range in feature space of the features of the training data during training. Therefore, it is more likely that the distribution range in feature space of the features of the input data input to the estimation model when the estimation model is applied will be included in the distribution range in feature space of the features of the training data during training. As a result, it is believed that a decrease in the accuracy of estimation using the estimation model can be suppressed. (9) According to a third aspect of the present disclosure, there is provided a program that causes a computer to perform the following functions: acquiring spectral information of an object from a multispectral image of the object, generating an estimation model that receives the spectral information of the object as an input and outputs amounts of components of the object, estimating the amounts of the components of the object using the estimation model, where the spectral information is acquired from a second multispectral image that is a multispectral image with a lower positional resolution than a first multispectral image that is a multispectral image from which the spectral information used as learning data when generating the estimation model was acquired, and outputting the estimated amounts of the components. According to the above aspect, when estimating the component amounts of components of an object, spectral information acquired from a second multispectral image having a lower positional resolution than the first multispectral image from which spectroscopic information used as training data during training was acquired is input to an estimation model to estimate the component amounts. As a result, the distribution range in feature space of the features of the input data input to the estimation model when the estimation model is applied is narrower than the distribution range in feature space of the features of the training data during training. Therefore, it is more likely that the distribution range in feature space of the features of the input data input to the estimation model when the estimation model is applied will be included in the distribution range in feature space of the features of the training data during training. As a result, it is believed that a decrease in the accuracy of estimation using the estimation model can be suppressed. [Explanation of symbols]

[0080] 1...information processing system, 100...information processing device, 110...memory, 120...interface circuit, 130...input device, 140...display device, 150...processor, 210...multispectral image acquisition unit, 220...spectrum extraction unit, 230...learning processing unit, 240...application processing unit, 250...output unit, 300...camera, BS...base unit, EL...organic, FX...fixing unit, IM0...original multispectral image, IM1...multispectral image, IM2...multispectral image, M1...estimation model, P1...program, PL...support, ST...stand, TL...tray, VG...vegetables

Claims

1. An information processing device, a spectral information acquisition unit that acquires spectral information of an object from a multispectral image obtained by capturing the object; a learning processing unit that receives the spectral information of the object as an input and generates an estimation model that outputs the component amounts of the components of the object; an application processing unit that estimates the component amounts of the object using the estimation model; an output unit that outputs the component amount estimated by the application processing unit; Equipped with The application processing unit inputting, as input data, the spectral information acquired from a second multispectral image, which is the multispectral image having a lower positional resolution than a first multispectral image, which is the multispectral image from which the spectral information used as learning data when generating the estimation model was acquired, into the estimation model to estimate the component amounts; Information processing device.

2. 2. The information processing device according to claim 1, a receiving unit that receives, from a user, a designation of a method for acquiring the spectral information of the second multispectral image; Furthermore, the spectral information acquisition unit acquires the spectral information from the second multispectral image as the input data using the acquisition method accepted by the accepting unit from the user. Information processing device.

3. 3. The information processing device according to claim 2, the receiving unit receives, from the user, a designation of the acquisition method of the spectral information for the first multispectral image; the spectral information acquisition unit acquires the spectral information from the first multispectral image as the learning data using the acquisition method accepted by the acceptance unit from the user. Information processing device.

4. 4. The information processing device according to claim 2, the receiving unit receives, as the acquisition method, designation of at least one of one or more partial regions indicating regions within the multispectral image from which the spectral information is to be acquired, pre-processing to be performed on the multispectral image before processing to acquire the spectral information, and processing to maintain consistency of the spectral information. Information processing device.

5. 5. The information processing device according to claim 4, a spectral image generating unit that generates the second multispectral image by performing a filter process on a third multispectral image, the third multispectral image being the multispectral image having the same positional resolution as the first multispectral image; Further provided with Information processing device.

6. 5. The information processing device according to claim 4, the second multispectral image is acquired by capturing an image of the object using a second image capturing device having a resolution lower than that of the first image capturing device that captured the first multispectral image; Information processing device.

7. 7. The information processing device according to claim 6, a display unit for displaying the multispectral image, The reception unit displaying the second multispectral image on the display unit; receiving, from the user, a designation regarding the positional resolution of the second multispectral image when inputting the second multispectral image into the estimation model; The application processing unit changing the positional resolution of the second multispectral image to a positional resolution designated by the user; displaying the second multispectral image with the changed positional resolution on the display unit; acquiring the spectroscopic information from the second multispectral image with a changed positional resolution; The acquired spectral information is input as the input data into the estimation model to estimate the component amounts. Information processing device.

8. 1. A method comprising: acquiring spectral information of an object from a multispectral image of the object; generating an estimation model that receives the spectral information of the object as an input and outputs the component amounts of the components of the object; a step of estimating the component amounts of the object using the estimation model, inputting, into the estimation model, the spectral information acquired from a second multispectral image, the second multispectral image having a lower positional resolution than a first multispectral image, the first multispectral image from which the spectral information used as learning data when generating the estimation model was acquired, and estimating the component amounts; outputting the estimated component amounts; A method comprising:

9. A program, a function of acquiring spectral information of an object from a multispectral image of the object; a function of generating an estimation model that receives the spectral information of the object as an input and outputs the component amounts of the components of the object; A function of estimating the component amounts of the object using the estimation model, a function of estimating the component amounts by inputting the spectral information acquired from a second multispectral image, which is the multispectral image having a lower positional resolution than a first multispectral image, which is the multispectral image from which the spectral information used as learning data when generating the estimation model was acquired, into the estimation model; and a function of outputting the estimated component amount; A program that makes a computer realize this.

Images