State estimation method, state estimation system, state estimation device, state estimation program, and recording medium on which same program is recorded

Abstract

An estimation system S executes processing including: a step in which an imaging data recognition unit 1a recognizes imaging data including a plurality of images respectively corresponding to different components and acquired for each of a plurality of light beams having mutually different prescribed wavelengths; a step in which a component recognition unit 1b recognizes, on the basis of the imaging data, a plurality of corresponding components that are components corresponding to each of the plurality of light beams having prescribed wavelengths in an object; and a step in which a state estimation unit 1g estimates the state of the object at a future prescribed time point on the basis of the plurality of corresponding components.

Description

State estimation method, state estimation system, state estimation device, state estimation program, and recording medium on which the program is recorded

[0001] The present invention relates to a state estimation method, a state estimation system, a state estimation device, a state estimation program, and a recording medium on which the program is recorded, for recognizing the state of an object including at least one of food and a member that may come into contact with the food.

[0002] Conventionally, food inspection devices have been known that irradiate food being transported on a transport means such as a belt conveyor with light of a predetermined wavelength, capture an image, and then detect foreign objects contained in the food based on the image (see, for example, Patent Document 1).

[0003] International Publication No. 2018 / 038123

[0004] The device described in Patent Document 1 inspects whether or not a food contains foreign matter at the time an image is captured. Therefore, there is a problem in that it is not possible to predict the state of the food at a predetermined time in the future (for example, whether the food will have spoiled after a predetermined period of time has passed). Similarly, there is a problem in that it is not possible to predict the state of components that may come into contact with the food.

[0005] The present invention has been made in consideration of the above points, and aims to provide a state estimation method, a state estimation system, a state estimation device, a state estimation program, and a recording medium on which the program is recorded, for estimating the state of an object including at least one of food and a component that may come into contact with the food at a predetermined point in time in the future.

[0006] The state estimation method of the present invention is a state estimation method for estimating the state of an object including at least one of food and a component that may come into contact with the food, and is characterized by comprising the steps of: a step in which an imaging data recognition unit recognizes imaging data including a plurality of images each corresponding to a different component and acquired for each of a plurality of predetermined wavelengths of light that are different from each other; a step in which a component recognition unit recognizes a plurality of corresponding components in the object that correspond to each of the plurality of predetermined wavelengths of light based on the imaging data; and a step in which a state estimation unit estimates the state of the object at a predetermined time in the future based on the plurality of corresponding components.

[0007] Here, the "object" includes at least one of food and a member that may come into contact with the food, as described above.

[0008] The "food" includes tea drinks such as black tea and green tea, coffee, fruit juice drinks, carbonated drinks, sports drinks, soft drinks such as amino acid drinks, alcoholic drinks such as wine, beer, and sake, dairy products such as milk, cheese, yogurt, and dairy drinks, soy milk, lactic acid bacteria drinks, refrigerated food and beverages, retort foods, frozen foods, fresh foods, fruits and vegetables, vegetables, fruits, seeds, beans, mushrooms, and processed products thereof, cut vegetables, cut fruit, meat, aged meat, processed meat products (hamburger steaks, meatballs, ham, sausages), fresh fish, fish, processed fish products, cooked foods This includes all foods and beverages distributed as processed oils and fats, such as ready-to-eat foods, bento lunches, cooked rice, processed cooked rice products, rice balls, daily necessities, confectionery, desserts, puddings, jellies, creams, cakes, bread, processed bread products, sandwiches, fermented foods, amazake (sweet sake), soy sauce, mirin (sweet sake), mentsuyu (noodle soup), dashi (soy sauce), kaeshi (seasoning), tare (sauce), ponzu (citrus sauce), soy sauce with dashi, ramen soup, instant foods, cup noodles, liquid seasonings, powdered seasonings, dried foods, freeze-dried foods, oils and fats, sesame oil, perilla oil, rapeseed oil, soybean oil, sunflower oil, corn oil, rice oil, linseed oil, olive oil, etc.

[0009] Furthermore, the "possible contact parts" include, for example, containers that hold food, as well as equipment used in food factories when the food is processed (e.g., belt conveyors, piping, ingredient extrusion nozzles, ingredient mixers, food cutting devices, blades, teeth, cutters, combs, smoking rooms, juicing devices, grinders, brewing tanks, barrels, pressing devices, dispensing machines, filling machines, tablet presses, food factory work tables, etc.), and clothing of food factory workers. Furthermore, it also includes not only food factories, but also work environments where food is processed, such as supermarkets, convenience stores, wholesalers, intermediate wholesalers, food manufacturer food development sites, restaurants, and central kitchens, as well as equipment used in these places (e.g., work tables, work benches, cooking utensils, knives, cutting boards, etc.), and clothing of workers.

[0010] Furthermore, "condition" refers to, for example, the ingredients contained in the food, the degree of contamination of the food (e.g., the type and amount of organic contamination, inorganic contamination, microbial contamination, and invading pests), the extent of the spread of contamination, the freshness of the food, and the quality of the food (e.g., the progress of fermentation, the storage state, changes in physical properties such as flavor, aroma, taste, and texture, changes in appearance, and the shelf life). Furthermore, "ingredients" includes not only the materials that make up the object, but also substances attached to the object, and microorganisms, viruses, or insects present on the surface or inside of the object.

[0011] As described above, the state estimation method of the present invention first acquires a plurality of images (so-called spectral images) of an object, each corresponding to a different component and for each of a plurality of predetermined wavelengths of light that are different from one another. In this case, each of the plurality of wavelengths corresponds to a different component. Therefore, according to this method, the components contained in the object at the current time can be easily and accurately grasped simply by acquiring the spectral images.

[0012] Then, this method estimates the state of the object at a predetermined time in the future (for example, a date and time set as the expiration date) based on the components thus identified. Here, since the spectroscopic image accurately indicates the components contained in the object at the current time, the estimation result is also highly accurate.

[0013] Therefore, according to the state estimation method of the present invention, the state of an object at a predetermined time in the future can be recognized with high accuracy simply by the simple task of acquiring a spectroscopic image.

[0014] In the state estimation method of the present invention, it is preferable that the state estimation unit compares the amounts of the plurality of corresponding components to estimate the state of the object at a predetermined time point in the future.

[0015] The components contained in the sample may affect each other. For example, a microorganism and a component required for the growth of the microorganism. Therefore, by comparing the amounts of each of the multiple components thus determined, it is possible to more accurately estimate the state of the target object at a predetermined time in the future, which will fluctuate based on the components.

[0016] Furthermore, the state estimation method of the present invention preferably includes a step in which an object attribute recognition unit recognizes object attributes that are attributes of the object, and a step in which a noise component recognition unit recognizes noise components that are components that should be removed as noise from among the plurality of corresponding components based on the object attributes, and the state estimation unit preferably estimates the state of the object at a predetermined time point in the future based on components other than the noise components from among the plurality of corresponding components.

[0017] When estimating the state of an object at a predetermined time in the future, it is not always advisable to consider all components (and therefore all spectroscopic images). For example, if the container is changed after the image is captured, it may be better not to consider the components of the container before the change.

[0018] Therefore, by recognizing noise components (e.g., components of packaging containers) according to the attributes of the object (e.g., storage method, etc.) and making an estimation based on components other than those noise components (i.e., excluding the noise components or reducing the proportion of noise components taken into consideration), the state of the object at a specified point in time in the future can be estimated more accurately.

[0019] Furthermore, in the state estimation method of the present invention, it is preferable that the state estimation unit estimates the state of the object at a predetermined time point in the future using a first prediction model that inputs the plurality of corresponding components and outputs a change in the state of the object over time.

[0020] The changes in the state of food vary in a complex manner depending on an extremely large number of factors, so by using this prediction model, it is possible to more accurately estimate the state of an object at a specified time in the future, even if the object changes in a complex manner.

[0021] Furthermore, in the state estimation method of the present invention, when the state of an object is estimated using a first prediction model, it is preferable that the state estimation unit uses as the first prediction model the multiple corresponding components in a sample containing a material that is the same as or similar to a material contained in the object and a change in the state of the sample over time as learning data, performs machine learning on the correlation between the multiple corresponding components and the change in the state of a substance containing the material over time, and outputs the state of the object at the specified future point in time from the multiple corresponding components of the input object.

[0022] By using such samples to train a predictive model, the predictive model can be used to accurately estimate the state of an object at a specified time in the future, even if it is a new object that has not been estimated before.

[0023] Furthermore, in the state estimation method of the present invention, when the state of an object is estimated using a first prediction model, it is preferable that the state estimation unit uses, as the first prediction model, the corresponding components in a sample containing a material that is the same as or similar to a constituent material of the object, changes in the state of the sample over time, and environmental information about the sample as learning data, and machine-learns the correlation between the changes in the state of a substance containing the plurality of corresponding components and the material over time and the environmental information, and outputs the state of the object at the predetermined future point in time from the input corresponding components of the object and the environmental information about the object.

[0024] Here, "environmental information" includes, for example, the date on which information about the sample or object or its surroundings was obtained, the temperature on that date, the humidity on that date, the number of days that have passed since a specified date until the date on which the information was obtained, the location of the sample or object from which the information was obtained, the surface condition of the sample or object when the information was obtained, etc.

[0025] The nature of changes in the state of food can vary significantly depending on the food itself or the environment surrounding it. Therefore, by using a prediction model that has been trained using environmental information, it is possible to more accurately estimate the state of an object at a given time in the future, even for such complexly changing objects.

[0026] Furthermore, in the state estimation method of the present invention, when the state of an object is estimated using a first prediction model, it is preferable that the method further comprises: a step in which a state recognition unit recognizes the state of the object at the current time and the state of the object at the predetermined time in the future; and a step in which a first learning processing unit adds information regarding changes over time based on the plurality of corresponding components corresponding to the object, the state of the object at the current time, and the state of the object at the predetermined time in the future to learning data, and causes the first prediction model to perform machine learning again.

[0027] In this way, by performing machine learning of a prediction model again using the actual state of the object, it is possible to make subsequent state estimations more accurate.

[0028] Furthermore, the state estimation method of the present invention preferably includes a step in which a recommended method recognition unit recognizes, based on the plurality of corresponding components, a recommended method, which is a processing method for the object that is currently recommended for improving the state of the object at the predetermined time point in the future, and a step in which a notification unit notifies a user of the recommended method.

[0029] Here, examples of "treatment methods" include methods for cleaning the object, methods for applying or coating chemicals that inhibit microbial growth on at least one of the surface and interior of the object, types of chemicals that are effective in preventing pest intrusion into the object and methods of using them, methods for adding ingredients necessary to promote fermentation, enzymatic reactions, or chemical reactions in the object, changes to the shelf life, changes to the storage temperature, humidity, illuminance, pressure, or air (oxygen, nitrogen, carbon dioxide concentrations, whether to create a vacuum, etc.), changes to product prices, and methods for disposing of, exchanging, or removing the object.

[0030] In this way, by presenting the current treatment method corresponding to the estimated condition, even people who have no knowledge of the condition can easily improve the condition (for example, preventing contamination, producing useful substances using fermentation, enzyme reactions, or chemical reactions, preventing the loss of freshness or spoilage of fresh foods, fruits and vegetables, and cooked foods at retail stores, etc.).

[0031] Furthermore, in the state estimation method of the present invention, when a recommended method is notified, it is preferable that the recommended method recognition unit recognizes the recommended method using a second prediction model that inputs the plurality of corresponding components and outputs the recommended method.

[0032] The changes in the state of food vary in complex ways depending on an extremely large number of factors. As a result, the recommended method for that state also varies in complex ways. Therefore, by using such a prediction model, it is possible to more accurately recognize the recommended method for the state of the object at a predetermined time in the future, even for such a complexly changing object.

[0033] Furthermore, in the state estimation method of the present invention, when the recommended method is recognized using a second prediction model, it is preferable that the recommended method recognition unit uses as the second prediction model the multiple corresponding components in a sample containing a material that is the same as or similar to a material contained in the object, a change in the state of the sample over time, and a first implementation method that is a processing method performed on the sample as learning data, and uses machine learning to determine the correlation between the multiple corresponding components and the change in the state of a substance containing the material over time and the first implementation method, and outputs the recommended method from the multiple corresponding components of the input object.

[0034] In this way, by using samples to train a predictive model, it is possible to accurately recognize the recommended method for the state of a new object at a specified future point in time, even if the object is a new object for which a recommended method has not previously been recognized.

[0035] Furthermore, in the state estimation method of the present invention, when the recommended method is recognized using a second prediction model, it is preferable that the method comprises: a step in which a state recognition unit recognizes the state of the object at the current time and the state of the object at the specified time in the future; a step in which an implementation method recognition unit recognizes a second implementation method that is a processing method implemented on the object; and a step in which a second learning processing unit adds the plurality of corresponding components corresponding to the object, the state of the object at the current time, information on changes over time based on the state of the object at the specified time in the future, and the second implementation method to learning data, and causes the second prediction model to perform machine learning again.

[0036] In this way, by re-machining the predictive model using the processing method applied to the actual object, the subsequent recognition of the recommended method can be made more accurate.

[0037] Furthermore, in the state estimation method of the present invention, the state estimation unit may estimate the state of the object at a predetermined time point in the future based on correlation data representing a correlation between the plurality of corresponding components and a change over time in the state of the object.

[0038] In the state estimation method of the present invention, the state of the object may be the content of at least one of organic soiling, inorganic soiling, mixed organic-inorganic soiling, components produced by autolysis, and microbial soiling in the object.

[0039] In addition, in the state estimation method of the present invention, when the state of the object is organic soiling, the organic soiling may be soiling based on at least one of lipids, proteins, allergens, carbohydrates, urea, amino acids, and nucleic acids.

[0040] Furthermore, in the state estimation method of the present invention, when the state of the object is inorganic dirt, the inorganic dirt may be dirt that is generated by the solidification of at least one of salt, calcium, magnesium, iron, soil components, heavy metals, and minerals.

[0041] Furthermore, in the state estimation method of the present invention, when the state of the object is microbial soiling, the microbial soiling may be soiling based on at least one of food poisoning bacteria, putrefactive bacteria, spoilage bacteria, lactic acid bacteria, yeast, mold, Escherichia coli, spore-forming bacteria, biofilm-forming bacteria, VBNC bacteria, drug-resistant bacteria, and viruses.

[0042] Here, "dirt" caused by the above-mentioned bacteria, etc., includes not only the adhesion of the bacteria, etc. themselves, but also the adhesion of secretions, toxins, and other metabolites produced by the bacteria, etc.

[0043] In addition, in the state estimation method of the present invention, the state of the object may be at least one of the freshness and quality of the food that is the object or the food that is in contact with the object's possible contact member.

[0044] Furthermore, in the state estimation method of the present invention, when the state of the object is freshness, the freshness may be a value based on at least one of the appearance, color, gloss, weight, and k value of the food whose freshness is to be estimated.

[0045] Furthermore, in the state estimation method of the present invention, when the state of the object is freshness, the freshness may be estimated based on at least one of the degree of spoilage, the degree of deterioration, and the degree of unpleasant odor estimated for the food whose freshness is to be estimated.

[0046] In the state estimation method of the present invention, when the state of the object is quality, the quality may be a value based on the degree of fermentation of the food whose quality is to be estimated.

[0047] In the state estimation method of the present invention, when the state of the object is quality, the quality may be a value based on components of the food whose quality is to be estimated.

[0048] Here, the above-mentioned components include, for example, components necessary for food fermentation, components produced by autolysis of food, etc. The value based on the component may be, for example, an absolute value, or a value indicating the excess or deficiency of the component relative to a predetermined threshold value.

[0049] Furthermore, the state estimation system of the present invention is a state estimation system for estimating the state of an object including at least one of food and a member that may come into contact with the food, and is characterized in that it is configured to execute any of the state estimation methods described above.

[0050] Furthermore, the state estimation device of the present invention is a state estimation device for estimating the state of an object including at least one of food and a member that may come into contact with the food, and is characterized in that it is configured to execute any of the state estimation methods described above.

[0051] Furthermore, the state estimation program of the present invention is a state estimation program that causes a computer to execute a state estimation method for estimating the state of an object that includes at least one of food and a component that may come into contact with the food, and is characterized in that it causes the computer to execute any of the above state estimation methods.

[0052] A recording medium according to the present invention is characterized in that it records the above-described state estimation program and the state estimation program is readable by the computer.

[0053] FIG. 1 is an explanatory diagram showing a schematic configuration of an estimation system according to an embodiment. FIG. 2 is a block diagram showing the configuration of a processing unit of the estimation system of FIG. 1. FIG. 3 is an image diagram showing an example of an image used during estimation processing in the estimation system of FIG. 1. FIG. 4 is an image diagram showing an example of a screen displayed on an administrator terminal when showing the results of estimation in the estimation system of FIG. 1. FIG. 5 is an image diagram showing an example of a screen displayed on a user terminal when showing the results of estimation in the estimation system of FIG. 1. FIG. 6 is an image diagram showing another example of a screen displayed on a user terminal when showing the results of estimation in the estimation system of FIG. 1. FIG. 7 is a flowchart showing processing executed by the estimation system of FIG. 1 when estimating an object.

[0054] A state estimation system according to an embodiment (hereinafter referred to as an "estimation system S") and a state estimation method executed using the same will be described below with reference to the drawings.

[0055] The estimation system S described below is a computer system for estimating the state of an object O at a predetermined time in the future. Specifically, the estimation system S estimates the state of the object O at a predetermined time in the future in the estimation system S owned by the service provider based on an image of the object O sent from a first user (e.g., a production manager at a food manufacturer) or a second user (e.g., a worker in charge of production at a food manufacturer) who is a user of the service, and presents the result of the estimation to the first user or the second user.

[0056] Here, the "object" includes at least one of food and a member that may come into contact with the food.

[0057] The "food" includes tea drinks such as black tea and green tea, coffee, fruit juice drinks, carbonated drinks, sports drinks, soft drinks such as amino acid drinks, alcoholic drinks such as wine, beer, and sake, dairy products such as milk, cheese, yogurt, and dairy drinks, soy milk, lactic acid bacteria drinks, refrigerated food and beverages, retort foods, frozen foods, fresh foods, fruits and vegetables, vegetables, fruits, seeds, beans, mushrooms, and processed products thereof, cut vegetables, cut fruit, meat, aged meat, processed meat products (hamburger steaks, meatballs, ham, sausages), fresh fish, fish, processed fish products, cooked foods This includes all foods and beverages distributed as processed oils and fats, such as ready-to-eat foods, bento lunches, cooked rice, processed cooked rice products, rice balls, daily necessities, confectionery, desserts, puddings, jellies, creams, cakes, bread, processed bread products, sandwiches, fermented foods, amazake (sweet sake), soy sauce, mirin (sweet sake), mentsuyu (noodle soup), dashi (soy sauce), kaeshi (seasoning), tare (sauce), ponzu (citrus sauce), soy sauce with dashi, ramen soup, instant foods, cup noodles, liquid seasonings, powdered seasonings, dried foods, freeze-dried foods, oils and fats, sesame oil, perilla oil, rapeseed oil, soybean oil, sunflower oil, corn oil, rice oil, linseed oil, olive oil, etc.

[0058] Furthermore, the "possible contact parts" include, for example, containers that hold food, as well as equipment used in food factories when the food is processed (e.g., belt conveyors, piping, ingredient extrusion nozzles, ingredient mixers, food cutting devices, blades, teeth, cutters, combs, smoking rooms, juicing devices, grinders, brewing tanks, barrels, pressing devices, dispensing machines, filling machines, tablet presses, food factory work tables, etc.), and clothing of food factory workers. Furthermore, it also includes not only food factories, but also work environments where food is processed, such as supermarkets, convenience stores, wholesalers, intermediate wholesalers, food manufacturer food development sites, restaurants, and central kitchens, as well as equipment used in these places (e.g., work tables, work benches, cooking utensils, knives, cutting boards, etc.), and clothing of workers.

[0059] Furthermore, "condition" refers to, for example, the ingredients contained in the food, the degree of contamination of the food (e.g., the type and amount of organic contamination, inorganic contamination, microbial contamination, and invading pests), the extent of the spread of contamination, the freshness of the food, and the quality of the food (e.g., the progress of fermentation, the storage state, changes in physical properties such as flavor, aroma, taste, and texture, changes in appearance, and the shelf life). Furthermore, "ingredients" includes not only the materials that make up the object, but also substances attached to the object, and microorganisms, viruses, or insects present on the surface or inside of the object.

[0060] [System Overview] The overview of the configuration of the estimation system S will be described below with reference to FIGS. 1 and 2. FIG.

[0061] As shown in FIG. 1, the estimation system S is configured by a server 1 owned by a provider of the service provided by the estimation system S.

[0062] The server 1 is configured to be able to communicate information with the camera 2, and the first user terminal 3 used by the first user, which may be a smartphone, tablet, or the like, and the second user terminal 4 used by the second user, via the Internet network, public lines, or the like. In the following description, the first user and the second user will be collectively referred to as "users." Furthermore, the first user terminal 3 and the second user terminal 4 will be collectively referred to as "user terminals."

[0063] The state estimation system of the present invention is not limited to being configured by a single server, but may be configured so that any of the terminals constituting the state estimation system is equipped with a processing unit described below.

[0064] Therefore, for example, the entire state estimation system may be configured by multiple servers. Alternatively, at least one of the processing units or at least part of the functions of the processing unit may be implemented in a user terminal, and the system may be configured by the user terminal and the server working together or by the user terminal alone.

[0065] Furthermore, in the estimation system S of this embodiment, the functions of the camera 2 or the user terminal may be provided in the terminal constituting the server 1, and configured as an independent state estimation device.

[0066] The camera 2 is a so-called hyperspectral camera that captures spectral images of an object to obtain image data and generates a data cube from the image data. The data cube is a compilation of image data of spectral images of the object O captured simultaneously or consecutively at a predetermined time.

[0067] The generated data cube, together with other data such as the date of photography, is transmitted as photography data to the server 1. In this embodiment, the camera 2 sequentially acquires image data for each of the multiple objects O being transported by the belt conveyor 5, generates data cubes, and sequentially transmits the data cubes to the server 1.

[0068] However, the camera 2 is not limited to such a hyperspectral camera, and may be any camera capable of acquiring data in a format that allows the imaging data recognition unit 1a (described later) to recognize imaging data including spectral images. Therefore, for example, the camera 2 may be a combination of multiple cameras capable of capturing images corresponding to light of one predetermined wavelength.

[0069] Each of the first user terminal 3 and the second user terminal 4 has, as functions (processing units) realized by at least one of the implemented hardware configuration and program, an output unit for presenting the content estimated by the estimation system S to the corresponding first user or second user, and an input unit for inputting the implementation method described below, the state of the object O at a predetermined time in the future, etc. (see Figures 4 and 5).

[0070] In this embodiment, the first user terminal 3 is a laptop computer, its output unit is the laptop computer's monitor, its input unit is a keyboard, the second user terminal 4 is a mobile phone or tablet, and its output unit and input unit are touch panels.

[0071] However, the user terminal of the present invention is not limited to such a configuration, and may be any device that can accept information input from a user and output information to present to the user. Therefore, for example, the user terminal may be configured to enable input and output using other devices such as a keyboard, microphone, camera, speaker, etc. in addition to a touch panel.

[0072] In addition, in this embodiment, the server 1, the camera 2, the first user terminal 3, and the second user terminal 4 are described as being configured as objects in real space. However, devices that cooperate with the state estimation system of the present invention or devices that input or output information to or from the state estimation system are not limited to objects in real space, but may be objects in virtual space, or may be configured as a combination of objects in real space and objects in virtual space.

[0073] Therefore, for example, in this embodiment, at least one of the output section of the first user terminal 3 and the output section of the second user terminal 4 may be virtual, and the first user terminal 3 or the second user terminal 4 may be configured to allow the user to input and output information in a virtual space or in a space that combines virtual space and real space.

[0074] In addition, in this embodiment, the camera 2, the first user terminal 3, and the second user terminal 4 are described as independent devices. However, the device that cooperates with the state estimation system of the present invention is not limited to such a configuration, and may also combine the functions of multiple devices. For example, instead of the camera 2 in this embodiment, a camera mounted on the first user terminal 3 or the second user terminal 4 may be used.

[0075] [Configuration of Each Processing Unit] Next, the processing units that make up the estimation system S will be described with reference to FIGS.

[0076] In the following description, the term "recognition" is used, and this "recognition" includes not only the process of obtaining the target information as is, but also various processes for obtaining the target information, such as the process of generating the target information by performing some processing on the base information. Furthermore, some or all of the processes required for the recognition may be performed by the processing unit that performs the recognition itself, or may be performed by using another system, processing unit, database, etc. that is separately provided inside or outside the device or system that includes the processing unit.

[0077] As shown in Figure 2, the server 1 has the following functions (processing units) realized by at least one of the implemented hardware configuration and program: an image data recognition unit 1a, a component recognition unit 1b, a corresponding component storage unit 1c, an object attribute recognition unit 1d, a noise component recognition unit 1e, a noise component storage unit 1f, a state estimation unit 1g, a first prediction model 1h, a recommended method recognition unit 1i, a second prediction model 1j, an information presentation unit 1k (alert unit), a state recognition unit 1l, a first learning processing unit 1m, an implementation information recognition unit 1n (implementation method recognition unit, environment recognition unit), and a second learning processing unit 1o.

[0078] The imaging data recognition unit 1a recognizes imaging data including a plurality of images acquired for each of a plurality of predetermined wavelengths of light, each of which corresponds to a different component. In the estimation system S, the imaging data recognition unit 1a recognizes a data cube generated from a spectroscopic image acquired by a camera 2 by photographing an object O.

[0079] The photographic data recognition unit of the present invention is not limited to this configuration, and may be any unit capable of recognizing photographic data. Therefore, for example, the photographic data recognition unit may be configured to acquire and recognize images captured in advance from a user terminal as photographic data, rather than images captured by a camera at the time of estimation. Furthermore, the photographic data may be acquired as images recognized from the beginning, or as images extracted from a video.

[0080] Furthermore, it is preferable that the imaging data recognized by the imaging data recognition unit is imaging data obtained after applying a reagent containing a predetermined luminescent substrate to the object O. This is because, if such a reagent is used, it becomes possible to selectively cause a component present in a small amount to emit light, thereby making it easy to detect that component. This in turn makes it possible to detect that component with high sensitivity and high selectivity, thereby enabling highly accurate state estimation.

[0081] Examples of luminescent substrates include bioluminescent substrates such as luciferin (firefly luciferin, coelenterazine), and chemiluminescent substrates such as luminol, acridinium, lucigenin, adamantyl dioxetane, dioxetane, peroxalate ester, carminic acid, and ruthenium complexes.

[0082] The component recognition unit 1b recognizes a plurality of corresponding components in the object O, which are components corresponding to light of a plurality of predetermined wavelengths, based on the imaging data recognized by the imaging data recognition unit 1a.

[0083] In the estimation system S, the component recognition unit 1b first extracts only images corresponding to wavelengths having a certain intensity or higher from the multiple images that make up a data cube for the object O, then generates an analysis image of the object O using these images, as shown in Figure 3, and obtains the components corresponding to the wavelengths that make up the analysis image from the corresponding component storage unit 1c to recognize the components contained in the object O.

[0084] Specifically, for example, the component recognition unit 1b first recognizes the absorbance of the object O in light of the wavelength corresponding to each of the multiple images constituting the data cube. Next, the component recognition unit 1b recognizes a calibration curve for absorbance by acquiring data that has been obtained in advance or by calculating it from reference data on other objects, etc. After that, the component recognition unit 1b estimates and recognizes the type, amount, concentration, etc. of the components contained in the object O using the absorbance recognized from the image and the recognized calibration curve. Performing recognition in this manner enables more quantitative recognition of the corresponding components.

[0085] The component recognition unit of the present invention is not limited to this configuration, but may be any unit that recognizes corresponding components in an object based on the photographed data.

[0086] Therefore, for example, the component recognition unit may be provided inside or outside the system, and may be configured to recognize the corresponding component in the object using a predictive model or the like that inputs image data of the object and outputs the corresponding component of the object.

[0087] When using such a prediction model, it is preferable that the learning data of the prediction model include data on components that may be contained in the target object. For example, if it is assumed that the target object may contain multiple types of oil, and if the presence of the oil types is not a problem and is not a problem depending on the type, using data on multiple types of oil in the learning data will enable appropriate estimation in the state estimation described below.

[0088] Specifically, for example, it is not a problem if a certain amount of oil is contained in the oil used to cook a food product, but even a small amount of oil used to operate the manufacturing equipment for the product can be a problem, so it is preferable to use learning data about these oils in training the predictive model.

[0089] Furthermore, the corresponding components recognized by the component recognition unit of the present invention are used as information for estimating a state by a state estimation unit described later. Therefore, the information about the corresponding components may include information about the characteristics of the recognized corresponding components in addition to information about the identity of the corresponding components.

[0090] Specifically, for example, if the target component is a microorganism and the state to be estimated is the amount of microorganisms (amount of proliferation), in addition to information on the type of microorganism, information on the characteristics of the microorganism, such as whether the microorganism is a gram-positive bacterium or a gram-negative bacterium and the fermentation characteristics of the microorganism, may also be used when estimating the state. In other words, the component recognition unit may be configured to recognize such characteristic information. This configuration enables even more accurate state estimation.

[0091] The corresponding component storage unit 1c stores, for each of a plurality of predetermined wavelengths of light, information about the component corresponding to that wavelength, such as the ratio of component x and component y contained in a portion where the intensity of the light of a predetermined wavelength exceeds a predetermined intensity.

[0092] The object attribute recognition unit 1 d recognizes object attributes, which are attributes of an object. In the estimation system S, the object attribute recognition unit 1 d recognizes object attributes based on information input by a user to the first user terminal 3.

[0093] Here, the "attributes" of an object include the type of object (for example, whether it is meat, fish, vegetables, or a processed product thereof), how the object is packaged (for example, what kind of container it is stored in), etc.

[0094] The object attribute recognition unit of the present invention may be any unit capable of recognizing object attributes. For example, the object attribute recognition unit may estimate and recognize the attributes of an object from an image of the object using a prediction model that receives image data of the object and outputs the attributes of the object.

[0095] The noise component recognition unit 1e recognizes noise components that should be removed as noise from among the multiple corresponding components recognized by the component recognition unit 1b, based on the object attributes recognized by the object attribute recognition unit 1d. In the estimation system S, the noise component recognition unit 1e acquires and recognizes information that should be considered as noise from the noise component storage unit 1f, based on the object O to be estimated and the object attributes. For example, if the object is food, components corresponding to plastic, paper, metal hooks, cigarette butts, cardboard, fabric, etc. are expected to be noise components.

[0096] The noise component recognition unit of the present invention may be configured to recognize noise components based on object attributes, and may be configured to recognize noise components using a prediction model that receives image data of an object and outputs noise components in the object.

[0097] The noise component storage unit 1f stores information about components that may be contained in the attribute and components that are unlikely to be contained (i.e., noise components) for each attribute of the object O. For example, if the object O is a food product, the information may indicate that proteins, lipids, calcium, etc. are components that may be contained in food products, and that plastics, etc. are noise components that are unlikely to be contained.

[0098] In this embodiment, the object O is meat, and the meat is packaged in a food tray and a film covering it, and the user wishes to estimate the state of the object meat at a predetermined time in the future. In such a case, the components of the food tray and the film covering it are not necessary for the estimation, and are therefore removed as noise components.

[0099] Furthermore, in the state estimation system of the present invention, if a predetermined noise component is found, it is preferable to configure the system so that, after or before performing subsequent processing such as state estimation, the user is notified that a component corresponding to the noise component has been detected.

[0100] The state estimation unit 1g estimates the state of the object O at a predetermined time in the future based on components other than the noise components recognized by the noise component recognition unit 1e among the multiple corresponding components recognized by the component recognition unit 1b.

[0101] Here, in this embodiment, the "condition" refers to the degree of contamination of the food (for example, the type and amount of organic contamination, inorganic contamination, microbial contamination, and invading pests) and the extent of the contamination. More specifically, in this embodiment, the condition of the object O is determined by the content of at least one of organic contamination, inorganic contamination, mixed organic-inorganic contamination, components produced by autolysis, and microbial contamination in the object O.

[0102] Organic soils are soils based on at least one of lipids, proteins, allergens, carbohydrates, urea, amino acids, and nucleic acids. Inorganic soils are soils formed by the coagulation of at least one of salt, calcium, magnesium, iron, soil components, heavy metals, and minerals. Microbial soils are soils based on at least one of food poisoning bacteria, putrefactive bacteria, spoilage bacteria, lactic acid bacteria, yeast, mold, Escherichia coli, spore-forming bacteria, biofilm-forming bacteria, VBNC bacteria, drug-resistant bacteria, and viruses.

[0103] Here, "dirt" caused by the above-mentioned bacteria, etc., includes not only the adhesion of the bacteria, etc. themselves, but also the adhesion of secretions, toxins, and other metabolites produced by the bacteria, etc.

[0104] It should be noted that the "condition" of an object in the present invention is not limited to the "soiling" of the object. For example, the "condition" may be at least one of the freshness and quality of the food object or the food object that is in contact with a potentially contactable member.

[0105] When the "condition" is "freshness," the freshness may be a value based on at least one of the appearance, color, luster, weight, and k value of the food whose freshness is to be estimated. Alternatively, the freshness may be a value indirectly estimated based on at least one of the degree of spoilage, deterioration, and off-flavor estimated using a system similar to that of the present embodiment.

[0106] Furthermore, when the "state" is "quality," the quality may be a value based on the degree of fermentation of the food whose quality is to be estimated. Furthermore, the quality may be a value based on the components of the food whose quality is to be estimated. Here, the components include, for example, components necessary for food fermentation, components produced by autolysis of the food, etc. The value based on the components may be, for example, an absolute value, or a value indicating the excess or deficiency of a component relative to a predetermined threshold value.

[0107] In the estimation system S, the state estimation unit 1g compares the amounts of each of the multiple corresponding components to estimate the state of the object O at a predetermined time point in the future.

[0108] This is because the components contained in the sample may affect each other, such as a microorganism and a component required for the growth of the microorganism. Therefore, by comparing the amounts of each of the identified components, it is possible to more accurately estimate the state of the target object at a predetermined time in the future, which will fluctuate based on the components.

[0109] The state estimation unit of the present invention is not limited to this configuration, and may be configured to estimate the state of an object at a predetermined time in the future based on a plurality of corresponding components. Therefore, for example, the state estimation unit may be configured to perform estimation based on the absolute amounts of the corresponding components, the types of the corresponding components, etc.

[0110] Furthermore, in the estimation system S, the state estimation unit 1g estimates the state of the object at a predetermined time point in the future based on the components other than the noise components among the multiple corresponding components, as described above.

[0111] This is because it is not necessarily better to consider all components (and therefore all spectroscopic images) when estimating the state of the object O at a predetermined time in the future. For example, if the container is changed after the image is captured, it may be better not to consider the components of the container before the change in the state when making the estimation.

[0112] Therefore, by recognizing noise components (e.g., components of the packaging container, etc.) according to the attributes of the object O (e.g., storage method, etc.) and making an estimation based on components other than those noise components (i.e., excluding the noise components or reducing the proportion of the noise components that are taken into consideration), the state of the object O at a specified point in time in the future can be estimated more accurately.

[0113] In this embodiment, the components of the food tray containing the object O, which is meat, and the film covering it (in this embodiment, plastic) are considered to be noise components, so when making an estimation, the object O is estimated based on the components excluding the plastic.

[0114] The state estimation unit of the present invention is not limited to this configuration, and may be configured to estimate the state of an object at a predetermined time in the future based on a plurality of corresponding components. Therefore, for example, the state estimation unit may be configured to perform estimation without performing processing to remove noise components when the object is not stored in a container, etc.

[0115] In addition, in the estimation system S, the state estimation unit 1g estimates the state of the object O at a predetermined future point in time using a first prediction model 1h that inputs multiple corresponding components and outputs the change in the state of the object over time.

[0116] The changes in the state of food vary in a complex manner depending on an extremely large number of factors, so by using the first prediction model 1h, it is possible to more accurately estimate the state of the object O at a predetermined time in the future, even for an object that changes in such a complex manner.

[0117] The state estimation unit of the present invention is not limited to this configuration, and may be configured to estimate the state of an object at a predetermined time in the future based on a plurality of corresponding components. Therefore, for example, the state estimation unit may be configured to estimate the state of an object at a predetermined time in the future using a prediction model or the like external to the system. Also, for example, the state estimation unit may be configured to estimate the state of an object at a predetermined time in the future based on correlation data representing the correlation between a plurality of corresponding components and a change over time in the state of the object.

[0118] The first prediction model 1h is a prediction model that uses corresponding components in a sample containing materials that are identical or similar to the constituent materials of the object O, changes in the state of the sample over time, and environmental information about the sample as learning data, and machine-learns the correlation between the changes in the state of a substance containing the multiple corresponding components and the materials over time and the environmental information, and outputs the state of the object O at a specified time in the future from the multiple corresponding components of the input object O and the environmental information about the object O.

[0119] If such samples are used to train a prediction model, the prediction model can be used to accurately estimate the state of an object at a predetermined time in the future, even if the object is a new object for which estimation has not been performed before. For this reason, the estimation system S employs such a prediction model as the first prediction model 1h.

[0120] Here, "environmental information" includes, for example, the date on which information about the sample or object or its surroundings was obtained, the temperature on that date, the humidity on that date, the number of days that have passed since a specified date until the date on which the information was obtained, the location of the sample or object from which the information was obtained, the surface condition of the sample or object when the information was obtained, etc.

[0121] The nature of changes in the state of food can vary significantly depending on the food itself or the environment surrounding the food. Therefore, using a prediction model trained using environmental information, even for such complexly changing objects, can more accurately estimate the state of the object at a predetermined time in the future. Therefore, the estimation system S employs such a prediction model as the first prediction model 1h.

[0122] The first prediction model of the present invention is not limited to this configuration and may be one that does not use environmental information. Therefore, for example, in the present embodiment, if the environment in which the object O to be estimated is located is kept constant, the estimation system S may adopt a prediction model that does not use environmental information as the first prediction model.

[0123] In this embodiment, the first prediction model 1 h uses the comparison result (and therefore the ratio) of the amounts of the corresponding components as input data. However, the first prediction model of the present invention is not limited to this configuration, and may use the absolute amounts of the corresponding components, the types of the corresponding components, or the like as input data instead of the ratios.

[0124] The recommended method recognition unit 1i recognizes a recommended method, which is a processing method for the object O that is currently recommended to improve the state of the object O at a predetermined time point in the future, based on a plurality of corresponding components.

[0125] Here, examples of "treatment methods" include methods for cleaning the object, methods for applying or coating chemicals that inhibit microbial growth on at least one of the surface and interior of the object, types of chemicals that are effective in preventing pest intrusion into the object and methods of using them, methods for adding ingredients necessary to promote fermentation, enzymatic reactions, or chemical reactions in the object, changes to the shelf life, changes to the storage temperature, humidity, illuminance, pressure, or air (oxygen, nitrogen, carbon dioxide concentrations, whether to create a vacuum, etc.), changes to product prices, and methods for disposing of, exchanging, or removing the object.

[0126] In this way, by presenting the current treatment method corresponding to the estimated condition, even people who have no knowledge of the condition can easily improve the condition (for example, preventing contamination, producing useful substances using fermentation, enzyme reactions, or chemical reactions, preventing the loss of freshness or spoilage of fresh foods, fruits and vegetables, and cooked foods at retail stores, etc.).

[0127] In this embodiment, the degree of contamination of the meat, which is the object O, at a predetermined time in the future is estimated, and therefore a method for suppressing the spread of the contamination is recognized as a treatment method. Specifically, a method for processing the meat, which is the object O, a method for cleaning trays and films for storing the meat, which is the object O, and the like are recognized as treatment methods.

[0128] In the estimation system S, the recommended method recognition unit 1i recognizes the recommended method using a second prediction model 1j that receives a plurality of corresponding components as input and outputs a recommended method.

[0129] The changes in the state of food vary in complex ways depending on an extremely large number of factors. Consequently, the recommended method for that state also varies in complex ways. Therefore, by using the second prediction model 1j, it is possible to more accurately recognize the recommended method for the state of an object at a predetermined time in the future, even for such complexly changing objects.

[0130] Note that the recommended method recognition unit of the present invention is not limited to this configuration, and may be configured to recognize a recommended method. Therefore, for example, the recommended method recognition unit may be configured to recognize a recommended method using a prediction model or the like external to the system. Furthermore, for example, the recommended method recognition unit may be configured to recognize a recommended method based on correlation data that represents the correlation between a plurality of corresponding components, a change in the state of the object over time, and the recommended method.

[0131] The second prediction model 1j is a prediction model that uses multiple corresponding components in a sample containing materials that are identical or similar to those contained in the target object O, the changes in the state of the sample over time, and a first implementation method, which is a processing method performed on the sample, as learning data, and machine-learns the correlation between the changes in the state of the multiple corresponding components and the material containing the first implementation method over time, and outputs a recommended method from the multiple corresponding components of the input target object O.

[0132] In this way, by using samples to train a prediction model, it is possible to accurately recognize a recommended method for the state of a new object at a predetermined time in the future, even if the object is a new object for which no recommended method has been recognized before. For this reason, the estimation system S employs such a prediction model as the second prediction model 1j.

[0133] In this embodiment, the second prediction model 1j uses the comparison result (and therefore the ratio) of the amounts of the corresponding components as input data. However, the second prediction model of the present invention is not limited to this configuration, and may use the absolute amounts of the corresponding components, the types of the corresponding components, or the like as input data instead of the ratios.

[0134] The information presentation unit 1k (notification unit) notifies the corresponding first user or second user via the first user terminal 3 and the second user terminal 4 by presenting the components of the object O at the current time recognized by the component recognition unit 1b, the state of the object O at the current time recognized by the state recognition unit 1l, the state of the object O at a predetermined time in the future estimated by the state estimation unit 1g, and the recommended method recognized by the recommended method recognition unit 1i.

[0135] In the presentation system S, as shown in Figure 4, the information presentation unit 1k displays on a monitor, which is the output unit of the first user terminal 3, the components of the object O at the current time (spectral information and spectroscopic image), the state of the object O at the current time (the dirt components displayed in the item ``The following dirt components have been detected.''), the state of the object O at a specified time in the future (the dirt components displayed in the item ``The dirt components are predicted to change as follows in xx hours.''), and a recommended method (the recommended method displayed in the item ``The following work is recommended to suppress these changes.'').

[0136] 4 is an example of the environmental information. In this embodiment, the environmental information is information input by the user to the first user terminal 3 or information acquired by the information presentation unit 1k from a predetermined database (e.g., data published by the Japan Meteorological Agency). However, the environmental information may be information recognized by other methods (e.g., information acquired from another system).

[0137] 5, in the presentation system S, the information presentation unit 1k displays the components (spectral image) of the object O at the current time, the current state of the object O (the dirt components displayed in the section "The following dirt components have been detected"), and also displays recommended methods (recommended methods displayed in the section "Please perform the following tasks") in a selectable manner on a touch panel, which is the output unit of the second user terminal 4. Here, when one of the recommended methods is selected by the user, a screen displaying a specific method for implementing the recommended method is presented to the user.

[0138] In addition, in the presentation system S, the information presentation unit 1k may be configured to display instructions regarding how to handle the product (for example, instructions such as "Please mark this as a sale item" or "Please collect it from the store within two hours") as a recommended method, as shown in Figure 6.

[0139] The state recognition unit 1l recognizes the state of the object O at the current time and the state of the object O at a predetermined time in the future. In the presentation system S, the state recognition unit 1l recognizes the state of the object O at the current time by performing the same processing as the state estimation unit 1g at the current time. Furthermore, the state recognition unit 1l recognizes the state of the object O at a predetermined time in the future estimated by the state estimation unit 1g by performing the same processing as the state estimation unit 1g at that time.

[0140] It should be noted that the state recognition unit of the present invention is not limited to such a configuration, and may be configured to recognize the state of the object at the present time and the state of the object at a predetermined time in the future. Therefore, for example, the state recognition unit may be configured to recognize the state of the object using a method different from that used by the state estimation unit, or may be configured to recognize information input by a user as the state of the object at that time.

[0141] The first learning processing unit 1m adds to the learning data multiple corresponding components corresponding to the object O recognized by the component recognition unit 1b, as well as information regarding the state of the object at the current time recognized by the state recognition unit 1l and changes over time based on the state of the object O at a specified time in the future, and causes the first prediction model 1h to perform machine learning again.

[0142] In this way, by performing machine learning of the first prediction model 1h again using the actual state of the object, it is possible to make subsequent state estimations more accurate.

[0143] Note that the state estimation system of the present invention is not limited to such a configuration, and may be any system capable of estimating the state of an object. Therefore, for example, the state estimation system may continue to use the first prediction model that has been previously machine-learned, without newly performing machine learning of the first prediction model using the actual state of the object.

[0144] The implementation information recognition unit 1n (implementation method recognition unit, environment recognition unit) recognizes the second implementation method, which is the processing method implemented on the object, and the environment information at the time when the method was implemented.

[0145] In the presentation system S, the implementation information recognition unit 1n recognizes the second implementation method based on information input via the first user terminal 3 or the second user terminal 4. For example, of the recommended methods presented in Fig. 5, the recommended method whose detailed information has been confirmed is recognized as the second implementation method.

[0146] In addition, in the presentation system S, the implementation information recognition unit 1n recognizes environmental information based on information input via the first user terminal 3 or the second user terminal 4, or information obtained from a specified database (for example, data published by the Japan Meteorological Agency).

[0147] The second learning processing unit 1o adds multiple corresponding components corresponding to the object O, the current state of the object O, information regarding changes over time based on the state of the object O at a specified future point in time, and the second implementation method to the learning data, and causes the second prediction model to perform machine learning again.

[0148] In this way, by performing machine learning of the second prediction model 1j again using the processing method applied to the actual object, the recognition of the recommended method performed thereafter can be made more accurate.

[0149] The state estimation system of the present invention is not limited to this configuration, and may be any system that recognizes a recommended method. Therefore, for example, the state estimation system may continue to use the second prediction model that was previously machine-learned, without re-machining the second prediction model using the actual state of the object. Furthermore, when re-machining the second prediction model, only the second implementation method may be used without using environmental information.

[0150] [Processing Executed by Each Processing Unit] Next, the processing executed by the estimation system S (i.e., the state estimation method) will be described with reference to FIGS.

[0151] In this process, first, the photographing data recognition unit 1a recognizes photographing data including a plurality of images (so-called spectroscopic images) each corresponding to a different component and acquired for each of a predetermined number of different wavelengths of light (FIG. 7 / STEP 01).

[0152] Specifically, the photographic data recognition unit 1 a recognizes a data cube generated from a spectroscopic image acquired by photographing the object O with the camera 2 .

[0153] Next, the component recognition unit 1b recognizes multiple corresponding components in the object O, which are components corresponding to each of multiple predetermined wavelengths of light, based on the shooting data recognized by the shooting data recognition unit 1a (Figure 7 / STEP 02).

[0154] Specifically, the component recognition unit 1b first extracts only those images corresponding to wavelengths having a certain intensity or higher from the multiple images that constitute the data cube for the object O, then generates an analysis image of the object O using these images, as shown in Figure 3, and obtains the components corresponding to the wavelengths that constitute the analysis image from the corresponding component storage unit 1c to recognize the components contained in the object O.

[0155] Next, the object attribute recognition unit 1d recognizes the object attributes, which are attributes of the object (FIG. 7 / STEP 03).

[0156] Specifically, the object attribute recognition unit 1d recognizes object attributes such as the type of object O (for example, whether it is meat, fish, vegetables, or a processed product thereof) and how the object O is packaged (for example, what kind of container it is stored in), based on information input by the user to the first user terminal 3.

[0157] Next, the noise component recognition unit 1e recognizes noise components that should be removed as noise from among the multiple corresponding components recognized by the component recognition unit 1b, based on the object attributes recognized by the object attribute recognition unit 1d (Figure 7 / STEP 04).

[0158] Specifically, the noise component recognition unit 1e acquires and recognizes information to be recognized as noise from the noise component storage unit 1f based on the object O to be estimated and the object attributes. In this embodiment, the noise component recognition unit 1e recognizes components of the food tray containing meat, which is the object O, and the film covering it, as noise components.

[0159] Next, the state estimation unit 1g estimates the state of the object O at a predetermined future point in time based on the components other than the noise components recognized by the noise component recognition unit 1e among the multiple corresponding components recognized by the component recognition unit 1b (Figure 7 / STEP 05).

[0160] Specifically, the state estimation unit 1g inputs the proportion of each of the amounts of a plurality of corresponding components into the first prediction model 1h, and estimates the state of the object O at a predetermined time point in the future.

[0161] Next, the recommended method recognition unit 1i recognizes a recommended method, which is a processing method for the object O that is currently recommended to improve the state of the object O at a specified time in the future, based on the multiple corresponding components (Figure 7 / STEP 06).

[0162] Specifically, the recommended method recognition unit 1i inputs the proportions of the amounts of each of the plurality of corresponding components into the second prediction model 1j to recognize the recommended method. In this embodiment, a method for processing the target object O, a method for cleaning trays and films for storing the target object O, and the like are recognized as processing methods.

[0163] Next, the information presentation unit 1k presents to the corresponding first user or second user via the first user terminal 3 and the second user terminal 4 the components of the object O at the current time recognized by the component recognition unit 1b, the state of the object O at the current time recognized by the state recognition unit 1l, the state of the object O at a predetermined time in the future estimated by the state estimation unit 1g, and the recommended method recognized by the recommended method recognition unit 1i (Figure 7 / STEP 07).

[0164] Specifically, as shown in Figure 4, the information presentation unit 1k displays on a monitor, which is the output unit of the first user terminal 3, the components of the object O at the current time (spectral information and spectroscopic image), the state of the object O at the current time (the dirt components displayed in the item "The following dirt components have been detected"), the state of the object O at a specified time in the future (the dirt components displayed in the item "The dirt components are predicted to change as follows in xx hours"), and a recommended method (the recommended method displayed in the item "The following work is recommended to suppress these changes").

[0165] In addition, as shown in Figure 5, the information presentation unit 1k displays the current components (spectral image) of the object O, the current state of the object O (the dirt components displayed in the item "The following dirt components have been detected"), and the recommended method (the recommended method displayed in the item "Please perform the following steps") on the touch panel, which is the output unit of the second user terminal 4.

[0166] Next, the first learning processing unit 1m determines whether or not the state at a predetermined time point in the future has been recognized (FIG. 7 / STEP 08).

[0167] Specifically, the first learning processing unit 1m recognizes that the state at a predetermined time in the future has been recognized when information regarding the state at a predetermined time in the future is input into the first user terminal 3 through the work of the first user, or when an image at a predetermined time in the future is captured by the camera 2.

[0168] If the state at a predetermined future point in time is not recognized (NO in STEP 08), the first learning processing unit 1m repeats the processing of STEP 08 at a predetermined control cycle for a certain period of time (for example, the period until the expiration date has passed).

[0169] On the other hand, if the state at a predetermined future time point is recognized (YES in STEP 08), the first learning processing unit 1m adds to the learning data the multiple corresponding components corresponding to the object O recognized by the component recognition unit 1b, as well as the state of the object at the current time recognized by the state recognition unit 1l, and information regarding changes over time based on the state of the object O at a predetermined future time point, and causes the first prediction model 1h to perform machine learning again (Figure 7 / STEP 09).

[0170] Next, the second learning processing unit 1o recognizes whether or not the recommended method has been executed (FIG. 7 / STEP 10).

[0171] Specifically, when detailed information about any of the recommended methods presented to the second user via the second user terminal 4 is confirmed, the second learning processing unit 1o recognizes that the recommended method has been executed and recognizes that method as the second implementation method.

[0172] If the execution of the recommended method is not recognized (NO in STEP 10), the estimation system S ends the current process. Note that the process of STEP 10 is repeatedly executed at a predetermined control cycle for a certain period of time.

[0173] On the other hand, if the execution of the recommended method is recognized (YES in STEP 10), the second learning processing unit 1o adds multiple corresponding components corresponding to the object O, the current state of the object O, information regarding changes over time based on the state of the object O at a specified future time, and the second implementation method to the learning data, and causes the second prediction model to perform machine learning again (Figure 6 / STEP 11), thereby terminating the current processing.

[0174] As described above, in the state estimation method implemented by the estimation system S, first, a plurality of images (so-called spectroscopic images) of a target object are acquired for each of a plurality of predetermined wavelengths of light, each of which corresponds to a different component. In this case, each of the plurality of wavelengths corresponds to a different component. Therefore, according to this method, the components currently contained in the target object O can be easily and accurately determined simply by acquiring the spectroscopic images.

[0175] Then, in this method, the state of the object O at a predetermined time in the future (for example, a date and time set as the expiration date) is estimated based on the components thus identified. Here, since the spectroscopic image accurately indicates the components contained in the object O at the current time, the estimation result is also highly accurate.

[0176] Therefore, according to this state estimation method, the state of the object O at a predetermined time in the future can be recognized with high accuracy simply by the simple task of acquiring a spectroscopic image.

[0177] By managing the condition of the object based on the estimated contamination level and the estimated freshness and quality of the food, it becomes possible to set best-before dates or expiration dates, control logistics in a planned manner, and so on for a wide range of foods.

[0178] Other Embodiments Although the illustrated embodiments have been described above, the present invention is not limited to these embodiments.

[0179] Also, for example, in the above embodiment, the state estimation unit 1g estimates the state of the object O at one predetermined time point in the future. However, the state estimation unit of the present invention is not limited to such a configuration, and may be configured to estimate the states of the object O at multiple predetermined time points in the future.

[0180] In the above embodiment, for example, the estimation system S is a single computer system. However, the present invention also includes a state estimation program for causing any one or more computers to execute the above-described state estimation method, and a recording medium on which the program is recorded and which is readable by a computer used by a user or the like.

[0181] 1...server, 1a...photographing data recognition unit, 1b...component recognition unit, 1c...corresponding component storage unit, 1d...object attribute recognition unit, 1e...noise component recognition unit, 1f...noise component storage unit, 1g...state estimation unit, 1h...first prediction model, 1i...recommended method recognition unit, 1j...second prediction model, 1k...information presentation unit (alert unit), 1l...state recognition unit, 1m...first learning processing unit, 1n...implementation information recognition unit (implementation method recognition unit, environment recognition unit), 1o...second learning processing unit, 2...camera, 3...first user terminal, 4...second user terminal, 5...belt conveyor, O...object, S...estimation system (state estimation system).

Claims

1. A state estimation method for estimating the state of an object containing at least one of food and a component that may come into contact with the food, comprising the steps of: a photographic data recognition unit recognizing photographic data containing a plurality of images each corresponding to a different component and acquired for each of a plurality of predetermined wavelengths of light that are different from one another; a component recognition unit recognizing, based on the photographic data, a plurality of corresponding components in the object that correspond to each of the plurality of predetermined wavelengths of light; and a state estimation unit estimating the state of the object at a predetermined time in the future based on the plurality of corresponding components.

2. A state estimation method according to claim 1, characterized in that the state estimation unit compares the amounts of each of the plurality of corresponding components to estimate the state of the object at a predetermined time in the future.

3. A state estimation method as claimed in claim 1, comprising: a step in which an object attribute recognition unit recognizes object attributes that are attributes of the object; and a step in which a noise component recognition unit recognizes noise components that are components that should be removed as noise from among the plurality of corresponding components based on the object attributes, wherein the state estimation unit estimates the state of the object at a predetermined time point in the future based on components other than the noise components from among the plurality of corresponding components.

4. A state estimation method as described in claim 1, characterized in that the state estimation unit estimates the state of the object at a predetermined time in the future using a first prediction model that inputs the multiple corresponding components and outputs the change in the state of the object over time.

5. A state estimation method according to claim 4, wherein the state estimation unit uses as the first prediction model the plurality of corresponding components in a sample containing a material that is the same as or similar to the material contained in the object and the change in the state of the sample over time as learning data, and performs machine learning to determine the correlation between the plurality of corresponding components and the change in the state of a substance containing the material over time, and uses a prediction model that outputs the state of the object at the specified future point in time from the plurality of corresponding components of the input object.

6. A state estimation method according to claim 5, wherein the state estimation unit uses as the first prediction model the corresponding components in a sample containing a material that is the same as or similar to the constituent material of the object, changes in the state of the sample over time, and environmental information about the sample as learning data, and performs machine learning to determine the correlation between the changes in the state of the plurality of corresponding components and substances containing the material over time and the environmental information, and uses the plurality of corresponding components of the object and the environmental information about the object that have been input as learning data to output the state of the object at the specified future point in time.

7. A state estimation method according to claim 5, characterized in that it comprises: a step in which a state recognition unit recognizes the state of the object at the current time and the state of the object at the predetermined time in the future; and a step in which a first learning processing unit adds to learning data information relating to changes over time based on the plurality of corresponding components corresponding to the object, the state of the object at the current time, and the state of the object at the predetermined time in the future, and causes the first prediction model to perform machine learning again.

8. A state estimation method as described in claim 1, characterized in that it comprises: a step in which a recommended method recognition unit recognizes, based on the plurality of corresponding components, a recommended method, which is a processing method for the object that is currently recommended in order to improve the state of the object at the predetermined future time; and a step in which a notification unit notifies a user of the recommended method.

9. A state estimation method according to claim 8, characterized in that the recommended method recognition unit recognizes the recommended method using a second prediction model that inputs the plurality of corresponding components and outputs the recommended method.

10. A state estimation method according to claim 9, wherein the recommended method recognition unit uses as the second prediction model the multiple corresponding components in a sample containing a material that is the same as or similar to the material contained in the object, changes in the state of the sample over time, and a first implementation method that is a processing method performed on the sample as learning data, and machine-learns the correlation between the multiple corresponding components and the changes in the state of a substance containing the material over time and the first implementation method, and uses a prediction model that outputs the recommended method from the multiple corresponding components of the input object.

11. A state estimation method according to claim 9, comprising: a step in which a state recognition unit recognizes the state of the object at the current time and the state of the object at the specified time in the future; a step in which an implementation method recognition unit recognizes a second implementation method which is a processing method implemented on the object; and a step in which a second learning processing unit adds the plurality of corresponding components corresponding to the object, the state of the object at the current time, the state of the object at the specified time in the future, and the second implementation method to learning data, and causes the second prediction model to perform machine learning again.

12. A state estimation method according to claim 1, characterized in that the state estimation unit estimates the state of the object at a predetermined time in the future based on correlation data representing the correlation between the plurality of corresponding components and changes in the state of the object over time.

13. A state estimation method as described in claim 1, characterized in that the state of the object is the content of at least one of organic soiling, inorganic soiling, mixed organic-inorganic soiling, components produced by self-decomposition, and microbial soiling in the object.

14. A state estimation method according to claim 13, characterized in that the organic soiling is soiling based on at least one of lipids, proteins, allergens, carbohydrates, urea, amino acids, and nucleic acids.

15. A condition estimation method according to claim 13, characterized in that the inorganic dirt is dirt that is produced by the solidification of at least one of salt, calcium, magnesium, iron, soil components, heavy metals, and minerals.

16. A state estimation method according to claim 13, characterized in that the microbial soil is soil based on at least one of food poisoning bacteria, putrefactive bacteria, spoilage bacteria, lactic acid bacteria, yeast, mold, Escherichia coli, spore-forming bacteria, biofilm-forming bacteria, VBNC bacteria, drug-resistant bacteria, and viruses.

17. A state estimation method according to claim 1, characterized in that the state of the object is at least one of the freshness and quality of the food that is the object or the food that is in contact with the object's possible contact member.

18. A state estimation method according to claim 17, characterized in that the freshness is a value based on at least one of the appearance, color, luster, weight, and k value of the food whose freshness is to be estimated.

19. A state estimation method as described in claim 17, characterized in that the freshness is estimated based on at least one of the degree of spoilage, the degree of deterioration, and the degree of unpleasant odor estimated for the food whose freshness is to be estimated.

20. A state estimation method according to claim 17, wherein the quality is a value based on the degree of fermentation of the food whose quality is to be estimated.

21. A state estimation method according to claim 17, wherein the quality is a value based on components in the food whose quality is to be estimated.

22. A state estimation system for estimating the state of an object including at least one of food and a component that may come into contact with the food, characterized in that the state estimation system is configured to execute the state estimation method described in any one of claims 1 to 21.

23. A state estimation device for estimating the state of an object including at least one of food and a member that may come into contact with the food, characterized in that the state estimation device is configured to execute the state estimation method described in any one of claims 1 to 21.

24. A state estimation program that causes a computer to execute a state estimation method for estimating the state of an object that includes at least one of food and a component that may come into contact with the food, characterized in that the state estimation program causes the computer to execute the state estimation method described in any one of claims 1 to 21.

25. A recording medium having the state estimation program according to claim 24 recorded thereon, said state estimation program being readable by said computer.

Images