Information processing apparatus, learning model creating apparatus, information processing method, learning model creating method, and program

Abstract

To provide an information processing device, a learning model creation device, an information processing method, a learning model creation method, and a program capable of stably grasping a state of a surface layer of a basis material.SOLUTION: The information processing device includes an image processing unit that creates an image-processed image including one or a plurality of regions indicating one or a plurality of objects included in an image based on a learned model obtained by machine-learning a relationship between an image of an inside of a glass melting furnace and an image including one or a plurality of regions indicating one or a plurality of objects included in the image, and the image of the inside of the glass melting furnace, and a derivation unit that derives information indicating a basis flow based on the image-processed image created by the image processing unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a learning model creation device, an information processing method, a learning model creation method, and a program. [Background technology]

[0002] Glass articles are manufactured using a glass melting furnace. The glass melting furnace includes a melting tank having a bottom and a side wall, and an upper structure covering the upper part of the melting tank. The glass melting furnace is operated while adjusting operation parameters based on predetermined control indices. Regarding glass manufacturing methods, a technology for manufacturing high-quality glass articles while suppressing deterioration of the quality of the glass articles is known (see, for example, Patent Document 1).

[0003] The quality of glass is greatly affected by the temperature and residence time of the glass. Therefore, in order to ensure stable glass quality, it is important to understand and control the flow of the glass. For example, by monitoring the inside of the kiln with a camera, the flow of the glass can be continuously monitored. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-081593 Summary of the Invention [Problem to be solved by the invention]

[0005] When assessing the flow of the base material, the condition of the surface layer of the base material changes depending on the timing of capturing images inside the kiln. Also, the results of assessing the condition of the surface layer of the base material may vary depending on the person.

[0006] The present disclosure has been made in consideration of these circumstances, and aims to provide an information processing device, a learning model creation device, an information processing method, a learning model creation method, and a program that can stably grasp the condition of the surface of a substrate. [Means for solving the problem]

[0007] One aspect of the present disclosure is an information processing device comprising: a trained model that has machine-learned the relationship between an image of the interior of a glass melting furnace and an image including one or more regions that indicate one or more objects contained in the image; an image processing unit that creates an image-processed image including one or more regions that indicate one or more objects contained in the image based on the image of the interior of the furnace; and a derivation unit that derives information indicating base material flow based on the image-processed image created by the image processing unit.

[0008] One aspect of the present disclosure is a learning model creation device that includes a processing unit that creates a learning model by machine learning the relationship between the image of the inside of a glass melting furnace and the image-processed image including one or more regions that indicate one or more objects included in the image, using the image of the inside of the furnace as an explanatory variable and the image-processed image including one or more regions that indicate one or more objects included in the image as a target variable, based on a learning dataset that includes an image of the inside of a glass melting furnace as learning data and an image-processed image including one or more regions that indicate one or more objects included in the image as a training data set.

[0009] One aspect of the present disclosure is an information processing method executed by a computer, which creates an image-processed image including one or more regions indicating one or more objects contained in the image based on an image of the inside of a glass melting furnace and a trained model that has been machine-learned to understand the relationship between an image of the inside of the furnace and an image including one or more regions indicating one or more objects contained in the image, and the image of the furnace, and derives information indicating the flow of the base material based on the created image-processed image.

[0010] One aspect of the present disclosure is a computer-implemented learning model creation method, which is based on a learning dataset that includes, as learning data, an image of the interior of a glass melting furnace and, as training data, an image-processed image that includes one or more regions that indicate one or more objects included in the image, and creates a learning model by machine learning the relationship between the image of the interior of the furnace and the image-processed image that includes one or more regions that indicate one or more objects included in the image, using the image of the furnace as an explanatory variable and the image-processed image that includes one or more regions that indicate one or more objects included in the image as a target variable.

[0011] One aspect of the present disclosure is a program that causes a computer to create an image-processed image including one or more regions indicating one or more objects contained in an image based on an image of the inside of a glass melting furnace and a trained model that has been machine-learned to understand the relationship between an image of the inside of the furnace and an image including one or more regions indicating one or more objects contained in the image, and the image of the furnace, and derive information indicating the flow of the base material based on the image-processed image that has been created.

[0012] One aspect of the present disclosure is a program that causes a computer to create a learning model by machine learning the relationship between the image of the inside of a glass melting furnace and the image-processed image including one or more regions that indicate one or more objects included in the image, using the image of the inside of the furnace as an explanatory variable and the image-processed image including one or more regions that indicate one or more objects included in the image as a target variable, based on a learning dataset that includes an image of the inside of a glass melting furnace as learning data and an image-processed image including one or more regions that indicate one or more objects included in the image as training data. [Effects of the Invention]

[0013] The information processing device, learning model creation device, information processing method, learning model creation method, and program disclosed herein have the effect of enabling the state of the surface layer of a substrate to be stably grasped. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a cross-sectional view showing an example of a glass melting furnace 100 according to the present embodiment. [Figure 2] 1 is a cross-sectional view showing an example of a glass melting furnace 100 according to the present embodiment. [Figure 3A] 1 is a diagram showing an example of a batch pile image of a glass melting furnace 100 according to the present embodiment. FIG. [Figure 3B] 1 is a diagram showing an example of a mirror image of a glass melting furnace 100 according to the present embodiment. FIG. [Figure 4] FIG. 1 is a diagram illustrating an example of an information processing device 200 according to the present embodiment. [Figure 5A] FIG. 2 is a diagram for explaining an example of the operation of the information processing device 200 according to the present embodiment. [Figure 5B] FIG. 2 is a diagram for explaining an example of the operation of the information processing device 200 according to the present embodiment. [Figure 6A] FIG. 2 is a diagram for explaining an example of the operation of the information processing device 200 according to the present embodiment. [Figure 6B] FIG. 2 is a diagram for explaining an example of the operation of the information processing device 200 according to the present embodiment. [Figure 7A] FIG. 2 is a diagram for explaining an example of the operation of the information processing device 200 according to the present embodiment. [Figure 7B] FIG. 2 is a diagram for explaining an example of the operation of the information processing device 200 according to the present embodiment. [Figure 8A] FIG. 2 is a diagram for explaining an example of the operation of the information processing device 200 according to the present embodiment. [Figure 8B] FIG. 2 is a diagram for explaining an example of the operation of the information processing device 200 according to the present embodiment. [Figure 9] FIG. 2 is a diagram for explaining an example of the operation of the information processing device 200 according to the present embodiment. [Figure 10] 10 is a flowchart showing an example of the operation of the information processing device 200 according to the present embodiment. [Figure 11] FIG. 10 is a diagram showing an example of an image of an area where a foam layer is visible. [Figure 12]FIG. 1 is a diagram illustrating an example of a learning model creation device 300 according to the present embodiment. [Figure 13A] FIG. 2 is a diagram for explaining an example of the operation of the learning model creation device 300 according to the present embodiment. [Figure 13B] FIG. 2 is a diagram for explaining an example of the operation of the learning model creation device 300 according to the present embodiment. [Figure 14A] FIG. 2 is a diagram for explaining an example of the operation of the learning model creation device 300 according to the present embodiment. [Figure 14B] FIG. 2 is a diagram for explaining an example of the operation of the learning model creation device 300 according to the present embodiment. [Figure 15] 10 is a flowchart showing an example of the operation of the learning model creation device 300 of this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0015] Next, an information processing device, a learning model creation device, an information processing method, a learning model creation method, and a program according to the present embodiment will be described with reference to the drawings. The embodiments described below are merely examples, and embodiments to which the present disclosure is applicable are not limited to the following embodiments. In all the drawings for explaining the embodiments, the same reference numerals are used for components having the same functions, and repeated explanations will be omitted.

[0016] (Embodiment) [Glass melting furnace] Fig. 1 is a cross-sectional view showing an example of a glass melting furnace 100 according to this embodiment. Fig. 1 shows a cross-sectional view of the glass melting furnace 100 in the xz plane. Fig. 2 is a cross-sectional view showing an example of a glass melting furnace 100 according to this embodiment. Fig. 2 shows a cross-sectional view of the glass melting furnace 100 in the yz plane. 1 and 2, the xy plane is the horizontal plane. The glass melting furnace 100 includes a melting tank 10 into which glass raw material 70 is supplied, a plurality of current-carrying electrodes 50, a plurality of thermocouples 40, a bubbler 90, an upper structure 20 covering the upper part of the melting tank 10, and a plurality of burners 60.

[0017] In the glass melting furnace 100, glass frit 70 is melted by firing the burner 60 and applying a voltage to the current-carrying electrode 50. The glass frit is supplied to a batch pile 80 from the right end of FIG. 2, and the molten glass frit flows in the positive direction of the y-axis and is taken out from the right end of FIG. 2. A plurality of burners 60 are arranged on both sides of the upper structure 20 in the y direction, which is the direction in which the frit 70 flows. Each burner 60 heats the surface of the frit 70 near the burner 60 by burning. Therefore, depending on the amount of gas supplied to each of the plurality of burners 60, the surface of the frit 70 in the flow direction of the frit 70 and the width direction of the melting tank 10 is heated, and the heat amount is distributed.

[0018] The current-carrying electrodes 50 are arranged so that a plurality of pairs to which a voltage is applied protrude from the bottom of the melting tank 10 in the y-axis direction, which is the direction in which the glass frit 70 flows, and one or more pairs in the x-axis direction, which is the width direction of the melting tank 10. When a voltage is applied to each pair of current-carrying electrodes 50, the glass frit 70 between the pair of current-carrying electrodes 50 is heated.

[0019] The frit material 70 is heated in the flow direction of the frit material 70 and in the width direction of the melting tank 10 by the voltage applied to each of the multiple pairs of powered electrodes 50, and the amount of heat is distributed. Furthermore, the melting tank 10 is heated in the depth direction by the amount of gas supplied to the burner 60 and the voltage applied to the pairs of powered electrodes 50, and the amount of heat is distributed. Although only one pair of powered electrodes 50 is shown in FIG. 1, multiple pairs of powered electrodes 50 may be arranged in the x-z cross section.

[0020] Each of one or more thermocouples 40 arranged at the bottom of the melting tank 10 measures the temperature of the glass frit 70 at the bottom of the melting tank 10. Furthermore, each of one or more thermocouples 40 arranged in the upper structure 20 measures either or both of the ambient temperature at the upper part of the glass melting furnace 100 and the temperature of the upper structure 20 as a substitute for the surface temperature of the glass frit 70. Alternatively, each of one or more thermocouples 40 arranged in the upper structure 20 may measure only the ambient temperature at the upper part of the glass melting furnace 100.

[0021] A plurality of thermocouples 40 are arranged at the bottom of the melting tank 10 in the y-axis direction, which is the flow direction of the frit material 70, and a plurality of thermocouples 40 are arranged in the x-axis direction, which is the width direction of the melting tank 10. A plurality of thermocouples 40 are arranged in the y-axis direction in the upper structure 20. Therefore, the plurality of thermocouples 40 arranged at the bottom of the melting tank 10 can measure the temperature distribution of the frit material 70 at the bottom of the melting tank 10 in the flow direction of the frit material 70 and in the width direction of the melting tank 10.

[0022] In addition, the temperature distribution of the surface of the frit 70 in the flow direction of the frit 70 can be measured by a plurality of thermocouples 40 arranged on the upper structure 20. Here, the temperature distribution of the surface of the frit 70 can include not only the temperature distribution of the surface of the molten frit 70 but also the temperature distribution of the surface of the frit before melting, which is supplied from the right end of Fig. 2. The temperature distribution of the surface of the frit before melting can be measured by the thermocouple 40 on the left end of Fig. 2, which is arranged near the supply of the frit 70.

[0023] Each pair of current-carrying electrodes 50 can measure the temperature of the frit material 70 between the pair by detecting the current or resistance between the pair. Therefore, by using multiple pairs of current-carrying electrodes 50, it is possible to measure the temperature distribution of the frit material 70 in the center of the melting tank 10 in the direction of flow of the frit material 70 and in the width direction of the melting tank 10. Furthermore, by combining two or more pairs of thermocouples 40 arranged at the bottom of the melting tank 10, thermocouples 40 arranged in the upper structure 20, and current-carrying electrodes 50, it is possible to obtain the temperature distribution in the depth direction of the melting tank 10.

[0024] One or more bubblers 90 are arranged at the bottom of the melting tank 10 in the x-axis direction, which is perpendicular to the flow direction of the glass frit 70. Each of the one or more bubblers 90 pumps air to continuously generate bubbles in the furnace.

[0025] One or more imaging devices (not shown) are also installed inside the melting tank 10. The imaging device takes images at any location where it can capture an image of the surface base material inside the kiln. For example, the imaging device captures images of the mirror surface MS and the batch pile BM. The imaging device is equipped with a communication device, and the communication device transmits image information of the captured image to an information processing device. The image information of the captured image may include information indicating the date and time of capture. Below, we will continue to explain the case where the image information of the captured image includes information indicating the date and time of capture.

[0026] 3A is a diagram showing an example of a batch pile image of the glass melting furnace 100 according to this embodiment. The batch pile image is an example of a captured image. In the example of the batch pile image, the area surrounded by a dashed line is the batch pile portion. FIG. 3B is a diagram showing an example of a mirror image of the glass melting furnace 100 according to this embodiment. The mirror image is an example of a captured image. In the example of the mirror image, the area surrounded by a dashed line is the mirror surface portion. The captured image is analyzed by an information processing device. The information processing device acquires information indicating the state of the surface layer of the substrate by analyzing the captured image. The information processing device will be described below.

[0027] [Information processing device] 4 is a diagram showing an example of an information processing device 200 according to this embodiment. The information processing device 200 is realized by including a personal computer, a tablet terminal device, or other information processing equipment. The information processing device 200 includes, for example, a communication unit 201, an image processing unit 203, a derivation unit 205, a creation unit 206, an output unit 208, and a storage unit 210.

[0028] The communication unit 201 is realized by a communication module. The communication unit 201 communicates with an external communication device such as an imaging device (not shown) via a network NW. The communication unit 201 may communicate using a wireless communication method such as wireless LAN, Bluetooth (registered trademark), or LTE (registered trademark). The communication unit 201 may also communicate using a communication method such as wired LAN. The communication unit 201 receives image information of the captured image of the interior of the glass melting furnace 100 transmitted by the imaging device.

[0029] The image processing unit 203 acquires image information of the interior of the glass melting furnace 100 from the communication unit 101. Based on the acquired image information of the interior of the glass melting furnace 100, the image processing unit 203 creates an image including one or more regions indicating one or more objects included in the image of the interior of the glass melting furnace 100. An example of an image including one or more regions indicating one or more objects included in the image of the interior of the glass melting furnace 100 is an image in which one or more objects included in the image of the interior of the glass melting furnace 100 are each divided into different colors. Some of the multiple objects may be divided into the same color.

[0030] For example, the image processing unit 203 has a trained model 204. The trained model 204 is a machine learning result of the relationship between an image of the interior of the glass melting furnace 100 and an image including one or more regions showing one or more objects contained in the image. The creation of the trained model 204 will be described later.

[0031] The image processing unit 203 inputs the image information of the acquired image of the interior of the glass melting furnace 100 into the trained model 204, and acquires image information of an image including one or more areas indicating one or more objects contained in the image of the interior of the furnace output by the trained model 204. Hereinafter, by inputting image information of an image inside the glass melting furnace 100 into the trained model 204, an image including one or more areas indicating one or more objects contained in the image inside the furnace output by the trained model 204 will be referred to as an image-processed image, and the image information will also be referred to as image-processed image information.

[0032] 5A and 5B are diagrams for explaining an example of the operation of the information processing device 200 according to this embodiment. 5A is a diagram showing an example of an image of a batch pile inside the glass melting furnace 100. The image processing unit 203 inputs image information of the image of the batch pile inside the furnace to the trained model 204, and acquires image-processed image information output by the trained model 204. 5B is a diagram showing an example of an image processed by the trained model 204. As shown in FIG. 5B, in the image processed image, the wall area, the base material area, the batch pile area, the frame area, and the wall area are divided into different colors. In FIG. 5B, the wall of the melting tank 10 and the wall of the superstructure 20 are divided into different colors.

[0033] 6A and 6B are diagrams for explaining an example of the operation of the information processing device 200 according to this embodiment. 6A is a diagram showing an example of an image of a mirror surface inside the glass melting furnace 100. The image processing unit 203 inputs image information of the mirror surface inside the glass melting furnace 100 to the trained model 204, and acquires image-processed image information output by the trained model 204. 6B is a diagram showing an example of an image-processed image output by the trained model 204. As shown in FIG. 6B, in the image showing the state of the mirror surface inside the glass melting furnace 100, an area showing the wall, an area showing the base material, an area showing the mirror surface, an area showing the bubble layer, and an area showing the wall are divided into different colors. In FIG. 6B, as in FIG. 5B, the wall of the melting tank 10 and the wall of the superstructure 20 are divided into different colors. Returning to FIG. 4, the explanation will be continued.

[0034] The derivation unit 205 acquires, from the image processing unit 203, information indicating the image capture date and time and image-processed image information included in the image information of the image inside the glass melting furnace 100. The derivation unit 205 derives information indicating the flow of the glass substrate based on the acquired image-processed image information. For example, the derivation unit 205 derives, based on the image-processed image, at least one of information indicating the size of the pile of glass, information indicating the area of ​​the pile of glass, and information indicating the orientation of the pile of glass, which is the direction in which the pile of glass extends.

[0035] 7A and 7B are diagrams for explaining an example of the operation of the information processing device 200 according to this embodiment. Fig. 7A is an example of image information of a pile of batches in a glass melting furnace 100, and Fig. 7B is an example of image information obtained from Fig. 7A after image processing. The derivation unit 205 derives the size of the pile of batches by deriving the length of the pile of batches in the flow direction of the glass raw material 70 from the image after image processing.

[0036] The derivation unit 205 derives the area of ​​the batch pile by deriving the area of ​​the batch pile region from the processed image. The derivation unit 205 derives the orientation of the batch pile by deriving the distance from the center of the kiln to the batch pile from the processed image.

[0037] For example, the deriving unit 205 derives the distance from the kiln center to the batch pile for each of the multiple regions that are obtained by dividing the batch pile in the y-axis direction, and for each of the multiple distances that are derived, the longer distance is designated as the outward direction. Figure 7B shows an example in which the distance from the kiln center to the batch pile is derived for each of the four regions that are obtained by dividing the batch pile in the y-axis direction. In this case, the derivation unit 205 determines that the longer of the derived distances d1 to d2 is the outward direction. The derivation unit 205 outputs at least one of the derived information indicating the size of the pile of batches, the information indicating the area of ​​the pile of batches, and the information indicating the orientation of the pile of batches to the output unit 208.

[0038] 8A and 8B are diagrams for explaining an example of the operation of the information processing device 200 according to this embodiment. Fig. 8A is an example of an image of the mirror surface inside the glass melting furnace 100, and Fig. 8B is an example of an image-processed image obtained from Fig. 8A. The derivation unit 205 derives at least one of information indicating the area of ​​the mirror surface and information indicating the mirror surface distance based on the image-processed image.

[0039] The derivation unit 205 derives the area of ​​the mirror surface by deriving the area of ​​the mirror surface region from the image-processed image. The derivation unit 205 derives the distance from each of the plurality of bubbler wicks to the bubble layer boundary portion, which is the boundary between the mirror surface region and the bubble layer region, from the image-processed image, thereby deriving the mirror surface distance. Here, each of the plurality of bubbler positions in the image-processed image is set in advance. The derivation unit 205 outputs at least one of the derived information indicating the area of ​​the mirror surface and the information indicating the mirror surface distance to the output unit 208. Returning to FIG.

[0040] The creation unit 206 acquires information indicating the image capture date and time, and at least one of information indicating the area of ​​the mirror surface and information indicating the mirror surface distance from the derivation unit 205. Here, as an example, the description will continue assuming that the creation unit 206 acquires information indicating the area of ​​the mirror surface. The creation unit 206 creates time-series data of the area of ​​the mirror surface based on the acquired information indicating the image capture date and time and the information indicating the area of ​​the mirror surface. FIG. 9 is a diagram for explaining an example of the operation of the information processing device 200 according to this embodiment. FIG. 9 shows an example of time-series data of the mirror surface area. In addition to the time-series data of the mirror surface area, FIG. 9 includes information indicating the flatness of a glass article such as a glass plate and information indicating the number of bubbles. According to FIG. 9, it can be seen that the mirror surface area is almost stable except for the days around 9 / 17, which are surrounded by a dashed line. Returning to FIG. 4, the explanation will continue.

[0041] The output unit 208 acquires at least one of the information indicating the size of the batch pile, the information indicating the area of ​​the batch pile, and the information indicating the orientation of the batch pile output by the derivation unit 205. The output unit 208 outputs at least one of the acquired information indicating the size of the batch pile, the information indicating the area of ​​the batch pile, and the information indicating the orientation of the batch pile. For example, the output unit 208 may display at least one of information indicating the size of the batch pile, information indicating the area of ​​the batch pile, and information indicating the orientation of the batch pile by outputting it to a display unit (not shown), or may output it by voice.

[0042] Furthermore, the output unit 208 acquires and outputs at least one of the information indicating the area of ​​the mirror surface and the information indicating the mirror surface distance output by the derivation unit 205. For example, the output unit 208 may output at least one of the information indicating the area of ​​the mirror surface and the information indicating the mirror surface distance to a display unit (not shown) for display, or may output it by voice. Furthermore, the output unit 208 acquires and outputs time-series data of at least one of information indicating the area of ​​the mirror surface and information indicating the mirror surface distance from the creation unit 206. For example, the output unit 208 may output time-series data of at least one of information indicating the area of ​​the mirror surface and information indicating the mirror surface distance to a display unit (not shown) to display the data.

[0043] The storage unit 210 is realized by a hard disk drive (HDD), flash memory, random access memory (RAM), read only memory (ROM), or the like, and stores information.

[0044] The image processing unit 203, derivation unit 205, creation unit 206, and output unit 208 are realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a computer program (software) stored in the storage unit 210. Furthermore, some or all of these functional units may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware.

[0045] The computer program may be stored in advance in a storage device such as a HDD or flash memory, or may be stored in a removable storage medium such as a DVD (Digital Versatile Disc) or CD-ROM, and installed by inserting the storage medium into a drive device.

[0046] (Operation of information processing device 200) FIG. 10 is a flowchart showing an example of the operation of the information processing device 200 according to this embodiment. (Step S1-1) The communication unit 201 receives image information of the inside of the glass melting furnace 100 transmitted by the imaging device. (Step S2-1) The image processing unit 203 acquires image information of the inside of the glass melting furnace 100 from the communication unit 101.

[0047] (Step S3-1) The image processing unit 203 creates image-processed image information based on the acquired image information of the interior of the glass melting furnace 100. (Step S4-1) The derivation unit 205 acquires the image-processed image information from the image processing unit 203. The derivation unit 205 derives information indicating the material flow based on the acquired image-processed image information. For example, the derivation unit 205 derives at least one of information indicating the size of the batch pile, information indicating the area of ​​the batch pile, and information indicating the orientation of the batch pile.

[0048] (Step S5-1) The creating unit 206 acquires information indicating the area of ​​the mirror surface from the deriving unit 205. The creating unit 206 creates time-series data of the area of ​​the mirror surface based on the acquired information indicating the area of ​​the mirror surface. (Step S6-1) The output unit 208 outputs information indicating the base material flow acquired from the derivation unit 205. For example, the output unit 208 outputs at least one of information indicating the size of the batch pile, information indicating the area of ​​the batch pile, and information indicating the orientation of the batch pile. The output unit 208 also outputs at least one of information indicating the area of ​​the mirror surface and information indicating the mirror surface distance acquired from the derivation unit 205. The output unit 208 also acquires time-series data on the area of ​​the mirror surface from the creation unit 206 and outputs the data.

[0049] In the above-described embodiment, the imaging device may capture an image of an area where the bubble layer is visible. Fig. 11 is a diagram showing an example of an image of an area where the bubble layer is visible. In this case, the communication unit 201 receives image information transmitted by the imaging device, which captures an image of an area where the bubble layer is visible inside the glass melting furnace 100. The image processing unit 203 acquires image information of an area in which a bubble layer is visible inside the glass melting furnace 100 from the communication unit 101. The image processing unit 203 creates image-processed image information based on the acquired image information of an area in which a bubble layer is visible inside the glass melting furnace 100. Here, the image-processed image is an image that includes one or more areas that show one or more bubble layers included in the image inside the furnace.

[0050] The derivation unit 205 acquires the image-processed image information from the image processing unit 203. Based on the acquired image-processed image information, the derivation unit 205 derives at least one of information indicating the number of bubbles in the surface foam layer and information indicating the area of ​​the surface foam layer. The derivation unit 205 outputs the derived at least one of the information indicating the number of bubbles in the surface foam layer and information indicating the area of ​​the surface foam layer to the output unit 208. The output unit 208 outputs at least one of the acquired information indicating the number of bubbles in the surface foam layer and the acquired information indicating the area of ​​the surface foam layer.

[0051] As described above, the information processing device 200 of this embodiment creates an image-processed image including one or more regions indicating one or more objects included in the image based on the trained model 204 and an image of the inside of the kiln. The information processing device 200 derives information indicating the base material flow based on the created image-processed image. The information processing device 200 can use the trained model 204 to create an image-processed image in which one or more objects included in the image of the inside of the kiln are divided into one or more regions. According to the information processing device 200 configured in this manner, information indicating the flow of the substrate can be derived based on the processed image, so that the state of the surface layer of the substrate can be grasped stably.

[0052] The information processing device 200 also creates multiple processed images based on multiple images of the inside of the kiln captured continuously by the imaging unit. The information processing device 200 configured in this way can create time-series data of information indicating the flow of the base material based on the multiple processed images.

[0053] The information processing device 200 further includes a creating unit that creates time-series data of the base material flow based on the information indicating the base material flow derived by the derivation unit. The information processing device 200 configured in this manner can acquire the obtained feature amounts as time-series data, making it possible to confirm the correlation between the quality data and operation data such as the kiln temperature and the gas amount.

[0054] Furthermore, the information processing device 200 creates an image-processed image including at least one of an area showing the frame, an area showing the batch pile, and an area showing the base material, based on an image of the inside of the kiln including an image of the batch pile. With the information processing device 200 configured in this way, it is possible to separate at least one of an area showing the frame, an area showing the batch pile, and an area showing the base material, based on an image of the inside of the kiln including an image of the batch pile.

[0055] Furthermore, the information processing device 200 creates an image-processed image including at least one of an area showing a mirror surface, an area showing a foam layer, and an area showing a base material, based on an image of the inside of the kiln including an image of the periphery of the bubbler. The information processing device 200 configured in this manner can distinguish at least one of an area showing a mirror surface, an area showing a foam layer, and an area showing a base material, based on an image of the inside of the kiln including an image of the periphery of the bubbler.

[0056] Furthermore, the information processing device 200 derives at least one of information indicating the size of the pile of batches, information indicating the area of ​​the pile of batches, and information indicating the orientation of the pile of batches based on the processed image. With the information processing device 200 configured in this manner, at least one of information indicating the size of the pile of batches, information indicating the area of ​​the pile of batches, and information indicating the orientation of the pile of batches can be derived from the processed image that includes at least one of an area indicating the frame, an area indicating the pile of batches, and an area indicating the substrate, making it possible to reliably grasp the condition of the surface of the substrate.

[0057] Furthermore, the information processing device 200 derives at least one of information indicating the area of ​​the mirror surface and information indicating the distance between the bubbler wicks and the bubble layer boundary portion based on the image-processed image. With the information processing device 200 configured in this way, at least one of information indicating the area of ​​the mirror surface and information indicating the distance between the bubbler wicks and the bubble layer boundary portion can be derived from an image-processed image of the kiln interior, including an image of the bubbler periphery, that includes at least one of an area indicating the mirror surface, an area indicating the bubble layer, and an area indicating the base material. This allows for a stable understanding of the surface condition of the base material.

[0058] Furthermore, the information processing device 200 derives at least one of information indicating the number of foam layers and the area of ​​the foam layers based on an image of the kiln interior including an image of the area where the foam layers are visible. The information processing device 200 configured in this manner can derive at least one of information indicating the number of foam layers and the area of ​​the foam layers from an image-processed image in which areas indicating one or more foam layers are segmented, obtained by inputting image information of the area where the foam layers are visible into the trained model 204.

[0059] (Learning model creation device) A method for creating the trained model 204 will be described. The trained model 204 is created by a learning model creation device. That is, the learning model creation device creates the trained model 204. Note that the information processing device 200 may include the learning model creation device. That is, the information processing device 200 may create the trained model 204.

[0060] 12 is a diagram showing an example of a learning model creation device 300 according to this embodiment. The learning model creation device 300 is realized by a device such as a personal computer, a server, a smartphone, a tablet computer, or an industrial computer. The learning model creation device 300 trains a learning model using a learning dataset in which image information of the interior of the glass melting furnace 100 is used as an input sample and image information of an image including one or more regions showing one or more objects included in the image of the interior of the furnace is used as an output sample, and creates a learned model 204. Here, the learning model is a model that is the basis of the learned model 204.

[0061] For example, the learning model creation device 300 uses algorithms such as a convolution neural network (CNN), a recurrent neural network (RNN), a long short-term memory (LSTM), a random forest, a support vector machine (SVM), or a neural network to construct the learned model 204. An input sample is data input to an input layer when training the learning model. An output sample is data (teacher data) that serves as a correct answer for comparison with an output value output from an output layer when training the learning model.

[0062] 13A and 13B are diagrams for explaining an example of the operation of the learning model creation device 300 according to this embodiment. Fig. 13A is an example of image information of a batch pile inside a glass melting furnace 100. Fig. 13B is an image processed based on Fig. 13A. As shown in Fig. 13B, in the image processed, an area showing the wall, an area showing the substrate, an area showing the batch pile, an area showing the frame, and an area showing the wall are divided into different colors.

[0063] 14A and 14B are diagrams for explaining an example of the operation of the learning model creation device 300 according to this embodiment. Fig. 14A is an example of image information of the mirror surface inside the glass melting furnace 100. Fig. 14B is an image processed based on Fig. 14A. As shown in Fig. 14B, in the processed image, the area representing the wall, the area representing the substrate, the area representing the mirror surface, the area representing the bubble layer, and the area representing the wall are divided into different colors. Returning to Fig. 12, the explanation will be continued.

[0064] The learning model creation device 300 includes an input unit 302 , a receiving unit 304 , a processing unit 306 , an output unit 308 , and a storage unit 310 . The input unit 302 inputs information. As an example, the input unit 302 may have an operation unit such as a keyboard and a mouse. In this case, the input unit 302 inputs information according to an operation performed by a user on the operation unit. As another example, the input unit 302 may input information from an external device. The external device may be, for example, a portable storage medium. A training dataset is input to the input unit 302.

[0065] The receiving unit 304 acquires a training dataset from the input unit 302 and accepts the acquired training dataset. The training dataset includes input samples and output samples, and the input samples and output samples are paired. The training dataset is made up of a plurality of pairs.

[0066] The processing unit 306 inputs an input sample into the input layer of the learning model 307, calculates the error between the output value output from the output layer and the output sample corresponding to the input sample, and changes the parameters of the learning model 307 so as to minimize the error, thereby creating the trained model 204. Here, the output sample is an example of training data. The learning model 307 is trained by changing the parameters of the learning model 307. The trained model 204 created as described above is received by the information processing device 200 from the output unit 308 via a network or a medium, and acquired by the image processing unit 203. When the learning model creation device 300 is included in the information processing device 200, the image processing unit 203 acquires the trained model 204 from the learning model creation device 300.

[0067] All or part of the input unit 302, the receiving unit 304, the processing unit 306 and the output unit 308 are functional units (hereinafter referred to as software functional units) that are realized, for example, by a processor such as a CPU executing a program stored in the memory unit 310. In addition, all or part of the input unit 302, the reception unit 304, the processing unit 306, and the output unit 308 may be realized by hardware such as an LSI, an ASIC, or an FPGA, or may be realized by a combination of a software function unit and hardware.

[0068] (Training dataset) The creation of a training dataset will now be described. As described above, the training dataset includes input samples and output samples. An example of an input sample is image information of the interior of a glass melting furnace 100. When creating the training model 307, the captured images are grayscale, and therefore, as the difference in color gradation between multiple objects decreases, they become harder to see. In such cases, at least one of brightness adjustment, color adjustment, and contrast adjustment is performed on the image of the interior of the furnace captured by the imaging device. By doing so, even if the image of the interior of the furnace captured by the imaging device has low contrast, it can be made easier to see.

[0069] (Operation of the learning model creation device 300) FIG. 15 is a flowchart showing an example of the operation of the learning model creation device 300 of this embodiment. (Step S1-2) The input unit 302 acquires a training dataset. (Step S2-2) The receiving unit 304 acquires the training data set from the input unit 302 and accepts the acquired training data set.

[0070] (Step S3-2) The processing unit 306 acquires a training dataset from the receiving unit 304. For every pair of an input sample and an output sample included in the training dataset, the processing unit 306 inputs the input sample to the input layer of the training model 307 and acquires an output value output from the output layer. The processing unit 306 calculates the error between the output value and the training data, which is the output sample corresponding to the input sample, and trains the training model 307 by changing the parameters of the training model 307 so as to minimize the error, thereby creating the trained model 204. (Step S4-2) The output unit 308 acquires the learning model 307 from the processing unit 306. The output unit 308 outputs the acquired learning model 307.

[0071] In the above-described embodiment, the case where the learning model creation device 300 creates the learning model 307 by performing supervised learning has been described, but this is not limiting. The learning method for machine learning is not limited to supervised learning, and may be any of unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods. Any learning method for machine learning is acceptable.

[0072] As described above, the learning model creation device 300 of this embodiment creates a learning model 307 by machine learning the relationship between the image of the inside of the kiln and the image-processed image including one or more regions indicating one or more objects contained in the image, using the image of the inside of the kiln as an explanatory variable and the image-processed image including one or more regions indicating one or more objects contained in the image as a target variable, based on the learning dataset. The learning model creation device 300 can create an image-processed image including one or more regions indicating one or more objects contained in the image, based on an image of the inside of a glass melting furnace. According to the learning model creation device 300 configured in this manner, a learned model 204 can be created, and therefore the information processing device 200 can use the learned model 204 created by the learning model creation device 300 to create an image-processed image in which one or more objects included in an image of the kiln interior are divided into one or more regions.

[0073] Furthermore, the learning model creation device 300 uses, as explanatory variables, images of the inside of the kiln captured by the imaging unit, including at least one of an image with brightness adjustment, an image with color adjustment, and an image with contrast adjustment, and as objective variables, image-processed images including one or more regions showing one or more objects included in the images, to create a learning model 307 by machine learning the relationship between the images of the inside of the kiln and image-processed images including one or more regions showing one or more objects included in the images. The learning model creation device 300 can create image-processed images including one or more regions showing one or more objects included in the images, based on images of the inside of the kiln captured by the imaging unit, including at least one of an image with brightness adjustment, an image with color adjustment, and an image with contrast adjustment. According to the learning model creation device 300 configured in this manner, even if the contrast of the image of the inside of the kiln captured by the imaging device is low, the contrast can be increased, thereby improving the accuracy of machine learning.

[0074] In addition, in the learning model creation device 300, the image inside the kiln includes an image of the batch pile, and the image-processed image includes at least one of an area showing the frame, an area showing the batch pile, and an area showing the base material. According to the learning model creation device 300, an image-processed image can be created based on an image of the kiln interior that includes an image of a batch pile, the image including at least one of an area showing the frame, an area showing the batch pile, and an area showing the base material included in the image.

[0075] In addition, in the learning model creation device 300, the image inside the kiln includes an image of the area around the bubbler, and the image-processed image includes at least one of an area showing a mirror surface, an area showing a bubble layer, and an area showing a base material. According to the learning model creation device 300, an image-processed image can be created based on an image of the kiln interior, including an image of the area around the bubbler, that includes at least one of an area showing the mirror surface, an area showing the bubble layer, and an area showing the base material.

[0076] Furthermore, in the learning model creation device 300, the image inside the kiln includes an image of the area where the foam layer is visible, and the image-processed image includes an area showing the foam layer. According to the learning model creation device 300, an image-processed image including an area showing the foam layer included in an image of the inside of the kiln including an image of an area where the foam layer is visible can be created based on the image of the inside of the kiln including an image of the area where the foam layer is visible.

[0077] The above has described in detail an embodiment of the present disclosure with reference to the drawings, but the specific configuration is not limited to this embodiment, and design changes and the like are also included within the scope that does not deviate from the gist of the present disclosure. Furthermore, for example, a computer program for realizing the functions of each of the above-described devices may be recorded on a computer-readable recording medium, and the computer program recorded on the recording medium may be read and executed by a computer system. Note that the "computer system" here may also include hardware such as an OS and peripheral devices.

[0078] In addition, "computer-readable recording medium" refers to writable non-volatile memory such as a flexible disk, optical magnetic disk, ROM, or flash memory, portable media such as a DVD (Digital Versatile Disc), or a storage device such as a hard disk built into a computer system. Furthermore, the term "computer-readable recording medium" also includes those that retain a program for a certain period of time, such as volatile memory (e.g., DRAM (Dynamic Random Access Memory)) within a computer system that serves as a server or client when a computer program is transmitted via a network such as the Internet or a communication line such as a telephone line.

[0079] The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be a program for implementing some of the functions described above, or may be a so-called differential file (differential program) that can implement the functions described above in combination with a program already stored in the computer system. [Explanation of symbols]

[0080] 10 melting tank, 20 upper structure, 40 thermocouple, 50 current-carrying electrode, 60 burner, 70 glass raw material, 80 batch pile, 90 bubbler, 100 glass melting furnace, 200 information processing device, 201 communication unit, 203 image processing unit, 204 trained model, 205 derivation unit, 206 creation unit, 208 output unit, 210 memory unit, 300 learning model creation device, 302 input unit, 304 reception unit, 306 processing unit, 307 learning model, 308 output unit, 310 memory unit

Claims

1. an image processing unit that creates an image-processed image including one or more regions showing one or more objects included in the image based on a trained model that has been machine-learned to understand the relationship between an image of the inside of a glass melting furnace and an image including one or more regions showing one or more objects included in the image, and the image of the inside of the furnace; a derivation unit that derives information indicating a material flow based on the image-processed image created by the image processing unit; An information processing device comprising:

2. The information processing device according to claim 1 , wherein the image processing unit creates a plurality of processed images based on a plurality of images of the inside of the kiln captured successively by an imaging unit.

3. a creating unit that creates time-series data of the material flow based on the information indicating the material flow derived by the deriving unit; The information processing device according to claim 2 , further comprising:

4. the image of the kiln interior includes an image of a batch pile; 4. The information processing device according to claim 1, wherein the image processing unit creates the processed image including at least one of an area showing a frame, an area showing a batch pile, and an area showing a base material based on the image of the inside of the kiln.

5. the image of the interior of the kiln includes an image of the area around a bubbler; The information processing device described in any one of claims 1 to 3, wherein the image processing unit creates the image-processed image including at least one of an area showing a mirror surface, an area showing a foam layer, and an area showing a base material based on the image of the inside of the kiln.

6. The information processing device according to claim 4 , wherein the deriving unit derives at least one of information indicating a size of the pile of batches, information indicating an area of ​​the pile of batches, and information indicating an orientation of the pile of batches based on the processed image.

7. The information processing device according to claim 5 , wherein the deriving unit derives at least one of information indicating an area of ​​a mirror surface and information indicating distances between a plurality of bubbler wicks and a boundary portion of a foam layer based on the processed image.

8. the image of the kiln interior includes an image of an area where a foam layer is visible; The information processing device according to claim 1 , wherein the deriving unit derives at least one of information indicating the number of foam layers and an area of ​​the foam layers based on the image of the oven interior.

9. a processing unit that creates a learning model by machine learning the relationship between the image of the inside of the furnace and the image-processed image including one or more regions showing one or more objects included in the image, using the image of the inside of the furnace as an explanatory variable and the image-processed image including one or more regions showing one or more objects included in the image as a target variable, based on a learning dataset that includes an image of the inside of a glass melting furnace as learning data and an image-processed image including one or more regions showing one or more objects included in the image as training data; A learning model creation device comprising:

10. The learning model creation device of claim 9, wherein the images of the inside of the kiln include at least one of an image with brightness adjustment, an image with color adjustment, and an image with contrast adjustment relative to the image of the inside of the kiln captured by the imaging unit.

11. the image of the kiln interior includes an image of a batch pile; The learning model creation device according to claim 9 , wherein the processed image includes at least one of an area representing a frame, an area representing a batch pile, and an area representing a substrate.

12. the image of the interior of the kiln includes an image of the area around a bubbler; The learning model creation device according to claim 9 , wherein the processed image includes at least one of an area representing a mirror surface, an area representing a foam layer, and an area representing a base material.

13. the image of the kiln interior includes an image of an area where a foam layer is visible; The learning model creation device according to claim 9 , wherein the image-processed image includes an area showing a foam layer.

14. 1. A computer-implemented information processing method, comprising: A trained model is created based on the image of the interior of a glass melting furnace, which has been machine-learned to understand the relationship between the image of the interior of the furnace and an image including one or more regions showing one or more objects included in the image, and the image of the furnace, to create an image-processed image including one or more regions showing one or more objects included in the image; An information processing method for deriving information indicating material flow based on the created image processed image.

15. The information processing method according to claim 14, wherein when the image-processed image is created, a plurality of image-processed images are created based on a plurality of images of the inside of the kiln captured continuously by an imaging unit.

16. The information processing method according to claim 14, further comprising creating time-series data of the material flow based on the information indicating the material flow.

17. the image of the kiln interior includes an image of a batch pile; The information processing method of claim 14, wherein when creating the image-processed image, the image-processed image is created based on the image of the inside of the kiln, and includes at least one of an area showing a frame, an area showing a batch pile, and an area showing the base material.

18. the image of the interior of the kiln includes an image of the area around a bubbler; An information processing method as described in claim 14, wherein when creating the image-processed image, the image-processed image is created based on the image inside the kiln, and includes at least one of an area showing a mirror surface, an area showing a foam layer, and an area showing a base material.

19. The information processing method according to claim 14, wherein when deriving the information indicating the material flow, at least one of information indicating the size of the batch pile, information indicating the area of ​​the batch pile, and information indicating the orientation of the batch pile is derived based on the image-processed image.

20. An information processing method as described in claim 14, wherein when deriving information indicating the base material flow, at least one of information indicating the area of ​​the mirror surface and information indicating the distance between multiple bubbler wicks and the bubble layer boundary portion is derived based on the image-processed image.

21. the image of the kiln interior includes an image of an area where a foam layer is visible; The information processing method according to claim 14, wherein when deriving the information indicating the base material flow, at least one of information indicating the number of foam layers and the area of ​​the foam layers is derived based on the image inside the kiln.

22. A computer-implemented learning model creation method, comprising: A learning model creation method, based on a learning dataset that includes an image of the inside of a glass melting furnace as learning data and an image-processed image that includes one or more regions that show one or more objects included in the image as teacher data, creates a learning model by machine learning the relationship between the image of the inside of the furnace and the image-processed image that includes one or more regions that show one or more objects included in the image, using the image of the inside of the furnace as an explanatory variable and the image-processed image that includes one or more regions that show one or more objects included in the image as a target variable.

23. The learning model creation method described in claim 22, wherein the images of the inside of the kiln include at least one of an image with brightness adjustment, an image with color adjustment, and an image with contrast adjustment relative to the image of the inside of the kiln captured by the imaging unit.

24. the image of the kiln interior includes an image of a batch pile; The learning model creation method according to claim 22, wherein the processed image includes at least one of an area representing a frame, an area representing a batch pile, and an area representing a substrate.

25. the image of the interior of the kiln includes an image of the area around a bubbler; The learning model creation method according to claim 22, wherein the processed image includes at least one of an area representing a mirror surface, an area representing a foam layer, and an area representing a base material.

26. the image of the kiln interior includes an image of an area where a foam layer is visible; The learning model creation method according to claim 22 , wherein the image-processed image includes an area showing a foam layer.

27. On the computer, A trained model is created based on the image of the interior of a glass melting furnace, which has been machine-learned to understand the relationship between the image of the interior of the furnace and an image including one or more regions showing one or more objects included in the image, and the image of the furnace, and the trained model is created based on the image of the interior of the furnace, which includes one or more regions showing one or more objects included in the image, and A program for deriving information indicating material flow based on the created image-processed image.

28. The program according to claim 27, wherein when creating the image-processed image, a plurality of image-processed images are created based on a plurality of images of the inside of the kiln captured continuously by an imaging unit.

29. 28. The program according to claim 27, which creates time-series data of the material flow based on the information indicating the material flow.

30. the image of the kiln interior includes an image of a batch pile; The program of claim 27, wherein when creating the image-processed image, the image-processed image is created based on the image of the inside of the kiln, and includes at least one of an area showing a frame, an area showing a batch pile, and an area showing a base material.

31. the image of the interior of the kiln includes an image of the area around a bubbler; The program of claim 27, wherein when creating the image-processed image, the image-processed image is created based on the image inside the kiln, and includes at least one of an area showing a mirror surface, an area showing a foam layer, and an area showing a base material.

32. The program according to claim 27, wherein when deriving the information indicating the material flow, at least one of information indicating the size of the batch pile, information indicating the area of ​​the batch pile, and information indicating the orientation of the batch pile is derived based on the image-processed image.

33. The program described in claim 27, wherein when deriving information indicating the base material flow, at least one of information indicating the area of ​​the mirror surface and information indicating the distance between the multiple bubbler wicks and the bubble layer boundary portion is derived based on the image-processed image.

34. the image of the kiln interior includes an image of an area where a foam layer is visible; The program according to claim 27, wherein when deriving the information indicating the base material flow, at least one of information indicating the number of foam layers and the area of ​​the foam layers is derived based on the image inside the kiln.

35. On the computer, A program that creates a learning model by machine learning the relationship between the image of the inside of the kiln and the image-processed image including one or more regions showing one or more objects included in the image, using the image of the inside of the kiln as an explanatory variable and the image-processed image including one or more regions showing one or more objects included in the image as a target variable, based on a learning dataset that includes an image of the inside of a glass melting kiln as learning data and an image-processed image including one or more regions showing one or more objects included in the image as a training data set.

36. The program according to claim 35, wherein the image of the inside of the kiln includes at least one of an image with brightness adjustment, an image with color adjustment, and an image with contrast adjustment for the image of the inside of the kiln captured by the imaging unit.

37. the image of the kiln interior includes an image of a batch pile; 36. The program according to claim 35, wherein the processed image includes at least one of an area representing a frame, an area representing a batch pile, and an area representing a substrate.

38. the image of the interior of the kiln includes an image of the area around a bubbler; The program according to claim 35, wherein the processed image includes at least one of an area showing a mirror surface, an area showing a foam layer, and an area showing a substrate.

39. the image of the kiln interior includes an image of an area where a foam layer is visible; The program of claim 35 , wherein the processed image includes an area showing a foam layer.

Images