Program and method for generating learned model

Abstract

To improve a technique for monitoring an operation state of a liquid treatment apparatus.SOLUTION: A non-transitory computer-readable storage medium storing a program causing a computer to execute a process, the process comprising: acquiring an image relating to a liquid treatment apparatus; generating additional output information based on the image; and outputting a video obtained by applying image processing to the image based on the additional output information, wherein the liquid treatment apparatus includes a tank configured to store a liquid and a scraping unit configured to scrape a floating substance floating on a liquid surface of the liquid to the outside of the tank, the image includes a region of the liquid surface, and the additional output information includes first information specifying a region of the floating substance that can be included in the image.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present disclosure relates to a program and a method for generating a trained model. [Background technology]

[0002] In the manufacturing industry, liquid treatment devices such as pressurized flotation tanks are primarily used to treat wastewater. For example, in a pressurized flotation tank, substances with low specific gravity can be made to float as suspended matter from the liquid stored in the tank by adding chemicals and pumping air using a pump. Furthermore, the suspended matter can be separated from the water by scraping it out of the tank using a scraping unit.

[0003] There has been a need for improved technology to monitor whether liquid treatment devices are operating properly. Meanwhile, a method for determining water quality abnormalities from images captured by a surveillance camera is known (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2023-44875 Summary of the Invention [Problem to be solved by the invention]

[0005] In the technique described in Patent Document 1, no particular consideration is given to the convenience of an operator when monitoring the operating status of a liquid treatment device.

[0006] In view of the above circumstances, an object of the present disclosure is to improve the technology for monitoring the operating status of a liquid treatment device. [Means for solving the problem]

[0007] (1) A program according to an embodiment of the present disclosure includes: On the computer, acquiring an image relating to the liquid treatment device; generating additional output information based on the image; and outputting a video obtained by performing image processing on the image based on the additional output information, The liquid treatment device includes a tank that stores a liquid, and a scraping unit that scrapes out floating matter floating on the surface of the liquid to the outside of the tank, the image includes a region of the liquid surface; The additional output information includes first information identifying regions of the floating matter that may be included in the image.

[0008] (2) A program according to an embodiment of the present disclosure is the program according to (1), the additional output information includes second information regarding an area of ​​the floating object region that may be included in the image; The image is an image in which the second information is superimposed on the area of ​​the floating matter.

[0009] (3) A program according to an embodiment of the present disclosure is the program according to (2), The second information is If the image includes an overlapping area where the floating matter is hidden by the scraping portion, the information is based on the area of ​​the overlapping area and the area of ​​the floating matter area.

[0010] (4) A program according to an embodiment of the present disclosure is the program according to (2), The second information is When the floating matter is divided into a plurality of divided regions by the scraping unit in the image, the information is based on the area of ​​each of the divided regions.

[0011] (5) A program according to an embodiment of the present disclosure is the program according to (2) or (3), The second information is If the image includes an overlapping area where the floating matter is hidden by the scraping section, and if the floating matter is divided into multiple separated areas in the image by the scraping section, the information is based on the area of ​​each separated area and the area of ​​the overlapping area apportioned among the multiple separated areas.

[0012] (6) A program according to an embodiment of the present disclosure is a program according to any one of (1) to (5), The additional output information is The image data includes at least one of third information that identifies the area of ​​the wall surface of the tank included in the image, and fourth information regarding the area of ​​the wall surface of the tank included in the image.

[0013] (7) A program according to an embodiment of the present disclosure is the program according to (6), the additional output information includes the third information, The image processing a first image processing for superimposing the first information on the image and displaying it; and third image processing for superimposing the third information on the image and displaying it, The first image processing and the third image processing are different image processing.

[0014] (8) A program according to an embodiment of the present disclosure is a program according to any one of (1) to (7), In generating the additional output information, the first information is generated by inference using a trained model.

[0015] (9) A program according to an embodiment of the present disclosure is the program according to (6) or (7), In generating the additional output information, the first information and the third information are generated by inference using a trained model.

[0016] (10) A program according to an embodiment of the present disclosure is the program according to (8) or (9), The trained model is This is a trained model generated by machine learning using an image that has been pre-processed to change the brightness of a captured image that includes the liquid surface area as a training image.

[0017] (10) A method for generating a trained model according to an embodiment of the present disclosure includes: A method for generating a trained model, executed by an information processing device, in which an image related to a liquid treatment device is input and output data related to the image, The liquid treatment device includes a tank that stores a liquid, and a scraping unit that scrapes out floating matter floating on the surface of the liquid to the outside of the tank, the image includes a region of the liquid surface; the output data includes information identifying regions of the floating matter that may be included in the image; The method for generating the trained model includes: acquiring an image obtained by performing preprocessing to change brightness of the captured image including the liquid surface region; The method includes generating the trained model through machine learning using the acquired image as a teacher image. [Effects of the Invention]

[0018] According to an embodiment of the present disclosure, techniques for monitoring the operational status of liquid treatment devices can be improved. [Brief explanation of the drawings]

[0019] [Figure 1] FIG. 1 is a schematic diagram of an information processing system including an information processing device that executes a program according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a diagram for explaining the liquid treatment device and the imaging device. [Figure 3] FIG. 3 is a diagram showing a schematic image relating to a liquid treatment device. [Figure 4] FIG. 4 is a block diagram showing the configuration of the information processing device. [Figure 5]FIG. 5 is a flowchart illustrating an example of processing executed by the information processing device. [Figure 6] FIG. 6 is a diagram for explaining a specific example of generation of the second information. [Figure 7] FIG. 7 is a diagram for explaining a specific example of generation of the second information. [Figure 8] FIG. 8 is a diagram schematically showing an output video. [Figure 9] FIG. 9 is a flowchart illustrating an example of processing executed by an information processing device to generate a trained model. DETAILED DESCRIPTION OF THE INVENTION

[0020] Hereinafter, a program according to an embodiment of the present disclosure will be described with reference to the drawings.

[0021] In each drawing, the same or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0022] 1 shows a schematic diagram of an information processing system 1 including an information processing device 10 that executes a program according to this embodiment. The information processing system 1 includes the information processing device 10 and an imaging device 20. The information processing device 10 and the imaging device 20 are connected so as to be able to communicate with each other.

[0023] The information processing device 10 is any device used by a user. For example, a personal computer, a server computer, a general-purpose electronic device, or a dedicated electronic device can be employed as the information processing device 10. The information processing device 10 may also be a server device installed in a data center, for example. For example, the information processing device 10 may be a server belonging to a cloud computing system or other computing system. Note that while FIG. 1 shows an example in which the information processing system 1 includes one information processing device 10, this is not limiting. The information processing system 1 may also include two or more information processing devices 10. The information processing device 10 can acquire images captured by the imaging device 20.

[0024] The imaging device 20 is any device capable of capturing an image of a target. The imaging device 20 includes a high-speed camera, such as a CCD (Charge-Coupled Device) camera or a CMOS (Complementary Metal-Oxide-Semiconductor) camera. While FIG. 1 illustrates an example in which the information processing system 1 includes one imaging device 20, the present invention is not limited to this. The information processing system 1 may include two or more imaging devices 20. Furthermore, in the present embodiment, an example in which the information processing device 10 and the imaging device 20 are separate entities will be described, but the present invention is not limited to this. That is, the information processing device 10 and the imaging device 20 may be integrated.

[0025] Next, a liquid treatment device 30 and an imaging device 20 according to the present disclosure will be described with reference to FIG.

[0026] Liquid treatment device 30 is a device used to treat liquid 2 and includes a tank 4 and a scraping unit 5. Tank 4 stores liquid 2. Liquid 2 includes floating matter 3 floating on the liquid surface. Scraping unit 5 serves to scrape floating matter 3 out of tank 4. In the present disclosure, examples of liquid treatment devices include, but are not limited to, a pressurized flotation device. Similarly, examples of tank 4 include, but are not limited to, a pressurized flotation tank. Examples of liquid 2 include, but are not limited to, water, particularly wastewater. Examples of floating matter 3 include, but are not limited to, oil. In addition, in FIG. 2, tank 4 is approximately cylindrical and scraping unit 5 is installed rotatably around the central axis of tank 4, but the shapes and operating principles of the tank and scraping unit are not limited thereto.

[0027] If the amount of suspended matter 3 is small, there is a risk that impurities may not be sufficiently separated from the liquid 2. On the other hand, if the amount of suspended matter 3 is large, the processing speed of the liquid 2 by the liquid treatment device 30 may not be able to keep up, and impurities may not be sufficiently separated from the liquid 2. Therefore, by accurately determining the amount of suspended matter 3, the operating conditions of the liquid treatment device can be appropriately controlled, allowing the liquid treatment device to operate effectively. Examples of operating conditions to be controlled include the operating speed (e.g., rotational speed) of the scraping unit 5. In addition, in the case of a pressurized flotation tank, examples of operating conditions include the operating conditions of the pump that sends gas to the liquid 2. The pump can send gas to the liquid 2 through pressurized water.

[0028] Although there are methods for visually checking the amount of suspended solids, traveling to the tank requires a lot of work. Furthermore, since there are no indicators for quantitatively checking the amount of suspended solids, judgment is subjective. Therefore, the technology disclosed herein performs image processing based on predetermined additional output information on an image including the liquid surface area, and then displays and outputs the image. Using the information processing device 10 to process liquids using the liquid processing device 30 improves the convenience of the operator when monitoring the operating status of the liquid processing device.

[0029] The imaging device 20 captures an image including the liquid surface area of ​​the liquid 2. The image captured by the imaging device 20 can be used to generate a video, which will be described later, and to generate a trained model. The imaging device 20 may be attached above the liquid surface, for example, to a deck spanning the top of the tank 4. Note that the installation position of the imaging device 20 is not limited in the present disclosure. Furthermore, the installation angle (image capture direction) of the imaging device 20 is not limited, but may be, for example, facing downward or diagonally downward. The imaging device 20 is preferably installed so as to capture an image including the liquid surface area of ​​the liquid 2 and the wall surface area of ​​the tank 4.

[0030] Fig. 3 schematically shows an image 100 relating to a liquid treatment device 30. Image 100 was captured by imaging device 20 and includes a liquid surface region 110. In Fig. 3, image 100 also includes a floating matter region 120, a scraping portion region 140, and a tank wall surface region 130. However, in the present disclosure, the floating matter region, the scraping portion region, and the wall surface region may not be included in the image relating to the liquid treatment device.

[0031] 4 is a diagram showing a configuration of the information processing device 10 according to an embodiment of the present disclosure. Each component of the information processing device 10 according to an embodiment of the present disclosure will be described in detail with reference to FIG.

[0032] As shown in FIG. 4, the information processing device 10 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, and an output unit 15.

[0033] The control unit 11 includes at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a central processing unit (CPU) or a graphics processing unit (GPU), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The control unit 11 executes processes related to the operation of the information processing device 10 while controlling each unit of the information processing device 10.

[0034] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory is, for example, a random access memory (RAM) or a read only memory (ROM). The RAM is, for example, a static random access memory (SRAM) or a dynamic random access memory (DRAM). The ROM is, for example, an electrically erasable programmable read only memory (EEPROM). The storage unit 12 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores data used in the operation of the information processing device 10 and data obtained by the operation of the information processing device 10.

[0035] The communication unit 13 includes at least one external communication interface. The communication interface may be either a wired communication interface or a wireless communication interface. In the case of wired communication, the communication interface is, for example, a LAN (Local Area Network) interface or a USB (Universal Serial Bus). In the case of wireless communication, the communication interface is, for example, an interface compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th generation), or 5G (5th generation), or an interface compatible with short-range wireless communication such as Bluetooth (registered trademark). The communication unit 13 receives data used in the operation of the information processing device 10 and transmits data obtained by the operation of the information processing device 10.

[0036] The input unit 14 includes at least one input interface. The input interface may be, for example, a physical key, a capacitance key, a pointing device, or a touch screen integrated with a display. The input interface may also be, for example, a microphone that accepts voice input, or a camera that accepts gesture input. The input unit 14 accepts an operation to input data used in the operation of the information processing device 10. The input unit 14 may be connected to the information processing device 10 as an external input device instead of being provided in the information processing device 10. Any connection method may be used, for example, a Universal Serial Bus (USB), a High-Definition Multimedia Interface (HDMI) (registered trademark), or Bluetooth (registered trademark).

[0037] The output unit 15 includes at least one output interface. The output interface is, for example, a display that outputs information as a video. The display is, for example, an LCD (liquid crystal display) or an organic EL (electro luminescence) display. The output unit 15 displays and outputs data obtained by the operation of the information processing device 10. The output unit 15 may be connected to the information processing device 10 as an external output device instead of being provided in the information processing device 10. As a connection method, any method such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used.

[0038] The functions of the information processing device 10 are realized by executing a program according to this embodiment on a processor corresponding to the information processing device 10. That is, the functions of the information processing device 10 are realized by software. The program causes a computer to execute the operations of the information processing device 10, thereby causing the computer to function as the information processing device 10. That is, the computer functions as the information processing device 10 by executing the operations of the information processing device 10 in accordance with the program.

[0039] In this embodiment, the program can be recorded on a computer-readable recording medium. The computer-readable recording medium includes non-transitory computer-readable media, such as a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory. The program can be distributed, for example, by selling, transferring, or lending a portable recording medium, such as a DVD (digital versatile disc) or a CD-ROM (compact disc read only memory), on which the program is recorded. The program can also be distributed by storing the program in the storage of an external server and transmitting the program from the external server to another computer. The program can also be provided as a program product.

[0040] Some or all of the functions of the information processing device 10 may be implemented by a dedicated circuit equivalent to the control unit 11. In other words, some or all of the functions of the information processing device 10 may be implemented by hardware.

[0041] (Operation of information processing device) The operation of the information processing device 10 according to this embodiment will be described with reference to FIG. FIG. 5 is a flowchart showing an example of processing executed by the information processing device 10 according to this embodiment.

[0042] Step S110: Control unit 11 of information processing device 10 acquires an image related to liquid processing device 30. Specifically, control unit 11 acquires image 100 including the liquid surface area. Any method can be used to acquire image 100. For example, control unit 11 may acquire image 100 captured by imaging device 20. Control unit 11 may receive image 100 via communication unit 13, or may receive image 100 without via communication unit 13. Memory unit 12 may store image 100.

[0043] Step S120: The control unit 11 generates additional output information based on the image 100. The additional output information includes first information specifying a region 120 of floating matter that may be included in the image 100. The additional output information may also include second information relating to the area of ​​the region 120 of floating matter. Furthermore, the additional output information may include at least one of third information specifying a region 130 of the wall surface of the tank 4 that is included in the image 100 and fourth information relating to the area of ​​the wall surface region 130. The memory unit 12 may store the generated additional output information.

[0044] The method for generating the first information and the third information is not limited. However, for example, the first information and the third information may be generated by inference using a trained model. That is, the first information, or the first information and the third information, may be generated by inputting input data based on the image 100 into the trained model. By using the trained model, it is possible to classify the image into two or more regions (e.g., floating object region 120, wall surface region 130, etc.) with higher accuracy. The trained model will be described later.

[0045] The input data to be input to the trained model may be image data. The image data may be data of the image 100 itself, or may be data obtained by subjecting the image 100 to image processing.

[0046] The storage unit 12 stores the input data and the trained model. Alternatively, one or both of the input data and the trained model are stored in an external device separate from the information processing device 10 and are received from the external device via the communication unit 13.

[0047] The second information may be generated based on the first information. The second information may be, for example, the area ratio of suspended matter. The area ratio of suspended matter may be determined, for example, by Equation (1). [Number 1] (Floating matter area ratio) = (Floating matter area) / (Reference area) (1) For example, if the floating matter region 120 is identified in pixel units, the area of ​​the floating matter region 120 may be the number of pixels in the floating matter region 120. The reference area may be determined arbitrarily. For example, the number of pixels in the image 100 may be the reference area. Alternatively, the reference area may be the area of ​​the liquid surface region 110 in the image 100 when it is assumed that the scraped-out region 140 and the floating matter region 120 do not exist in the image 100, specifically the number of pixels in the liquid surface region 110 in the image 100. Furthermore, the reference area may be the area of ​​the entire liquid surface when the entire liquid surface of the liquid 2 is interpolated from the image 100, specifically the number of pixels corresponding to that area.

[0048] The fourth information may be generated based on the third information. The fourth information may be, for example, the wall surface area ratio. The wall surface area ratio may be determined by, for example, Equation (2). [Number 2] (Wall area ratio) = (Wall area) / (Reference area) (2) For example, if the wall surface region 130 is specified in pixel units, the area of ​​the wall surface region 130 may be the number of pixels in the wall surface region 130. The reference area may be determined arbitrarily. The reference area in the above formula (1) and the reference area in the above formula (2) may be the same or different.

[0049] A specific example of generating the second information will be described with reference to Figures 6 and 7. The method described below can also be applied to generating the fourth information.

[0050] The image shown in FIG. 6 includes not only the floating matter region 121 but also an overlapping region 122 in which the floating matter 3 floating on the liquid surface is hidden by the scraping unit 5. In this case, the second information may be information based on the area of ​​the overlapping region and the area of ​​the floating matter region, or may be information based on the area of ​​the floating matter region. That is, the area of ​​the floating matter region 121 may be used as the "area of ​​the floating matter region" in the above formula (1), or the area of ​​region 123 obtained by adding together the floating matter region 121 and the overlapping region 122 may be used. For example, if the area of ​​the floating matter region 121 is 12.8% and the area of ​​the overlapping region 122 is 0.4% of the entire image 100, "12.8%" may be used as the second information, or "13.2%" obtained by calculating 12.8% + 0.4% may be used as the second information.

[0051] In the image shown in FIG. 7, the floating matter 3 floating on the liquid surface is divided into multiple divided regions (divided region 124 and divided region 125) by the scraping unit 5. The image also includes an overlapping region 126 in which the floating matter 3 floating on the liquid surface is hidden by the scraping unit 5. In this case, the second information may be information based on the area of ​​the overlapping region and the area of ​​the floating matter region, or may be information based on the area of ​​each divided region. That is, the "area of ​​the floating matter region" in the above formula (1) may be the area of ​​each of the divided region 124 and the divided region 125, or the area of ​​region 129 obtained by adding together the divided region 124, the divided region 125, and the overlapping region 126. The information may also be based on the area of ​​each divided region and the area of ​​the overlapping region proportionally divided among the multiple divided regions. The "area of ​​the floating matter region" in the above formula (1) may be the combined area of ​​divided region 124 and region 127, or the combined area of ​​divided region 125 and region 128. Here, regions 127 and 128 may be equal divisions of overlapping region 126. For example, if the area of ​​divided region 124 is 15.3%, the area of ​​divided region 125 is 3.7%, and the area of ​​the overlapping region is 0.4% of the entire image 100, "15.3%, 3.7%" may be used as the second information, or "19.4%" calculated by 15.3% + 3.7% + 0.4% may be used as the second information. Alternatively, "15.5%, 3.9%" calculated by 15.3% + 0.4% ÷ 2 and 3.7% + 0.4% ÷ 2 may be used as the second information.

[0052] The control unit 11 may estimate an overlapping region (e.g., overlapping region 122 or overlapping region 126) or its area where the floating matter 3 floating on the liquid surface is hidden by the scraping unit 5. That is, the control unit 11 may generate information identifying the overlapping region or information based on the area of ​​the overlapping region. The memory unit 12 may store the information. The overlapping region may be estimated based on information identifying the scraping unit region 140 that may be included in the image 100 and the first information. For example, when the floating matter region and the scraping unit region contact each other, the overlapping region may be estimated so that the tangent line is included in the boundary line of the overlapping region. In FIG. 3, the floating matter region 120 and the scraping unit region 140 do not contact each other, so it may be estimated that no overlapping region exists. In FIG. 6, the floating matter region 121 and the scraping unit region 140 contact each other. Therefore, overlapping region 122 can be estimated by connecting the ends of the tangent lines (dotted lines in the figure) with a virtual line (straight line or curved line). In FIG. 7, division region 124 and scraped-out region 140 are in contact with each other, and division region 125 and scraped-out region 140 are in contact with each other. Therefore, overlapping region 126 can be estimated by connecting the ends of one tangent line and the other tangent line with a virtual line (straight line or curved line) for the two tangent lines (dotted lines in the figure). Control unit 11 may generate information that identifies scraped-out region 140 that may be included in image 100 based on image 100. For example, the information may be generated by inputting input data based on image 100 into a trained model.

[0053] Step S130: The control unit 11 of the information processing device 10 outputs a video obtained by performing image processing on the image 100 based on the additional output information. The video may be output by controlling the output unit 15. The control unit 11 may generate the video and control the output unit 15 to output the generated video. The storage unit 12 may store the generated video.

[0054] By displaying the first information and preferably also the second information as images, an operator can monitor whether the floating matter is being properly scraped out in the liquid treatment device, change the operating conditions as necessary, and determine whether there is an abnormality in the liquid treatment device.

[0055] By displaying the third or fourth information as an image, the liquid level in the tank can be grasped. Because it becomes difficult to scrape out floating matter when the liquid level drops, the operator can use the grasped liquid level to monitor whether the liquid level is sufficient to properly scrape out the floating matter and change the operating conditions as necessary. In addition, the liquid level can be used to determine abnormalities in the liquid treatment device. This method eliminates the need to install a separate gauge to grasp the liquid level, thereby reducing the cost of installing the gauge and eliminating the need to check the gauge separately.

[0056] The video will be described in detail with reference to the drawings. Fig. 8 shows a schematic diagram of an output video 200. The video 200 is an example of a video generated from the image 100 shown in Fig. 3, and image processing has been performed on the image 100. The image processing includes a first image processing 210 based on first information, a second image processing 220 based on second information, a third image processing 230 based on third information, and a fourth image processing 240 based on fourth information.

[0057] As an example of the first image processing 210, as shown in Fig. 8, a mask process may be performed on the floating matter region 120 with a predetermined color and transparency. Similarly, as the third image processing 230, a mask process may be performed on the wall surface region 130 with a predetermined color and transparency. Here, by using different image processing for the first image processing 210 and the third image processing 230, the worker can easily visually distinguish between the floating matter region 120 and the wall surface region 130. For example, it is preferable to perform mask processing using different colors for the floating matter region 120 and the wall surface region 130.

[0058] The manner in which the second information is displayed is not limited, but it is preferable to display it superimposed on the floatable matter region 120. As an example of the second image processing 220, as shown in FIG. 8, the second information may be displayed in numbers and units representing a percentage superimposed on the floatable matter region 120. The second image processing is not limited to this, and the second information may be displayed as numbers, letters, symbols, etc. superimposed on the floatable matter region 120. Furthermore, the first image processing may also serve as the second image processing. For example, when a mask process is performed on (superimposed on) the floatable matter region 120 as the first image processing, the second information may be displayed using color intensity, a color scale, etc.

[0059] The manner in which the fourth information is displayed is not limited, but it is preferable to display it by superimposing it on the wall surface region 130. As an example of the fourth image processing 240, as shown in FIG. 8, the fourth information may be displayed by numbers and units representing a ratio superimposed on the wall surface region 130. The fourth image processing is not limited to this, and the fourth information may be displayed by numbers, letters, symbols, etc. superimposed on the wall surface region 130. Furthermore, the third image processing may also serve as the fourth image processing. For example, when a mask process is performed on (superimposed on) the wall surface region 130 as the third image processing, the fourth information may be displayed by color density, a color scale, etc.

[0060] (Generating a trained model) Next, a technique for generating a trained model that can be used to generate additional output information according to the present disclosure will be described with reference to the accompanying drawings. The trained model receives an image including a liquid surface area as input and outputs output data including information identifying a floating matter area that may be included in the image. The output data preferably also includes information identifying a tank wall area that may be included in the image. The trained model can be generated by machine learning using an image obtained by preprocessing a captured image including a liquid surface area to change brightness as a training image.

[0061] FIG. 9 is a flowchart showing an example of processing executed by the information processing device 10 according to this embodiment to generate a trained model.

[0062] Step S210: The control unit 11 acquires an image that has been subjected to preprocessing to change the brightness of the captured image including the liquid surface area. For example, the control unit 11 may acquire the image from a host computer or the like via the communication unit 13. The control unit 11 may also acquire the captured image including the liquid surface area from an imaging device. In this case, the control unit 11 may perform preprocessing to change the brightness of the acquired captured image. The control unit 11 may refer to a database to perform the preprocessing. The storage unit 12 may store the database. The control unit 11 may also refer to an input based on a user operation via the input unit 14 to perform the preprocessing.

[0063] Examples of brightness that may be changed in preprocessing include brightness in the HSV color space, luminance in the HLS color space, and luminance in the YUV color space. Preprocessing may also be processing that sets the average and variance of the brightness distribution of the captured images to predetermined values. Preprocessing may also be processing that makes the brightness uniform across multiple captured images.

[0064] Step S220: The control unit 11 acquires information identifying a region of floating matter that may be included in the acquired image. For example, the control unit 11 may acquire the information from a host computer or the like via the communication unit 13. Alternatively, the control unit 11 may acquire the information based on a user operation via the input unit 14.

[0065] In order to further use the trained model to generate third information, the control unit 11 may further acquire information that identifies the area of ​​the tank wall that may be included in the acquired image.

[0066] Step S230: The control unit 11 generates a trained model by machine learning using the acquired image as a teacher image. The trained model is generated by associating the teacher image with the information acquired in step S220.

[0067] The trained model may be, for example, a trained model for segmentation, such as a trained model generated by a machine learning algorithm such as Mask R-CNN, Unet, or UperNet.

[0068] Although the present disclosure has been described based on the drawings and examples, it should be noted that those skilled in the art can easily make various modifications and alterations based on the present disclosure. Therefore, it should be noted that these modifications and alterations are included in the scope of the present disclosure. For example, the functions included in each means or step can be rearranged so as not to be logically inconsistent, and multiple means or steps can be combined or divided into one.

[0069] 5 and 9 show examples in which steps are processed serially, but this is not limiting, and each step may be processed in parallel, for example.

[0070] Furthermore, for example, in the above-described embodiment, an embodiment is also possible in which the configuration and operation of the information processing device 10 are distributed among a plurality of computers that can communicate with each other. [Explanation of symbols]

[0071] 1. Information Processing Systems 2 liquid 3 Floating objects 4 Tank 5. Scraping section 10. Information processing equipment 11 Control section 12 Storage section 13 Communications Department 14 Input section 15 Output section 20 Imaging device 30 Liquid treatment equipment 100 images 110 Liquid Surface Area 120 Floating Area 121 Floating Area 122, 126 overlapping area 123, 129 area 124, 125 Divided Area 127, 128 area 130 Wall Area 140 scraping area 200 videos 210 First Image Processing 220 Second Image Processing 230 Third Image Processing 240 The Fourth Image Processing

Claims

1. On the computer, acquiring an image relating to the liquid treatment device; generating additional output information based on the image; and outputting a video obtained by performing image processing on the image based on the additional output information, The liquid treatment device includes a tank that stores a liquid, and a scraping unit that scrapes out floating matter floating on the surface of the liquid to the outside of the tank, the image includes a region of the liquid surface; The program, wherein the additional output information includes first information that identifies regions of the floating matter that may be included in the image.

2. the additional output information includes second information regarding an area of ​​the floating object region that may be included in the image; The program according to claim 1 , wherein the image is an image in which the second information is superimposed on an area of ​​the floating object.

3. The second information is The program according to claim 2 , wherein, when the image includes an overlapping area where the floating matter is hidden by the scraping unit, the information is based on an area of ​​the overlapping area and an area of ​​the floating matter area.

4. The second information is The program according to claim 2 , wherein, when the floating matter is divided into a plurality of divided regions by the scraping unit in the image, the information is based on an area of ​​each of the divided regions.

5. The second information is The program of claim 2, wherein if the image includes an overlapping area where the floating matter is hidden by the scraping section, and if the floating matter is divided into multiple divided areas in the image by the scraping section, the information is based on the area of ​​each of the divided areas and the area of ​​the area obtained by dividing the overlapping area by the multiple divided areas.

6. The additional output information is The program according to any one of claims 1 to 5, further comprising at least one of third information that identifies an area of ​​the wall surface of the tank included in the image, and fourth information regarding an area of ​​the wall surface of the tank included in the image.

7. the additional output information includes the third information, The image processing a first image processing for superimposing the first information on the image and displaying it; and third image processing for superimposing the third information on the image and displaying it, The program according to claim 6 , wherein the first image processing and the third image processing are different image processing.

8. The program according to any one of claims 1 to 5, wherein in generating the additional output information, the first information is generated by inference using a trained model.

9. The program according to claim 6 , wherein in generating the additional output information, the first information and the third information are generated by inference using a trained model.

10. The program according to claim 7 , wherein in generating the additional output information, the first information and the third information are generated by inference using a trained model.

11. The trained model is The program according to claim 8, wherein the program is a trained model generated by machine learning using an image that has been pre-processed to change the brightness of a captured image including the liquid surface region as a training image.

12. The trained model is The program according to claim 9, wherein the program is a trained model generated by machine learning using an image that has been pre-processed to change the brightness of a captured image including the liquid surface region as a training image.

13. The trained model is The program according to claim 10, wherein the program is a trained model generated by machine learning using an image obtained by preprocessing an image including the liquid surface region to change its brightness as a training image.

14. A method for generating a trained model, executed by an information processing device, in which an image related to a liquid treatment device is input and output data related to the image, The liquid treatment device includes a tank that stores a liquid, and a scraping unit that scrapes out floating matter floating on the surface of the liquid to the outside of the tank, the image includes a region of the liquid surface; the output data includes information identifying regions of the floating matter that may be included in the image; The method for generating the trained model includes: acquiring an image obtained by performing preprocessing to change brightness of the captured image including the liquid surface region; A method for generating a trained model, comprising: generating the trained model through machine learning using the acquired image as a teacher image.

Images