How to generate programs and trained models

Abstract

Improve technology for monitoring the operation of liquid treatment equipment. [Solution] A program that causes a computer to acquire an image related to a liquid treatment device, generate additional output information based on the image, and output a video obtained by applying image processing to the image based on the additional output information, wherein the liquid treatment device has a tank for storing liquid and a scraping unit for scraping floating matter floating on the surface of the liquid to the outside of the tank, the image includes an area of ​​the liquid surface, and the additional output information includes first information that identifies an area of ​​the floating matter that may be included in the image.

Description

[Technical field]

[0001] The present disclosure relates to a program and a method for generating a trained model. [Background technology]

[0002] In the manufacturing industry, liquid treatment devices such as pressurized flotation tanks are used mainly for the purpose of treating wastewater. For example, in a pressurized flotation tank, substances with a low specific gravity can be made to float as suspended matter from the liquid stored in the tank by adding chemicals and feeding air using a pump. In addition, the suspended matter can be scraped out of the tank using a scraping unit, whereby the substance can be separated from the water.

[0003] There has been a demand for improved technology for monitoring whether liquid treatment devices are operating properly. Meanwhile, a method for determining water quality abnormalities from images captured by a surveillance camera is known (for example, Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2023-44875 A Summary of the Invention [Problem to be solved by the invention]

[0005] In the technique described in Patent Document 1, no particular consideration is given to the convenience for an operator to monitor the operating status of the liquid treatment device.

[0006] In view of the above circumstances, an object of the present disclosure is to improve the technology for monitoring the operating status of a liquid treatment device. [Means for solving the problem]

[0007] (1) A program according to an embodiment of the present disclosure, On the computer, acquiring an image relating to the liquid treatment device; generating additional output information based on the image; and outputting a video obtained by performing image processing on the image based on the additional output information, The liquid treatment device includes a tank for storing a liquid, and a scraping unit for scraping out floating matter floating on the surface of the liquid to the outside of the tank, the image includes a region of the liquid surface; The additional output information includes first information identifying regions of the floating object that may be included in the image.

[0008] (2) A program according to an embodiment of the present disclosure is the program according to (1), the additional output information includes second information regarding an area of ​​the floating object region that may be included in the image; The image is an image in which the second information is displayed superimposed on the area of ​​the floating object.

[0009] (3) A program according to an embodiment of the present disclosure is the program according to (2), The second information is When the image includes an overlapping region in which the floating matter is hidden by the scraping portion, the information is based on the area of ​​the overlapping region and the area of ​​the floating matter region.

[0010] (4) A program according to an embodiment of the present disclosure is the program according to (2), The second information is In the case where the floating matter is divided into a plurality of divided regions by the scraping portion in the image, the information is based on the area of ​​each of the divided regions.

[0011] (5) A program according to an embodiment of the present disclosure is the program according to (2) or (3), The second information is When the image includes an overlapping area in which the floating matter is hidden by the scraping-out section, and when the floating matter is divided into a plurality of divided areas in the image by the scraping-out section, the information is based on the area of ​​each of the divided areas and the area of ​​the overlapping area prorated among the plurality of divided areas.

[0012] (6) A program according to an embodiment of the present disclosure is a program according to any one of (1) to (5), The additional output information is The information includes at least one of third information that identifies the region of the wall surface of the tank included in the image, and fourth information regarding the area of ​​the region of the wall surface of the tank included in the image.

[0013] (7) A program according to an embodiment of the present disclosure is the program according to (6), the additional output information includes the third information, The image processing includes: a first image processing for superimposing the first information on the image and displaying the first information; and a third image processing for superimposing the third information on the image and displaying the third information, The first image processing and the third image processing are different image processing.

[0014] (8) A program according to an embodiment of the present disclosure is a program according to any one of (1) to (7), In generating the additional output information, the first information is generated by inference using a trained model.

[0015] (9) A program according to an embodiment of the present disclosure is the program according to (6) or (7), In generating the additional output information, the first information and the third information are generated by inference using a trained model.

[0016] (10) A program according to an embodiment of the present disclosure is the program according to (8) or (9), The trained model is This is a trained model generated by machine learning using an image that has been pre-processed to change the brightness of a captured image that includes the liquid surface area as a teacher image.

[0017] (10) A method for generating a trained model according to an embodiment of the present disclosure includes: A method for generating a trained model, which is executed by an information processing device, inputting an image related to a liquid treatment device and outputting output data related to the image, The liquid treatment device includes a tank for storing a liquid, and a scraping unit for scraping out floating matter floating on the surface of the liquid to the outside of the tank, the image includes a region of the liquid surface; the output data includes information identifying regions of the floating object that may be included in the image; The method for generating the trained model includes: acquiring an image obtained by performing pre-processing to change brightness of a captured image including the liquid surface area; The method includes generating the trained model through machine learning using the acquired image as a teacher image. Effect of the Invention

[0018] SUMMARY OF THE DISCLOSURE According to an embodiment of the present disclosure, techniques for monitoring the operational status of liquid treatment devices can be improved. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a schematic diagram of an information processing system including an information processing device that executes a program according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram for explaining the liquid treatment device and the imaging device. [Diagram 3] FIG. 3 is a schematic diagram showing an image relating to a liquid treatment device. [Figure 4] FIG. 4 is a block diagram showing the configuration of the information processing device. [Diagram 5]FIG. 5 is a flowchart illustrating an example of processing executed by the information processing device. [Figure 6] FIG. 6 is a diagram for explaining a specific example of generation of the second information. [Figure 7] FIG. 7 is a diagram for explaining a specific example of generation of the second information. [Figure 8] FIG. 8 is a diagram showing a schematic diagram of an output video. [Figure 9] FIG. 9 is a flowchart illustrating an example of a process executed by an information processing device to generate a trained model. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] Hereinafter, a program according to an embodiment of the present disclosure will be described with reference to the drawings.

[0021] In each drawing, the same or corresponding parts are denoted by the same reference numerals. In the description of this embodiment, the description of the same or corresponding parts will be omitted or simplified as appropriate.

[0022] 1 shows a schematic diagram of an information processing system 1 including an information processing device 10 that executes a program according to this embodiment. The information processing system 1 includes the information processing device 10 and an imaging device 20. The information processing device 10 and the imaging device 20 are connected to each other so as to be able to communicate with each other.

[0023] The information processing device 10 is any device used by a user. For example, a personal computer, a server computer, a general-purpose electronic device, or a dedicated electronic device can be adopted as the information processing device 10. The information processing device 10 may also be a server device installed in, for example, a data center. For example, the information processing device 10 may be a server belonging to a cloud computing system or other computing system. Note that, although an example in which the information processing system 1 includes one information processing device 10 is shown in FIG. 1, this is not limiting. The information processing system 1 may include two or more information processing devices 10. The information processing device 10 can acquire an image captured by the imaging device 20.

[0024] The imaging device 20 is any device capable of capturing an image of a target. The imaging device 20 includes a high-speed camera, such as a CCD (Charge-Coupled Device) camera or a CMOS (Complementary Metal-Oxide-Semiconductor) camera. FIG. 1 shows an example in which the information processing system 1 includes one imaging device 20, but is not limited to this. The information processing system 1 may include two or more imaging devices 20. In addition, in this embodiment, an example in which the information processing device 10 and the imaging device 20 are separate entities will be described, but is not limited to this. That is, the information processing device 10 and the imaging device 20 may be integrated.

[0025] Next, the liquid treatment device 30 and the imaging device 20 according to the present disclosure will be described with reference to FIG.

[0026] The liquid treatment device 30 is a device used to treat the liquid 2, and has a tank 4 and a scraping unit 5. The tank 4 stores the liquid 2. The liquid 2 includes floating matter 3 floating on the liquid surface. The scraping unit 5 has a role of scraping the floating matter 3 to the outside of the tank 4. In the present disclosure, the liquid treatment device may be a pressurized flotation device, but is not limited thereto. Similarly, the tank 4 may be a pressurized flotation tank, but is not limited thereto. The liquid 2 may be water, particularly wastewater, but is not limited thereto. The floating matter 3 may be, for example, oil, but is not limited thereto. In FIG. 2, the tank 4 is substantially cylindrical, and the scraping unit 5 is installed so as to be rotatable about the central axis of the tank 4, but the shape and operating principle of the tank and the scraping unit are not limited thereto.

[0027] When the amount of the suspended matter 3 is small, there is a risk that the impurities cannot be sufficiently separated from the liquid 2. On the other hand, when the amount of the suspended matter 3 is large, the processing speed of the liquid 2 by the liquid treatment device 30 cannot keep up, and the impurities may not be sufficiently separated from the liquid 2. Therefore, by accurately grasping the amount of the suspended matter 3, the operating conditions of the liquid treatment device can be appropriately controlled, and the liquid treatment device can be operated effectively. Examples of the operating conditions to be controlled include the operating speed (e.g., rotation speed) of the scraping unit 5. In addition, in the case of a pressurized flotation tank, examples of the operating conditions include the operating conditions of the pump that sends gas to the liquid 2. The pump can send gas to the liquid 2 through pressurized water.

[0028] Although there is a method for visually checking the amount of floating matter, it requires a lot of man-hours to move to the tank. In addition, since there is no index for quantitatively checking the amount of floating matter, the judgment is subjective. Therefore, in the technology disclosed herein, an image including the liquid surface area is subjected to image processing based on predetermined additional output information and displayed. By using the information processing device 10 to process the liquid using the liquid processing device 30, the convenience of the operator in monitoring the operating status of the liquid processing device is improved.

[0029] The imaging device 20 captures an image including the liquid surface area of ​​the liquid 2. The image captured by the imaging device 20 can be used for generating a video to be described later and for generating a trained model. The imaging device 20 may be attached above the liquid surface, for example, to a deck spanning the top of the tank 4. Note that the installation position of the imaging device 20 is not limited in the present disclosure. The installation angle (shooting direction) of the imaging device 20 is also not limited, but may be, for example, downward or diagonally downward. The imaging device 20 is preferably provided so as to capture an image including the liquid surface area of ​​the liquid 2 and the wall surface area of ​​the tank 4.

[0030] Fig. 3 shows a schematic diagram of an image 100 relating to a liquid treatment device 30. The image 100 is captured by the imaging device 20, and includes a liquid surface area 110. In Fig. 3, the image 100 also includes a floating matter area 120, a scraping part area 140, and a tank wall area 130. However, in the present disclosure, the floating matter area, the scraping part area, and the wall area may not be included in the image relating to the liquid treatment device.

[0031] Fig. 4 is a diagram showing a configuration of the information processing device 10 according to an embodiment of the present disclosure. Each component of the information processing device 10 according to an embodiment of the present disclosure will be described in detail with reference to Fig. 4.

[0032] As shown in FIG. 4, the information processing device 10 includes a control unit 11, a storage unit 12, a communication unit 13, an input unit 14, and an output unit 15.

[0033] The control unit 11 includes at least one processor, at least one dedicated circuit, or a combination of these. The processor is a general-purpose processor such as a central processing unit (CPU) or a graphics processing unit (GPU), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, a field-programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The control unit 11 executes processes related to the operation of the information processing device 10 while controlling each part of the information processing device 10.

[0034] The storage unit 12 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory is, for example, a random access memory (RAM) or a read only memory (ROM). The RAM is, for example, a static random access memory (SRAM) or a dynamic random access memory (DRAM). The ROM is, for example, an electrically erasable programmable read only memory (EEPROM). The storage unit 12 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 12 stores data used in the operation of the information processing device 10 and data obtained by the operation of the information processing device 10.

[0035] The communication unit 13 includes at least one external communication interface. The communication interface may be either a wired communication interface or a wireless communication interface. In the case of wired communication, the communication interface is, for example, a LAN (Local Area Network) interface or a USB (Universal Serial Bus). In the case of wireless communication, the communication interface is, for example, an interface compatible with mobile communication standards such as LTE (Long Term Evolution), 4G (4th generation), or 5G (5th generation), or an interface compatible with short-range wireless communication such as Bluetooth (registered trademark). The communication unit 13 receives data used in the operation of the information processing device 10, and transmits data obtained by the operation of the information processing device 10.

[0036] The input unit 14 includes at least one input interface. The input interface is, for example, a physical key, a capacitive key, a pointing device, or a touch screen integrally provided with a display. The input interface may also be, for example, a microphone that accepts voice input, or a camera that accepts gesture input. The input unit 14 accepts an operation to input data used for the operation of the information processing device 10. The input unit 14 may be connected to the information processing device 10 as an external input device instead of being provided in the information processing device 10. As a connection method, any method such as USB (Universal Serial Bus), HDMI (registered trademark) (High-Definition Multimedia Interface), or Bluetooth (registered trademark) can be used.

[0037] The output unit 15 includes at least one output interface. The output interface is, for example, a display that outputs information as a video. The display is, for example, a liquid crystal display (LCD) or an organic EL (electro luminescence) display. The output unit 15 displays and outputs data obtained by the operation of the information processing device 10. The output unit 15 may be connected to the information processing device 10 as an external output device instead of being provided in the information processing device 10. As a connection method, any method such as USB, HDMI (registered trademark), or Bluetooth (registered trademark) can be used.

[0038] The functions of the information processing device 10 are realized by executing a program according to this embodiment on a processor equivalent to the information processing device 10. That is, the functions of the information processing device 10 are realized by software. The program causes a computer to execute the operations of the information processing device 10, thereby causing the computer to function as the information processing device 10. That is, the computer functions as the information processing device 10 by executing the operations of the information processing device 10 in accordance with the program.

[0039] In this embodiment, the program can be recorded on a computer-readable recording medium. The computer-readable recording medium includes a non-transitory computer-readable medium, such as a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory. The program is distributed, for example, by selling, transferring, or lending a portable recording medium such as a DVD (digital versatile disc) or a CD-ROM (compact disc read only memory) on which the program is recorded. The program may also be distributed by storing the program in the storage of an external server and transmitting the program from the external server to another computer. The program may also be provided as a program product.

[0040] A part or all of the functions of the information processing device 10 may be realized by a dedicated circuit corresponding to the control unit 11. In other words, a part or all of the functions of the information processing device 10 may be realized by hardware.

[0041] (Operation of information processing device) The operation of the information processing device 10 according to the present embodiment will be described with reference to FIG. FIG. 5 is a flowchart showing an example of processing executed by the information processing device 10 according to the present embodiment.

[0042] Step S110: The control unit 11 of the information processing device 10 acquires an image related to the liquid processing device 30. Specifically, the control unit 11 acquires an image 100 including the liquid surface area. Any method can be adopted for acquiring the image 100. For example, the control unit 11 may acquire the image 100 captured by the imaging device 20. The control unit 11 may receive the image 100 via the communication unit 13, or may receive it without via the communication unit 13. The memory unit 12 may store the image 100.

[0043] Step S120: The control unit 11 generates additional output information based on the image 100. The additional output information includes first information specifying a region 120 of floating matter that may be included in the image 100. The additional output information may also include second information related to the area of ​​the region 120 of floating matter. Furthermore, the additional output information may include at least one of third information specifying a region 130 of the wall surface of the tank 4 included in the image 100, and fourth information related to the area of ​​the wall surface region 130. The memory unit 12 may store the generated additional output information.

[0044] The method of generating the first information and the third information is not limited. However, for example, the first information and the third information may be generated by inference using a trained model. That is, the first information, or the first information and the third information, may be generated by inputting input data based on the image 100 into the trained model. By using the trained model, it is possible to classify the image into two or more regions (for example, a region 120 of floating objects, a region 130 of a wall surface, etc.) with higher accuracy. The trained model will be described later.

[0045] The input data to be input to the trained model may be image data. The image data may be the data of the image 100 itself, or may be data obtained by subjecting the image 100 to image processing.

[0046] The storage unit 12 stores the input data and the trained model. Alternatively, one or both of the input data and the trained model are stored in an external device other than the information processing device 10 and are received from the external device via the communication unit 13.

[0047] The second information may be generated based on the first information. The second information may be, for example, the area ratio of the suspended matter. The area ratio of the suspended matter may be determined by, for example, Equation (1). [Number 1] (Floating matter area ratio) = (Floating matter area) / (Reference area) (1) For example, when the floating matter region 120 is specified in pixel units, the area of ​​the floating matter region 120 may be the number of pixels of the floating matter region 120. The reference area may be determined arbitrarily. For example, the number of pixels of the image 100 may be the reference area. Alternatively, the area of ​​the liquid surface region 110 in the image 100 when it is assumed that the scraped-out region 140 and the floating matter region 120 do not exist in the image 100, specifically, the number of pixels of the liquid surface region 110 in the image 100, may be the reference area. In addition, the reference area may be the area of ​​the entire liquid surface when the entire liquid surface of the liquid 2 is complemented from the image 100, specifically, the number of pixels corresponding to the area.

[0048] The fourth information may be generated based on the third information. The fourth information may be, for example, a wall surface area ratio. The wall surface area ratio may be determined by, for example, Equation (2). [Number 2] (Wall area ratio) = (Wall area) / (Reference area) (2) For example, when the wall surface region 130 is specified in pixel units, the area of ​​the wall surface region 130 may be the number of pixels of the wall surface region 130. The reference area may be determined arbitrarily. The reference area in the above formula (1) and the reference area in the above formula (2) may be the same or different.

[0049] A specific example of the generation of the second information will be described with reference to Figures 6 and 7. Note that the method described below is also applicable to the generation of the fourth information.

[0050] In addition to the floating matter region 121, the image shown in FIG. 6 includes an overlapping region 122 in which the floating matter 3 floating on the liquid surface is hidden by the scraping unit 5. In this case, the second information may be information based on the area of ​​the overlapping region and the area of ​​the floating matter region, or may be information based on the area of ​​the floating matter region. That is, as the "area of ​​the floating matter region" in the above formula (1), the area of ​​the floating matter region 121 may be used, or the area of ​​the region 123 obtained by adding the floating matter region 121 and the overlapping region 122 may be used. For example, when the area of ​​the floating matter region 121 is 12.8% and the area of ​​the overlapping region 122 is 0.4% of the entire image 100, "12.8%" may be used as the second information, or "13.2%" obtained by the calculation of 12.8%+0.4% may be used as the second information.

[0051] In the image shown in FIG. 7, the floating matter 3 floating on the liquid surface is divided into a plurality of divided regions (division region 124 and division region 125) by the scraping unit 5. The image also includes an overlapping region 126 in which the floating matter 3 floating on the liquid surface is hidden by the scraping unit 5. In this case, the second information may be information based on the area of ​​the overlapping region and the area of ​​the floating matter region, or may be information based on the area of ​​each divided region. That is, as the "area of ​​the floating matter region" in the above formula (1), the area of ​​each of the divided region 124 and the divided region 125 may be used, or the area of ​​the region 129 obtained by adding up the divided region 124, the divided region 125, and the overlapping region 126 may be used. Also, the information may be information based on the area of ​​each divided region and the area of ​​the overlapping region apportioned among the plurality of divided regions. As the "area of ​​the floating matter region" in the above formula (1), the area of ​​the region obtained by adding up the divided region 124 and the region 127 and the area of ​​the region obtained by adding up the divided region 125 and the region 128 may be used. Here, the region 127 and the region 128 may be regions obtained by equally dividing the overlap region 126. For example, when the area of ​​the divided region 124 is 15.3% of the entire image 100, the area of ​​the divided region 125 is 3.7%, and the area of ​​the overlap region is 0.4%, "15.3%, 3.7%" may be used as the second information, or "19.4%" obtained by calculating 15.3%+3.7%+0.4% may be used as the second information. In addition, "15.5%, 3.9%" obtained by calculating 15.3%+0.4%÷2 and 3.7%+0.4%÷2 may be used as the second information.

[0052] The control unit 11 may estimate an overlapping region (for example, overlapping region 122 or overlapping region 126) or its area in which the floating matter 3 floating on the liquid surface is hidden by the scraping unit 5. That is, the control unit 11 may generate information that specifies the overlapping region or information based on the area of ​​the overlapping region. The storage unit 12 may store the information. The overlapping region may be estimated based on information that specifies the scraping-out region 140 that may be included in the image 100 and the first information. For example, when the floating matter region and the scraping-out region are in contact with each other, the overlapping region may be estimated so that the tangent line is included in the boundary line of the overlapping region. In FIG. 3, the floating matter region 120 and the scraping-out region 140 are not in contact with each other, so it may be estimated that there is no overlapping region. In FIG. 6, the floating matter region 121 and the scraping-out region 140 are in contact with each other. Therefore, the overlapping area 122 may be estimated by connecting the ends of the tangents (dotted and dashed lines in the figure) with a virtual line (straight line or curved line). In FIG. 7, the divided area 124 and the scraped-out area 140 are in contact with each other, and the divided area 125 and the scraped-out area 140 are in contact with each other. Therefore, for the two tangents (dotted and dashed lines in the figure), the overlapping area 126 may be estimated by connecting the ends of one tangent and the other tangent with a virtual line (straight line or curved line). The control unit 11 may generate information that specifies the scraped-out area 140 that may be included in the image 100 based on the image 100. For example, the information may be generated by inputting input data based on the image 100 to the trained model.

[0053] Step S130: The control unit 11 of the information processing device 10 outputs a video obtained by performing image processing on the image 100 based on the additional output information. The video may be output by controlling the output unit 15. The control unit 11 may generate the video and control the output unit 15 to output the generated video. The storage unit 12 may store the generated video.

[0054] By displaying the first information and preferably also the second information as images, an operator can monitor whether the floating matter is being properly scraped out in the liquid treatment device, change the operating conditions as necessary, and can also determine whether there is an abnormality in the liquid treatment device.

[0055] By displaying the third information or the fourth information by image, the liquid level in the tank can be grasped. Since it becomes difficult to scrape out the floating objects when the liquid level drops, the operator can use the grasped liquid level to monitor whether the liquid level is sufficient to properly scrape out the floating objects, and change the operating conditions as necessary. In addition, the liquid level can be used to determine abnormalities related to the liquid treatment device. According to this method, there is no need to install a separate gauge to grasp the liquid level, so the installation cost of the gauge can be reduced, and the effort of checking the gauge separately can be eliminated.

[0056] The video will be described in detail with reference to the drawings. Fig. 8 shows a schematic diagram of a video 200 to be output. The video 200 is an example of a video generated from the image 100 shown in Fig. 3, and image processing has been performed on the image 100. The image processing includes a first image processing 210 based on the first information, a second image processing 220 based on the second information, a third image processing 230 based on the third information, and a fourth image processing 240 based on the fourth information.

[0057] As an example of the first image processing 210, as shown in FIG. 8, a mask process is performed on the floating object region 120 with a predetermined color and transparency. Similarly, as an example of the third image processing 230, a mask process is performed on the wall surface region 130 with a predetermined color and transparency. Here, by using different image processing for the first image processing 210 and the third image processing 230, the worker can easily visually distinguish between the floating object region 120 and the wall surface region 130. For example, it is preferable to perform mask processing using different colors for the floating object region 120 and the wall surface region 130.

[0058] The manner in which the second information is displayed is not limited, but it is preferable to display it by superimposing it on the floating matter region 120. As an example of the second image processing 220, as shown in FIG. 8, the second information may be displayed by superimposing it on the floating matter region 120 in numbers and units expressing a ratio. The second image processing is not limited to this, and the second information may be displayed by numbers, letters, symbols, etc. by superimposing it on the floating matter region 120. Furthermore, the first image processing may also serve as the second image processing. For example, when a mask process is performed (superimposed) on the floating matter region 120 as the first image processing, the second information may be displayed by color density, color scale, etc.

[0059] The manner in which the fourth information is displayed is not limited, but it is preferable to display it by superimposing it on the wall surface region 130. As an example of the fourth image processing 240, as shown in FIG. 8, the fourth information may be displayed by superimposing it on the wall surface region 130 in numbers and units representing a ratio. The fourth image processing is not limited to this, and the fourth information may be displayed by numbers, letters, symbols, etc. by superimposing it on the wall surface region 130. Furthermore, the third image processing may also serve as the fourth image processing. For example, when a mask process is performed (superimposed) on the wall surface region 130 as the third image processing, the fourth information may be displayed by color density, a color scale, etc.

[0060] (Generating a trained model) Next, a generation technique of a trained model that can be used to generate additional output information according to the present disclosure will be described with reference to the drawings. The trained model receives an image including a liquid surface area as input, and outputs output data including information identifying a floating matter area that may be included in the image. The output data preferably includes information identifying a tank wall area that may be included in the image. The trained model can be generated by machine learning using an image obtained by performing preprocessing to change the brightness of a captured image including a liquid surface area as a teacher image.

[0061] FIG. 9 is a flowchart showing an example of processing executed by the information processing device 10 according to this embodiment to generate a trained model.

[0062] Step S210: The control unit 11 acquires an image that has been subjected to pre-processing to change the brightness of the captured image including the liquid surface area. For example, the control unit 11 may acquire the image from a higher-level computer or the like via the communication unit 13. The control unit 11 may also acquire the captured image including the liquid surface area from an imaging device. In this case, the control unit 11 may perform pre-processing to change the brightness of the acquired captured image. The control unit 11 may refer to a database to perform the pre-processing. The storage unit 12 may store the database. The control unit 11 may also refer to an input based on a user's operation via the input unit 14 to perform the pre-processing.

[0063] Examples of brightness that may be changed in the pre-processing include brightness in the HSV color space, luminance in the HLS color space, and luminance in the YUV color space. The pre-processing may be a process that sets the average and variance of the brightness distribution of the captured images to predetermined values. The pre-processing may also be a process that makes the brightness uniform among multiple captured images.

[0064] Step S220: The control unit 11 acquires information for identifying an area of ​​floating matter that may be included in the acquired image. For example, the control unit 11 may acquire information from a higher-level computer or the like via the communication unit 13. The control unit 11 may also acquire information based on a user's operation via the input unit 14.

[0065] In order to further use the trained model to generate third information, the control unit 11 may further acquire information that identifies an area of ​​the tank wall that may be included in the acquired image.

[0066] Step S230: The control unit 11 generates a trained model by machine learning using the acquired image as a teacher image. The trained model is generated by associating the teacher image with the information acquired in step S220.

[0067] As the trained model, for example, a trained model for segmentation can be used. Examples of the trained model include trained models generated by machine learning algorithms such as Mask R-CNN, Unet, and UperNet.

[0068] Although the present disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art can easily make various modifications and corrections based on the present disclosure. Therefore, it should be noted that these modifications and corrections are included in the scope of the present disclosure. For example, the functions included in each means or step can be rearranged so as not to be logically inconsistent, and multiple means or steps can be combined or divided into one.

[0069] 5 and 9 show examples in which steps are processed in series, but the present invention is not limited to this example and may be modified in such a way that each step is processed in parallel.

[0070] Furthermore, for example, in the above-described embodiment, an embodiment is also possible in which the configuration and operation of the information processing device 10 are distributed among a plurality of computers capable of communicating with each other. [Explanation of symbols]

[0071] 1. Information Processing Systems 2 liquid 3. Floating Objects 4. Tank 5 Scraping section 10. Information processing device 11 Control section 12 Storage section 13. Communications Department 14 Input section 15 Output section 20 Imaging device 30 Liquid treatment equipment 100 images 110 Liquid Surface Area 120 Floating Object Area 121 Floating Area 122, 126 overlapping area 123, 129 area 124, 125 Divided Areas 127, 128 area 130 Wall Area 140 Scraping area 200 videos 210 First Image Processing 220 Second Image Processing 230 Third Image Processing 240 The Fourth Image Processing

Claims

1. On the computer, acquiring an image relating to the liquid treatment device; generating additional output information based on the image; and outputting a video obtained by performing image processing on the image based on the additional output information, The liquid treatment device includes a tank for storing a liquid, and a scraping unit for scraping out floating matter floating on the surface of the liquid to the outside of the tank, the image includes an area of ​​the liquid surface and an area of ​​a wall surface of the tank; The additional output information includes first information identifying a region of the floating object that may be included in the image; and at least one of third information that identifies a region of the wall surface of the tank included in the image and fourth information regarding an area of ​​the region of the wall surface of the tank included in the image.

2. the additional output information includes second information regarding an area of ​​the floating object region that may be included in the image; The program according to claim 1 , wherein the image is an image in which the second information is superimposed on an area of ​​the floating object.

3. The second information is The program according to claim 2 , wherein, when the image includes an overlapping region in which the floating matter is hidden by the scraping unit, the information is based on an area of ​​the overlapping region and an area of ​​the floating matter region.

4. The second information is The program according to claim 2 , wherein when the floating matter is divided into a plurality of divided regions by the scraping unit in the image, the information is based on an area of ​​each of the divided regions.

5. The second information is The program of claim 2, wherein when the image includes an overlapping area in which the floating matter is hidden by the scraping section, and when the floating matter is divided into a plurality of divided areas in the image by the scraping section, the information is based on the area of ​​each of the divided areas and the area of ​​the overlapping area prorated among the plurality of divided areas.

6. the additional output information includes the third information, The image processing includes: a first image processing for superimposing the first information on the image and displaying the first information; and a third image processing for superimposing the third information on the image and displaying the third information, The program according to any one of claims 1 to 5, wherein the first image processing and the third image processing are different image processing.

7. The program according to any one of claims 1 to 5, wherein in generating the additional output information, the first information is generated by inference using a trained model.

8. The program according to claim 6 , wherein in generating the additional output information, the first information and the third information are generated by inference using a trained model.

9. The trained model is The program according to claim 7 , which is a trained model generated by machine learning using an image that has been pre-processed to change the brightness of an image including the liquid surface area as a teacher image.

10. The trained model is The program according to claim 8 , which is a trained model generated by machine learning using an image that has been pre-processed to change the brightness of an image including the liquid surface region as a teacher image.

11. The additional output information includes the third information and the fourth information, the fourth information is generated based on the third information, The program according to claim 1 , wherein the fourth information is an area ratio of a wall surface of the tank.

12. A method for generating a trained model, which is executed by an information processing device, inputting an image related to a liquid treatment device and outputting output data related to the image, The liquid treatment device includes a tank for storing a liquid, and a scraping unit for scraping out floating matter floating on the surface of the liquid to the outside of the tank, the image includes an area of ​​the liquid surface and an area of ​​a wall surface of the tank; the output data includes information for identifying an area of ​​the floating matter that may be included in the image and information for identifying an area of ​​the wall surface of the tank that is included in the image; The method for generating the trained model includes: acquiring an image obtained by performing pre-processing to change brightness of a captured image including the liquid surface area; A method for generating a trained model, comprising: generating the trained model through machine learning using the acquired image as a teacher image.

Images