Information processing device

Abstract

To provide a technique which is less likely to be influenced by meteorological conditions and can determine an adhesion state of dust and refuse to a screen even under relatively bad communication conditions.SOLUTION: An information processing method includes: step S1 of acquiring a still image 200 from a camera for shooting a water intake installed with a screen; step S2 of detecting a region R1 reflecting a water surface, one or more regions R2 reflecting dust and refuse, and a region R3 reflecting a screen in the still image 200; step S3 of identifying end points PTa and PTb in a boundary between a composite region R4 of a region R1 and one or more regions R2 and a region R3; step S4 of setting a screen near-field region RA including a line segment connecting the end points PTa and PTb as a part of a contour and becoming convex on the composite region R4 side; and step S5 of determining the adhesion state of dust and refuse to a screen on the basis of a region R2 located in the screen near-field region RA among one or more regions R2.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a monitoring system, an information processing method, and a program. [Background technology]

[0002] There are many deciduous broadleaf trees around the waterway of a small hydroelectric power plant, and in the fall, large amounts of leaves and other debris flow into the waterway. If the debris adheres to the intake screen installed in the waterway, it can cause the waterway to become clogged. Therefore, it is preferable for workers to remove the debris as soon as it becomes clogged.

[0003] Japanese Patent Application Laid-Open Publication No. 2022-17900 (Patent Document 1) discloses an information processing device that applies dense optical flow to video images taken by a camera capturing an intake port to distinguish between flowing and stagnant debris. The information processing device disclosed in Patent Document 1 determines the state of debris adhesion to the screen based on the stagnant debris and the position of the screen. From this determination result, workers can determine when a blockage in the waterway has occurred. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2022-17900 Summary of the Invention [Problem to be solved by the invention]

[0005] In the method of Patent Document 1, a dense optical flow is applied to a moving image. However, due to the influence of weather conditions (e.g., the angle of incidence of sunlight, brightness, etc.), the dense optical flow may not be able to accurately distinguish between flowing and stagnant debris.

[0006] Furthermore, when acquiring video from a camera using a wireless method, depending on the communication conditions at the location where the camera is installed, it may not be possible to acquire the video from the camera, or it may not be possible to accurately determine the degree of dust adhesion to the screen based on the video acquired from the camera. Generally, when transmitting video, it is necessary to transmit the necessary data (data indicating the difference from the previous frame) within a certain period of time. If communication conditions are poor and the necessary data cannot be transmitted, a process of thinning out the data is performed. As a result, data with some missing data is transmitted, causing block noise in the video. If a video with block noise is used, it is not possible to accurately determine the degree of dust adhesion to the screen.

[0007] The present disclosure has been made in consideration of the above-mentioned problems, and its purpose is to provide a technology that is less affected by weather conditions and can determine the degree of dust adhesion to a screen even in an environment with relatively poor communication conditions. [Means for solving the problem]

[0008] An information processing device according to an aspect of the present disclosure includes an acquisition unit, an area detection unit, an identification unit, a setting unit, and a determination unit. The acquisition unit acquires a still image from a camera that captures a location of a waterway having an intake with a screen installed therein. The area detection unit detects, in the still image, a first area that captures the water surface, one or more second areas that capture dust particles, and a third area that captures the screen. The identification unit identifies a first end point and a second end point of a boundary between a combined area of ​​the first area and the one or more second areas and the third area. The setting unit sets a near-screen area that has a first line segment connecting the first end point and the second end point as part of its outline and is convex toward the combined area. The determination unit determines the degree of dust adhesion to the screen based on a second area among the one or more second areas that is located within the near-screen area.

[0009] A monitoring system according to another aspect of the present disclosure includes a camera, a server device, and a terminal device. The camera captures an image of a location of a waterway having an intake port with a screen installed thereon, and transmits the captured still image to the server device. The server device receives the still image from the camera. The server device detects, in the still image, a first area showing the water surface, one or more second areas showing debris, and a third area showing the screen. The server device identifies a first end point and a second end point of the boundary between a combined area of ​​the first area and the one or more second areas and the third area. The server device sets a near-screen area that has a line segment connecting the first end point and the second end point as part of its outline and is convex toward the combined area. The server device determines the degree of debris adhesion to the screen based on the second area among the one or more second areas that is located within the near-screen area, and transmits the determination result to the terminal device. The terminal device receives the determination result from the server device and displays the determination result.

[0010] An information processing method according to yet another aspect of the present disclosure includes first to fifth steps. The first step is a step in which one or more processors acquire a still image from a camera that captures a location of a waterway having an intake with a screen installed therein. The second step is a step in which one or more processors detect, in the still image, a first area in which the water surface is captured, one or more second areas in which debris is captured, and a third area in which the screen is captured. The third step is a step in which one or more processors identify a first end point and a second end point of a boundary between a combined area of ​​the first area and the one or more second areas and the third area. The fourth step is a step in which one or more processors set a screen near-area that has a line segment connecting the first end point and the second end point as part of its outline and is convex toward the combined area. The fifth step is a step in which one or more processors determine a state of debris adhesion to the screen based on a second area of ​​the one or more second areas that is located within the screen near-area.

[0011] A program according to yet another aspect of the present disclosure causes a processor of an information processing device to execute the first to fifth steps described above. [Effects of the Invention]

[0012] According to the technology of the present disclosure, it is possible to determine the degree of dust adhesion to the screen even in an environment that is less susceptible to the influence of weather conditions and where communication conditions are relatively poor. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating a waterway. [Figure 2] FIG. 1 is a diagram illustrating a system configuration of a monitoring system. [Figure 3] FIG. 3 is a schematic diagram showing the state of garbage at the water intake shown in FIGS. 1 and 2. [Figure 4] FIG. 10 is a diagram showing an example of a still image captured by a camera. [Figure 5] FIG. 2 is a diagram illustrating a hardware configuration of a server device. [Figure 6] FIG. 2 illustrates an example of a functional configuration of a server device. [Figure 7] 10 is a flowchart showing an outline of the processing flow of the server device in the operation phase. [Figure 8] FIG. 2 is a diagram illustrating registration information in the first embodiment. [Figure 9] 10 is a flowchart showing the first half of the processing flow in the first embodiment. [Figure 10] 10 is a flowchart showing the second half of the processing flow in the first embodiment. [Figure 11] FIG. 10 is a diagram illustrating steps S12 to S17 shown in FIG. [Figure 12] 11 is a diagram illustrating steps S18, S21, and S24 shown in FIG. 10. FIG. [Figure 13] FIG. 11 is a diagram illustrating steps S20 and S23 shown in FIG. [Figure 14] FIG. 11 is a diagram illustrating the processing content of step S29 shown in FIG. [Figure 15] FIG. 10 is a diagram showing an example of a stagnant dust area extracted from a still image. [Figure 16] FIG. 10 is a diagram for explaining the length Ls. [Figure 17] FIG. 2 is a diagram showing an example of a near-screen region set in the present embodiment. [Figure 18] 10 is a flowchart showing the first half of the processing flow in the second embodiment. [Figure 19] 10 is a flowchart showing the second half of the processing flow in the second embodiment. [Figure 20] FIG. 10 is a diagram illustrating the processing in step S44. [Figure 21] FIG. 10 is a diagram showing the relationship between a range considered as a scatter plot and a separating line. [Figure 22] FIG. 10 is a diagram illustrating the processing of steps S45 to S49. [Figure 23] 10 is a flowchart showing the first half of the processing flow according to the third embodiment. [Figure 24] 11 is a flowchart showing the second half of the processing flow in the third embodiment. [Figure 25] FIG. 10 is a diagram illustrating steps S61 to S67. [Figure 26] 13 is a flowchart showing the first half of the processing flow in the fourth embodiment. [Figure 27] 13 is a flowchart showing the second half of the processing flow in the fourth embodiment. [Figure 28] FIG. 10 is a diagram illustrating steps S71 to S74. DETAILED DESCRIPTION OF THE INVENTION

[0014] A monitoring system according to an embodiment of the present invention will be described below with reference to the drawings. In the following description, the same components are denoted by the same reference numerals. The names and functions of these components are also the same. Therefore, detailed descriptions thereof will not be repeated.

[0015] In the following description, fallen leaves will be taken as an example of floating debris in waterways (floating debris), although debris is not limited to fallen leaves.

[0016] <Waterway> FIG. 1 is a diagram illustrating a waterway. In the example shown in FIG. 1, waterway 50 is a waterway branching off from river 40. Water flowing through waterway 50 is supplied to small hydroelectric power plant 100. In small hydroelectric power plant 100, this water is used as power generation water for the power generation equipment. The water is then released into river 40. Note that typically, power of 10,000 kW or less is referred to as "small hydropower."

[0017] The catchment area of ​​the river 40 is usually covered with deciduous broad-leaved trees. Therefore, in the fall, a large amount of fallen leaves flows into the river 40. The fallen leaves (i.e., garbage) also flow into the waterway 50 for the small hydroelectric power plant 100. The garbage adheres to the screen 60 of the intake 70 and causes clogging, resulting in a decrease in power generation (also known as "overflow"). The screen 60 is typically a metal fence.

[0018] <Monitoring system> Fig. 2 is a diagram illustrating the system configuration of a monitoring system. As shown in Fig. 2, monitoring system 1 includes server device 10, terminal device 20, and camera 30. Camera 30 and server device 10 are connected to each other so as to be able to communicate with each other via a wired or wireless system. The following describes a case where a wireless system is included in part of the communication line between camera 30 and server device 10. Terminal device 20 is connected to server device 10 so as to be able to communicate with each other via a network.

[0019] The terminal device 20 is, for example, a desktop PC (Personal Computer). However, the terminal device 20 is not limited to a desktop PC, and may be, for example, a laptop PC, a tablet terminal, or a smartphone.

[0020] Camera 30 is installed near waterway 50 to photograph waterway 50. Specifically, camera 30 is installed near water intake 70 of waterway 50. A screen 60 is provided at water intake 70 to prevent large foreign objects of a certain shape (driftwood, etc.) from flowing into the power generation facility. Camera 30 is installed from the upstream side of waterway 50 so as to photograph the water surface and screen 60.

[0021] Still images captured by camera 30 are sent to server device 10. Camera 30 transmits still image data to server device 10 using, for example, TCP (Transmission Control Protocol) / IP (Internet Protocol). With TCP / IP, even if communication conditions between camera 30 and server device 10 are relatively poor, retransmissions are repeated until all still image data is transmitted successfully. Furthermore, with TCP / IP, the data sender and receiver mutually confirm whether the data exchange was successful. Therefore, even if communication conditions between camera 30 and server device 10 are relatively poor, still image data is transmitted from camera 30 to server device 10 without any degradation in image quality.

[0022] The server device 10 is an information processing device that executes processing using a still image captured by the camera 30. Specifically, the server device 10 determines the state of dust adhesion on the screen 60 based on an area in the still image in which dust remaining near the screen 60 is captured. The server device 10 notifies the terminal device 20 of the determination result. For example, the server device 10 may send an email including the determination result to the terminal device 20, or may send an email containing an address for accessing the determination result to the terminal device 20.

[0023] The terminal device 20 receives the determination result from the server device 10 and displays the determination result. Based on the determination result displayed on the terminal device 20, workers at the small hydroelectric power plant 100 can remove debris when it becomes clogged in the water intake 70. Therefore, by monitoring water intakes 70 in locations that are difficult for workers to reach, the convenience of the monitoring system 1 is increased. Note that the waterway 50 is not limited to one used for a power plant. The waterway 50 may be a waterway for a large-scale power plant or a waterway for purposes other than power generation, such as an agricultural waterway.

[0024] <Dust adhesion condition> Figure 3 is a schematic diagram showing the state of debris at the water intake shown in Figures 1 and 2. As shown in Figure 3, debris 80 that has flowed into waterway 50 spreads from the bottom of the water to the surface and floats there. In this example, "floating" includes floating in a stagnant state and floating in a flowing state. Some debris 80 floats on the surface of the water, while other debris 80 floats below the surface (underwater).

[0025] Some of the debris 80 floating in the waterway is carried away in the direction of the water flow (the direction of arrow A1). Some of this debris 80 adheres to the screen 60. As the water flows through the screen 60, the debris 80 adheres to the screen 60, avoiding areas where debris 80 is already attached. In the example shown in the figure, the debris 80 is carried away in the direction of arrow A3, not in the direction of arrow A2.

[0026] 4 is a diagram showing an example of a still image captured by a camera. The still image output from camera 30 is represented by the coordinate system of camera 30. The coordinate system of camera 30 is an XY coordinate system with the upper left corner of the field of view of camera 30 as the origin O, the horizontal direction of the field of view as the X axis, and the vertical direction of the field of view as the Y axis. Therefore, each pixel of the still image is represented by XY coordinates.

[0027] 4A shows a still image 201 when there is little dust 80 adhering to the screen 60. FIG. 4B shows a still image 202 when the entire screen 60 is covered with dust 80.

[0028] 4, the dust particles 80 do not overlap with areas where dust particles 80 are already attached, but instead attach to areas where water flows due to gaps. As a result, the dust particles 80 attach to the screen 60 in a thin layer and evenly from the water surface to the bottom.

[0029] <Hardware configuration> 5 is a diagram illustrating the hardware configuration of the server device. As shown in FIG. 5, the server device 10 includes, as its main components, a processor 151 that executes a program, a ROM (Read Only Memory) 152 that stores data in a non-volatile manner, a RAM (Random Access Memory) 153 that volatilely stores data generated by the execution of the program by the processor 151 or input data, an HDD (Hard Disk Drive) 154 that stores data in a non-volatile manner, a communication IF (Interface) 155, an input device 156, a power supply circuit 157, and a display 158. The components are interconnected by a data bus. The communication IF 155 is an interface for communicating with other devices.

[0030] The processing in the server device 10 is realized by software (programs) executed by each piece of hardware and the processor 151. Such software may be pre-stored in the HDD 154. Alternatively, the software may be stored in other storage media and distributed as a program product. Alternatively, the software may be provided as a downloadable program product by an information provider connected to the Internet. Such software is read from the storage media by a reading device or downloaded via the communication IF 155 or the like, and then temporarily stored in the HDD 154. The software is read from the HDD 154 by the processor 151 and stored in the RAM 153 in the form of an executable program. The processor 151 executes the program.

[0031] The components constituting the server device 10 shown in the figure are common. Therefore, it can be said that the essential parts of the present invention are the RAM 153, the HDD 154, the software stored in the storage medium, or the software downloadable via the network. Note that the operation of each piece of hardware in the server device 10 is well known, so detailed description will not be repeated.

[0032] Since the terminal device 20 has the same configuration as the server device 10, the hardware configuration of the terminal device 20 will not be described repeatedly.

[0033] <Server device functional configuration> Fig. 6 is a diagram illustrating an example of the functional configuration of a server device. As shown in Fig. 6, the server device 10 includes an acquisition unit 110, an area detection unit 111, an identification unit 112, a setting unit 113, a determination unit 114, a notification unit 115, a registration unit 116, and a storage unit 120. The area detection unit 111, the identification unit 112, the setting unit 113, the determination unit 114, and the registration unit 116 are realized by a processor 151 shown in Fig. 5 executing a program. The acquisition unit 110 and the notification unit 115 are realized by a communication IF 155 shown in Fig. 5 and the processor 151 executing a program. The storage unit 120 is realized by a RAM 153 or an HDD 154 shown in Fig. 5.

[0034] The processing of the server device 10 may include processing in a preparation phase and processing in an operation phase. The processing in the preparation phase is executed after the installation of each device constituting the monitoring system 1 is completed and before the start of processing in the operation phase. The processing in the operation phase is executed after the processing in the preparation phase.

[0035] The acquisition unit 110 operates in both the advance preparation phase and the operation phase. The area detection unit 111, the identification unit 112, the setting unit 113, the determination unit 114, and the notification unit 115 operate in the operation phase. The registration unit 116 operates in the advance preparation phase.

[0036] In the preparation phase, the acquisition unit 110 acquires (receives) a still image for preparation (hereinafter referred to as a "preparation image") from the camera 30 in response to user input received by the input device 156 (see Figure 5), and outputs the acquired preparation image to the registration unit 116.

[0037] In the operation phase, the acquisition unit 110 acquires (receives) still images from the camera 30 periodically (for example, every 5 or 10 minutes), and outputs the acquired still images to the area detection unit 111.

[0038] The registration unit 116 displays the preparation image received from the acquisition unit 110 on the display 158 (see FIG. 5 ), and generates registration information that can be used for processing by the identification unit 112 in accordance with a user input to the input device 156. The registration unit 116 stores the generated registration information in the storage unit 120.

[0039] The region detection unit 111 detects a first region (hereinafter referred to as "region R1") showing the water surface, one or more second regions (hereinafter referred to as "region R2") showing dust particles 80, and a third region (hereinafter referred to as "region R3") showing the screen 60 in the still image received from the acquisition unit 110. Specifically, the region detection unit 111 uses the trained model 420 to perform semantic segmentation, which labels each pixel in the still image as a water surface, dust particle, screen, or other. The region detection unit 111 detects a pixel block labeled as a water surface as region R1. The region detection unit 111 detects a pixel block labeled as a dust particle as region R2. The region detection unit 111 detects a pixel block labeled as a screen 60 as region R3.

[0040] The trained model 420 is typically composed of a deep neural network structure and parameters. The trained model 420 is typically generated using backpropagation. The trained model 420 may be generated by the server device 10 or another device.

[0041] The trained model 420 is generated in advance using training data. The training data includes a plurality of still image data and a label indicating, for each still image data, whether it is a water surface, dust 80, screen 60, or other, on a pixel-by-pixel basis. The trained model 420 is generated by training the labeled still image data.

[0042] The identification unit 112 identifies a first endpoint (hereinafter referred to as "end point PTa") and a second endpoint (hereinafter referred to as "end point PTb") of the boundary between a combined area R4 of area R1 and one or more areas R2 and area R3. The identification unit 112 stores, in the storage unit 120, endpoint information indicating the coordinates of the identified endpoint PTa (hereinafter referred to as "first endpoint coordinates") and the coordinates of the identified endpoint PTb (hereinafter referred to as "second endpoint coordinates"). The identification unit 112 updates the endpoint information each time it identifies endpoint PTa, PTb.

[0043] As will be described later, the identification unit 112 may identify the endpoints PTa and PTb using the registration information stored in the storage unit 120, or may identify the endpoints PTa and PTb without using the registration information. When the identification unit 112 identifies the endpoints PTa and PTb without using the registration information, the registration unit 116 is omitted.

[0044] The setting unit 113 sets a near-screen area RA that has a line segment connecting the end points PTa and PTb as part of its contour and that is convex toward the synthesis area R4. The shape of the near-screen area RA is not particularly limited and may be a semi-ellipse, a rectangle, or the like. However, as will be described later, it is preferable that the near-screen area RA be a semi-ellipse, which can be easily drawn using an open-source computer vision library.

[0045] The determining unit 114 determines the adhesion state of the dust 80 on the screen 60 based on the area R2 located within the screen vicinity area RA among the one or more areas R2.

[0046] The notification unit 115 notifies the terminal device 20 of the determination result by the determination unit 114. As a result, the determination result is displayed on the terminal device 20. Alternatively, the notification unit 115 may display the determination result on the display 158.

[0047] <Outline of the server device processing flow> FIG. 7 is a flowchart showing an outline of the processing flow of the server device in the operation phase.

[0048] As shown in FIG. 7, first, the processor 151 operating as the acquisition unit 110 acquires the still image 200 from the camera 30 (step S1).

[0049] Next, processor 151 operating as region detection unit 111 detects, in still image 200, region R1 where the water surface is reflected, one or more regions R2 where dust 80 is reflected, and region R3 where screen 60 is reflected (step S2).

[0050] Next, the processor 151 operating as the identification unit 112 identifies the endpoints PTa and PTb of the boundary between the combined area R4 of the area R1 and one or more areas R2 and the area R3 (step S3).

[0051] Next, the processor 151 operating as the setting unit 113 sets a near-screen area RA that has a line segment connecting the end points PTa and PTb as part of its contour and that is convex toward the synthesis area R4 (step S4).

[0052] Next, the processor 151 operating as the determination unit 114 determines the adhesion state of the dust 80 to the screen 60 based on the area R2 located within the screen vicinity area RA among the one or more areas R2 (step S5). Finally, the processor 151 operating as the notification unit 115 notifies the terminal device 20 of the determination result (step S6).

[0053] <Specific Examples of Steps S1 to S6> A specific example of steps S1 to S6 shown in FIG. 7 will be described in detail below.

[0054] (First Example) In the first embodiment, in the preparation phase, the registration unit 116 stores registration information indicating the first to fourth reference coordinates in the storage unit 120 in accordance with a user input for the preparation image acquired from the camera 30.

[0055] 8 is a diagram for explaining registration information in Example 1. The registration unit 116 displays the preparation image 250 received from the acquisition unit 110 on the display 158, and receives designation of four reference pixels PX1 to PX4.

[0056] The reference pixel PX1 is the pixel that appears on the left edge of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50 when the water surface height is at a first level (e.g., low level). The registration unit 116 registers the coordinates of the reference pixel PX1 as the first reference coordinates.

[0057] The reference pixel PX2 is a pixel that appears on the right edge of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50 when the water surface height is at a first level (e.g., low level). The registration unit 116 registers the coordinates of the reference pixel PX2 as second reference coordinates.

[0058] The reference pixel PX3 is the pixel that appears at the left end of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50 when the water surface height is at a second level (e.g., high level) different from the first level. The registration unit 116 registers the coordinates of the reference pixel PX3 as third reference coordinates.

[0059] The reference pixel PX4 is the pixel that appears on the right edge of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50 when the water surface height is at the second level (e.g., high level). The registration unit 116 registers the coordinates of the reference pixel PX4 as the fourth reference coordinates.

[0060] The registration unit 116 may prompt the user to select a pre-preparation image when the water surface height is at the first level from among the multiple pre-preparation images received from the acquisition unit 110 in the pre-preparation phase, and display the selected pre-preparation image on the display 158. The registration unit 116 may then accept the selection of the reference pixel PX1 and the reference pixel PX2 in the displayed pre-preparation image.

[0061] Similarly, the registration unit 116 may prompt the user to select from the plurality of preparation images a preparation image for when the water surface height is at the second level, and display the selected preparation image on the display 158. Then, the registration unit 116 may accept the designation of the reference pixel PX3 and the reference pixel PX4 in the displayed preparation image.

[0062] Fig. 9 is a flowchart showing the first half of the processing flow of Example 1. Fig. 10 is a flowchart showing the second half of the processing flow of Example 1. In Figs. 9 and 10, step S12 corresponds to step S1 in Fig. 7, step S13 corresponds to step S2 in Fig. 7, steps S11 and S14 to S28 correspond to step S3 in Fig. 7, step S29 corresponds to step S4 in Fig. 7, and step S30 corresponds to steps S5 and S6 in Fig. 7.

[0063] First, the processor 151 operating as the identification unit 112 reads the first to fourth reference coordinates registered in advance (step S11). After step S11, steps S12 to S17 are executed.

[0064] Steps S12 to S17 will be described in detail with reference to Fig. 11. Fig. 11 is a diagram for explaining steps S12 to S17 shown in Fig. 9.

[0065] In step S12, the processor 151 operating as the acquisition unit 110 acquires a still image 200 from the camera 30 (see (a) of FIG. 11).

[0066] Next, in step S13, the processor 151 operating as the area detection unit 111 uses the trained model 420 to detect an area R1 reflecting the water surface, one or more areas R2 reflecting dust, and an area R3 reflecting the screen 60 from the still image 200 (see (b) of Figure 11).

[0067] Next, in step S14, processor 151 operating as identification unit 112 separates regions R1 and R2 from the remaining regions and generates combined region R4 by combining regions R1 and R2 (see (c) of FIG. 11). After step S14, the process proceeds to step S15.

[0068] In step S15, the processor 151 expands the synthesis region R4 using, for example, a 5×5 kernel (see FIG. 11(d)).

[0069] Furthermore, step S16 is executed in parallel with steps S14 and S15. In step S16, processor 151 separates region R3 from the remaining regions (see (e) of FIG. 11), and expands region R3 using, for example, a 5×5 kernel (see (f) of FIG. 11).

[0070] After steps S15 and S16, the process proceeds to step S17. In step S17, processor 151 separates a fourth region where expanded combined region R4 and expanded region R3 overlap (hereinafter referred to as "boundary region R5") from the other regions (see FIG. 11(g)). That is, processor 151 extracts boundary region R5 by performing an AND operation on expanded combined region R4 and expanded region R3. After step S17, steps S18 to S28 are executed.

[0071] Details of steps S18 to S28 will be described with reference to Fig. 12 and Fig. 13. Fig. 12 is a diagram illustrating steps S18, S21, and S24 shown in Fig. 10. Fig. 13 is a diagram illustrating steps S20 and S23 shown in Fig. 10.

[0072] In step S18, processor 151 draws a first edge line (hereinafter referred to as "edge line La") connecting the point of the first reference coordinate (hereinafter referred to as "reference point PT1") and the point of the third reference coordinate (hereinafter referred to as "reference point PT3") on the image showing boundary region R5 (see FIG. 12). Furthermore, processor 151 draws a second edge line (hereinafter referred to as "edge line Lb") connecting the point of the second reference coordinate (hereinafter referred to as "reference point PT2") and the point of the fourth reference coordinate (hereinafter referred to as "reference point PT4") on the image showing boundary region R5.

[0073] In the next step S19, the processor 151 determines whether or not the edge line La intersects with the boundary region R5.

[0074] If the end line La does not intersect with the boundary region R5 (NO in step S19), the processor 151 identifies the point at the first end point coordinate indicated by the end point information (i.e., the coordinate of the end point PTa saved in step S28 of the flow for the previous still image) as the end point PTa (step S20).

[0075] 13, the still image 200 was captured at night, so the shape of the area R3 detected from the still image 200 is smaller than the original shape of the screen 60. Therefore, the boundary area R5 where the expanded combined area R4 and the expanded area R3 overlap does not intersect with the edge line La. In such a case, the point of the first edge point coordinates indicated by the edge point information is identified as the edge point PTa.

[0076] Immediately after the start of the operation phase, the end point information indicates null. In this case, in step S20, the processor 151 specifies the point indicated by the first reference coordinates (reference point PT1) as the end point PTa.

[0077] If the edge line La intersects with the boundary region R5 (YES in step S19), in step S21, the processor 151 identifies the intersection of the boundary region R5 and the edge line La, i.e., a point (pixel) on the edge line La in the boundary region R5, as the edge point PTa (see FIG. 12). Note that the boundary region R5 can be a linear region having a width of multiple pixels. In this case, there are multiple points (pixels) on the edge line La in the boundary region R5. If there are multiple points on the edge line La in the boundary region R5, the processor 151 may identify one of the multiple points (for example, the point with the largest X coordinate and Y coordinate) as the edge point PTa, or may identify the center of gravity of the multiple points as the edge point PTa.

[0078] As described above, in the preparation image, the first and third reference coordinates are the coordinates of the left end of the intersection line between the water surface and the screen 60 when the water surface height is at the first and second levels, respectively, when the screen 60 is viewed from the upstream side of the waterway 50. Therefore, the endpoint PTa, which is the intersection point between the endpoint line La connecting the reference points PT1 and PT3 and the boundary region R5, corresponds to the left end point of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50 in the still image 200.

[0079] Furthermore, steps S22 to S24 are executed in parallel with steps S19 to S21. In step S22, processor 151 determines whether or not edge line Lb intersects with boundary region R5.

[0080] If the edge line Lb and boundary region R5 do not intersect (NO in step S22), the processor 151 identifies the point of the second edge point coordinates indicated by the edge point information (i.e., the coordinates of the edge point PTb saved in step S28 of the flow for the previous still image) as the edge point PTb (step S23). In the example shown in FIG. 13, the boundary region R5 where the expanded combined region R4 and the expanded region R3 overlap does not intersect with the edge line Lb. Therefore, the point of the second edge point coordinates indicated by the edge point information is identified as the edge point PTb.

[0081] Immediately after the start of the operation phase, the end point information indicates null. In this case, in step S23, the processor 151 specifies the point indicated by the second reference coordinates as the end point PTb.

[0082] If the edge line Lb intersects with the boundary region R5 (YES in step S22), in step S24, the processor 151 identifies the intersection of the boundary region R5 and the edge line Lb, i.e., the point (pixel) on the edge line Lb in the boundary region R5, as the edge point PTb (see FIG. 12). Note that if there are multiple points (pixels) on the edge line Lb in the boundary region R5, the processor 151 may identify one of the multiple points (for example, the point with the largest X coordinate and Y coordinate) as the edge point PTb, or may identify the center of gravity of the multiple points as the edge point PTb.

[0083] As described above, in the preparation image, the second and fourth reference coordinates are the coordinates of the right end of the intersection line between the water surface and the screen 60 when the water surface height is at the first and second levels, respectively, when the screen 60 is viewed from the upstream side of the waterway 50. Therefore, the end point PTb, which is the intersection point between the end line Lb connecting the reference points PT2 and PT4 and the boundary region R5, corresponds to the right end point of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50 in the still image 200.

[0084] When step S20 or step S21 and step S23 or step S24 are completed, the process proceeds to step S25. In step S25, processor 151 calculates the distance between the center of the identified endpoints PTa and PTb and the center of the first and second endpoint coordinates indicated by the endpoint information (i.e., the center of endpoints PTa and PTb identified for the previous still image).

[0085] In the next step S26, the processor 151 determines whether the calculated distance is equal to or greater than a predetermined threshold value Th1.

[0086] Generally, the height of the water surface of the waterway 50 does not change suddenly. Therefore, if the calculated distance is equal to or greater than the threshold value Th1 (YES in step S26), it is highly likely that points significantly different from the endpoints of the intersection line between the water surface and the screen 60 have been identified as the endpoints PTa and PTb in the still image 200 acquired in step S12. Therefore, if the answer is YES in step S26, in step S27, the processor 151 corrects the identified endpoints PTa and PTb to the first and second endpoint coordinates indicated by the endpoint information. The first and second endpoint coordinates indicated by the endpoint information are the endpoints PTa and PTb used to set the near-screen area RA for the previous still image. Therefore, the processor 151 corrects the endpoints PTa and PTb identified for the latest still image 200 so that they are identical to the endpoints PTa and PTb used to set the near-screen area RA for the previous still image. This makes it possible to prevent points that are significantly different from the end points of the intersection line between the water surface and the screen 60 in the still image 200 from being identified as the end points PTa and PTb.

[0087] Immediately after the start of the operation phase, the endpoint information indicates null. If the endpoint information indicates null (that is, in the case of the first flow immediately after the start of the operation phase), steps S25 to S27 are omitted.

[0088] After step S27, or if step S26 is NO, the process proceeds to step S28. In step S28, processor 151 updates the endpoint information to indicate the coordinates of the identified endpoint PTa (first endpoint coordinates) and the coordinates of the identified endpoint PTb (second endpoint coordinates). Note that, if endpoints PTa and PTb have been corrected in step S27, processor 151 updates the endpoint information to indicate the corrected coordinates of endpoints PTa and PTb.

[0089] In step S29 after step S28, the processor 151 operating as the setting unit 113 sets a screen near-area RA that has a line segment connecting the endpoints PTa and PTb of the coordinates indicated by the updated endpoint information as part of its contour and that is convex toward the synthesis area R4.

[0090] Fig. 14 is a diagram illustrating the processing content of step S29 shown in Fig. 10. As shown in the lower part of Fig. 14, the processor 151 sets a line segment 85 connecting the end points PTa and PTb as the major axis a, and sets an area surrounded by an elliptical arc 86 drawn clockwise from the end point PTb to the end point PTa and the line segment 85 as the near-screen area RA. Such a near-screen area RA can be drawn using OpenCV (Open Source Computer Vision Library), which is an open-source library for computer vision.

[0091] The function cv2.ellipse() for drawing an ellipse is shown at the top of Figure 14. To draw an ellipse using the function cv2.ellipse(), the image (img), the center coordinates of the ellipse (center), the major and minor axes of the ellipse (Axes), the rotation angle of the ellipse (angle), the start angle, the end angle, the color, and the width are set.

[0092] The processor 151 sets the still image 200 acquired in step S12 as the image (img).

[0093] Processor 151 uses the coordinates of the identified endpoints PTa and PTb to calculate the center coordinates (center), the major and minor axes (axes) of the ellipse, and the rotation angle (angle) of the ellipse. Specifically, processor 151 calculates the coordinates of the midpoint M between endpoints PTa and PTb as the center coordinates (center) of the ellipse. Processor 151 calculates the length of line segment 85 connecting endpoints PTa and PTb as the major axis a of the ellipse, and calculates the product of major axis a and a predetermined coefficient (e.g., 0.3) as the minor axis b of the ellipse. Processor 151 calculates the angle θ between unit vector V0 pointing in the positive direction of the X-axis and vector V1 starting from midpoint M and ending at endpoint PTb as the rotation angle (angle). The angle θ is expressed as a value in which the clockwise direction is positive.

[0094] As described above, the end points PTa and PTb correspond to the left and right ends of the intersection line between the water surface and the screen 60 in the still image 200 when the screen 60 is viewed from the upstream side of the waterway 50. Therefore, a semi-ellipse that has the line segment connecting the end points PTa and PTb as part of its contour and is convex toward the synthesis area R4 is drawn clockwise from the end point PTb toward the end point PTa, with the midpoint M as its center. Therefore, in the function cv2.ellipse(), the start angle and end angle are set to "0°" and "180°", respectively.

[0095] Furthermore, a predetermined value (for example, "255") is set as the color of the drawn semi-ellipse, and "-1" is set as the width to fill the semi-ellipse.

[0096] In this way, fixed values ​​can be preset for the start angle, end angle, color, and width.

[0097] By setting the function cv2.ellipse() in this way, the line segment 85 connecting the endpoints PTa and PTb has the major axis a, and an elliptical arc 86 drawn clockwise from endpoint PTb to endpoint PTa, and a near-screen area RA surrounded by the line segment 85, are drawn.

[0098] In step S30 after step S29, the processor 151 operating as the determination unit 114 extracts, from among one or more regions R2, a region R2 located within the screen vicinity region RA as a region of dust 80 retained by the screen 60 (hereinafter referred to as a "retained dust region R0"). Specifically, the processor 151 calculates the center of gravity of each region R2, and extracts the region R2 whose center of gravity is located within the screen vicinity region RA as a retained dust region R0. Then, the processor 151 determines the adhesion state of dust 80 to the screen 60 based on the retained dust region R0. Furthermore, the processor 151 operating as the notification unit 115 and the communication IF 155 notify the terminal device 20 of the determination result. After step S30, the process returns to step S12.

[0099] Details of step S30 will be described with reference to Figures 15 and 16. Figure 15 is a diagram showing an example of a stagnant dust area extracted from a still image. In Figure 15, (A) shows the stagnant dust area R0 extracted from the still image 201 of Figure 4(A), and (B) shows the stagnant dust area R0 extracted from the still image 202 of Figure 4(B).

[0100] As shown in (A) of FIG. 15, in a still image 201 when there is little dust 80 adhering to the screen 60, the length along the line segment connecting the end points PTa and PTb in the stagnant dust region R0 (hereinafter referred to as "length Ls") is short. Line segment 85 corresponds to the intersection between the screen 60 and the water surface. Length Ls is the cumulative length of the stagnant dust region R0 aligned in the direction of line segment 85. On the other hand, as shown in (B) of FIG. 15, in a still image 202 when dust 80 is adhering to the entire screen 60, length Ls is long.

[0101] The bottom of the water is also captured in still images 201 and 202. From these still images 201 and 202, it can be determined that "the length of the accumulated debris area R0 along the line segment 85 in the direction of the intersection line (cumulative horizontal length) is proportional to the state of debris adhesion on the entire screen 60."

[0102] FIG. 16 is a diagram for explaining length Ls. In FIG. 16, the arrow indicates the direction of line segment 85. Length Ls is the sum of the lengths L1 to Ln of the accumulating dust areas R0 lined up along line segment 85. Note that n is the number of accumulating dust areas R0. In this way, length Ls is the cumulative sum of the lengths L1 to Ln of the accumulating dust areas R0.

[0103] The processor 151 may calculate a ratio by dividing the length Ls by the length of the line segment 85 as a determination result of the adhesion state of the dust 80 on the screen 60. Alternatively, the processor 151 may output a result of comparing the ratio with a predetermined threshold value as a determination result of the adhesion state of the dust 80 on the screen 60.

[0104] Fig. 17 is a diagram showing an example of a near-screen region set in this embodiment. In Fig. 17, the upper part shows a still image 200 when the water surface height (water level) is at a medium level, a region R2 detected from the still image 200, and a near-screen region RA. The lower part shows a still image 200 when the water surface height is at a high level, a region R2 detected from the still image 200, and a near-screen region RA.

[0105] 17, the position of the intersection line between the screen 60 and the water surface changes depending on the height of the water surface. Specifically, as the water surface becomes higher, the distance between the line segment L0 connecting reference points PT1 and PT2 (two points corresponding to pixels that reflect the ends of the intersection line between the water surface at a low height and the screen 60) and the intersection line between the screen 60 and the water surface becomes longer.

[0106] According to this embodiment, even if the position of the intersection line between the screen 60 and the water surface changes depending on the height of the water surface, the screen near-area RA is automatically set near the intersection line. As a result, the area R2 whose center of gravity is located within the screen near-area RA is extracted as the accumulated dust area R0, and the adhesion status of the dust 80 on the screen 60 is accurately determined based on the accumulated dust area R0.

[0107] (Second Example) In the second embodiment, in the preparation phase, the processor 151 operating as the registration unit 116 stores registration information indicating first and second reference coordinates in the storage unit 120 in response to user input for the preparation image acquired from the camera 30. As described above, the first reference coordinates are the coordinates of the pixel appearing on the left edge of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50. The second reference coordinates are the coordinates of the pixel appearing on the right edge of the intersection line when the screen 60 is viewed from the upstream side of the waterway 50.

[0108] Fig. 18 is a flowchart showing the first half of the processing flow of Example 2. Fig. 19 is a flowchart showing the second half of the processing flow of Example 2. In Figs. 18 and 19, step S12 corresponds to step S1 in Fig. 7, step S13 corresponds to step S2 in Fig. 7, steps S14 to S17, S25 to S28, and S41 to S51 correspond to step S3 in Fig. 7, step S29 corresponds to step S4 in Fig. 7, and step S30 corresponds to steps S5 and S6 in Fig. 7.

[0109] 18 and 19, the processing of the second embodiment differs from the processing of the first embodiment in that it includes steps S42 to S44, step S41 instead of step S11, and steps S45 to S51 instead of steps S18 to S24. Therefore, steps S41 to S51 will be described below.

[0110] In step S41, the processor 151 operating as the identification unit 112 reads the first reference coordinates and the second reference coordinates that have been registered in advance.

[0111] In step S42 after step S41, processor 151 calculates a reference length and a reference angle based on the first reference coordinate and the second reference coordinate. The reference length is the length of a line segment connecting a point on the first reference coordinate (reference point PT1) and a point on the second reference coordinate (reference point PT2). The reference angle is the angle between a unit vector V0 pointing in the positive direction of the X axis and a vector starting from reference point PT1 and ending at reference point PT2. Step S12 is executed after step S42.

[0112] Step S43 is executed after step S17. In step S43, processor 151 acquires the coordinates of a plurality of points belonging to boundary region R5. For example, processor 151 acquires the coordinates of each point on the contour of boundary region R5.

[0113] Step S44 is executed in parallel with steps S14 to S17 and S43. In step S44, processor 151 identifies a separation line that separates combined area R4 from area R3 where screen 60 is projected.

[0114] FIG. 20 is a diagram illustrating the processing of step S44. Conventionally, linear classification models (also referred to as linear classifiers) generated using machine learning are known. A linear classification model classifies each piece of data into one of two or more classes based on the value of a linear combination of one or more types of feature quantities. As shown on the right side of FIG. 20, a linear two-class classification model that classifies input data into two classes shows a separating line 190 for classifying the two classes in a scatter plot on which the input data is plotted. The horizontal and vertical axes of the scatter plot correspond to the two types of feature quantities.

[0115] In the second example, a linear two-class classification technique is applied to the still image 200 acquired in step S12. That is, as shown on the left side of FIG. 20 , the processor 151 regards the still image 200 acquired in step S12 as a scatter plot and obtains a separating line 90 for classifying the combined region R4 and the region R3. Specifically, the processor 151 performs machine learning using first supervised answer data indicating the X and Y coordinates and the label "R4" of each pixel belonging to the combined region R4, and second supervised answer data indicating the X and Y coordinates and the label "R3" of each pixel belonging to the region R3. Through this machine learning, the processor 151 obtains a separating line 90 of a linear classification model that receives the X and Y coordinates as input and outputs either the label "R3" or "R4."

[0116] As shown in FIG. 20, the separation line 90 substantially coincides with the boundary line between the combined region R4 and the region R3.

[0117] 20, the entire still image 200 is regarded as a scatter plot, and the linear two-class classification technique is applied to the entire image. However, only a portion of the still image 200 may be regarded as a scatter plot, and the linear two-class classification technique may be applied to that portion.

[0118] Fig. 21 is a diagram showing the relationship between the range regarded as a scatter plot and the separation line 90. Fig. 21(A) shows a separation line 90 when the entire still image 200 is regarded as a scatter plot.

[0119] On the other hand, as shown in (B) of FIG. 21, processor 151 sets a range 92 of a predetermined size that includes a point of the first reference coordinate (reference point PT1) and a point of the second reference coordinate (reference point PT2), which are part of still image 200. The predetermined size is, for example, a fraction of the size of still image 200, and is determined in advance. Processor 151 then determines a separating line 90 when range 92 is regarded as a scatter plot. (B) of FIG. 21 shows separating line 90 of a linear classification model generated by machine learning using first supervised answer data indicating the X and Y coordinates of each point in range 92 of combined region R4, and second supervised answer data indicating the X and Y coordinates of each point in range 92 of region R3.

[0120] As described above, in the preparation image 250, the reference points PT1 and PT2 correspond to the pixels at the left and right ends of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50. Therefore, as shown in Figure 21, by regarding only the range 92 including the reference points PT1 and PT2 as a scatter plot, it becomes easier to obtain a separation line 90 that approximately coincides with the boundary line between the combined area R4 and the area R3.

[0121] The range 92 may be registered in advance in response to a user input during the advance preparation phase.

[0122] As shown in FIGS. 18 and 19, after both step S43 and step S44 are executed, steps S45 to S49 are executed in order.

[0123] The processing of steps S45 to S49 will be described with reference to Fig. 22. Fig. 22 is a diagram for explaining the processing of steps S45 to S49. Fig. 22 shows an enlarged image of boundary region R5.

[0124] As shown in Fig. 22, the boundary region R5 may have a distorted shape. In particular, depending on the conditions of the reflected light, the edge of the region R3 of the screen 60 may be significantly distorted, which affects the boundary region R5.

[0125] In contrast, the separating line 90 obtained by applying the linear two-class classification technique is less susceptible to distortion at the edge of region R3 of screen 60. Therefore, in order to reduce the influence of distortion in boundary region R5, in step S45, processor 151 projects the points of the coordinates acquired in step S43 (for example, each point on the contour of boundary region R5) onto separating line 90. In FIG. 22, P1 to Pn indicate projected points obtained by projecting the points on the contour of boundary region R5 onto separating line 90.

[0126] In the next step S46, the processor 151 calculates a distance D1 between the reference point PT1 of the first reference coordinate and each of the projection points P1 to Pn. Furthermore, in step S47, the processor 151 calculates a distance D2 between the reference point PT2 of the second reference coordinate and each of the projection points P1 to Pn.

[0127] In the next step S48, the processor 151 identifies the projection point at which the distance D2 is maximum as the end point PTa. In the example shown in Fig. 22, the projection point P1 is identified as the end point PTa. Furthermore, in step S49, the processor 151 identifies the projection point at which the distance D1 is maximum as the end point PTb. In the example shown in Fig. 22, the projection point Pn is identified as the end point PTb.

[0128] As described above, the first reference coordinates are the coordinates of the pixel in the preparation image that appears at the left end of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50. Therefore, the projection point (end point PTb) at which the distance D1 from the reference point PT1 of the first reference coordinates is the longest corresponds to the right end point of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50.

[0129] On the other hand, the second reference coordinates are the coordinates of the pixel in the preparation image that appears at the right end of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50. Therefore, the projection point (end point PTa) at which the distance D2 from the reference point PT2 of the second reference coordinates is the longest corresponds to the left end point of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50.

[0130] In the next step S50, processor 151 calculates the length of the line segment connecting endpoint PTa (i.e., the projection point where distance D2 is maximum) and endpoint PTb (i.e., the projection point where distance D1 is maximum), and the angle of this line segment with respect to unit vector V0 facing in the positive direction of the X axis. The angle of this line segment is the angle between unit vector V0 facing in the positive direction of the X axis and vector V2 that has endpoint PTa as its start point and endpoint PTb as its end point.

[0131] In the next step S51, processor 151 determines whether the difference between the length and angle calculated in step S50 and the reference length and reference angle calculated in step S42 is equal to or greater than a threshold. If at least one of the following is true: the difference between the length calculated in step S50 and the reference length is equal to or greater than threshold Th2, or the difference between the angle calculated in step S50 and the reference angle is equal to or greater than threshold Th3, processor 151 determines YES in step S51. If NO in step S51, the process proceeds to step S25.

[0132] In the coordinate system of the camera 30, the length and orientation of the intersection line between the water surface and the screen 60 fluctuate depending on the height of the water surface. However, the amount of this fluctuation is small. Therefore, if the answer to step S51 is YES, there is a high possibility that points in the still image 200 that are significantly different from the endpoints of the intersection line between the water surface and the screen 60 have been identified as the endpoints PTa and PTb. Therefore, if the answer to step S51 is YES, the process proceeds to step S27. This makes it possible to prevent points that are significantly different from the endpoints of the intersection line between the water surface and the screen 60 from being identified as the endpoints PTa and PTb.

[0133] (Third Example) In the third embodiment, the registration information is not used to identify the endpoints PTa and PTb, so the registration unit 116 may be omitted.

[0134] Fig. 23 is a flowchart showing the first half of the processing flow of Example 3. Fig. 24 is a flowchart showing the second half of the processing flow of Example 3. In Figs. 23 and 24, step S12 corresponds to step S1 in Fig. 7, step S13 corresponds to step S2 in Fig. 7, steps S14 to S17, S25 to S28, S43 to S49, and S61 to S67 correspond to step S3 in Fig. 7, step S29 corresponds to step S4 in Fig. 7, and step S30 corresponds to steps S5 and S6 in Fig. 7.

[0135] 23 and 24, the processing of the third embodiment differs from the processing of the second embodiment in that steps S41, S42, S50, and S51 are omitted and steps S61 to S67 are included. Therefore, steps S61 to S67 will be described below with reference to Fig. 25. Fig. 25 is a diagram for explaining steps S61 to S67.

[0136] Step S61 is executed after step S13 and before step S16. In step S61, the processor 151 identifies the center of gravity SC of the region R3 of the screen 60.

[0137] Step S62 is executed after step S44. In step S62, processor 151 identifies two intersections Pc1 and Pc2 between separation line 90 obtained in step S42 and the edge of still image 200 (image edge).

[0138] Step S63 is executed after steps S62 and S43 are completed. In step S63, the processor 151 identifies a point PT0 which is the foot of a perpendicular line drawn from the center of gravity SC to the separation line 90.

[0139] In the next step S64, processor 151 calculates, as viewed from point PTO, an angle θ1 between the direction of center of gravity SC and the direction of intersection Pc1, and an angle θ2 between the direction of center of gravity SC and the direction of intersection Pc2. The angles θ1 and θ2 range from -180° to 180°. When the direction from center of gravity SC to intersection Pc1 is clockwise around point PTO, angle θ1 is positive. When the direction from center of gravity SC to intersection Pc1 is counterclockwise around point PTO, angle θ1 is negative. Similarly, when the direction from center of gravity SC to intersection Pc2 is clockwise around point PTO, angle θ2 is positive. When the direction from center of gravity SC to intersection Pc2 is counterclockwise around point PTO, angle θ2 is negative. In the example shown in FIG. 25, angle θ1 is negative and angle θ2 is positive.

[0140] In the next step S65, the processor 151 determines whether or not the angle θ1<0 is satisfied.

[0141] If the determination in step S65 is YES, in step S66, the processor 151 sets the intersection point Pc1 as the reference point PT1 and sets the intersection point Pc2 as the reference point PT2.

[0142] If NO in step S65, in step S67, the processor 151 sets the intersection point Pc2 as the reference point PT1, and sets the intersection point Pc1 as the reference point PT2.

[0143] By steps S63 to S67, the processor 151 can set the intersection point Pc1, Pc2 between the separation line 90 and the edge of the still image 200, which is on the right side as viewed from the center of gravity SC of the region R3, as the reference point PT1, and the intersection point on the left side as viewed from the center of gravity SC as the reference point PT2.

[0144] After step S66 or step S67, steps S45 to S49 are executed in order, and after step S49, step S25 is executed.

[0145] According to the third embodiment, of the two intersection points Pc1 and Pc2 between the separation line 90 and the edge of the still image 200, the intersection point on the right side as viewed from the center of gravity SC of the region R3 is automatically set as the reference point PT1, and the intersection point on the left side as viewed from the center of gravity SC is automatically set as the reference point PT2. Therefore, there is no need for advance preparation to register the first reference coordinates and the second reference coordinates.

[0146] Then, steps S46 to S49 are executed using the automatically set reference points PT1 and PT2. As a result, of the multiple projection points obtained by projecting the points of the contour of boundary region R5 onto separation line 90, the leftmost projection point as viewed from the upstream side of waterway 50 is identified as end point PTa, and the rightmost projection point is identified as end point PTb. As a result, in step S29, processor 151 can easily set a semi-elliptical near-screen region RA that has a line segment connecting end points PTa and PTb as part of its contour and is convex toward the synthesis region R4, using function cv2.ellipse() shown in FIG.

[0147] In the third embodiment, points on the edges of the still image 200 (image edges) are set as the reference points PT1 and PT2. Therefore, in step S48, the processor 151 may identify the projection point whose distance D1 from the reference point PT1 is the smallest as the edge point PTa. Similarly, in step S49, the processor 151 may identify the projection point whose distance D2 from the reference point PT2 is the smallest as the edge point PTb. In this way, of the multiple projection points obtained by projecting the points of the contour of the boundary region R5 onto the separation line 90, the leftmost projection point as viewed from the upstream side of the waterway 50 is identified as the edge point PTa, and the rightmost projection point is identified as the edge point PTb.

[0148] (Fourth Example) Fig. 26 is a flowchart showing the first half of the processing flow of Example 4. Fig. 27 is a flowchart showing the second half of the processing flow of Example 4. In Figs. 26 and 27, step S12 corresponds to step S1 in Fig. 7, step S13 corresponds to step S2 in Fig. 7, steps S11, S14 to S18, S20, S23, S25 to S28, and S71 to S74 correspond to step S3 in Fig. 7, step S29 corresponds to step S4 in Fig. 7, and step S30 corresponds to steps S5 and S6 in Fig. 7.

[0149] 26 and 27, the processing of the fourth embodiment differs from the processing of the first embodiment in that it includes step S44, which is the same as that of the second embodiment, and it includes steps S71 to S74 instead of steps S19, S21, S22, and S24. Therefore, steps S71 to S74 will be described below with reference to Fig. 28. Fig. 28 is a diagram illustrating steps S71 to S74.

[0150] In step S71 after step S18, the processor 151 determines whether the edge line La and the separation line 90 intersect with each other.

[0151] If the edge line La and the separation line 90 intersect (YES in step S19), in step S72, the processor 151 identifies the intersection of the edge line La and the separation line 90 as the edge point PTa (see FIG. 28). If the edge line La and the separation line 90 do not intersect (NO in step S19), the above-mentioned step S20 is executed, and the point at the first edge point coordinates indicated by the edge point information (i.e., the coordinates of the edge point PTa saved in step S28 of the flow for the previous still image) is identified as the edge point PTa.

[0152] Furthermore, steps S73, S74, and S23 are executed in parallel with steps S71, S72, and S20. In step S73, processor 151 determines whether edge line Lb and separation line 90 intersect with each other.

[0153] If the edge line Lb intersects with the separation line 90 (YES in step S73), in step S74, the processor 151 identifies the intersection of the edge line Lb and the separation line 90 as the edge point PTb (see FIG. 28). If the edge line Lb and the separation line 90 do not intersect with each other (NO in step S73), the above-mentioned step S23 is executed, and the point at the second edge point coordinates indicated by the edge point information (i.e., the coordinates of the edge point PTb saved in step S28 of the flow for the previous still image) is identified as the edge point PTb.

[0154] <Modification> In the above description, the endpoint information indicates null immediately after the start of the operation phase. However, the processor 151 may provisionally set the endpoint information based on the registration information during the advance preparation phase.

[0155] For example, when the registration information indicates first to fourth reference coordinates, processor 151 provisionally sets the coordinates of a point on end line La connecting the point of the first reference coordinate (reference point PT1) and the point of the third reference coordinate (reference point PT3) (for example, a point ¼ of the length of end line La away from reference point PT1) as the first end point coordinate. Furthermore, processor 151 provisionally sets the coordinates of a point on end line Lb connecting the point of the second reference coordinate (reference point PT2) and the point of the fourth reference coordinate (reference point PT4) (for example, a point ¼ of the length of end line Lb away from reference point PT2) as the second end point coordinate.

[0156] When the registration information indicates the first and second reference coordinates, the processor 151 provisionally sets the first and second reference coordinates as the first and second endpoint coordinates, respectively.

[0157] Alternatively, in the case of a third embodiment in which registration information is not registered, the processor 151 may provisionally set the coordinates of the identified endpoint PTa as the first endpoint coordinates in step S48 of the first flow (see FIGS. 23 and 24) immediately after the start of the operation phase. Furthermore, the processor 151 may provisionally set the coordinates of the identified endpoint PTb as the second endpoint coordinates in step S49 of the first flow immediately after the start of the operation phase.

[0158] <Advantages> As described above, the server device 10 of this embodiment includes an acquisition unit 110, a region detection unit 111, an identification unit 112, a setting unit 113, and a determination unit 114. The acquisition unit 110 acquires a still image 200 from a camera 30 that captures a location of a waterway 50 having a water intake 70 with a screen 60 installed therein, the location of the water intake 70. The region detection unit 111 detects, in the still image 200, a region R1 that captures the water surface, one or more regions R2 that capture debris 80, and a region R3 that captures the screen 60. The identification unit 112 identifies endpoints PTa and PTb of the boundary between a combined region R4 of the region R1 and the one or more regions R2 and the region R3. The setting unit 113 sets a near-screen region RA that has a line segment connecting the endpoints PTa and PTb as part of its contour and that is convex toward the combined region R4. The determining unit 114 determines the adhesion state of the dust 80 on the screen 60 based on the area R2 located within the screen vicinity area RA among the one or more areas R2.

[0159] The debris 80 exists on the water surface or in the water. Therefore, the boundary between the combined region R4 and region R3 corresponds to the intersection line between the screen 60 and the water surface. Therefore, the screen vicinity region RA, which is set using the endpoints PTa and PTb of the boundary between the combined region R4 and region R3, is an area that is convex upstream from the intersection line between the screen 60 and the water surface. This makes it possible to accurately determine the adhesion status of the debris 80 to the screen 60 based on the region R2 located within the screen vicinity region RA.

[0160] The near-screen area RA is automatically set using endpoints PTa and PTb of the boundary between the synthesis area R4 and the area R3, which are identified from the still image. In other words, there is no need to use optical flow technology for moving images. In a moving object detection method such as the dense optical flow described in Patent Document 1, the accuracy of the still-motion map may be reduced due to the influence of water surface waves caused by strong winds, for example, and the accumulated dust area may not be accurately determined. However, in this embodiment, there is no need to use optical flow technology for moving images, so it is less susceptible to the influence of weather conditions such as wind and waves, as with optical flow, and it is possible to determine the dust adhesion status on the screen even in environments with relatively poor communication conditions.

[0161] In the first to third examples, the specification unit 112 expands the combined region R4 and the region R3. Then, the specification unit 112 extracts a boundary region R5 where the expanded combined region R4 and the expanded region R3 overlap, and specifies the end points PTa and PTb based on the boundary region R5.

[0162] As described above, the boundary between the combined region R4 and region R3 corresponds to the intersection line between the screen 60 and the water surface. Therefore, this intersection line is included in the boundary region R5. This allows both ends of the intersection line between the screen 60 and the water surface to be identified as endpoints PTa and PTb. Furthermore, the boundary region R5 can be easily extracted by performing an AND operation on the expanded combined region R4 and the expanded region R3.

[0163] In the first embodiment, the server device 10 includes a registration unit 116 that registers first to fourth reference coordinates in response to user input for a preparation image acquired from the camera 30. The first and second reference coordinates are the coordinates of pixels on the left and right edges of a line of intersection between the water surface at a first height level and the screen 60, respectively, when the screen 60 is viewed from the upstream side of the waterway 50. The third and fourth reference coordinates are the coordinates of pixels on the left and right edges of a line of intersection between the water surface at a second height level different from the first height level and the screen 60, respectively, when the screen 60 is viewed from the upstream side of the waterway 50. The identification unit 112 identifies, within the boundary region R5, a point on an edge line La connecting the point at the first reference coordinate and the point at the third reference coordinate as an edge point PTa. Furthermore, the identification unit 112 identifies, within the boundary region R5, a point on an edge line Lb connecting the point at the second reference coordinate and the point at the fourth reference coordinate as an edge point PTb.

[0164] As a result, in the still image 200, the left and right end points of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50 are identified as the end points PTa and PTb, respectively.

[0165] Note that the identification unit 112 identifies the end points PTa and PTb for the still image 200 each time the acquisition unit 110 acquires a still image 200. The identification unit 112 may identify the end point PTa identified for the previous still image as the end point PTa in the latest still image in response to the boundary region R5 and end line La not intersecting in the latest still image. Similarly, the identification unit 112 may identify the end point PTb identified for the previous still image as the end point PTb in the latest still image in response to the boundary region R5 and end line Lb not intersecting in the latest still image.

[0166] This allows provisional determination of the dust adhesion state even when the boundary region R5 and the edge lines La and Lb do not intersect.

[0167] In the second to fourth examples, the identification unit 112 performs machine learning using first supervised answer data indicating the X-coordinates and Y-coordinates of each point belonging to at least a part of the composite region R4 and the label "R4", and second supervised answer data indicating the X-coordinates and Y-coordinates of each point belonging to at least a part of the region R3 and the label "R3", thereby obtaining a separating line 90 of a linear classification model that receives input of the X-coordinates and the Y-coordinates and outputs either the label "R4" or "R3".

[0168] This makes it possible to easily obtain a separation line 90 that substantially coincides with the boundary line between the combined region R4 and the region R3.

[0169] In the second and third embodiments, the identification unit 112 identifies, among the multiple projected points obtained by projecting multiple points belonging to the boundary region R5 (for example, points on the contour of the boundary region R5) onto the separation line 90, the two projected points located at both ends as the end points PTa and PTb.

[0170] 22, boundary region R5 may have a distorted shape. However, by identifying the ends of multiple projected points obtained by projecting each point belonging to boundary region R5 onto separation line 90 as end points PTa and PTb, it becomes easier to identify points closer to the ends of the intersection line between the water surface and screen 60 as end points PTa and PTb.

[0171] In the second embodiment, the server device 10 includes a registration unit 116 that registers first and second reference coordinates in response to a user input for a preparation image acquired from the camera 30. The first and second reference coordinates are the coordinates of the pixels that appear at the left and right ends, respectively, of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50. The identification unit 112 identifies, among the multiple projection points, the projection point that has the longest distance D2 from the point of the second reference coordinates (reference point PT2) as the end point PTa, and identifies the projection point that has the longest distance D1 from the point of the first reference coordinates (reference point PT1) as the end point PTb.

[0172] As a result, in the still image 200, the left and right end points of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50 are identified as the end points PTa and PTb, respectively.

[0173] The determination unit 112 determines whether a first condition is satisfied, that is, the difference between the length of the line segment connecting the endpoints PTa and PTb identified for the latest still image 200 and the reference length (the length of the line segment connecting the reference points PT1 and PT2) is equal to or greater than a threshold value Th2. Furthermore, the determination unit 112 determines whether a second condition is satisfied, that is, the angle formed between the line segment connecting the endpoints PTa and PTb identified for the latest still image 200 and the line segment connecting the reference points PT1 and PT2 is equal to or greater than a threshold value Th3. The angle formed between the line segment connecting the endpoints PTa and PTb and the line segment connecting the reference points PT1 and PT2 is the difference between the angle calculated in step S50 and the reference angle calculated in step S42. If at least one of the first and second conditions is satisfied, the determination unit 112 corrects the endpoints PTa and PTb identified for the latest still image so that they are identical to the endpoints PTa and PTb used to set the near-screen area RA for the previous still image. Then, when the end points PTa and PTb are corrected, the setting unit 113 sets the near-screen area RA using the corrected end points PTa and PTb.

[0174] This makes it possible to prevent points that are significantly different from the end points of the intersection line between the water surface and the screen 60 from being identified as the end points PTa and PTb.

[0175] In the third embodiment, of the two intersection points Pc1, Pc2 between the separating line 90 and the edge of the still image 200, the identification unit 112 sets the intersection point on the right side of the center of gravity SC of region R3 as reference point PT1, and sets the intersection point on the left side of the center of gravity SC as reference point PT2. Then, of the multiple projection points, the identification unit 112 identifies the projection point with the smallest distance D1 from the reference point PT1 or the projection point with the largest distance D2 from the reference point PT2 as end point PTa. Furthermore, of the multiple projection points, the identification unit 112 identifies the projection point with the largest distance D1 from the reference point PT1 or the projection point with the smallest distance D2 from the reference point PT2 as end point PTb.

[0176] As a result, in the still image 200, the left and right end points of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50 are identified as the end points PTa and PTb, respectively. Also, in the third embodiment, there is no need to register registration information in advance.

[0177] In the fourth embodiment, the identification unit 112 identifies the intersection of the separation line 90 with an edge line La connecting the point of the first reference coordinate (reference point PT1) and the point of the third reference coordinate (reference point PT3) as the edge point PTa. Furthermore, the identification unit 112 identifies the intersection of the separation line 90 with an edge line Lb connecting the point of the second reference coordinate (reference point PT2) and the point of the fourth reference coordinate (reference point PT4) as the edge point PTb.

[0178] As a result, in the still image 200, the left and right end points of the intersection line between the water surface and the screen 60 when the screen 60 is viewed from the upstream side of the waterway 50 are identified as the end points PTa and PTb, respectively.

[0179] Note that the specification unit 112 may specify the endpoint PTa specified for the previous still image as the endpoint PTa in the latest still image in response to the fact that the separating line 90 and the edge line La do not intersect in the latest still image. Similarly, the specification unit 112 may specify the endpoint PTb specified for the previous still image as the endpoint PTb in the latest still image in response to the fact that the separating line 90 and the edge line Lb do not intersect in the latest still image.

[0180] This makes it possible to provisionally determine the state of adhesion of dust even when the separation line 90 and the edge lines La and Lb do not intersect.

[0181] The specifying unit 112 corrects the end points PTa and PTb when the distance between the midpoint of the end points PTa and PTb specified for the latest still image and the midpoint M (see FIG. 14 ) of the line segment 85 of the near-screen area RA set for the previous still image is equal to or greater than a threshold value Th1. Specifically, the specifying unit 112 corrects the end points PTa and PTb specified for the latest still image so that they are the same as the end points PTa and PTb used to set the near-screen area RA for the previous still image. Then, when the end points PTa and PTb have been corrected, the setting unit 113 sets the near-screen area RA using the corrected end points PTa and PTb.

[0182] This makes it possible to prevent points that are significantly different from the end points of the intersection line between the water surface and the screen 60 from being identified as the end points PTa and PTb.

[0183] In the second and fourth embodiments, the identification unit 112 preferably sets a range 92 of a predetermined size that is a part of the still image 200 and includes a point of the first reference coordinate (reference point PT1) and a point of the second reference coordinate (reference point PT2). The first supervised answer data indicates the X and Y coordinates of each point within the range 92 in the combined region R4. Furthermore, the second supervised answer data indicates the X and Y coordinates of each point within the range 92 in the region R3.

[0184] This makes it easier to obtain a separation line 90 that substantially coincides with the boundary line between the combined region R4 and the region R3.

[0185] The setting unit 113 sets the area surrounded by the line segment 85 (see FIG. 14) and an elliptical arc 86 drawn clockwise from end point PTb to end point PTa, with the line segment 85 as the major axis, as the near-screen area RA.

[0186] This makes it easy to set the near-screen area RA using the OpenCV function cv2.ellipse().

[0187] The embodiments disclosed herein should be considered to be illustrative in all respects and not restrictive. The scope of the present invention is defined by the claims, not by the above description, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0188] 1 Monitoring system, 10 Server device, 20 Terminal device, 30 Camera, 40 River, 50 Waterway, 60 Screen, 70 Water intake, 80 Garbage, 85, L0 Line segment, 86 Elliptical arc, 90, 190 Separation line, 92 Range, 100 Small hydroelectric power plant, 110 Acquisition unit, 111 Area detection unit, 112 Identification unit, 113 Setting unit, 114 Judgment unit, 115 Notification unit, 116 Registration unit, 120 Memory unit, 151 Processor, 153 RAM, 155 Communication IF, 156 Input device, 157 Power supply circuit, 158 Display, 200, 201, 202 Still image, 250 Preparatory image, 420 Trained model, La, Lb Edge line, M Midpoint, P1~Pn Projection point, PT1, PT2, PT3, PT4 reference points, PTa, PTb end points, PX1, PX2, PX3, PX4 reference pixels, Pc1, Pc2 intersection points, R0 accumulated dust area, R1, R2, R3 areas, R4 synthesis area, R5 boundary area, RA area near the screen.

Claims

1. an acquisition unit that acquires a still image from a camera that photographs a location where the water intake is installed in a waterway having the water intake with a screen installed; an area detection unit that detects, in the still image, a first area in which a water surface is reflected, one or more second areas in which dust is reflected, and a third area in which the screen is reflected; an identifying unit that identifies a first end point and a second end point of a boundary between the third area and a combined area of ​​the first area and the one or more second areas; a setting unit that sets a near-screen area that has a first line segment connecting the first end point and the second end point as part of a contour and that is convex toward the synthesis area; and a determination unit that determines a state of adhesion of the dust to the screen based on a second area located within the screen vicinity area among the one or more second areas.

2. The identification unit Expanding the composite region; Expanding the third region; extracting a fourth region where the expanded combined region and the expanded third region overlap; The information processing apparatus according to claim 1 , wherein the first end point and the second end point are identified based on the fourth region.

3. a registration unit that registers first to fourth reference coordinates in response to a user input for the advance preparation image acquired from the camera; the first reference coordinates are the coordinates of a pixel on the left edge of an intersection line between the water surface, which is at a first level, and the screen when the screen is viewed from the upstream side of the waterway; the second reference coordinates are the coordinates of a pixel on the right edge of the intersection line between the water surface, whose height is the first level, and the screen when the screen is viewed from the upstream side of the waterway; the third reference coordinates are the coordinates of a pixel on the left edge of an intersection between the water surface, which is at a second level different in height from the first level, and the screen when the screen is viewed from the upstream side of the waterway; the fourth reference coordinates are the coordinates of a pixel on the right edge of the intersection line between the water surface, whose height is the second level, and the screen when the screen is viewed from the upstream side of the waterway; The identification unit a point on a first end line connecting the point of the first reference coordinate and the point of the third reference coordinate in the fourth region is identified as the first end point; The information processing apparatus according to claim 2 , wherein a point on a second end line connecting the point of the second reference coordinate and the point of the fourth reference coordinate in the fourth region is identified as the second end point.

4. The identification unit each time the acquisition unit acquires the still image, the first endpoint and the second endpoint are identified for the still image; specifying the first endpoint identified for the previous still image as the first endpoint in the latest still image in response to the fourth region and the first endpoint line not intersecting in the latest still image; 4. The information processing device according to claim 3, wherein the second end point identified for the previous still image is identified as the second end point in the latest still image in response to the fourth region and the second end line not intersecting in the latest still image.

5. The identification unit performing machine learning using first supervised answer data indicating the coordinates and first labels of each point belonging to at least a portion of the composite region and second supervised answer data indicating the coordinates and second labels of each point belonging to at least a portion of the third region, thereby obtaining a separating line of a linear classification model that receives input of coordinates and outputs either the first label or the second label; The information processing device according to claim 2 , wherein, of a plurality of projected points obtained by projecting a plurality of points belonging to the fourth region onto the separating line, two projected points located at both ends are identified as the first end point and the second end point.

6. a registration unit that registers first and second reference coordinates in response to a user input for the preparation image acquired from the camera; the first reference coordinates are the coordinates of a pixel on the left edge of an intersection line between the water surface and the screen when the screen is viewed from the upstream side of the waterway, The second reference coordinates are the coordinates of a pixel on the right edge of the intersection line when the screen is viewed from the upstream side of the waterway, The identification unit Among the plurality of projection points, a projection point having a maximum distance from the point of the second reference coordinate is identified as the first end point; The information processing apparatus according to claim 5 , wherein the projection point having the greatest distance from the point of the first reference coordinates is identified as the second end point among the plurality of projection points.

7. The identification unit each time the acquisition unit acquires the still image, the first endpoint and the second endpoint are identified for the still image; correcting the first and second endpoints identified for the latest still image to be identical to the first and second endpoints used to set the near-screen region for the previous still image, in response to at least one of the following being satisfied: a difference between a length of a second line segment connecting the first endpoint and the second endpoint identified for the latest still image and a length of a third line segment connecting the point of the first reference coordinate and the point of the second reference coordinate is equal to or greater than a first threshold; and an angle formed by the second line segment and the third line segment is equal to or greater than a second threshold. The information processing device according to claim 6 , wherein, when the first endpoint and the second endpoint are corrected, the setting unit sets the near-screen region using the corrected first endpoint and the corrected second endpoint.

8. The identification unit Of the two intersections between the separation line and an edge of the still image, the intersection on the right side as viewed from the center of gravity of the third area is set as a first reference point, and the intersection on the left side as viewed from the center of gravity is set as a second reference point; Among the plurality of projection points, a projection point having a minimum distance from the first reference point or a projection point having a maximum distance from the second reference point is identified as the first end point; The information processing device according to claim 5 , wherein, of the plurality of projection points, a projection point having a maximum distance from the first reference point or a projection point having a minimum distance from the second reference point is identified as the second end point.

9. a registration unit that registers first to fourth reference coordinates in response to a user input for the advance preparation image acquired from the camera; the first reference coordinates are the coordinates of a pixel on the left edge of an intersection line between the water surface, which is at a first level, and the screen when the screen is viewed from the upstream side of the waterway; the second reference coordinates are the coordinates of a pixel on the right edge of an intersection line between the water surface, whose height is the first level, and the screen when the screen is viewed from the upstream side of the waterway; the third reference coordinates are the coordinates of a pixel on the left edge of an intersection between the water surface, which is at a second level different in height from the first level, and the screen when the screen is viewed from the upstream side of the waterway; the fourth reference coordinates are the coordinates of a pixel on the right edge of an intersection line between the water surface, whose height is the second level, and the screen when the screen is viewed from the upstream side of the waterway; The identification unit performing machine learning using first supervised answer data indicating the coordinates and first labels of each point belonging to at least a portion of the composite region and second supervised answer data indicating the coordinates and second labels of each point belonging to at least a portion of the third region, to obtain a separating line of a linear separation model that receives input of coordinates and outputs either the first label or the second label; an intersection of the separating line and a first end line connecting the point of the first reference coordinate and the point of the third reference coordinate is identified as the first end point; The information processing apparatus according to claim 1 , wherein an intersection of the separating line and a second end line connecting the point of the second reference coordinate and the point of the fourth reference coordinate is identified as the second end point.

10. The identification unit each time the acquisition unit acquires the still image, the first endpoint and the second endpoint are identified for the still image; specifying the first endpoint identified for the previous still image as the first endpoint in the latest still image in response to the separation line and the first endpoint line not intersecting in the latest still image; 10. The information processing device according to claim 9, wherein the second endpoint identified for the previous still image is identified as the second endpoint in the latest still image in response to the separation line and the second endpoint line not intersecting in the latest still image.

11. The identification unit each time the acquisition unit acquires the still image, the first endpoint and the second endpoint are identified for the still image; correcting the first endpoint and the second endpoint identified for the latest still image to be identical to the first endpoint and the second endpoint used to set the near-screen region for the previous still image, in response to a distance between a first midpoint of the first endpoint and the second endpoint identified for the latest still image and a second midpoint of the first line segment of the near-screen region set for the previous still image being equal to or greater than a third threshold; The information processing device according to any one of claims 1 to 3, 5, 6, 8 and 9, wherein when the first endpoint and the second endpoint are corrected, the setting unit sets the near-screen area using the corrected first endpoint and the corrected second endpoint.

12. the specifying unit sets a range of a predetermined size that is a part of the still image and that includes the point of the first reference coordinates and the point of the second reference coordinates; the first correct answer data indicates coordinates of each point within the range of the combined area, The information processing device according to claim 6 , wherein the second correct answer data indicates coordinates of each point within the range of the third region.

13. The information processing device according to claim 1 , wherein the near-screen area has a semi-elliptical shape with the first line segment as a major axis.

14. 10. The information processing device according to claim 3, wherein the setting unit sets, as the near-screen area, an area surrounded by an elliptical arc whose major axis is the first line segment and which is drawn clockwise from the second end point to the first end point, and the first line segment.

15. A surveillance system including a camera, a server device, and a terminal device, the camera photographs a location of the waterway having a water intake with a screen installed thereon, and transmits a still image obtained by photographing the location of the water intake to the server device; The server device receiving the still image from the camera; In the still image, a first area in which a water surface is reflected, one or more second areas in which dust particles are reflected, and a third area in which the screen is reflected are detected; Identifying a first end point and a second end point of a boundary between the third region and a combined region of the first region and the one or more second regions; a near-screen region having a line segment connecting the first end point and the second end point as part of a contour and convex toward the synthesis region; determining a state of adhesion of the dust to the screen based on a second region located within the screen vicinity region among the one or more second regions; Transmitting the determination result to the terminal device; The terminal device receives the determination result from the server device and displays the determination result.

16. acquiring, by one or more processors, a still image from a camera photographing a location of a waterway having a water intake with a screen installed therein; The one or more processors detect, in the still image, a first region in which a water surface is captured, one or more second regions in which dust is captured, and a third region in which the screen is captured; the one or more processors identifying first and second endpoints of a boundary between the third region and a combined region of the first region and the one or more second regions; a step in which the one or more processors set a near-screen region that has a line segment connecting the first endpoint and the second endpoint as part of a contour and that is convex toward the synthesis region; and a step in which the one or more processors determine the state of adhesion of the dust to the screen based on a second area among the one or more second areas that is located within the screen vicinity area.

17. A program for controlling an information processing device, A step of acquiring a still image from a camera that photographs a location where a water intake with a screen installed in the waterway having the water intake is installed; detecting a first area in which a water surface is reflected, one or more second areas in which dust particles are reflected, and a third area in which the screen is reflected in the still image; identifying a first end point and a second end point of a boundary between the third region and a combined region of the first region and the one or more second regions; setting a near-screen region that has a line segment connecting the first end point and the second end point as part of its contour and that is convex toward the synthesis region; and determining the state of adhesion of the dust to the screen based on a second area located within the screen vicinity area among the one or more second areas.

Images