River Monitoring System

Abstract

To provide a river monitoring system that makes it possible to accurately perform a prediction about an overflow of a river.SOLUTION: A river monitoring system of the present example comprises: an upstream camera 10 and downstream camera 15 that are installed with respect to rivers of a monitoring object; an image processing device 20 that performs an optical flow analysis of images of the rivers photographed by these cameras; and a river analysis device 30 that performs a prediction about an overflow of the river on the basis of a result of the optical flow analysis. The river analysis device 30 has a function that changes a threshold about a state of the river on the basis of the result of the optical flow analysis, and is configured to perform the prediction about the overflow of the river using the threshold.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a river monitoring system for monitoring the state of a river. [Background technology]

[0002] Conventionally, various countermeasures against disasters such as river flooding and levee collapse have been considered. For example, water levels at key points in rivers are observed, and if necessary, water is diverted to a reservoir, and the necessity of evacuating nearby residents is judged. In this specification, "river" includes not only natural and man-made rivers, but also waterways for specific purposes such as agricultural irrigation channels and power generation channels.

[0003] The prior art in the technical field of the present invention includes the following: For example, Patent Document 1 discloses an invention that calculates the flow speed of a river from the optical flow of a river image at the time of determination, based on the correspondence between the flow speed of the river measured in advance using a current meter and the optical flow of the river image taken at the time of measurement. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-047156 Summary of the Invention [Problem to be solved by the invention]

[0005] In recent years, the increasing severity of floods caused by heavy rains has become a problem. Conventional flood forecasts collect information such as rainfall at observation points, river water levels, and weather radar observations, and predict the occurrence of floods based on this information. However, if the environment of the river changes and its shape or width changes, there is a concern that the accuracy of the flood forecast data will be compromised, and there is also a concern that management will become difficult because the thresholds used for flood forecasting will need to be reset. Furthermore, with existing technology, future conditions at individual observation points are predicted based on information collected from each point, but there are disasters that cannot be predicted based on information from observation points alone (for example, flash floods caused by earth dams). For this reason, there is a possibility of secondary disasters in which people will be exposed to danger if they evacuate according to evacuation warnings based on conventional flood forecasts.

[0006] The present invention has been made in consideration of the above-mentioned conventional circumstances, and has an object to provide a river monitoring system capable of making accurate predictions regarding river flooding. [Means for solving the problem]

[0007] In order to achieve the above object, a river monitoring system according to one aspect of the present invention is configured as follows: That is, the river monitoring system according to the present invention includes a camera installed with respect to a river, an image processing device that performs optical flow analysis on an image of the river captured by the camera, and a river analysis device that predicts river flooding based on the results of the optical flow analysis, and the river analysis device has a function of changing a threshold value related to the state of the river based on the results of the optical flow analysis, and predicts river flooding using the threshold value.

[0008] Here, in the river monitoring system of the present invention, the river analysis device can be configured to calculate a threshold value related to the state of the river by analyzing the results of the optical flow analysis using a predetermined learning model.

[0009] Furthermore, the river monitoring system according to the present invention may be configured to further include a distribution device that distributes the prediction result of river flooding when the river analysis device predicts river flooding.

[0010] In addition, in the river monitoring system according to the present invention, the distribution device can be configured to distribute, as a result of predicting river flooding, a flood prediction image in which river flood prediction information is superimposed on an image of the river.

[0011] In addition, in the river monitoring system according to the present invention, the distribution device can be configured to distribute, as a result of prediction of river flooding, a flood forecast map in which dangerous areas due to river flooding are superimposed on a map.

[0012] In addition, in the river monitoring system of the present invention, the distribution device can be configured to control the distribution of river flood prediction results based on location information of terminal devices carried by residents and / or congestion information at evacuation destinations for residents. Effect of the Invention

[0013] According to the present invention, it is possible to provide a river monitoring system capable of making accurate predictions regarding river flooding. [Brief description of the drawings]

[0014] [Figure 1] 1 is a diagram showing an example of the configuration of a river monitoring system according to an embodiment of the present invention; [Diagram 2] FIG. 2 is a diagram showing an example of a processing flow by the river monitoring system of FIG. 1. [Diagram 3] FIG. 2 is a diagram showing an example of a processing sequence by the river monitoring system of FIG. 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] An embodiment of the present invention will be described with reference to the drawings. Fig. 1 shows an example of the configuration of a river monitoring system according to an embodiment of the present invention. The river monitoring system in Fig. 1 includes an upstream camera 10, a downstream camera 15, an image processing device 20, a river analysis device 30, a distribution device 40, and a terminal device 50. These devices are connected to each other so as to be able to communicate with each other via a network line. Furthermore, these devices include hardware resources such as a processor and memory, and are configured such that the processor executes a program for realizing each function according to the present invention.

[0016] The upstream camera 10 and downstream camera 15 are surveillance cameras for photographing the river to be monitored, and photograph the river by user operation or automatically. The upstream camera 10 is installed at a point on the upstream side of the river, and the downstream camera 15 is installed at a point on the downstream side of the river. Note that the configuration is not limited to installing two cameras, the upstream camera 10 and the downstream camera 15, on the river as in this example, and a midstream camera may be added at one or more points between the upstream camera 10 and the downstream camera 15. Also, while a single surveillance camera may be installed on the river, it is preferable to install multiple surveillance cameras from the viewpoint of increasing the accuracy of predicting river flooding.

[0017] The image processing device 20 performs image analysis using optical flow on images of the river captured by surveillance cameras such as the upstream camera 10 and downstream camera 15. Optical flow captures the movement of a subject and visualizes it using vector representation, and can be calculated by comparing and analyzing multiple consecutive frames of images.

[0018] The image processing device 20 detects the state of the river at the upstream point by, for example, calculating the optical flow at an upstream point based on an image of the river captured by the upstream camera 10 and analyzing this optical flow. Examples of the state of the river include flow speed, river width, flow direction, etc. The flow speed can be calculated based on the movement speed of the optical flow, for example, as described in Patent Document 1. The river width can be calculated based on the width of the area where the optical flow is obtained, for example. The flow direction can be calculated based on the direction of the optical flow, for example. The image processing device 20 also performs similar processing on images from other monitoring cameras (in this example, the downstream camera 15).

[0019] Furthermore, the image processing device 20 also performs processing to generate distribution data for informing local residents of a predicted river flooding by the river analysis device 30 described below. Examples of distribution data include a flood prediction image in which flood prediction information showing predicted flooding is superimposed on an image of a river, and a flood prediction map in which dangerous areas due to river flooding are superimposed on a map. The flood prediction information superimposed on the image of a river may be in text format, or in a schematic image format. The flood prediction map may include information showing safe and dangerous evacuation routes when multiple evacuation routes are assumed.

[0020] The river analysis device 30 predicts river flooding based on the result of optical flow analysis of the river image. The result of the optical flow analysis includes an optical flow calculated based on the river image, and the state of the river detected from the optical flow (flow speed, river width, flow direction, etc.). For example, when the state of the river detected from the optical flow exceeds a threshold value set for the state of the river, the river analysis device 30 predicts that there is a possibility of river flooding.

[0021] Furthermore, the river analysis device 30 has a function of calculating a threshold value related to the state of the river based on the result of optical flow analysis of the river image, and changing the setting of the threshold value used in predicting river flooding. That is, the river analysis device 30 has a function of automatically or manually changing the threshold value used in predicting river flooding, taking into consideration cases where the river environment has changed and the river shape, width, etc. have changed. The threshold value may be reset, for example, periodically (about once a month) based on the normal state. This allows the threshold value to always be set based on the latest river state.

[0022] The thresholds used in predicting river flooding (i.e., thresholds related to river conditions) should be set based on values ​​measured over a certain period of time when calculated according to the type of threshold (flow velocity, river width, flow direction, etc.).

[0023] In this example, the river analysis device 30 calculates thresholds for the state of the river by analyzing the results of optical flow analysis using AI (Artificial Intelligence) that uses a learning model of how the river width, water level, flow speed, flow direction, etc. change depending on the river state (normal, flooded, flooded, etc.). That is, the river analysis device 30 is configured to automatically change the thresholds used for predicting river flooding by using threshold estimation by AI.

[0024] When the river analysis device 30 predicts a river flood (i.e., when the state of the river detected from the optical flow exceeds a threshold), it creates flood prediction information regarding the predicted flood and transmits it to the image processing device 20. The flood prediction information includes the location, scale, time, etc. of the predicted flood.

[0025] The river analysis device 30 can predict flooding of a river based on other conditions in addition to comparing the state of the river detected from the optical flow with a threshold value. For example, the river analysis device 30 may predict flooding of a river when it is determined that a backflow is occurring based on the flow direction detected from the optical flow. In this case, a more accurate prediction can be made by predicting flooding of a river while taking into consideration the conditions of the state of the river, such as the confluence of the rivers. The river analysis device 30 may also predict flooding of a river by further taking into consideration other information, such as weather information obtained from a weather radar.

[0026] When a river flood is predicted by the river analysis device 30, the distribution device 40 distributes distribution data (flood prediction image, flood prediction map, etc.) created by the image processing device 20 to the terminal device 50 to notify local residents of the prediction. The distribution device 40 can control the distribution of the distribution data based on the location information of the terminal device 50 carried by the resident and congestion information at the resident's evacuation destination.

[0027] The location information of the terminal device 50 is acquired, for example, by a GPS (Global Positioning System) function of the terminal device 50 and periodically transmitted to the distribution device 40. The congestion information of the evacuation destination is acquired, for example, by a shelter terminal installed at the evacuation destination and periodically transmitted to the distribution device 40. The congestion information can be generated by input to the shelter terminal or by video of a surveillance camera. The input information from the shelter terminal is managed in a database by inputting, for example, personal information such as the gender and age of the evacuees, location information, and evacuation status to the shelter terminal, and the congestion status of each shelter can be grasped. In addition, as a method of detecting congestion using surveillance video, for example, there is a method disclosed in International Publication No. 2018 / 061976. In this method, a congestion degree estimation unit of an image processing device estimates the congestion status in the video acquired from an image input unit.

[0028] The terminal devices 50 are terminals owned by local residents, and are capable of displaying distribution data distributed from the distribution device 40. In this example, N terminal devices 50(1) to 50(N) are assumed, but the number of terminal devices 50 is arbitrary. The terminal devices 50 are not limited to portable terminals such as mobile phones, tablets, and laptops, and may be stationary terminals such as desktops.

[0029] The operation of the river monitoring system of this embodiment will be described with reference to the example of the processing sequence shown in Fig. 2. Here, it is assumed that flooding of a river is predicted for each installation point of a monitoring camera based on images captured by the monitoring camera.

[0030] Surveillance cameras such as the upstream camera 10 and downstream camera 15 capture images of the river to be monitored and transmit the images of the river to the image processing device 20 (step S11). The image processing device 20 performs optical flow analysis on the received river images to detect the state of the river (flow speed, river width, flow direction, etc.) (step S12), and transmits analysis data including the results of the optical flow analysis to the river analysis device 30 (step S13).

[0031] The river analysis device 30 calculates and sets thresholds for the state of the river (flow speed, river width, flow direction, etc.) as necessary based on the analysis data received from the image processing device 20, and then predicts flooding of the river (step S14). If it is predicted that flooding of the river will occur as a result, the river analysis device 30 generates flood prediction information and transmits it to the image processing device 20 (step S15).

[0032] The image processing device 20 generates a flood prediction image by superimposing the flood prediction information on an image of the river based on the flood prediction information created by the river analysis device 30 (step S16). The image processing device 20 further generates a flood prediction map by superimposing areas at risk of flooding of the river on the map (step S17). The flood prediction image and the flood prediction map are transmitted from the image processing device 20 to the distribution device 40 (step S18).

[0033] The distribution device 40 determines whether or not a notification of predicted river flooding is necessary for each area based on location information of the point where flooding is predicted (i.e., location information of the surveillance camera that took the image used for the prediction) (step S19), and distributes a flood prediction image and / or a flood prediction map to the terminal devices 50 of residents in the area where it is determined that a notification is necessary (step S20).

[0034] This allows flood forecast images and flood forecast maps to be displayed on the terminal devices 50 of residents in areas at risk of flooding. In addition, the distribution device 40 can distribute flood forecast images and flood forecast maps to terminal devices 50 in other areas in response to requests from those terminal devices 50.

[0035] In the above explanation using Fig. 2, flooding of rivers is predicted for each monitoring camera installation point based on the images from the monitoring camera, but images from monitoring cameras at other points may also be taken into consideration. The processing in this case will be explained below with reference to the example of the processing flow shown in Fig. 3.

[0036] First, the image processing device 20 receives an image of the upstream point of the river taken by the upstream camera 10 (step S31), and performs optical flow analysis on the image of the upstream point of the river (step S32). Next, the river analysis device 30 changes the calculation and setting of thresholds related to the state of the river (flow speed, river width, flow direction, etc.) as necessary based on the result of the optical flow analysis, and then determines whether the state of the upstream point of the river exceeds the threshold (step S33).

[0037] If it is determined that the state of the upstream point of the river exceeds the threshold (i.e., river flooding is predicted), the image processing device 20 generates flood prediction information for the upstream camera 10 (step S34). The image processing device 20 further predicts flooding at the downstream point of the river based on the flood prediction information for the upstream camera 10 and preset river information (upstream-downstream length, river width under normal conditions, etc.), and further creates flood prediction information for the downstream camera 15 (step S35). For example, flooding at the downstream point of the river is predicted based on the distance to the downstream camera and flood prediction information analyzed by the upstream camera. At this time, flooding at the downstream point of the river may be predicted by further considering the result of optical flow analysis of the image captured by the downstream camera 15.

[0038] The flood prediction information generated by the river analysis device 30 is transmitted to the image processing device 20 and used to generate a flood prediction image and a flood prediction map by the image processing device 20 (step S36). The flood prediction image and the flood prediction map generated by the image processing device 20 are distributed to the terminal devices 50 of local residents under the control of the distribution device 40 (step S37).

[0039] As described above, the river monitoring system of this example includes an upstream camera 10 and a downstream camera 15 installed in relation to the river to be monitored, an image processing device 20 that performs optical flow analysis on the images of the river captured by these cameras, and a river analysis device 30 that predicts river flooding based on the results of the optical flow analysis. In this way, since predictions of river flooding are made based on the results of optical flow analysis of the river images, it is possible to accurately estimate river information and make highly accurate predictions of river flooding. In addition, since the flow direction can be detected by optical flow analysis, it is also possible to predict river flooding due to backflow, which was previously difficult to observe.

[0040] In addition, in the river monitoring system of this example, the river analysis device 30 has a function of changing the threshold value related to the state of the river based on the result of optical flow analysis, and is configured to perform a prediction related to the flooding of the river using the threshold value. Therefore, when the environment of the river changes and its shape, width, etc. change, the threshold value used for predicting the flooding of the river is automatically changed, so it is possible to suppress the deterioration of the accuracy of the prediction related to the flooding of the river.

[0041] In addition, in the river monitoring system of this example, as described with reference to FIG. 3, the result of optical flow analysis on the image of the upstream camera 10 is configured to be used for predicting the flooding of the river at the location of the downstream camera 15. In this way, when predicting the flooding of the river at a certain location, by considering the result of optical flow analysis on the image of the monitoring camera at another location, it is possible to predict disasters that could not be predicted conventionally (for example, the occurrence of a water cannon due to an earth dam, etc.). That is, it is possible to perform not only temporal prediction of the river situation (prediction of future flooding at the same location) but also geographical prediction (prediction of future flooding at another location).

[0042] In addition, the river monitoring system of this example further includes a distribution device 40 that distributes a flood prediction image or a flood prediction map as a flood prediction result of the river to the terminal device 50 of the local residents when the flooding of the river is predicted by the river analysis device 30. Therefore, it is possible to promptly notify the local residents near the river that the flooding of the river has been predicted. In addition, there was a possibility that the reliability of the flooding was judged to be low for the residents only based on the conventional disaster information and evacuation information. However, by providing the residents with a flood prediction image in which the flood prediction information is superimposed on the image of the actual river, it is possible to accurately make the residents recognize the danger of the flooding. In addition, by providing the residents with a flood prediction map including the evacuation route, it is also possible to help the residents select the evacuation route.

[0043] In addition, in the river monitoring system of this example, the distribution device 40 is configured to control the distribution of flood prediction images and flood prediction maps based on the location information of the terminal device 50 carried by the resident and congestion information of the resident's evacuation destination. In this way, by performing distribution in conjunction with the location information and congestion information, it is possible to avoid crowded evacuation shelters and to make an evacuation plan that takes into account the living environment after evacuation. For example, when there are multiple evacuation shelters in the neighborhood, the resident can select a more appropriate evacuation shelter based on the congestion information received by the terminal device 50. In addition, since the flood prediction information of the river is also received, it is possible to move to the evacuation shelter by detouring the vicinity of the flooded river based on the flood prediction information. In addition, it is also possible to apply it to road congestion prediction, train congestion prediction, automatic driving of vehicles, etc., by taking into account the flow of people and traffic.

[0044] Here, in the river monitoring system of this example, the image processing device 20 and the river analysis device 30 are configured as separate devices, but they may be configured as an integrated device. Similarly, in the river monitoring system of this example, the distribution device 40 may also be configured as an integrated device with the image processing device 20 or the river analysis device 30. Alternatively, the functions of these devices may be distributed across multiple devices.

[0045] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents.

[0046] Furthermore, the present invention can be provided not only as devices such as those described above or as a system composed of these devices, but also as methods executed by these devices, programs for causing a processor to realize the functions of these devices, and storage media for storing such programs in a computer-readable format.

Industrial Applicability

[0047] The present invention can be used in a river monitoring system for monitoring the state of a river.

Explanation of Signs

[0048] 10: Upstream camera, 15: Downstream camera, 20: Image processing device, 30: River analysis device, 40: Distribution device, 50(1) to 50(N): Terminal device

Claims

1. A camera installed on the river, an image processing device that performs optical flow analysis on the image of the river captured by the camera; a river analysis device that predicts flooding of the river based on a result of the optical flow analysis and a threshold value related to a state of the river, The river analysis device is a river monitoring system characterized by having the function of analyzing the results of the optical flow analysis using a predetermined learning model that has learned how the state of the river changes under multiple conditions, including normal times, thereby recalculating the threshold value for the state of the river based on normal conditions and changing the threshold value for the state of the river.

2. The river monitoring system according to claim 1, A river monitoring system further comprising a distribution device that distributes a result of a prediction of a flood of the river when the flood of the river is predicted by the river analysis device.

3. The river monitoring system according to claim 2, The river monitoring system is characterized in that the distribution device distributes a flood prediction image in which the river flood prediction information is superimposed on an image of the river as a prediction result of the river flooding.

4. The river monitoring system according to claim 2, The river monitoring system is characterized in that the distribution device distributes a flood prediction map on which information on dangerous areas due to flooding of the river is superimposed as a prediction result of flooding of the river.

5. The river monitoring system according to any one of claims 2 to 4, A river monitoring system characterized in that the distribution device controls the distribution of the river flood prediction results based on location information of terminal devices carried by residents and / or congestion information at evacuation destinations for the residents.

Images