Flood depth estimation device, flood depth estimation method, and program

Abstract

To provide a flood depth estimation device, etc. that enable accurate estimation of a flood depth.SOLUTION: A flooded object estimation device has: a storage part that stores a learned model that has been learned to output an approximate flood depth of a flooded object represented in an image and a confidence level for the flooded object when an image representing the flooded object is inputted; and an estimation part that estimates a flood depth in the input image based on the approximate flood depth and the confidence level to be output from the learned model when the input image is inputted to the learned model.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present invention relates to a flood depth estimation device, a flood depth estimation method, and a program. [Background technology]

[0002] In recent years, damage caused by flooding has been increasing. In order to prevent damage caused by flooding, it is necessary to respond by issuing evacuation orders at an appropriate time. To this end, research is being conducted into estimating the flood depth of objects using images acquired from security cameras, SNS (Social Networking Services), and the like. For example, Non-Patent Document 1 describes a method of detecting the flood depth of flooded objects shown in images using R-CNN (Regional Convolutional Neural Network). In Non-Patent Document 1, the flood depth is indicated by a flood level that is preset with a granularity of about 10 cm to 20 cm. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] Flood-Water Level Estimation From Social Media Images, P.Chaudhary, et al., ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol.IV-2 / W5, 2019 Summary of the Invention [Problem to be solved by the invention]

[0004] It is generally known that when the water depth exceeds 45cm, there is a high possibility of flooding above floor level, and when it exceeds 50cm, it becomes rapidly difficult for adults to evacuate. In other words, even slight differences in water depth can affect the flood damage situation and the response that should be taken. Therefore, there is a need to estimate the water depth more accurately and with finer granularity.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to provide a water depth estimation device, a water depth estimation method, and a program that enable accurate estimation of water depth. [Means for solving the problem]

[0006] A flood depth estimation device according to an embodiment of the present invention is characterized in that it has a memory unit that stores a trained model that has been trained to output, when an image showing a submerged object is input, an approximate flood depth of the submerged object shown in the image and a certainty factor for the submerged object, and an estimation unit that estimates the flood depth in the input image based on the approximate flood depth and certainty factor output from the trained model when an input image is input to the trained model.

[0007] In addition, it is preferable that the estimation unit estimates the flood depth by combining the approximate flood depths of multiple floodable objects output from the trained model when an input image representing multiple floodable objects is input to the trained model based on the confidence level for each floodable object.

[0008] In addition, it is preferable that the trained model is trained to further output a flooded object area corresponding to the flooded object shown in the input image, and the estimation unit estimates the flooded depth by combining the approximate flooded depths of multiple flooded objects where the flooded object areas output from the trained model overlap based on the confidence level for each flooded object.

[0009] In addition, it is preferable that the estimation unit estimates the flood depth by weighting and adding up the approximate flood depths of multiple floodable objects whose floodable object areas overlap with each other according to the confidence level for each floodable object.

[0010] In addition, it is preferable that the trained model is trained to further output the type of submerged object represented in the input image, and the estimation unit estimates the type of submerged object represented in the input image based on the type of submerged object output from the trained model and the confidence level for the submerged object.

[0011] A flood depth estimation method according to an embodiment of the present invention is characterized in that it includes storing a trained model that has been trained to output, when an image showing a submerged object is input, an approximate flood depth of the submerged object shown in the image and a certainty factor for the submerged object, and estimating the flood depth for the input image based on the approximate flood depth and certainty factor output from the trained model when an input image is input into the trained model.

[0012] A program according to an embodiment of the present invention is a computer program having a memory unit that stores a trained model that has been trained to output, when an image showing a submerged object is input, the approximate flood depth of the submerged object shown in the image and a certainty factor for the submerged object, and is characterized in that the program causes the computer to estimate the flood depth for the input image based on the approximate flood depth and certainty factor output from the trained model when an input image is input into the trained model. Effect of the Invention

[0013] The flood depth estimation device, flood depth estimation method, and program according to the present invention make it possible to estimate flood depth with high accuracy. [Brief description of the drawings]

[0014] [Figure 1] FIG. 2 is an exemplary functional block diagram of the flood depth estimation device 1. [Diagram 2] FIG. 2 is a diagram showing an example of the data structure of learning data D1. [Diagram 3] FIG. 13 is a diagram showing an example of the data structure of an approximate flood depth table T1. [Figure 4] FIG. 11 is a flow diagram showing an example of the flow of a learning process. [Diagram 5]FIG. 11 is a flow diagram showing an example of the flow of a flood depth output process. [Figure 6] FIG. 11 is a flow diagram showing an example of the flow of an estimation process. [Figure 7] FIG. 11 is a schematic diagram for explaining an example of the contents of an estimation process. [Figure 8] FIG. 13 is a diagram showing an example of a flood depth display image IM2. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, various embodiments of the present invention will be described with reference to the drawings. It should be noted that the technical scope of the present invention is not limited to these embodiments, but covers the inventions described in the claims and their equivalents.

[0016] Figure 1 is an exemplary functional block diagram of a flood depth estimation device 1 according to an embodiment of the present invention. The flood depth estimation device 1 estimates the flood depth in an input image showing a submerged object. The flood depth estimation device 1 is an information processing device such as a server, a PC (Personal Computer), a tablet terminal, a smartphone, or a mobile phone. The flood depth estimation device 1 has a memory unit 11, a communication unit 12, a display unit 13, an operation unit 14, and a processing unit 15.

[0017] The storage unit 11 is configured to store programs and data, and includes, for example, a semiconductor memory. The storage unit 11 stores, as programs, an operating system program, a driver program, an application program, and the like used in processing by the processing unit 15. The programs are installed in the storage unit 11 from a computer-readable, non-transitory, portable storage medium such as a CD-ROM (Compact Disc Read Only Memory) or a DVD-ROM (Digital Versatile Disc Read Only Memory).

[0018] The communication unit 12 is configured to enable the flood depth estimation device 1 to communicate with other devices, and includes a communication interface circuit. The communication interface circuit is a communication interface circuit for a wired LAN (Local Area Network) or a wireless LAN, etc. The communication unit 12 receives data from other devices and supplies the data to the processing unit 15, and also transmits data supplied from the processing unit 15 to other devices.

[0019] The display unit 13 is configured to display an image, and includes a display such as a liquid crystal display or an organic EL (Electro-Luminescence) display. The display unit 13 displays an image based on the display data supplied from the processing unit 15.

[0020] The operation unit 14 is configured to accept operations for the flood depth estimation device 1, and includes input devices such as a keyboard, a keypad, and a mouse. The operation unit 14 may include a touch panel integrated with the display unit 13. The operation unit 14 supplies a signal corresponding to the accepted operation to the processing unit 15.

[0021] The processing unit 15 is configured to comprehensively control the operation of the flood depth estimation device 1, and comprises one or more processors and their peripheral circuits. The processing unit 15 comprises, for example, a CPU (Central Processing Unit). The processing unit 15 may also comprise a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), etc. The processing unit 15 controls the operation of each component and executes various processes so that the various processes of the flood depth estimation device 1 are executed in an appropriate order based on the programs stored in the memory unit 11.

[0022] The processing unit 15 has, as functional blocks, a learning unit 151, an acquisition unit 152, an estimation unit 153, and an output unit 154. These functional blocks are functional modules that are realized by the processing unit 15 executing a program. These functional blocks may be implemented in the flood depth estimation device 1 as dedicated processing circuits.

[0023] 2 is a diagram showing an example of the data structure of the learning data D1 stored in the storage unit 11 and used for learning the trained model. The learning data includes learning images and labels of each learning image associated with each other. The labels include a submerged object region and a class.

[0024] The flooded object area is an area corresponding to a flooded object shown in a training image. For example, the flooded object area is a circumscribing rectangle of the flooded object. In this case, the flooded object area is specified by the center coordinates, width, and height of the circumscribing rectangle, as shown in FIG. 2. When one training image represents multiple flooded objects, multiple flooded object areas are associated with the training image. In the example shown in FIG. 2, two flooded object areas are associated with the training image "Img1", one flooded object area is associated with the training image "Img2", and two flooded object areas are associated with the training image "Img3". The class indicates the type of flooded object and the approximate flood depth of the flooded object. The approximate flood depth is an approximate value of the flood depth set at a predetermined granularity. In the example shown in FIG. 2, the class "Car_10" indicates a car with an approximate flood depth of 10 cm, the class "Person_20" indicates a person with an approximate flood depth of 20 cm, and the class "Person_40" indicates a person with an approximate flood depth of 40 cm. In other words, even if multiple floodable objects are of the same type, if their approximate flood depths are different, they are associated with different classes. Similarly, the class "Cone_30" indicates a road cone with an approximate flood depth of 30 cm, and the class "Bicycle_30" indicates a bicycle with an approximate flood depth of 30 cm. In other words, even if multiple floodable objects are of the same approximate flood depth, if their types are different, they are associated with different classes.

[0025] FIG. 3 is a diagram showing an example of the data structure of class table T1 stored in memory unit 11 and showing the correspondence between classes, types of submergible objects, and approximate flood depths. As described above, a class is a combination of a type of submergible object and an approximate flood depth. The approximate flood depth is set in advance with a predetermined granularity based on the flood depth of the submergible object shown in the learning image. In the example shown in FIG. 3, the predetermined granularity is 10 cm. Furthermore, the approximate flood depth is a value included in the corresponding range of flood depths, and is set to a value approximately halfway between the lower limit and the upper limit of the range of flood depths.

[0026] The learning data is generated by an operator associating a label with a learning image. The submerged object area included in the label is set by the operator who has viewed the learning image. The type of submerged object and the approximate flood depth are estimated by the operator based on the situation when the learning image was taken, the shape of the submerged object shown in the learning image, etc. The operator selects a class corresponding to the estimated type of submerged object and the approximate flood depth from the classes stored in the class table T1 and sets it as a label. In this way, a class that is a combination of the type of submerged object and the approximate flood depth is associated as a label with the learning image, so that the trained model can output the approximate flood depth of the submerged object shown in the image.

[0027] As mentioned above, the approximate flood depth is set by the worker who looks at the image. Therefore, the granularity of the approximate flood depth is the granularity of the flood depth that the worker can estimate from the image. Generally, this granularity is approximately 10 cm or more.

[0028] 4 is a flow diagram showing an example of the flow of the learning process executed by the flood depth estimation device 1. The learning process is executed in a state where an unlearned learning model is stored in the storage unit 11. The learning model is, for example, Faster R-CNN, Mask R-CNN or Libra R-CNN. Without being limited to these examples, the learning model may be any object detection model that, when an image is input, outputs an object region corresponding to an object shown in the image, the type of the object, and a confidence level for the object.

[0029] First, the learning unit 151 acquires learning data (step S11). For example, the learning unit 151 acquires learning data stored in the storage unit 11 in advance.

[0030] Next, the learning unit 151 trains the learning model so that, when an image showing a submerged object is input, it outputs a submerged object region corresponding to the submerged object shown in the image, a submerged object class, and a certainty factor for the submerged object (step S12). The learning unit 151 trains the learning model by updating the parameters of the learning model so that the value of the error function between the submerged object region and submerged object class output from the learning model when a learning image is input to the learning model and the submerged object region and submerged object class included in the label becomes small. The error function includes an error related to the position and shape of the submerged object region, and an error related to the certainty factor for each class. For example, the error related to the position of the submerged object region is the L1 error, and the error related to the certainty factor is the cross entropy error. The parameters are updated by an error backpropagation method using a gradient descent method, or the like.

[0031] The certainty indicates the likelihood of the class for each floodable object region. In other words, the certainty is a value that represents the likelihood that the combination of floodable object type and approximate flood depth in each floodable object region (for example, that a certain floodable object region is an image of a vehicle with an approximate flood depth of 10 cm) is a true value. For example, the range of certainty is 0 to 1.0. In this case, the closer to 1.0 a combination of floodable object type and approximate flood depth is output in each floodable object region, the more likely it is to be a true value, and the closer to 0 a combination of floodable object type and approximate flood depth is output, the less likely it is to be a true value.

[0032] The learning unit 151 calculates an error related to the certainty by assuming that the certainty of the class included in the label is 1.0 and the certainty of the other classes is 0.

[0033] Next, the learning unit 151 stores the learned model in the storage unit 11 (step S13). This ends the learning process.

[0034] 5 is a flow diagram showing an example of the flow of a water depth output process executed by the water depth estimation device 1. The water depth output process is executed in a state where the learned model is stored in the memory unit 11.

[0035] First, the acquisition unit 152 acquires an input image (step S21). The acquisition unit 152 acquires the input image by receiving it via the communication unit 12. For example, the acquisition unit 152 receives an image captured by a security camera as the input image. The acquisition unit 152 may receive an image posted on a website such as an SNS as the input image. The acquisition unit 152 may also acquire an image stored in advance in the storage unit 11 as the input image.

[0036] Next, the estimation unit 153 executes an estimation process (step S22). The estimation process is a process for estimating the flood depth in the input image based on the approximate flood depth and the confidence level output from the trained model when the input image is input to the trained model. The estimation process will be described in detail later.

[0037] Next, the output unit 154 outputs the estimated result of the flood depth in the input image (step S23). The estimated result includes a flooded object area corresponding to the flooded object, as well as the type of flooded object and the flood depth. For example, the output unit 154 outputs the estimated result of the flood depth by displaying on the display unit 13 a flood depth display image in which the flooded object area, the type of flooded object, and the flood depth are superimposed on the input image. The output unit 154 may display the estimated result of the flood depth, or may output the flood depth display image or the estimated result of the flood depth by transmitting it to another device. This completes the flood depth output process.

[0038] 6 is a flow diagram showing an example of the flow of the estimation process. The estimation process is executed in step S22 of the flood depth output process.

[0039] First, the estimation unit 153 acquires the submerged object area corresponding to the submerged object shown in the input image, the class of the submerged object, and the confidence level for the submerged object from the trained model (step S31). The estimation unit 153 inputs the input image to the trained model, and acquires the submerged object area, class, and confidence level output from the trained model. For example, the trained model outputs the confidence level for each class for each submerged object area. In this case, the acquisition unit 152 acquires the class with the highest confidence level for each submerged object area together with the confidence level. The estimation unit 153 refers to the class table T1 to acquire the type of submerged object and the approximate submerged depth corresponding to the class output from the trained model. Hereinafter, the submerged object acquired in step S31 may be referred to as the acquired submerged object.

[0040] Next, the estimation unit 153 extracts some of the acquired submerged objects based on the submerged object area corresponding to each acquired submerged object and the certainty of each acquired submerged object (step S32). For example, the estimation unit 153 extracts acquired submerged objects whose submerged object area area is included in a predetermined range and whose certainty is equal to or greater than a threshold. The predetermined area range is, for example, 0.02% or more and 50% or less of the area of ​​the input image. An acquired submerged object whose submerged object area area is too large is likely to be a false positive, and an acquired submerged object whose submerged object area area is too small is likely to be a distant submerged object that is not suitable for estimating the submerged depth. By setting the predetermined range as described above, acquired submerged objects whose submerged object area area is too large or too small are removed. The certainty threshold is, for example, 0.2. By setting the certainty threshold in this way, acquired submerged objects whose certainty is likely to be a false positive are removed. Hereinafter, the acquired submerged object extracted in step S32 may be referred to as an extracted submerged object.

[0041] Next, the estimation unit 153 selects, from among the extracted submerged objects, the extracted submerged object with the highest degree of certainty as a target submerged object for which the flood depth is to be estimated (step S33).

[0042] Next, the estimation unit 153 selects other extracted submerged objects having a submerged object area overlapping with the submerged object area of ​​the target submerged object as overlapping submerged objects (step S34). For example, the estimation unit 153 calculates the IoU (Intersection over Union) between the submerged object area of ​​the target submerged object and the submerged object area of ​​the other extracted submerged objects. The estimation unit 153 selects extracted submerged objects having an IoU greater than a threshold as overlapping submerged objects whose submerged object areas overlap with the target submerged object by a predetermined ratio or more. The threshold value of IoU is, for example, 0.3.

[0043] Next, the estimation unit 153 determines whether or not at least one extracted submerged object has been selected as an overlapping submerged object (step S35).

[0044] If at least one extracted submerged object is selected as an overlapping submerged object (step S35-Yes), the estimation unit 153 estimates the flood depth in the input image based on the approximate flood depth and the certainty factor of the target submerged object and the overlapping submerged object (step S36). The estimation unit 153 estimates the flood depth of the target submerged object by combining the approximate flood depths of the target submerged object and the overlapping submerged object based on the certainty factor for the target submerged object and the overlapping submerged object. For example, the estimation unit 153 combines the approximate flood depths of the target submerged object and the overlapping submerged object by adding them together while weighting them by the certainty factor.

[0045] If none of the extracted submerged objects is selected as an overlapping submerged object (step S35-No), the estimation unit 153 estimates the submerged depth in the input image based on the approximate submerged depth of the target submerged object (step S37). That is, the estimation unit 153 sets the value of the approximate submerged depth of the target submerged object as the estimated value of the submerged depth of the target submerged object.

[0046] Following step S36 or S37, the estimation unit 153 determines whether or not all of the extracted water-filled objects have been selected as the target water-filled objects or the overlapping water-filled objects (step S38).

[0047] If any of the extracted submersible objects is not selected as the target submersible object or an overlapping submersible object (step S38-No), the estimation process returns to step S33, and the estimation unit 153 selects the extracted submersible object with the highest confidence level as the target submersible object among the extracted submersible objects that have not been selected as the target submersible object or an overlapping submersible object.

[0048] If all of the extracted submersible objects are selected as target submersible objects or overlapping submersible objects (step S38-Yes), the estimation unit 153 generates an estimation result of the submersible depth in the input image (step S39). The estimation unit 153 generates an estimation result including the submersible object area and submersible object type for each of the extracted submersible objects selected as the target submersible objects in step S33, and the submersible object depth estimated in step S36 or S37 for each target submersible object. That is, for a group of extracted submersible objects whose submersible object areas overlap at a predetermined rate or more, the estimation unit 153 sets the type of the target submersible object that is the extracted submersible object with the highest degree of certainty as the type of submersible object included in the estimation result. This ends the estimation process.

[0049] Fig. 7 is a schematic diagram for explaining an example of the contents of the estimation process. In Fig. 7, a road cone RC is shown on the left side of the input image IM1 as a flooded object, and a bicycle BC is shown on the right side as a flooded object. In Fig. 7, five flooded object regions A1 to A5 output from the trained model are shown by dashed lines, and the class and confidence of the acquired flooded object corresponding to each flooded object region are shown.

[0050] 7, the class of the submerged object region A1 is "Cone_10" and the confidence level is 0.1. The acquired submerged object corresponding to the submerged object region A1 is not extracted in step S32 because the confidence level is less than the threshold value. Therefore, the acquired submerged object corresponding to the submerged object region A1 is not included in the estimation result generated in step S39.

[0051] The class of the floodable object area A2 is "Cone_20" and the confidence level is 0.9. The acquired floodable object corresponding to the floodable object area A2 is extracted in step S32 because the confidence level is equal to or greater than the threshold. The extracted floodable object corresponding to the floodable object area A2 is selected as the target floodable object in step S33. Furthermore, since there are no other extracted floodable objects having a floodable object area overlapping with the floodable object area A2, in step S37, the flood depth of the target floodable object corresponding to the floodable object area A2 is estimated to be 20 cm, which is the approximate flood depth value. Therefore, the floodable object corresponding to the floodable object area A2 is included in the estimation result as being a road cone of type and having an estimated flood depth of 20 cm.

[0052] The class of the flooded object region A3 is "Car_10" and the confidence level is 0.2. The class of the flooded object region A4 is "Bicycle_20" and the confidence level is 0.8. The class of the flooded object region A5 is "Bicycle_30" and the confidence level is 0.7. The acquired flooded objects corresponding to the flooded object regions A3 to A5 are all extracted in step S32 because their confidence levels are equal to or higher than the threshold. Of the three extracted flooded objects corresponding to the flooded object regions A3 to A5, the extracted flooded object corresponding to the flooded object region A4, which has the highest confidence level, is selected as the target flooded object in step S33. At this time, the IoU of the flooded object regions A3 and A5 calculated in step S34 are both equal to or higher than the threshold, so the extracted flooded objects corresponding to the flooded object regions A3 and A5 are selected as overlapping flooded objects.

[0053] The estimated flood depth of the target floodable object corresponding to the floodable object area A4 is calculated in step S36 by adding up the approximate flood depths of the target floodable object and the overlapping floodable object weighted by the confidence level. Since the sum of the confidence levels of the floodable object areas A3 to A5 is 1.7, the weighted value of the approximate flood depth of the floodable object area A3 is 10cm x 0.2 / 1.7 = 1.2cm, the weighted value of the approximate flood depth of the floodable object area A4 is 20cm x 0.8 / 1.7 = 9.4cm, and the weighted value of the approximate flood depth of the floodable object area A5 is 30cm x 0.7 / 1.7 = 12.4cm. The flood depth of the target floodable object is estimated to be 23cm, which is the sum of these. Therefore, the floodable object corresponding to the floodable object area A4 is included in the estimation result as a bicycle of type with an estimated flood depth of 23cm.

[0054] Fig. 8 is a diagram showing an example of a flood depth display image IM2 displayed on the display unit 13 in step S23 of the flood depth output process. The example shown in Fig. 8 is an example of a flood depth display image based on the estimation result generated in the estimation process described using Fig. 7.

[0055] The flood depth display image IM2 displays the flooded object area, type of flooded object, and estimated flood depth included in the estimation result generated by the estimation process, superimposed on the input image. In the example shown in Fig. 8, of the flooded object areas A1 to A5 in Fig. 7, flooded object areas A2 and A4 that were selected as target flooded objects and correspond to the flooded objects included in the estimation result are displayed superimposed on the input image. In addition, the type of flooded object and the estimated flood depth are displayed near each flooded object area.

[0056] As described above, in the flood depth estimation device 1, the estimation unit 153 estimates the flood depth of a flooded object shown in an input image by weighting and adding the approximate flood depths of multiple flooded objects whose flooded object areas overlap, output from the trained model when an input image is input to the trained model, according to the certainty of each flooded object. This allows the flood depth estimation device 1 to accurately estimate the flood depth of a flooded object at a finer granularity than the approximate flood depth.

[0057] In general, it is difficult to read the flood depth of a flooded object at a fine level of granularity from an image. Even when learning images in which rough-grained approximate flood depths (e.g., granularity of 10 cm or more) are set as labels are used, the flood depth estimation device 1 can accurately estimate the flood depth at a finer level of granularity than the approximate flood depth.

[0058] The flood depth estimation device 1 may have the following modified examples.

[0059] In the above-described embodiment, the labels of the learning data include classes indicating combinations of the type of submergible object and the approximate flood depth, and the trained model outputs the class of the submergible object. This example is not limited to the above, and the trained model may output the type of submergible object and the approximate flood depth, respectively. In this case, in the learning process, the parameters of the learning model are updated so that the value of an error function including an error related to the type of submergible object and an error related to the approximate flood depth becomes smaller. Even in this way, the flood depth estimation device 1 can accurately estimate the flood depth of the submergible object at a finer granularity than the approximate flood depth.

[0060] In the above-described embodiment, the storage unit 11 stores one trained model that outputs a class indicating a combination of the type of submergible object and the approximate flood depth. The present invention is not limited to this example, and the storage unit 11 may store a plurality of trained models corresponding to each of a plurality of types of submergible objects. In this case, each trained model outputs a confidence level for the approximate flood depth of the corresponding type of submergible object. In addition, in step S31 of the estimation process, the estimation unit 153 acquires the submergible object region corresponding to the submergible object shown in the input image, the class of the submergible object, and the confidence level for the submergible object by synthesizing the approximate flood depth for each type of submergible object obtained by inputting the input image into each trained model. Even in this way, the submergible object's flood depth can be estimated with precision at a finer granularity than the approximate flood depth. However, when a plurality of trained models are used, the degree of learning of each trained model may differ, so it is preferable to use one trained model that outputs a class.

[0061] In the above-described embodiment, in step S36 of the estimation process, the estimation unit 153 estimates the flood depth in the input image based on the approximate flood depth and the certainty factor of the target floodable object and the overlapping floodable object. This example is not limited to the above example, and the estimation unit 153 may further estimate the flood depth based on the type of the target floodable object and the overlapping floodable object. For example, the estimation unit 153 may estimate the flood depth by combining only the approximate flood depth of the overlapping floodable object of the same type as the target floodable object based on the certainty factor. The estimation unit 153 may combine the approximate flood depth of the overlapping floodable object of the same type as the target floodable object by making the weight of the approximate flood depth of the overlapping floodable object of the same type as the target floodable object greater than the weight of the approximate flood depth of the overlapping floodable object of a different type. By doing so, only the approximate flood depth of the overlapping floodable object that is the same type as the target floodable object and therefore unlikely to be erroneously detected is used for the estimation, and the flood depth of the floodable object can be more appropriately estimated. However, since the certainty level of overlapping floodable objects of a different type from the target floodable object is generally low, by appropriately setting the certainty level threshold in step S32, the flood depth can be appropriately estimated even if the type of flood depth is not used.

[0062] In the above embodiment, in step S36, the estimation unit 153 adds the approximate inundation depths of the target floodable object and the overlapping floodable objects weighted by the confidence level. This example is not limited to the above example, and the estimation unit 153 may extract a predetermined number of overlapping floodable objects in descending order of confidence level, and combine the approximate inundation depths of the target floodable object and the overlapping floodable objects based on the confidence level by averaging the approximate inundation depths of the target floodable object and the extracted overlapping floodable objects. This method also makes it possible to estimate the inundation depth of the floodable object at a finer granularity than the approximate inundation depth.

[0063] In the above-described embodiment and modified example, the flood depth is estimated by combining the approximate flood depth of the target floodable object and the approximate flood depth of the overlapping floodable object. The present invention is not limited to such an example, and the approximate flood depth of multiple floodable objects whose floodable object areas do not overlap within the same image may be combined based on the confidence level to estimate the flood depth. If it is known that the shooting range of the input image is a flat area, the flood depth of the floodable object can be estimated in this way with finer granularity than the approximate flood depth and with high accuracy.

[0064] In the above-described embodiment and modified example, the approximate flood depths of multiple submergible objects are weighted and added according to the confidence level to estimate the flood depth. This is not a limitation, and the approximate flood depth may be synthesized using other methods. For example, the average value of the approximate flood depths of multiple submergible objects whose confidence level is equal to or greater than a predetermined value (e.g., 0.7 or greater) may be used as the estimated value of the flood depth, or other calculation methods may be used. The predetermined value is a value greater than the confidence level threshold in step S32.

[0065] The above-mentioned functions of the flood depth estimation device 1 may be realized by multiple devices. For example, the learning process may be executed by a learning device that communicates with the flood depth estimation device 1. In this case, the learning unit 151 of the flood depth estimation device 1 receives the trained model from the learning device and stores it in the memory unit 11.

[0066] It should be understood by those skilled in the art that various changes, substitutions, and modifications can be made to the present invention without departing from the scope of the present invention. For example, the above-described embodiments and modifications may be appropriately combined within the scope of the present invention. [Explanation of symbols]

[0067] 1 Flood depth estimation device 151 Learning Department 152 Acquisition Department 153 Estimation Department 154 Output section

Claims

1. A memory unit that stores a trained model that has been trained to output, when an image showing a flooded object is input, an approximate flood depth of the flooded object shown in the image and a certainty factor for the flooded object; an estimation unit that estimates the flood depth in an input image based on an approximate flood depth and a confidence level output from the trained model when the input image is input to the trained model; A flood depth estimation device comprising:

2. The estimation unit estimates the flood depth by synthesizing approximate flood depths of the multiple floodable objects output from the trained model when an input image representing multiple floodable objects is input to the trained model based on a certainty factor for each floodable object. The flood depth estimation device according to claim 1.

3. The trained model is trained to further output a submerged object region corresponding to a submerged object represented in the input image; The estimation unit estimates the flood depth by synthesizing approximate flood depths of multiple floodable objects whose floodable object regions overlap each other, based on a certainty factor for each floodable object, output from the trained model. The flood depth estimation device according to claim 2.

4. The estimation unit estimates the flood depth by weighting and adding up the approximate flood depths of the multiple flooded objects whose flooded object regions overlap with each other by the confidence level for each flooded object. The flood depth estimation device according to claim 3.

5. The trained model is trained to further output the type of submerged object represented in the input image, The estimation unit estimates a type of the submerged object shown in the input image based on the type of the submerged object and a certainty factor for the submerged object output from the trained model. A flood depth estimation device according to any one of claims 1 to 4.

6. A trained model is stored that is trained to output, when an image showing a flooded object is input, an approximate flood depth of the flooded object shown in the image and a certainty factor for the flooded object; When an input image is input to the trained model, the flood depth for the input image is estimated based on the approximate flood depth and the confidence level output from the trained model. A method for estimating flood depth comprising the steps of:

7. A computer program having a memory unit that stores a trained model that has been trained to output, when an image showing a flooded object is input, an approximate flood depth of the flooded object shown in the image and a certainty factor for the flooded object, When an input image is input to the trained model, the flood depth for the input image is estimated based on the approximate flood depth and the confidence level output from the trained model. A program for causing the computer to execute the above steps.

Images