Collapse risk estimation apparatus

Abstract

To provide a collapse risk estimation apparatus which can attain efficiency improvement or higher accuracy.SOLUTION: An image capturing unit 2 captures an object and outputs an optical captured image. This optical captured image is acquired in optical captured image acquisition processing 4. The risk of elements (element risk) related to collapse is estimated based on the acquired optical captured image of the object in collapse element risk estimation processing 6. A point group data measurement unit 8 measures the object by the irradiation of a laser or the like, and outputs three-dimensional point group data. This three-dimensional point group data is acquired in three-dimensional point group data acquisition processing 10. The risk of the elements (element risk) related to the collapse is estimated based the acquired three-dimensional point group data of the object in collapse element risk estimation processing 12. The collapse risk of the object is estimated based on the element risk based on the optical captured image and the element risk based on the three-dimensional point group data in collapse risk estimation processing 14.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a device for estimating the degree of risk of collapse of a feature or the like, and to devices related thereto. [Background technology]

[0002] The risk of collapse of objects such as land features has been estimated. For example, Patent Document 1 discloses the following building structure diagnostic system.

[0003] This system extracts damaged areas such as cracks based on images of buildings and structures and marks them on the images, making it easier for workers to visually assess the extent of the damage.

[0004] Moreover, Patent Document 2 discloses a diagnostic method for determining the risk of concrete peeling based on an infrared image of an object.

[0005] These systems and methods can improve the efficiency of diagnostic work. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Patent Publication No. 2021-51022 [Patent Document 2] Patent Publication No. 2016-6398 Summary of the Invention [Problem to be solved by the invention]

[0007] However, the above-mentioned conventional techniques cannot estimate the risk of collapse of an object.

[0008] Furthermore, it was not possible to estimate the extent of the impact that the collapse would have on roads and other infrastructure.

[0009] Furthermore, in order to determine the risk of collapse (or for other purposes), it is necessary to identify the type of object itself, but the above-mentioned conventional techniques did not provide a concrete method for doing so.

[0010] An object of the present invention is to provide an apparatus that can solve any of the above problems and achieve greater efficiency or higher accuracy. [Means for solving the problem]

[0011] Independently applicable features of the present invention are listed below.

[0012] (1)(2) The collapse risk estimation device of the present invention acquires an optical image of an object captured by an imaging unit, acquires three-dimensional point cloud data of the object measured by a point cloud data measurement unit, estimates an element risk of elements related to the collapse of the object based on appearance image features including dirt on the object in the optical image, estimates an element risk of elements related to the collapse of the object based on appearance shape features including the inclination of the object in the three-dimensional point cloud data, and estimates a risk of the object collapsing based on the element risk based on the appearance image features and the element risk based on the appearance shape features.

[0013] Since the degree of risk is determined based on both the optically captured image and the three-dimensional point cloud data, estimation can be performed with greater accuracy.

[0014] (3) The collapse risk estimation device of the present invention is characterized in that the imaging unit and the point cloud data measurement unit are mounted on the same vehicle so that the imaging and measurement positions are linked and the same object can be imaged and measured simultaneously while driving.

[0015] Therefore, optically captured images and three-dimensional point cloud data can be obtained efficiently while traveling.

[0016] (4) The collapse risk estimation device of the present invention is characterized in that it extracts an object from an optically captured image, obtains three-dimensional point cloud data at measurement positions near the imaging position from which the object is extracted, and, if the extracted object is present in the three-dimensional point cloud data, estimates the risk of collapse of the object.

[0017] Therefore, the possibility of erroneously recognizing the object can be reduced.

[0018] (5) The collapse risk estimation device of the present invention is characterized in that, for objects near a road having a collapse risk of a predetermined level or higher, the height is calculated based on the three-dimensional point cloud data, the shape of the object is determined based on the three-dimensional point cloud data, the possible collapse direction of the object is calculated based on the shape, the estimated collapse range on the road is estimated based on the possible collapse direction and the height, and the possibility of road closure is estimated based on the estimated collapse range on the road.

[0019] Therefore, it is possible to estimate the possibility of a road being blocked by an object that is at high risk of collapse.

[0020] (6) The collapse risk estimation device of the present invention is characterized in that it estimates the type of object based on the optically captured image, estimates the type of object based on the three-dimensional point cloud data, and finally estimates the type of object based on the estimation of the type of object based on the optically captured image and the estimation of the type of object based on the three-dimensional point cloud data, and estimates the collapse risk taking into account the finally estimated type of object.

[0021] Therefore, the risk of collapse can be estimated with greater accuracy.

[0022] (7)(8) The object type estimation device of the present invention acquires an optically captured image of an object captured by an imaging unit, acquires three-dimensional point cloud data of the object captured by a point cloud data measurement unit, estimates the type of the object based on the optically captured image, estimates the type of the object based on the three-dimensional point cloud data, and finally estimates the type of the object based on the estimation of the type of the object based on the optically captured image and the estimation of the type of the object based on the three-dimensional point cloud data.

[0023] Therefore, the type of the object can be estimated more accurately.

[0024] (9) The object type estimation device of the present invention is characterized in that when the type of object estimated based on an optically captured image matches the type of object estimated based on the three-dimensional point cloud data, the type of object is finally estimated as the corresponding type.

[0025] Therefore, the type of the object can be estimated more accurately.

[0026] (10)(11) The collapse area estimation device of the present invention acquires three-dimensional point cloud data of features near a road measured by a point cloud data measurement unit, calculates the height of the feature based on the three-dimensional point cloud data, determines the shape of the feature based on the three-dimensional point cloud data, calculates the possible collapse direction of the feature based on the shape, and estimates the estimated collapse area on the road based on the possible collapse direction and the height.

[0027] Therefore, the expected collapse range can be estimated with high accuracy depending on the shape of the object.

[0028] (12) The collapse range estimation device according to the present invention is characterized in that it estimates the possibility of road closure based on the estimated collapse range on the road.

[0029] Therefore, the possibility of a road closure can be estimated with high accuracy.

[0030] In the embodiment, the "optical image acquisition process" corresponds to step S2 and step S51.

[0031] In the embodiment, the "three-dimensional point cloud data acquisition process" corresponds to step S12 and step S53.

[0032] In the embodiment, the "collapse element risk estimation process" j corresponds to steps S6 and S13.

[0033] In this embodiment, the "collapse risk estimation process" corresponds to step S14.

[0034] In the embodiment, the "target document tentative estimation process" corresponds to steps S52 and S54.

[0035] In this embodiment, the "object type estimation process" corresponds to step S56.

[0036] In the embodiment, the "point cloud data recording process" corresponds to step S101.

[0037] In this embodiment, step S102 corresponds to the "feature height calculation process."

[0038] In this embodiment, step S103 corresponds to the "collapse potential direction estimation process."

[0039] In this embodiment, the "collapse range estimation process" corresponds to step S104.

[0040] The term "apparatus" is a concept that includes not only one consisting of one computer, but also one consisting of multiple computers connected via a network, etc. Therefore, when the means of the present invention (or a part of the means) is distributed among multiple computers, these multiple computers correspond to the apparatus.

[0041] The term "program" is a concept that includes not only programs that can be executed directly by a CPU, but also programs in source format, compressed programs, encrypted programs, programs that work in conjunction with an operating system to perform their functions, and the like. [Brief description of the drawings]

[0042] [Figure 1] 2 shows the functional configuration of a collapse risk estimation device. [Diagram 2] This is the hardware configuration. [Diagram 3] FIG. [Figure 4] 13 is a flowchart of a collapse risk estimation program. [Diagram 5] 13 is a flowchart of a collapse risk estimation program. [Figure 6] 1 is an example of an optically captured image. [Figure 7] 1 is an example of three-dimensional point cloud data. [Figure 8A] 13 is a diagram showing a block wall recognized in an optically captured image. FIG. [Figure 8B] 13 is a diagram for explaining the process of determining whether or not the blocks are the same block walls. FIG. [Figure 9] This is an item for judging the risk of collapse based on optically captured images. [Figure 10] This is an example of a see-through block and a block with a cap. [Figure 11] This shows the judgment results for each frame with respect to the judgment items. [Figure 12] This is a criterion for determining the risk of collapse based on three-dimensional point cloud data. [Figure 13] This is a map showing block walls that are at risk of collapsing. [Figure 14] 13 illustrates a functional configuration of an object type estimation device according to a second embodiment. [Figure 15] 13 is a flowchart of an object type estimation process. [Figure 16]13 illustrates a functional configuration of a collapse extent estimation device according to a third embodiment. [Figure 17] 13 is a flowchart of a collapse range estimation process. [Figure 18] FIG. 13 is a diagram for explaining collapse range estimation. [Figure 19] FIG. 13 is a diagram for explaining collapse range estimation. [Figure 20] FIG. 13 is a diagram for determining whether or not a road is closed. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0043] 1. First embodiment 1.1 Overall process flow 1 shows the overall configuration of a collapse risk estimation device 20 according to one embodiment of the present invention. An imaging unit 2 images an object and outputs an optically captured image. In an optically captured image acquisition process 4, this optically captured image is acquired. In a collapse element risk estimation process 6, the risk of elements related to collapse (element risk) is estimated based on the acquired optically captured image of the object.

[0044] A point cloud data measurement unit 8 measures the object by irradiating it with a laser or the like, and outputs three-dimensional point cloud data. This three-dimensional point cloud data is acquired in a three-dimensional point cloud data acquisition process 10. In a collapse element risk estimation process 12, the risk of elements related to collapse (element risk) is estimated based on the acquired three-dimensional point cloud data of the object.

[0045] The collapse risk estimation process 14 estimates the collapse risk of the object based on the element risk based on the optically captured image and the element risk based on the three-dimensional point cloud data.

[0046] In this way, the risk of collapse of an object is estimated based on the optically captured image and the three-dimensional point cloud data, so that the risk of collapse can be estimated more accurately.

[0047] 1.2 Hardware Configuration 2 shows the hardware configuration of the collapse risk estimation device 20. To the CPU 30, a memory 32, a display 34, an SSD 36, a DVD-ROM drive 38, a keyboard / mouse 40, and a communication circuit 42 are connected.

[0048] The communication circuit 42 is a circuit for connecting to a network such as the Internet. An operating system 44 and a collapse risk estimation program 46 are recorded in the SSD 36. The collapse risk estimation program 46 performs its functions in cooperation with the operating system 44. These programs were recorded on a DVD-ROM 52 and installed in the SSD 36 via the DVD-ROM drive 38.

[0049] The SSD 36 also stores optically captured images 48 and three-dimensional point cloud data 50. These images were captured and measured while traveling using a three-dimensional laser measuring device and a digital camera mounted on a vehicle or the like. That is, in this embodiment, the images captured using an MMS (Mobile Mapping System) are used.

[0050] 3 shows a vehicle 60 used in the mobile mapping system. Optical cameras 62, 64, and a LiDAR 66 are mounted on the ceiling of the vehicle 60. The optical camera 62 is installed so as to capture an image of the right diagonal front with respect to the traveling direction of the vehicle 60, and the camera 64 is installed so as to capture an image of the left diagonal front with respect to the traveling direction of the vehicle 60. Images captured while the vehicle 60 is traveling are stored in a recording medium such as an SSD (not shown).

[0051] The LiDAR 66 outputs three-dimensional point cloud data obtained by measuring the surface shape of an object based on the time it takes for the laser to be reflected. This three-dimensional point cloud data is stored in a recording medium such as an SSD (not shown).

[0052] 1.3 Collapse risk estimation process Fig. 4 shows a flowchart of the collapse risk estimation program 46. The CPU 30 acquires the optically captured image 48 and the three-dimensional point cloud data 50 from the SSD 36. These data are data acquired while the traveling vehicle 60 shown in Fig. 3 is traveling (data acquired by the MMS) and recorded in the SSD 36 via a portable recording medium.

[0053] In this embodiment, an optically captured image 48 is obtained as a video including several frames per second. Each frame of the image is recorded in association with the latitude and longitude of the point where the measurement was performed.

[0054] 6A and 6B show examples of optically captured images, in which FIG. 6A is an image of the left front captured by the camera 64, and FIG.

[0055] The 3D point cloud data is recorded as 3D data represented by latitude and longitude by integrating the data measured at each point in time. Figure 7 shows the recorded 3D point cloud data.

[0056] The CPU 30 acquires one frame of an optically captured image (step S2). For example, the leftmost image in FIG. 6A is acquired. Next, the CPU 30 extracts a block wall, if any, from the acquired optically captured image (step S3). For this extraction process, for example, a trained model in which an estimation model based on the YOLACT algorithm is trained using an image of a block wall can be used. As a result, the block wall is extracted surrounded by a frame, as shown in FIG. 8A.

[0057] If the block wall is not found, the CPU 30 performs the same process as above for the next frame. If the block wall is found, it is determined whether or not the block wall has newly appeared (step S5).

[0058] An example of determining whether or not a block wall is new is shown in Figures 8B to 8G. Starting from Figure 8B, six frames are shown in sequence, one frame at a time, up to Figure 8G. In Figure 8C, the starting end S of the block wall (this can be determined as the left end on the extracted image of the block wall. In the case of the image taken by the camera on the right, it will be the right end) appears, so it can be determined that this block wall is a newly appeared one.

[0059] The frames progress through Figures 8D, E, and 8F, and at the end of Figure 8F, the end E of the block wall (which can be determined as the right end on the extracted image of the block wall. In the case of the image taken by the camera on the right side, it would be the left end) has disappeared. Therefore, it can be determined that this block wall is part of the frame images of Figures 8C to 8F.

[0060] If the CPU 30 determines that it is a new block wall, it assigns a new block ID to the block wall (step S8). If the CPU 30 determines that it is not a new block wall, it determines that the block wall of the previous frame is a continuation of the block wall.

[0061] The CPU 30 extracts an image of only the block wall portion and calculates the elemental risk level (step S6).

[0062] In this embodiment, the element risk level is calculated based on items such as those shown in Fig. 9. Items used for judgment include whether the exterior of the block wall has moss, dirt, or partial damage, whether there are decorative blocks with holes in them as shown in Fig. 10A, and whether a coping is provided at the top of the block wall as shown in Fig. 10B.

[0063] Whether or not there is moss on a block wall is estimated by inputting an image of the block wall portion into a trained estimation model generated as follows: In this embodiment, the estimation model is trained by being given optically captured images of a block wall with moss and optically captured images of a block wall without moss, and a trained estimation model is generated.

[0064] Similarly, for cases where there is dirt, a watermark block, a capping, or partial damage, a trained estimation model is generated and estimation is performed based on this.

[0065] As described above, when the presence or absence of an item is determined for one frame image, the next frame is read out and the same process is repeated (steps S1 to S6).

[0066] As mentioned above, a single block wall is imaged over multiple frames, so whether or not it falls under each of the above items is recorded for each frame, and if each item applies to a specified percentage or more of the total frames (for example, 50%), the item is considered to apply to the entire frame in the calculation.

[0067] Figure 11 shows an example of the judgment of each item in each frame for the block wall with block ID "0126." In this example, "moss" is "present" in 3 out of 5 frames, so it is judged to be "present" overall. Similarly, "dirt" is "present" overall, "watermark blocks" is "present" overall, "cap rails" are "absent" overall, and "partial damage" is "present" overall.

[0068] Based on the above-mentioned judgment result, the CPU 30 calculates the danger level according to FIG. 9. If there is moss on the exterior of the block wall, it is highly likely that deterioration is progressing, so the danger level is increased by +1. If there is dirt, it is also highly likely that deterioration is progressing, so the danger level is increased by +1. If there is partial damage, it is highly likely that there is a problem with the strength, so the danger level is increased by +1. If there is a decorative block with a hole as shown in FIG. 10A, there is a high risk of collapse because no reinforcing bars are passed through the interior, so the danger level is increased by +2. Note that, if a coping is provided on the top of the block wall as shown in FIG. 10B, rainwater is prevented from entering the interior, so the danger level is decreased by -1. With optically captured images, it is easier to judge such appearances compared to three-dimensional point cloud data.

[0069] In the example shown in Figure 11, the element risk level of this block wall is 3 points. Although the image taken by the left camera shown in Figure 6A has been described above, the same processing is also carried out on the image taken by the right camera shown in Figure 6B.

[0070] In this manner, the CPU 30 determines the elemental risk level of the block wall based on the optically captured image.

[0071] Furthermore, the CPU 30 acquires the imaging position (latitude and longitude of the vehicle when the block wall is imaged) for each block wall (step S10). Next, the CPU 30 determines whether or not there is a block wall near the road on the imaging direction side of the position (latitude and longitude) in the three-dimensional point cloud data (step S11).

[0072] The process of extracting block walls from 3D point cloud data can be achieved using an algorithm similar to YOLACT for camera images. In other words, it can be achieved by using an estimation model trained on a large amount of 3D point cloud data including block walls and 3D point cloud data (ground truth data) that encloses the block walls with a frame.

[0073] If the block wall is not found in the three-dimensional point cloud data, it is determined that the block wall identified in the optically captured image was incorrect, and processing for that block wall ends.

[0074] If the exterior wall of a house has a lattice pattern, it may be mistaken for a block wall in an optically captured image. In this embodiment, by checking the three-dimensional point cloud data (three-dimensional point cloud data can clearly distinguish whether it is a house wall or a fence based on its three-dimensional shape), such a misjudgment can be prevented.

[0075] When a block wall is found in the three-dimensional point cloud data, the CPU 30 acquires the three-dimensional point cloud data of the block wall (step S12). Based on the three-dimensional point cloud data of the block wall, the element risk is calculated (step S13).

[0076] In this embodiment, the element risk level is calculated based on items such as those shown in Fig. 12. If the height of the block wall is 2m or more, the risk level is increased by +1. If the inclination to the horizontal direction is 5 degrees or more (average inclination), the risk level is increased by +3. If there is a significant deformation (protruding), the risk level is increased by +1. The above items are extremely difficult to judge using optical images, but can be calculated relatively easily using 3D point cloud data.

[0077] In this manner, the element risk level can be calculated based on the three-dimensional point cloud data.

[0078] Next, the CPU 30 calculates a risk level by adding up the element risk levels based on the optical images and the three-dimensional point cloud data, and if the value is equal to or greater than a predetermined value (for example, 5), it determines that there is a risk of collapse.

[0079] The above process is repeated for each block wall, the degree of danger for each wall is calculated, and the presence or absence of a risk of collapse is determined (steps S9 to S15).

[0080] 1.4 Variations (Other) (1) In the above embodiment, the recorded optically captured images and three-dimensional point cloud data are imported into the collapse risk estimation device 20. However, the camera and the three-dimensional point cloud data measuring device may be directly connected to the collapse risk estimation device 20.

[0081] (2) In the above embodiment, the collapse risk estimation device 20 is configured as an independent device. However, it may also be configured as a server device to transmit optically captured images and three-dimensional point cloud data from a terminal device and receive the judgment results.

[0082] (3) In the above embodiment, the trained estimation model is used to determine whether or not the judgment items in Fig. 9 are met. However, the judgment may be made based on the characteristics of the image.

[0083] (4) In the above embodiment, a determination is made for each frame of the optically captured image as to whether or not there is a match with the items, and the determination is made by tallying up all frames that include the block wall.

[0084] However, an image of the entire block wall may be generated based on the block wall image in each frame, and a judgment as to whether or not the item matches may be made based on this image.

[0085] (5) In the above embodiment, the element risk level based on the optically captured image and the element risk level based on the three-dimensional point cloud data are summed to calculate the risk level.

[0086] However, the risk may be calculated by taking a simple average or a weighted average of both.

[0087] (6) In the above embodiment, the degree of collapse risk is calculated for each block wall. Based on this, the degree of collapse risk (and whether or not the blocks have collapsed) for blocks in each area may be listed. Furthermore, as shown in Figure 13, a hazard map may be output on a map showing block walls D that are at risk of collapse.

[0088] (7) In the above embodiment, the risk of collapse of a block wall is judged.

[0089] However, the present invention can be similarly applied to other structures such as fences made of wires. In the case of a fence, the element risk level can be determined by optically captured images to see if the wire is frayed or partially broken, and the element risk level can be determined by 3D point cloud data to see if the height is above a predetermined level, if the inclination is above a predetermined level, etc.

[0090] (8) In the above embodiment, a LiDAR is used as a measuring device for measuring the distance to each point on the surface of an object and generating three-dimensional point cloud data. However, three-dimensional point cloud data may be generated using a stereo camera.

[0091] Also, instead of measuring the distance to each point on the surface of the object, it is possible to use SFM (Structure from Motion) to generate three-dimensional point cloud data of the object from multiple images taken from different angles, and use this data.

[0092] (9) In the above embodiment, the risk of collapse of block walls, fences, etc. is estimated. However, the present invention can also be applied to other objects such as utility poles.

[0093] In addition, the estimation of the risk of collapse is changed depending on the type of object, and therefore it is important to accurately identify the type of object. From this perspective, the identification of the object type according to the second embodiment may be used in this embodiment.

[0094] (10) The above-described embodiments and modifications can be implemented in any combination.

[0095] 2. Second embodiment 2.1 Overall process flow 14 shows the functional configuration of an object type estimation device according to the second embodiment. The imaging unit 2 captures an object and outputs an optically captured image. In an optically captured image acquisition process 4, this optically captured image is acquired. In an object type tentative estimation process 96, the type of the object is estimated based on the acquired optically captured image.

[0096] A point cloud data measurement unit 8 measures the object by irradiating it with a laser or the like, and outputs three-dimensional point cloud data. This three-dimensional point cloud data is acquired in a three-dimensional point cloud data acquisition process 10. In a provisional object type estimation process 92, the type of the object is estimated based on the acquired three-dimensional point cloud data.

[0097] In the object document estimation process 94, if the estimation of the object type based on the optically captured image and the estimation of the object type based on the 3D point cloud data match, the object is finally estimated. If they do not match, the object is not identified.

[0098] In this way, since the type of the object is estimated based on the optically captured image and the three-dimensional point cloud data, the type of the object can be more accurately identified.

[0099] 2.2 Hardware Configuration The hardware configuration is the same as that of the first embodiment, except that the SSD 36 stores an object type estimation program instead of the collapse risk estimation program 46.

[0100] 2.3 Feature type estimation process A flowchart of the feature type estimation program is shown in Fig. 15. The CPU 30 acquires an optically captured image showing an object (step S51). The CPU 30 estimates what the object is based on this image (step S52). For example, such an estimation can be performed by learning an estimation model using the YOLACT algorithm.

[0101] Next, the CPU 30 acquires three-dimensional point cloud data showing the same object as the optically captured image (step S53). The CPU 30 estimates what the object is based on the three-dimensional point cloud data (step S54). For this, too, a trained estimation model can be used, as in the first embodiment.

[0102] Next, the CPU 30 judges whether the estimation result of the object type based on the optically captured image matches the estimation result of the object type based on the three-dimensional point cloud data (step S55). If they match, the estimation result is determined as the object type (step S56). If they do not match, the object type is not identified.

[0103] 2.4 Modifications (Other) (1) In the above embodiment, if the estimation results do not match, the type of the object is not identified. However, it is also possible to output both the estimation result of the object type based on the optically captured image and the estimation result of the object type based on the 3D point cloud data. In this case, it is preferable to clearly indicate that the two estimation results do not match.

[0104] (2) The above-described embodiments and modifications may be implemented in any combination with each other. In addition, the above-described embodiments and modifications may be implemented in any combination with other embodiments and their modifications.

[0105] 3. Third embodiment 3.1 Overall process flow Fig. 16 shows the functional configuration of a collapse extent estimation device according to the third embodiment. In point cloud data acquisition process 102, three-dimensional point cloud data of the feature is acquired. In feature height calculation process 104, the height of the feature is calculated. In potential collapse direction estimation process 106, the potential collapse direction of the feature is determined based on the shape of the feature. In collapse extent estimation process 108, the potential collapse extent is estimated based on the potential collapse direction and height.

[0106] Therefore, the extent of collapse due to features can be estimated in advance based on the three-dimensional point cloud data.

[0107] 3.2 Hardware Configuration The hardware configuration is the same as that of the first embodiment, except that a collapse range estimation program is recorded in the SSD 36 instead of the collapse risk estimation program 46.

[0108] 3.3 Collapsed area estimation process A flowchart of the collapsed area estimation program is shown in Fig. 17. The CPU 30 acquires three-dimensional point cloud data in the vicinity of a road (step S101). For example, the three-dimensional point cloud data as shown in Fig. 7 is acquired.

[0109] The CPU 30 extracts features such as utility poles and block walls from the three-dimensional point cloud data. The feature extraction can be performed using a trained estimation model that has been trained with an estimation model having the same algorithm as YOLACT for normal images.

[0110] Next, the height of each feature from the road surface is calculated (step S102). Note that which is the road surface can be determined using a learned estimation model that has been trained using a cross simulation method or deep learning (learning is performed using 3D point cloud data including the road surface and 3D point cloud data labeled with the road surface (ground truth data) as training data). The height of features such as utility poles and block walls can be easily calculated from the coordinate difference from the road surface.

[0111] Next, the CPU 30 estimates the collapse potential direction for each feature (step S103). In this embodiment, the collapse potential direction is obtained by carrying out the following process.

[0112] First, calculate the principal component direction of the distribution of 3D point cloud data for each feature (principal component analysis). If the direction of principal component M is up and down as shown in the side view of Figure 18A, it is estimated to be a shape like a utility pole, and the possible directions for its collapse are 360 ​​degrees in all directions as shown in Figure 18B.

[0113] Therefore, a circle ARE1 having a radius of the collapse distance according to the height (height x a predetermined value (for example, 2.5)) can be estimated as the collapse range.

[0114] On the other hand, for features such as block walls, the direction of principal component M is horizontal, as shown in the side view of Figure 19A. In this case, the possible collapse direction is set to a direction perpendicular to the principal component direction, as shown in the plan view of Figure 19B. Therefore, the collapse range can be estimated as a rectangle ARE2 with a collapse distance on both sides according to the height (height x a predetermined value (e.g. 2.5)). Note that if the height of the block wall changes along the way, the collapse range will also change accordingly.

[0115] The CPU 30 superimposes the collapse areas ARE1, ARE2 calculated as described above on the road surface LS. For example, they can be superimposed as shown in Fig. 20. When the road surface is completely blocked, as shown in ARE1, the CPU 30 determines that there is a possibility that the road will be closed. When there is a part that is not blocked, as in ARE2, the CPU 30 determines whether this width FF is passable for people (or vehicles), and if it is not passable, determines that there is a possibility that the road will be closed (step S105).

[0116] In this manner, it is possible to estimate the possibility of road closure when an object collapses due to an earthquake, etc. The CPU 30 may output a map indicating the locations where road closures are likely to occur as a hazard map.

[0117] 3.4 Modifications (Other) (1) In the above embodiment, the extent of collapse is estimated for all features, but it is also possible to estimate the extent of collapse only for features that are at high risk of collapse (for example, only for features that are determined to have the possibility of collapse by the first embodiment).

[0118] (2) The above-described embodiments and modifications may be implemented in any combination with each other. In addition, the above-described embodiments and modifications may be implemented in any combination with other embodiments and their modifications.

Claims

1. Acquire an optically captured image of the object captured by the imaging unit; Acquire three-dimensional point cloud data of the object measured by a point cloud data measurement unit; Estimating an element risk of an element related to the collapse of the object based on an appearance image feature including dirt of the object in the optically captured image; estimating an element risk of an element related to the collapse of the object based on an external shape feature including an inclination of the object in the three-dimensional point cloud data; A collapse risk estimation device that estimates a collapse risk of the object based on an element risk based on the appearance image features and an element risk based on the appearance shape features.

2. A collapse risk estimation program for implementing a collapse risk estimation device by a computer, Acquire an optically captured image of the object captured by the imaging unit; Acquire three-dimensional point cloud data of the object measured by a point cloud data measurement unit; Estimating an element risk of an element related to the collapse of the object based on an appearance image feature including dirt of the object in the optically captured image; estimating an element risk of an element related to the collapse of the object based on an external shape feature including an inclination of the object in the three-dimensional point cloud data; A collapse risk estimation program having instructions for estimating a collapse risk of the object based on an element risk based on the appearance image features and an element risk based on the appearance shape features.

3. In the device of claim 1 or the program of claim 2, An apparatus or program characterized in that the imaging unit and the point cloud data measurement unit are mounted on the same vehicle so that the imaging and measurement positions are linked and the same object can be simultaneously imaged and measured while driving.

4. In the device or program of claim 3, Extracting an object from the optically captured image; Acquire three-dimensional point cloud data at a measurement position near the imaging position where the object is extracted; If the extracted object is present in the three-dimensional point cloud data, the device or program estimates the degree of risk of collapse of the object.

5. In the device of claim 1 or the program of claim 2, Calculating heights of objects near roads that have a predetermined collapse risk level or higher based on the three-dimensional point cloud data; determining a shape of the object based on the three-dimensional point cloud data, and calculating a direction in which the object may collapse based on the shape; estimating an estimated collapse range on the road based on the possible collapse direction and the height; A device or program for estimating the possibility of road closure based on an estimated collapse area on the road.

6. In the device of claim 1 or the program of claim 2, Estimating a type of the object based on the optically captured image; Estimating a type of the object based on the three-dimensional point cloud data; A final estimation of the type of the object is performed based on the estimation of the type of the object based on the optically captured image and the estimation of the type of the object based on the three-dimensional point cloud data; A device or program for estimating the risk of collapse taking into consideration the type of the object finally estimated.

7. Acquire an optically captured image of the object captured by the imaging unit; Acquire three-dimensional point cloud data of the object imaged by a point cloud data measurement unit; Estimating a type of the object based on the optically captured image; Estimating a type of the object based on the three-dimensional point cloud data; An object type estimation device that finally estimates the type of the object based on an estimation of the type of the object based on the optically captured image and an estimation of the type of the object based on the three-dimensional point cloud data.

8. A collapse risk estimation program for implementing a collapse risk estimation device by a computer, Acquire an optically captured image of the object captured by the imaging unit; Acquire three-dimensional point cloud data of the object imaged by a point cloud data measurement unit; Estimating a type of the object based on the optically captured image; Estimating a type of the object based on the three-dimensional point cloud data; A collapse risk estimation program having instructions for finally estimating the type of object based on an estimation of the type of object based on the optically captured image and an estimation of the type of object based on the three-dimensional point cloud data.

9. In the device of claim 7 or the program of claim 8, When the type of object estimated based on the optically captured image matches the type of object estimated based on the three-dimensional point cloud data, the type of the object is finally estimated as that type.

10. Acquire three-dimensional point cloud data of features near the road measured by the point cloud data measurement unit; Calculating the height of the feature based on the three-dimensional point cloud data; determining a shape of the feature based on the three-dimensional point cloud data, and calculating a direction in which the feature may collapse based on the shape; A collapse range estimation device that estimates an estimated collapse range on the road based on the possible collapse direction and the height.

11. A road closure possibility estimation program for implementing a collapse range estimation device by a computer, comprising: Acquire three-dimensional point cloud data of features near the road measured by the point cloud data measurement unit; Calculating the height of the feature based on the three-dimensional point cloud data; determining a shape of the feature based on the three-dimensional point cloud data, and calculating a direction in which the feature may collapse based on the shape; estimating an estimated collapse range on the road based on the possible collapse direction and the height; A collapse extent estimation program that estimates the possibility of road closure based on an estimated collapse extent of the road.

12. In the device of claim 10 or the program of claim 11, A device or program for estimating the possibility of road closure based on an estimated collapse area on the road.

Images