Information processing device, status inspection system, information processing method and program

Abstract

To efficiently evaluate the state of a structure compared to before.SOLUTION: An evaluation system receives photographed image data obtained by a photographing device photographing a structure, and sensor data acquired by a three-dimensional sensor (for example, angle sensor) according to the photographing of the photographed image data performed by the photographing device, and based on the received photographed image data and sensor data, detects a shape including three-dimensional information of the structure. The evaluation system creates an evaluation report indicating a result of evaluation of the state of the structure based on the photographed image data and shape data showing the detected shape.SELECTED DRAWING: Figure 20

Description

[Technical field]

[0001] The present invention relates to an evaluation system, a state inspection system, an evaluation method, and a program. [Background technology]

[0002] In recent years, the deterioration of road structures that have been in place for a long time has been remarkable, and the need for inspection and maintenance of such structures has increased. Therefore, road structures that may cause damage to vehicles or people while they are in operation are inspected periodically, and the inspection results are reported to the national or local governments (see Patent Document 1).

[0003] Furthermore, with the development of infrastructure, many road earthwork structures (slopes) have come into existence on roads in various locations. Therefore, in addition to road surfaces, tunnels, and bridges, slopes are also subject to regular inspections, and inspections are conducted on the progress of deterioration and damage to structures such as mortar spraying, slope frames, retaining walls, anchors, and rockfall protection fences. Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional inspections of structures are conducted by visual inspection by experts, and for example, it is difficult to grasp the displacement or deformation that is a precursor to the collapse of a slope by human visual inspection or by using images, making it difficult to quantitatively evaluate the progress of deterioration and damage. Therefore, the conventional method has the problem that it is not possible to efficiently evaluate the deterioration and damage of structures. [Means for solving the problem]

[0005] In order to solve the above-mentioned problems, the invention of claim 1 is an evaluation system for evaluating the condition of a structure, comprising: a receiving means for receiving image data of the structure photographed by an imaging device and sensor data acquired by a three-dimensional sensor in response to the photographing of the image data by the imaging device; a detection means for detecting a shape including three-dimensional information of the structure based on the received photographed image data and the sensor data; and a report generation means for generating an evaluation report showing an evaluation result of the condition of the structure based on shape data showing the detected shape and the photographed image data. Effect of the Invention

[0006] According to the present invention, the condition of a structure can be evaluated more efficiently than in the past. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram showing an example of an overall configuration of a state inspection system according to an embodiment; [Diagram 2] FIG. 13 is a diagram showing an example of a state in which a slope condition is inspected using the mobile body system according to the embodiment. [Diagram 3] FIG. 2 is a diagram illustrating an example of a hardware configuration of a data acquisition device. [Figure 4] FIG. 2 is a diagram illustrating an example of a hardware configuration of an evaluation device and a data management device. [Diagram 5] FIG. 2 is a diagram illustrating an example of a functional configuration of a state inspection system. [Figure 6] FIG. 13 is a conceptual diagram illustrating an example of a status type management table. [Figure 7] FIG. 1A is a conceptual diagram showing an example of an acquired data management table, and FIG. 1B is a conceptual diagram showing an example of a processed data management table. [Figure 8] FIG. 11 is a sequence diagram showing an example of a data acquisition process using a mobile system. [Figure 9] FIG. 2 is a diagram for explaining a captured image acquired by a mobile system. [Figure 10] FIG. 11 is a sequence diagram showing an example of a process for generating evaluation target data. [Figure 11] FIG. 11 is a sequence diagram showing an example of a process for generating a report that is an evaluation result of a slope condition. [Figure 12] FIG. 13 is a diagram showing an example of an evaluation screen displayed on the evaluation device. [Figure 13] FIG. 13 is a diagram showing an example of an evaluation screen on which processing data is displayed. [Figure 14] 13 is a flowchart showing an example of a process for detecting a slope state. [Figure 15] FIG. 13 is a diagram showing an example of an evaluation screen on which detection results of shape data are displayed. [Figure 16] FIG. 13 is a diagram showing an example of a display screen showing a damage detection result. [Figure 17] FIG. 11 is a diagram showing an example of a cross-sectional image of a detected slope shape. [Figure 18] FIG. 2 is a diagram illustrating an example of map information. [Figure 19] FIG. 13 is a diagram showing an example of an evaluation report generated by the evaluation device. [Figure 20] FIG. 13 is a diagram showing an example of an evaluation report generated by the evaluation device. [Figure 21] FIG. 13 is a diagram showing an example of an evaluation report generated by the evaluation device. [Figure 22] FIG. 13 is a diagram showing another example of a display screen showing a damage detection result. [Figure 23] FIG. 11 is a diagram showing another example of a cross-sectional image of the detected shape of a slope. [Figure 24] FIG. 11 is a diagram showing an example of how a slope condition is inspected using a mobile body system according to the first modification. [Diagram 25] FIG. 11 is a diagram showing an example of a state in which a slope condition is inspected using a mobile body system according to Modification 2. [Figure 26] FIG. 11 is a diagram showing an example of how a slope condition is inspected using a mobile body system according to Modification 3. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals, and duplicated description will be omitted.

[0009] First embodiment System Overview First, an overview of the condition inspection system will be described with reference to Fig. 1 and Fig. 2. Fig. 1 is a diagram showing an example of the overall configuration of a condition inspection system according to an embodiment. The condition inspection system 1 shown in Fig. 1 is a system for inspecting the condition of road earthwork structures using various data acquired by a mobile system 60. Road earthwork structures are a general term for structures made mainly of ground materials such as soil and rocks that are constructed to build roads, and structures associated with them, and refer to cut and slope stabilization facilities, embankments, culverts, and similar structures. Hereinafter, road earthwork structures are referred to as slopes.

[0010] The condition inspection system 1 is composed of a mobile system 60 and an evaluation system 4. The mobile system 60 is composed of a data acquisition device 9 and a mobile body 6 such as a vehicle equipped with the data acquisition device 9. In the mobile system 60, the data acquisition device 9 has an imaging device 7, an angle sensor 8a, and a GNSS (Global Navigation Satellite System) sensor 8b. GNSS is a general term for satellite positioning systems such as GPS (Global Positioning System) or Quasi-Zenith Satellite System (QZSS).

[0011] The photographing device 7 is a line camera equipped with a line sensor in which photoelectric conversion elements are arranged in one or more rows. The photographing device 7 photographs a position along a predetermined photographing range on a photographing surface along the traveling direction of the moving body 6. The photographing device is not limited to a line camera, and may be a camera equipped with an area sensor in which photoelectric conversion elements are arranged in a planar manner. The photographing device may also be composed of multiple cameras.

[0012] The angle sensor 8a is a gyro sensor or the like for detecting the angle (attitude) or angular velocity (or each acceleration) of the shooting direction of the shooting device 7. The GNSS sensor 8b is a positioning means for measuring a position on the earth by receiving signals transmitted at each time from a plurality of GNSS satellites and calculating the distance to the satellite from the difference in the time of receiving each signal. The positioning means may be a device dedicated to positioning, or may be an application dedicated to positioning installed on a PC (Personal Computer), a smartphone, or the like. The angle sensor 8a and the GNSS sensor 8b are examples of sensor devices. The angle sensor 8a is also an example of a three-dimensional sensor.

[0013] Fig. 2 is a diagram showing an example of a state in which a slope condition is inspected using a mobile body system according to an embodiment. As shown in Fig. 2, the mobile body system 6 photographs a predetermined range of the slope with an imaging device 7 while a mobile body 6 equipped with a data acquisition device 9 travels on a road.

[0014] As shown in Figure 2, a cut slope is called a cut slope, and a piled-up slope is called a bank slope. In addition, the slope on the side of a road that runs along the side of a mountain is called a natural slope. Cut slopes and bank slopes can be made more durable by planting plants on the surface, and can be left unchanged for decades. However, this is not always the case. When cut slopes, bank slopes, and natural slopes deteriorate due to wind and rain, surface collapses occur, causing rocks and soil to fall, or the mountain collapses, causing road closures. To prevent such situations, methods are used to spray mortar on the surface of the slope (mortar spraying) or to install and harden concrete structures to slow down the rate at which the slope deteriorates when exposed to wind and rain. Structures constructed using such methods are called earthwork structures. Earthwork structures include retaining walls that are installed between natural slopes and roads, and rockfall protection fences that prevent rocks from falling onto the road. Both of these are intended to prevent road closures or human injury due to the outflow of soil, sand, falling rocks, etc. onto the road.

[0015] In recent years, the deterioration of earthwork structures that have been in place for decades has become significant, and the development of social infrastructure has become a major issue. Therefore, it is important to detect the deterioration of earthwork structures early and to inspect and maintain them to ensure their longevity. Conventional inspections of natural slopes and earthwork structures involve visual inspections by experts to investigate rock falls, collapses, landslides, or debris flows on the slopes and to create repair plans.

[0016] However, visual inspections by experts have problems with efficiency, such as the inability to inspect all of the large number of earthwork structures throughout Japan in a given period of time and the inability to inspect embankments at high altitudes or along rivers. In addition, visual inspections are unable to quantitatively grasp the degree of deterioration, such as cracks or peeling, that has occurred on the surface of earthwork structures.

[0017] Therefore, the condition inspection system 1 according to the embodiment acquires photographed image data of the slope of an earthwork structure using the photographing device 7, and acquires sensor data including three-dimensional information using three-dimensional sensors such as an angle sensor 8a. The evaluation system 4 then combines the acquired photographed image data and sensor data to evaluate the condition of the slope, thereby detecting shape data indicating the three-dimensional shape of the slope and detecting abnormalities such as cracks and peeling. This enables the condition inspection system 1 to efficiently perform evaluations that are difficult to inspect visually by humans.

[0018] Returning to FIG. 1, the evaluation system 4 is constructed by an evaluation device 3 and a data management device 5. The evaluation device 3 and the data management device 5 constituting the evaluation system 4 can communicate with each other via a communication network 100. The communication network 100 is constructed by the Internet, a mobile communication network, a LAN (Local Area Network), or the like. The communication network 100 may include not only wired communication but also networks using wireless communication such as 3G (3rd Generation), 4G (4th Generation), 5G (5th Generation), Wi-Fi (Wireless Fidelity) (registered trademark), WiMAX (Worldwide Interoperability for Microwave Access), or LTE (Long Term Evolution). The evaluation device 3 and the data management device 5 may also have a communication function using a short-range communication technology such as NFC (Near Field Communication) (registered trademark).

[0019] The data management device 5 is a computer such as a PC that manages various data acquired by the data acquisition device 9. The data management device 5 receives various acquired data from the data acquisition device 9, and transfers the received various acquired data to the evaluation device 3 that performs data analysis. Note that the method of transferring the various acquired data from the data management device 5 to the evaluation device 3 may be manual transfer using a USB (Universal Serial Bus) memory or the like.

[0020] The evaluation device 3 is a computer such as a PC that evaluates the condition of the slope based on various acquired data transferred from the data management device 5. The evaluation device 3 has a dedicated application program installed therein for evaluating the condition of the slope. The evaluation device 3 detects the type or structure of the slope from the photographed image data and the sensor data, extracts shape data, and performs a detailed analysis by detecting the presence or absence of deformation and the degree of deformation. The evaluation device 3 also generates a report to be submitted to a road administrator such as a national government, a local government, or a commissioned business operator, using the photographed image data and the sensor data, the evaluation target data, and the detailed analysis results. The data of the report generated by the evaluation device 3 is submitted to the road administrator as electronic data or printed on a document. The report generated by the evaluation device 3 is called an investigation record sheet, an inspection sheet, an investigation ledger, or a report. The evaluation device 3 is not limited to a PC, and may be a smartphone or a tablet terminal. The evaluation system 4 may be configured to construct the evaluation device 3 and the data management device 5 as a single device or terminal.

[0021] ●Hardware configuration Next, the hardware configuration of each device constituting the state inspection system 1 will be described with reference to Figures 3 and 4. Note that components may be added or deleted from the hardware configuration shown in Figures 3 and 4 as necessary.

[0022] ○Hardware configuration of data acquisition device○ 3 is a diagram showing an example of a hardware configuration of a data acquisition device 9. The data acquisition device 9 includes a controller 900 that controls the processing or operation of the data acquisition device 9, together with the image capture device 7 and the sensor device 8 as shown in FIG.

[0023] The controller 900 includes an imaging device I / F (Interface) 901, a sensor device I / F 902, a bus line 910, a CPU (Central Processing Unit) 911, a ROM (Read Only Memory) 912, a RAM (Random Access Memory) 913, a HD (Hard Disk) 914, a HDD (Hard Disk Drive) controller 915, a network I / F 916, a DVD-RW (Digital Versatile Disk Rewritable) drive 918, a media I / F 922, an external device connection I / F 923 and a timer 924.

[0024] Of these, the photographing device I / F 901 is an interface for transmitting and receiving various data or information to and from the photographing device 7. The sensor device I / F 902 is an interface for transmitting and receiving various data or information to and from the sensor device 8. The bus line 910 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 911 shown in FIG.

[0025] Furthermore, the CPU 911 controls the operation of the entire data acquisition device 9. The ROM 912 stores programs used to drive the CPU 911, such as the IPL. The RAM 913 is used as a work area for the CPU 911. The HD 914 stores various data such as programs. The HDD controller 915 controls the reading and writing of various data from and to the HD 914 under the control of the CPU 911. The network I / F 916 is an interface for data communication using the communication network 100. The DVD-RW drive 918 controls reading and writing of various data from and to a DVD-RW 917 as an example of a removable recording medium. The medium is not limited to a DVD-RW, and may be a DVD-R or a Blu-ray (registered trademark) Disc. The media I / F 922 controls reading and writing (storing) of data from and to a recording medium 921 such as a flash memory. The external device connection I / F 923 is an interface for connecting an external device such as an external PC 930. The timer 924 is a measuring device having a time measurement function. The timer 924 may be a software timer implemented by a computer.

[0026] ○Hardware configuration of evaluation device○ Fig. 4 is a diagram showing an example of the hardware configuration of the evaluation device. Each hardware component of the evaluation device 3 is indicated by a reference number in the 300 series. As shown in Fig. 4, the evaluation device 3 is constructed by a computer, and includes a CPU 301, a ROM 302, a RAM 303, a HD 304, a HDD controller 305, a display 306, an external device connection I / F 308, a network I / F 309, a bus line 310, a keyboard 311, a pointing device 312, a DVD-RW drive 314, and a media I / F 316.

[0027] Among these, the CPU 301 controls the operation of the entire evaluation device 3. The ROM 302 stores programs such as IPL used to drive the CPU 301. The RAM 303 is used as a work area for the CPU 301. The HD 304 stores various data such as programs. The HDD controller 305 controls reading or writing of various data from the HD 304 under the control of the CPU 301. The display 306 displays various information such as a cursor, a menu, a window, a character, or an image. The display 306 is an example of a display unit. The external device connection I / F 308 is an interface for connecting various external devices. In this case, the external device is, for example, a USB memory or a printer. The network I / F 309 is an interface for data communication using the communication network 100. The bus line 310 is an address bus, a data bus, or the like for electrically connecting each component such as the CPU 301 shown in FIG. 4.

[0028] The keyboard 311 is a type of input means having a plurality of keys for inputting characters, numbers, various instructions, etc. The pointing device 312 is a type of input means for selecting and executing various instructions, selecting a processing target, moving a cursor, etc. The DVD-RW drive 314 controls reading and writing of various data from a DVD-RW 313 as an example of a removable recording medium. Note that this is not limited to a DVD-RW, and may be a DVD-R or Blu-ray Disc, etc. The media I / F 316 controls reading and writing (storing) of data from a recording medium 315 such as a flash memory.

[0029] ○Hardware configuration of data management device○ Fig. 4 is a diagram showing an example of the hardware configuration of the data management device. Each hardware component of the data management device 5 is indicated by a reference number in the 500 range in parentheses. As shown in Fig. 4, the data management device 5 is constructed by a computer, and has the same configuration as the evaluation device 3, as shown in Fig. 4, so a description of each hardware component will be omitted.

[0030] Each of the above programs may be recorded in a computer-readable recording medium as an installable or executable file and distributed. Examples of the recording medium include a CD-R (Compact Disc Recordable), a DVD (Digital Versatile Disk), a Blu-ray Disc, an SD card, and a USB memory. The recording medium may be provided domestically or internationally as a program product. For example, the evaluation system 4 according to the embodiment realizes the evaluation method according to the present invention by executing the program according to the present invention.

[0031] ●Function configuration Next, the functional configuration of the state inspection system according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a diagram showing an example of the functional configuration of the state inspection system according to the first embodiment. Fig. 5 shows devices related to the processes or operations described below among those shown in Fig. 1.

[0032] ○Functional configuration of data acquisition device○ First, the functional configuration of the data acquisition device 9 will be described with reference to Fig. 5. The data acquisition device 9 has a communication unit 91, a determination unit 92, a photographing device control unit 93, a sensor device control unit 94, a photographed image data acquisition unit 95, a sensor data acquisition unit 96, a time data acquisition unit 97, a request acceptance unit 98, and a storage / readout unit 99. Each of these units is a function or means realized by any of the components shown in Fig. 3 operating according to an instruction from the CPU 911 in accordance with a program for the data acquisition device expanded from the HD 914 onto the RAM 913. The data acquisition device 9 also has a storage unit 9000 constructed by the ROM 912 and HD 914 shown in Fig. 3.

[0033] The communication unit 91 is mainly realized by processing of the CPU 911 for the network I / F 916, and communicates various data or information with other devices via the communication network 100. The communication unit 91 transmits, for example, data acquired by the photographed image data acquisition unit 95 and the sensor data acquisition unit 96 to the data management device 5. The judgment unit 92 is realized by processing of the CPU 911, and makes various judgments.

[0034] The imaging device control unit 93 is mainly realized by the processing of the CPU 911 for the imaging device I / F 901, and controls the imaging processing by the imaging device 7. The sensor device control unit 94 is mainly realized by the processing of the CPU 911 for the sensor device I / F 902, and controls the data acquisition processing for the sensor device 8.

[0035] The captured image data acquisition unit 95 is mainly realized by the processing of the CPU 911 for the photographing device I / F 901, and acquires captured image data related to an image photographed by the photographing device 7. The sensor data acquisition unit 96 is mainly realized by the processing of the CPU 911 for the sensor device I / F 902, and acquires sensor data which is the detection result by the sensor device 8. The time data acquisition unit 97 is mainly realized by the processing of the CPU 911 for the timer 924, and acquires time data indicating the time when data was acquired by the photographed image data acquisition unit 95 or the sensor data acquisition unit 96.

[0036] The request receiving unit 98 is mainly realized by the processing of the CPU 911 for the external device connection I / F 923, and receives a predetermined request from the external PC 930 or the like.

[0037] The storage / reading unit 99 is realized mainly by the processing of the CPU 911 , and stores various data (or information) in the storage unit 9000 and reads various data (or information) from the storage unit 9000 .

[0038] ○Functional configuration of evaluation device○ Next, the functional configuration of the evaluation device 3 will be described with reference to Fig. 5. The evaluation device 3 has a communication unit 31, a reception unit 32, a display control unit 33, a judgment unit 34, an evaluation target data generation unit 35, a detection unit 36, a map data management unit 37, a report generation unit 38, and a storage / readout unit 39. Each of these units is a function or means realized by any of the components shown in Fig. 4 being loaded from HD 304 onto RAM 303 and operating according to an instruction from CPU 301 in accordance with a program for the evaluation device. The evaluation device 3 also has a storage unit 3000 constructed by ROM 302 and HD 304 shown in Fig. 4.

[0039] The communication unit 31 is mainly realized by the processing of the CPU 301 for the network I / F 309, and communicates various data or information with other devices via the communication network 100. The communication unit 31 communicates with the data management device 5, for example, with respect to the evaluation of the slope condition. Sends and receives various data.

[0040] The reception unit 32 is mainly realized by the processing of the CPU 301 on the keyboard 311 or the pointing device 312, and receives various selections or inputs from the user. The reception unit 32 receives various selections or inputs on an evaluation screen 400, which will be described later. The display control unit 33 is mainly realized by the processing of the CPU 301, and causes the display 306 to display various images. The display control unit 33 causes the display 306 to display the evaluation screen 400, which will be described later. The judgment unit 34 is mainly realized by the processing of the CPU 301, and performs various judgments.

[0041] The evaluation target data generating unit 35 is realized by the processing of the CPU 301, and generates data to be evaluated. The detection unit 36 ​​is mainly realized by the processing of the CPU 301, and performs a process of detecting the state of the slope using the evaluation target data generated by the evaluation target data generating unit 35. The map data management unit 37 is mainly realized by the processing of the CPU 301, and manages map information acquired from an external server or the like.

[0042] The report generation unit 38 is mainly realized by the processing of the CPU 301, and generates an evaluation report to be submitted to the road administrator based on the evaluation results.

[0043] The storage / reading unit 39 is realized mainly by the processing of the CPU 301 , and stores various data (or information) in the storage unit 3000 and reads various data (or information) from the storage unit 3000 .

[0044] Status type management table Fig. 6 is a conceptual diagram showing an example of a condition type management table. The condition type management table is a table for managing teacher data for detecting the condition type of a slope. A condition type management DB 3001 configured by a condition type management table as shown in Fig. 6 is constructed in the storage unit 3000. In this condition type management table, a type name indicating a condition type, a teacher image, and a remarks column are associated and managed for each type number.

[0045] Among these, the type name is a name indicating a condition type for identifying the condition of the slope. Here, the condition types include retaining wall, crest, sprayed mortar, wire mesh, fence, pole, utility pole, sign, or billboard, etc. Also, the teacher image is a teacher image used in machine learning for determining the condition type of the slope from the captured image data. Furthermore, the remarks column indicates information that serves as a detection criterion for detecting the type of condition.

[0046] ○Functional configuration of data management device○ Next, the functional configuration of the data management device 5 will be described with reference to Fig. 5. The data management device 5 has a communication unit 51, a judgment unit 52, a data management unit 53, and a storage / readout unit 59. Each of these units is a function or means realized by any of the components shown in Fig. 4 being loaded from HD 504 onto RAM 503 and operating according to an instruction from CPU 501 in accordance with a program for the data management device. The data management device 5 also has a storage unit 5000 constructed by ROM 502 and HD 504 shown in Fig. 4.

[0047] The communication unit 51 is mainly realized by the processing of the CPU 501 for the network I / F 509, and communicates various data or information with other devices via the communication network 100. The communication unit 51 receives, for example, photographed image data and sensor data transmitted from the data acquisition device 9. The communication unit 51 also transmits and receives, for example, various data related to the evaluation of the slope condition with the evaluation device 3. The judgment unit 52 is realized by the processing of the CPU 501, and makes various judgments.

[0048] The data management unit 53 is mainly realized by the processing of the CPU 501, and manages various data related to the evaluation of the slope condition. For example, the data management unit 53 registers the photographed image data and the sensor data transmitted from the data acquisition device 9 in the acquired data management DB 5001. In addition, the data management unit 53 registers, for example, the data processed or generated by the evaluation device 3 in the processed data management DB 5003.

[0049] The storage / reading unit 59 is realized mainly by the processing of the CPU 501 , and stores various data (or information) in the storage unit 5000 and reads various data (or information) from the storage unit 5000 .

[0050] ○ Acquired data management table Fig. 7(A) is a conceptual diagram showing an example of an acquired data management table. The acquired data management table is a table for managing various acquired data acquired by the data acquisition device 9. An acquired data management DB 5001 configured by an acquired data management table as shown in Fig. 7(A) is constructed in the storage unit 5000. In this acquired data management table, the photographed image data, the sensor data, and the acquisition time are associated and managed for each folder.

[0051] Among these, the photographed image data and the sensor data are data files of acquired data transmitted from the data acquisition device 9. The acquisition time indicates the time when the photographed image data and the sensor data were acquired by the data acquisition device 9. Data acquired in one inspection process is stored in the same folder.

[0052] Processing data management table Fig. 7(B) is a conceptual diagram showing an example of a processing data management table. The processing data management table is a table for managing various processing data processed by the evaluation device 3. A processing data management DB 5003 configured by a processing data management table as shown in Fig. 7(B) is constructed in the storage unit 5000. In this processing data management table, evaluation target data, evaluation data, positioning data, and comments are associated and managed for each folder.

[0053] Among these, the evaluation target data is a data file used for detection evaluation of the slope condition by the evaluation device 3. Moreover, the evaluation data is a data file showing the evaluation result by the evaluation device 3. Furthermore, the positioning data is data showing the position information measured by the GNSS sensor 8b. Moreover, the comment is bibliographic information input by the evaluator for the evaluation target data or the evaluation data.

[0054] Processing or operation of the embodiment ○Data acquisition process○ Next, the processing or operation of the state inspection system 1 according to the embodiment will be described with reference to Fig. 8 to Fig. 23. First, the data acquisition process using the mobile body system 60 will be described with reference to Fig. 8 and Fig. 9. An operator inspecting the slope state gets on the mobile body 6, photographs the slope existing on the road, and uploads the acquired data to the data management device 5. A detailed description will be given below.

[0055] FIG. 8 is a sequence diagram showing an example of data acquisition processing using a mobile system. First, an inspection worker performs a predetermined input operation or the like on the external PC 330, and the request receiving unit 98 of the data acquisition device 9 receives a data acquisition start request (step S11). Then, the data acquisition device 9 executes data acquisition processing using the photographing device 7 and the sensor device 8 (step S12). Specifically, the photographing device control unit 93 issues a photographing request to the photographing device 7 to start photographing processing for a predetermined area. In addition, the sensor device control unit 94 starts detection processing by the angle sensor 8a and the GNSS sensor 8b in synchronization with the photographing processing by the photographing device 7. Then, the photographed image data acquisition unit 95 acquires photographed image data acquired by the photographing device 7, and the sensor data acquisition unit 96 acquires sensor data acquired by the angle sensor 8a and the GNSS sensor 8b. In addition, the time data acquisition unit 97 acquires time data indicating the time when various data are acquired by the photographed image data acquisition unit 95 and the sensor data acquisition unit 96.

[0056] Next, the inspection worker performs a predetermined input operation on the external PC 330 or the like, and the request receiving unit 98 receives a request to upload the various acquired data (step S13). Then, the communication unit 91 uploads (transmits) the captured image data, sensor data, and time data, which are the acquired data acquired in step S12, to the data management device 5 (step S14). As a result, the communication unit 51 of the data management device 5 receives the acquired data transmitted from the data acquisition device 9. Then, the data management unit 53 of the data management device 5 registers the acquired data received in step S14 in the acquired data management DB 5001 (see FIG. 7(A)) (step S15). The data management unit 53 stores the captured image data and the sensor data in one folder in association with time data indicating the acquisition time of each data included in the acquired data.

[0057] Here, an outline of the data acquisition process using the mobile body system 60 will be described with reference to FIG. 9. FIG. 9 is a diagram for explaining the captured images acquired by the mobile body system. The mobile body system 60 captures an image of a slope on a road using the image capture device 7 provided in the data acquisition device 9 while the mobile body 6 is traveling. As the mobile body 6 travels, the data acquisition device 9 acquires the captured image 1 and the captured image 2 in time series as shown in FIG. 9. At this time, the image capture device 7 and the sensor device 8 are time-synchronized, and the tilt correction (image correction) of the captured image is performed based on the attitude of the vehicle at the time of shooting, and the image data and the positioning data (north latitude and east longitude) are linked based on the time of the captured image.

[0058] In this way, the mobile system 60 acquires photographed image data of the slope and sensor data acquired in response to photography by the photography device 7 while driving the vehicle serving as the mobile body 6, and uploads the data to the data management device 5.

[0059] ○Evaluation of slope conditions○ ○Generating data to be evaluated Next, a process in which the evaluation system 4 evaluates the slope condition using the acquired data stored in the data management device 5 will be described with reference to Fig. 10 and Fig. 23. First, a process for generating evaluation target data used in the evaluation process of the slope condition will be described with reference to Fig. 10. Fig. 10 is a sequence diagram showing an example of the process for generating evaluation target data.

[0060] First, the communication unit 31 of the evaluation device 3 transmits a request to generate evaluation target data to the data management device 5 (step S31). This request includes the name of the folder in which the data to be generated is stored. As a result, the communication unit 51 of the data management device 5 receives the request to generate transmitted from the evaluation device 3.

[0061] Next, the storage / read unit 59 of the data management device 5 searches the acquired data management DB 5001 using the folder name included in the creation request received in step S31 as a search key to read out the acquired data associated with the folder name included in the creation request (step S32). Then, the communication unit 51 transmits the acquired data read out in step S32 to the evaluation device 3 (step S33). This acquired data includes the photographed image data, the sensor data, and the time data, and thus the communication unit 31 of the evaluation device 3 receives the acquired data transmitted from the data management device 5.

[0062] Next, the evaluation target data generating unit 35 of the evaluation device 3 generates evaluation target data using the acquired data received in step S33 (step S34). Specifically, the evaluation target data generating unit 35 performs tilt correction of the captured image data from the attitude of the image capturing device 7 (moving body 6) at the time of capturing, based on the received sensor data of the angle sensor 8a. In addition, the evaluation target data generating unit 35 links the captured image data to the positioning data, which is the sensor data received from the GNSS sensor 8b, based on the received time data. Furthermore, the evaluation target data generating unit 35 performs a process of synthesizing multiple captured image data into one image data.

[0063] In this way, the evaluation target data generating unit 35 has a tilt correction function for image data, a function for linking image data with position information, and a function for combining image data. The evaluation target data generating unit 35 uses the acquired data received from the data management device 5 to perform image correction on the received captured image data so that processing by the detection unit 36 ​​and report generating unit 38 described below can be easily performed.

[0064] Next, the communication unit 31 of the evaluation device 3 transmits the generated data generated in step S34 to the data management device 5 (step S35). This generated data includes the evaluation target data, positioning data, and comments generated by the evaluation target data generation unit 35. As a result, the communication unit 51 of the data management device 5 receives the generated data transmitted from the evaluation device 3. Then, the data management unit 53 of the data management device 5 stores the generated data received in step S35 in the processing data management DB 5003 (see FIG. 7(B)) (step S33). Specifically, the data management unit 53 associates the evaluation target data, positioning data, and comments included in the generated data and stores them in one folder.

[0065] In this way, the evaluation system 4 performs image processing based on the various data (captured image data, sensor data, and time data) acquired from the data acquisition device 9, thereby generating evaluation target data to be used for evaluating the slope condition.

[0066] Generate evaluation report Next, the process of generating an evaluation report to be submitted to the road administrator in the evaluation system 4 will be described with reference to Fig. 11 and Fig. 21. The evaluator evaluates the condition of the slope using the photographed image data and sensor data acquired by the data acquisition device 9, and generates an evaluation report showing the evaluation results. This will be described in detail below.

[0067] FIG. 11 is a sequence diagram showing an example of a process for generating a report that is an evaluation result of the slope condition. First, the display control unit 33 of the evaluation device 3 causes the display 306 to display an evaluation screen 400 for performing an evaluation process of the slope condition (step S51). FIG. 12 is a diagram showing an example of an evaluation screen displayed on the evaluation device. The evaluation screen 400 shown in FIG. 12 includes a selection area 410 for evaluation target data, an evaluation item selection area 430 for selecting an evaluation item for detecting the slope condition, a shape data display area 460 for displaying shape data, an "upload" button 491 pressed when uploading the evaluation result to the data management device 5, and a "report generation" button 493 pressed when generating an evaluation report. Among these, the selection area 410 includes a "folder designation" button 411 for designating a folder in which the evaluation target data is stored, a display area 413 in which the designated folder name is displayed, and an "OK" button 415 pressed when requesting download of the evaluation target data stored in the designated folder.

[0068] Next, the evaluator specifies a folder using the "specify folder" button 411, and the reception unit 32 of the evaluation device 3 accepts the selection of the evaluation target data (step S52). For example, in the case of the example of Fig. 12, the reception unit 32 accepts the selection of the evaluation target data stored in "folder 0615".

[0069] Next, the communication unit 31 receives a read request for the evaluation target data selected in step S52 from the data management device 5 (step S53). This read request includes the folder name selected in step S52. As a result, the communication unit 51 of the data management device 5 receives the read request transmitted from the evaluation device 3.

[0070] Next, the storage / read unit 59 of the data management device 5 searches the processing data management DB 5003 (see FIG. 7(B)) using the folder name included in the read request received in step S53 as a search key, thereby reading out the processing data associated with the folder name included in the read request (step S54). Then, the communication unit 51 transmits the processing data read out in step S54 to the evaluation device 3 (step S55). This processing data includes the evaluation target data, the positioning data, and the comments. As a result, the communication unit 31 of the evaluation device 3 receives the processing data transmitted from the data management device 5.

[0071] Next, the display control unit 33 of the evaluation device 3 displays the processing data received in step S54 in the evaluation item selection area 430 of the evaluation screen 400 (step S56). FIG. 13 is a diagram showing an example of the evaluation screen on which the processing data is displayed. The evaluation item selection area 430 shown in FIG. 13 includes an image display area 431 that displays an image of the evaluation target data, which is the processing data transmitted from the data management device 5, a bibliographic information display area 433 that displays bibliographic information of the evaluation target data, and a "Back" button 437 and a "Next" button 439 that are pressed when switching the image displayed in the image display area 431. The evaluation item selection area 430 also includes a "shape detection" button 451 that is pressed when detecting the shape of the slope, a "damage detection" button 453 that is pressed when detecting the damage state of the slope, and a "map information" button 455 that is pressed when generating map information.

[0072] In addition, in the image display area 431, evaluation areas 435a and 435b are displayed superimposed on the image of the evaluation target data. The evaluation areas 435a and 435b indicate an evaluation range in a process of detecting the state of a slope, which will be described later. The evaluator moves the evaluation areas 435a and 435b and enlarges or reduces the evaluation areas 435a and 435b by input operations such as tapping, dragging, swiping, pinching in, and pinching out on the evaluation areas 435a and 435b. The number of evaluation areas 435a and 435b is not limited to this, and may be one, or three or more. In addition, the evaluation areas 435a and 435b may not be displayed in the image display area 431, and the entire image display area 431 may be used as the evaluation range.

[0073] Next, the evaluation device 3 performs a process of detecting the slope condition using the evaluation target data (step S57). Here, the process of detecting the slope condition will be described in detail with reference to Fig. 14. Fig. 14 is a flowchart showing an example of the process of detecting the slope condition.

[0074] First, the evaluator presses the "shape detection" button 451 included in the evaluation item selection area 430, and the receiving unit 32 receives a shape detection request (step S71). Next, the detection unit 36 ​​performs a shape detection process using the evaluation target data (step S72). Here, the shape data indicating the shape of the slope is represented by three-dimensional information such as the extension, height, and inclination angle of the slope, as well as position information. The extension of the slope is the length of the slope in a plan view (the length in the depth direction of the cross section where the inclination of the slope can be seen). The shape data also includes information indicating the type of slope, whether it is a natural slope or an earthwork structure. Furthermore, the slope In the case of an earthwork structure, the shape data also includes information on the type of the earthwork structure, such as a retaining wall, a crenellated frame, a mortar sprayed wall, whether an anchor is used, or an embankment.

[0075] Specifically, the detection unit 36 ​​detects the extension, height, and inclination angle of the slope based on the image data and three-dimensional data included in the evaluation target data. The detection unit 36 ​​also detects the type of slope shown in the image, which is the evaluation target data, using the condition type management DB 3001 (see FIG. 6). In this case, the detection unit 36 ​​detects the type of slope by image matching processing using the teacher image shown in the condition type management table.

[0076] Next, the display control unit 33 displays the shape data, which is the detection result in step S72, in the shape data display area 460 of the evaluation screen 400 (step S73). Fig. 15 is a diagram showing an example of the evaluation screen on which the detection result of the shape data is displayed. The shape data display area 460 shown in Fig. 15 includes a display area 461 of bibliographic information of the shape data showing the shape detection result by the detection unit 36, and a "display details" button 463 that is pressed when displaying detailed data of the shape detection result. Of these, the display area 461 displays, for example, the total length of the slope and the proportion of each detected type in the entire slope.

[0077] Next, when the evaluator presses the "Damage Detection" button 453 included in the evaluation item selection area 430, the reception unit 32 transitions the process to step S75 if it receives a damage detection request for detecting damage to the slope condition (YES in step S74). On the other hand, when the damage detection request is not received (NO in step S74), the reception unit 32 transitions the process to step S77. The detection unit 36 ​​performs damage detection processing of the slope condition for the evaluation target data (step S75).

[0078] Here, the slope condition damage detection process detects the presence or absence of deformation on the slope or the degree of deformation as damage data representing the degree of damage to the slope. The degree of deformation indicates the degree of deterioration of the deformation, and is the width of the crack, the size of the separation, or the size of the lift. The detection unit 36 ​​detects the presence or absence of deformation on the slope or the degree of deformation based on the image data and sensor data included in the evaluation target data. The detection unit 36 ​​also detects whether the degree of deformation exceeds a predetermined value using a predetermined detection formula for the degree of deterioration of deformation, etc. In this case, the detection unit 36 ​​determines whether the width of the crack is equal to or larger than a certain value, whether the size of the peeling is equal to or larger than a certain value, or whether the lift is large, etc.

[0079] Next, the display control unit 33 causes the display 306 to display a display screen 470 showing the damage detection result in step S75 (step S76). FIG. 16 is a diagram showing an example of a display screen showing the damage detection result. The display screen 470 shown in FIG. 16 includes a display image area 480 showing the detected position of the damage on the entire slope to be evaluated, a detailed information display area 485 showing a photographed image corresponding to the position of the detected damage, and a "Cross-Section" button 489 to be pressed when displaying a cross-section of the detected slope. Among these, the display image area 480 shows a plan view in which images (P1 to P4) showing the position of the detected damage are drawn on an image showing the two-dimensional shape of the slope to be evaluated. The display image area 480 also shows position coordinates (positioning data) showing the position of the slope to be evaluated.

[0080] Furthermore, when the evaluator presses a "Cross-Section" button 489, the display control unit 33 causes the display 306 to display a cross-sectional image 475 shown in Fig. 17. The cross-sectional image 475 shown in Fig. 17 shows a cross-sectional view of the slope to be evaluated, drawn based on shape data detected by the detection unit 36. The shape data is detected using sensor data from the angle sensor 8a (three-dimensional sensor), and therefore, as shown in Fig. 17, it is possible to express in detail, including three-dimensional information such as the slope inclination or height of the slope, which cannot be calculated from a two-dimensional image alone.

[0081] Next, when the evaluator presses the "Map Information" button 455 included in the evaluation item selection area 430, the reception unit 32 advances the process to step S78 if the reception unit 32 receives a map information acquisition request (YES in step S77). On the other hand, when the reception unit 32 does not receive a map information acquisition request (NO in step S77), the process ends. The detection unit 36 ​​generates map information indicating the position of the slope state to be evaluated (step S78). Specifically, the detection unit 36 ​​generates map information in which an image indicating the position of the slope is added to the position (latitude, longitude) indicated by the positioning data acquired in step S55, which corresponds to map data available using a predetermined service or application provided by an external WEB server or the like. The map data provided by the external WEB server or the like is managed by the map data management unit 37.

[0082] Next, the display control unit 33 causes the display 306 to display the map information 490 generated in step S78 (step S79). FIG. 18 is a diagram showing an example of map information. As described above, in the map information 490, images 491a and 491b showing the position of the slope to be evaluated are drawn on the map data. In the map information 490 shown in FIG. 26, images 491a and 491b are drawn at the start and end positions of the inspection, and the range of the slope to be inspected is shown. Note that, when it is desired to show the position of the slope more accurately, it is preferable that the detection unit 36 ​​generates the map information by collating the positioning data with positioning map data held by the Geospatial Information Authority of Japan or the like.

[0083] In this way, the evaluation system 4 detects the shape of the slope including three-dimensional information, the degree of damage to the slope, and the position of the slope to be evaluated in order to evaluate the slope condition.

[0084] Returning to FIG. 11, when the evaluator presses the "upload" button 491 included in the evaluation screen 400, the reception unit 32 receives a request to upload the evaluation results (step S58). Then, the communication unit 31 uploads (transmits) the evaluation results to the data management device 5 (step S59). As a result, the communication unit 51 of the data management device 5 receives the evaluation data transmitted from the evaluation device 3. Then, the data management unit 53 of the data management device 5 registers the evaluation data received in step S59 in the processing data management DB 5003 (see FIG. 7(B)) (step S60). In this case, the data management unit 53 stores the evaluation data in one folder in association with the evaluation target data that was evaluated, etc.

[0085] When the evaluator presses a "Generate Report" button 493 included in the evaluation screen 400, the reception unit 32 receives a request to generate an evaluation report (step S61). The report generation unit 38 then generates an evaluation report based on the detection result of the slope condition by the detection unit 36 ​​(step S62). The report generation unit 38 generates an evaluation report by arranging the evaluation data indicating the above-mentioned evaluation results based on the inspection guidelines issued by the government or a format in accordance with the request of the road administrator.

[0086] 19 to 21 are diagrams showing an example of an evaluation report generated by the evaluation device. The first page of the evaluation report shown in Fig. 19 shows bibliographical information such as the route name at the top, and a location map showing the location of the slope to be inspected and the evaluation results at the bottom. The location map shown in Fig. 19 includes map information (see Fig. 18) generated by the detection unit 36.

[0087] Moreover, the second page of the evaluation report shown in Fig. 20 shows a plan view and a cross-sectional view. The evaluation report shown in Fig. 20 is generated based on the damage detection results and shape detection results by the detection unit 36. Of these, the plan view includes an image showing the damage analysis results by the detection unit 36 ​​(see Fig. 16), and the cross-sectional view includes a cross-sectional view of the slope to be evaluated (see Fig. 17) drawn based on the shape detection results by the detection unit 36.

[0088] Furthermore, the third page of the evaluation report shown in Fig. 21 shows image data of the slope to be evaluated. The evaluation report shown in Fig. 21 visually shows the condition and shape of the slope shown in the plan view and cross section of Fig. 20.

[0089] In this way, the evaluation device 3 uses the evaluation results of the slope condition to create an evaluation report to be provided to the road administrator. Note that the contents or layout of the evaluation report are not limited to the above.

[0090] As described above, the evaluation system 4 generates a report indicating the shape of the slope and the location and extent of damage by evaluating the slope condition using the photographed image data, sensor data (three-dimensional data), and positioning data acquired by the mobile system 60. This enables the evaluation system 4 to improve the quality and efficiency of the report generation function as an image assessment service used for slope inspection or a slope shape or damage assessment service.

[0091] 14, it is sufficient that the condition detection process by the detection unit 36 ​​is configured to perform at least the shape detection process shown in steps S71 to S73. The evaluator can write detailed evaluation results in the evaluation report by performing the damage detection process shown in steps S74 to S76 and the map information generation process shown in steps S77 to S79 ​​in addition to the shape detection process as necessary.

[0092] ○ Variations in slope condition evaluation Here, a modified example of the evaluation process of the slope condition will be described with reference to Fig. 22 and Fig. 23. The mobile system 60 may be configured to include a distance sensor 8c that measures the distance to the subject photographed by the photographing device 7, instead of the angle sensor 8a or together with the angle sensor 8a, as a three-dimensional sensor. The distance sensor 8c is a LiDAR (Light Detection and Ranging) sensor. By using the distance sensor 8c, the mobile system 60 can obtain three-dimensional information that is difficult to obtain from a two-dimensional image, such as the height, inclination angle, or overhang of the slope.

[0093] FIG. 22 is a diagram showing a display screen showing the damage detection result when a distance sensor is used, and FIG. 23 is a diagram showing a cross-sectional image of the shape of the slope detected by using the distance sensor. In addition to the examples shown in FIG. 15 and FIG. 16, a newly detected deformation (P5) is drawn on the display screen 470 shown in FIG. 22 and the cross-sectional image 475 shown in FIG. 23. A protrusion (uplift) is detected at the position of P5. In this way, by performing detection using the distance sensor, more detailed three-dimensional information can be obtained compared to the case where only the angle sensor 8a is used. The distance sensor 8c may be a radar sensor, a TOF (Time Of Flight) sensor, a stereo camera, or the like.

[0094] Modified examples of mobile systems ○Variation 1○ Next, modified examples of the mobile body system 60 will be described with reference to Fig. 24 to Fig. 26. First, Fig. 24 is a diagram showing an example of a state in which a slope condition is inspected using a mobile body system according to Modification 1. The mobile body system 60 according to Modification 1 is a system in which a data acquisition device 9 is fixed to a pole installed on the top surface of a mobile body 6 to enable photography at high altitudes.

[0095] The photographing device 7 of the above-mentioned embodiment is low in height from the ground, and it is difficult to photograph the berms on the retaining wall, the berms on the crenellation, or the berms on the sprayed mortar as shown in FIG. 24. In addition, the berms of the current road earthwork structures are not covered as shown in FIG. 24, and there is a risk of problems such as dead leaves accumulating and clogging the waterway, and they require regular cleaning. Therefore, by using the mobile body system 60 according to the first modification capable of photographing from a high place, even in cases where it is difficult for a person to climb a slope to check the degree of clogging of the waterway, for example, the confirmation can be made by the photographing process accompanying the traveling movement of the mobile body 6, and the efficiency of inspection can be significantly improved.

[0096] ○Variation 2○ 25 is a diagram showing an example of a state in which a slope condition is inspected using a mobile body system according to Modification 2. The mobile body system 60 (60a, 60b) according to Modification 2 is a system that uses a drone equipped with a data acquisition device 9 as an example of a mobile body 6 to photograph an embankment slope at a high place or below the roadside that cannot be photographed even by the pole-mounted camera of Modification 1.

[0097] The drone as the moving body 6 is equipped with not only the imaging device 7 but also a data acquisition device 9 equipped with sensor devices such as an angle sensor 8a, a GNSS sensor 8b, or a distance sensor 8c, and is therefore capable of evaluating the condition of high places and embankments that could not be evaluated by a vehicle as the moving body 6. In particular, embankments and high places are places where it is difficult for humans to approach and visually inspect them up close, and therefore it is desirable to photograph them using a drone as in the second modified example. In addition, the slopes of embankments and high places are often overgrown with vegetation such as trees and grass. For this reason, it is preferable that the data acquisition device 9 is equipped with an imaging device 7 capable of photographing wide-angle images.

[0098] ○Variation 3○ FIG. 26 is a diagram showing an example of a state in which a slope condition is inspected using a mobile system according to the third modification. As shown in FIG. 26, a slope has a complex structure, unlike a tunnel or a bridge, which is a structure on a road. For example, a slope may be undulating rather than flat (for example, an earthwork structure in which mortar is sprayed onto a quay), covered with vegetation, or covered with wire mesh. For this reason, the mobile system 60 (60a, 60b, 60c) according to the third modification includes a spectral camera, an infrared camera, or an expanded depth of field camera (EDof (Expanded Depth of Field) camera) capable of acquiring wavelength information as the sensor device 8 in order to distinguish between objects such as plants and wire mesh and the shape of the slope.

[0099] Moreover, it is preferable that the mobile system 60 according to the third modification is configured to be capable of photographing slopes under various conditions such as weather, sunlight, etc. by mounting a lighting device on the data acquisition device 9, in addition to providing a tool for distinguishing the shape of the slope. In this case, the lighting device is preferably a line lighting device that illuminates an area corresponding to the range photographed by the photographing device 7, or a time-sharing lighting device synchronized with the photographing device 7 and the sensor device 8.

[0100] Furthermore, in order to process the data acquired by the mobile body system 60 according to the third modification, the evaluation target data generating unit 35 of the evaluation device 3 preferably has image processing functions such as a camera shake correction function, a focal depth correction function (blur correction function), a distortion correction function, or a contrast enhancement function so as not to miss even small abnormalities. Also, the evaluation target data generating unit 35 preferably has a function of removing noise that hides abnormalities on earthwork structures such as grass, moss, or wire mesh, or a function of distinguishing between shadows of grass, etc. and abnormalities such as cracks. In this way, by using the mobile body system 60 according to the third modification, the condition inspection system 1 can accurately evaluate the condition of the slope even in places with complex structures or places where grass, moss, wire mesh, etc. are present.

[0101] Effect of the embodiment As described above, the condition inspection system 1 can quantitatively detect the shape of an earthwork structure, as well as changes in the shape of the surface layer of the earthwork structure and abnormalities occurring in the surface layer of the earthwork structure, based on data acquired using three-dimensional sensors such as the camera 7 and the angle sensor 8a, and can automatically generate evaluation reports such as those shown in Figures 19 to 21 based on the evaluation results, thereby significantly improving the efficiency of slope inspection. Furthermore, by using the mobile system 60, the condition inspection system 1 can inspect high places, embankments, and other places where humans cannot go.

[0102] Furthermore, the condition inspection system 1 can grasp the position of the slope to be evaluated by combining map data and the evaluation results of the slope condition, so that it can extract earthwork structure slopes that do not need to be visually inspected and improve the inspection speed per slope. Also, by registering the evaluation results in the data management device 5, the condition inspection system 1 can quantitatively accumulate the changes in the condition over time, making the soundness diagnosis rational and efficient.

[0103] In the above embodiment, an example of inspecting and evaluating the condition of a slope has been described, but the condition inspection system 1 may be used not only to inspect the condition of a slope, but also in combination with inspecting other structures such as road surfaces, tunnels, bridges, etc. The condition inspection system 1 can further improve the efficiency of infrastructure inspection by inspecting various structures on the road using the mobile system 60.

[0104] ●Summary● As described above, the evaluation system according to one embodiment of the present invention is an evaluation system 4 that evaluates the condition of a structure (e.g., a road earthwork structure), and receives photographed image data of the structure photographed by the photographing device 7 and sensor data acquired by a three-dimensional sensor (e.g., angle sensor 8a) in response to photographing the photographed image data by the photographing device 7, and detects a shape including three-dimensional information of the structure based on the received photographed image data and sensor data. Then, the evaluation system 4 generates an evaluation report showing the evaluation result of the condition of the structure based on the shape data indicating the detected shape and the photographed image data. This allows the evaluation system 4 to efficiently generate an evaluation report showing the evaluation result of the shape including three-dimensional information of the structure.

[0105] Furthermore, the evaluation system 4 according to one embodiment of the present invention detects the degree of damage to the structure based on shape data indicating the shape including three-dimensional information of the detected structure, and generates an evaluation report indicating the evaluation result including the shape data and damage data indicating the detected degree of damage. This allows the evaluation system 4 to efficiently generate an evaluation report including the degree of damage to the structure.

[0106] Furthermore, the evaluation system 4 according to an embodiment of the present invention generates an evaluation report in which an image showing the position of the detected damage is drawn against an image showing the detected shape, so that the evaluation system 4 can visually show the shape of the slope and the position of the damage on the slope in the evaluation report.

[0107] Moreover, the condition inspection system according to one embodiment of the present invention is a condition inspection system 1 including an evaluation system 4 and a mobile object 6 equipped with a data acquisition device 9. The data acquisition device 9 includes a photographing device 7 that acquires photographed image data and a three-dimensional sensor (e.g., an angle sensor 8a, a distance sensor 8c) that acquires sensor data, and acquires photographed image data and sensor data according to the travel of the mobile object 6. This allows the condition inspection system 1 to acquire data on structures that are difficult for humans to visually inspect, and allows for efficient inspection of the structures.

[0108] ●Additional Information● Each function of the above-described embodiment can be realized by one or more processing circuits. Here, the "processing circuit" in the present embodiment includes a processor programmed to execute each function by software, such as a processor implemented by an electronic circuit, and devices such as an ASIC (Application Specific Integrated Circuit), a DSP (digital signal processor), an FPGA (field programmable gate array), a SOC (System on a chip), a GPU (Graphics Processing Unit), and a conventional circuit module designed to execute each function described above.

[0109] In addition, the various tables in the above-described embodiment may be generated by the learning effect of machine learning, and the data of each associated item may be classified by machine learning, making it unnecessary to use tables. Here, machine learning is a technology for making a computer acquire human-like learning capabilities, and refers to a technology in which a computer autonomously generates an algorithm required for judgment such as data identification from learning data previously acquired, and applies this to new data to make predictions. The learning method for machine learning may be any of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or may be a combination of these learning methods, and any learning method for machine learning may be used.

[0110] So far, an evaluation system, a status inspection system, an evaluation method, and a program according to one embodiment of the present invention have been described. However, the present invention is not limited to the above-described embodiment, and can be modified within the scope of what a person skilled in the art can conceive, such as adding, changing, or deleting other embodiments. In any aspect, as long as the function and effect of the present invention are achieved, it is included in the scope of the present invention. [Explanation of symbols]

[0111] 1. Condition Inspection System 3. Evaluation equipment 4. Rating System 5 Data management device 6. Mobile 7. Imaging Equipment 8 Sensor device 8a Angle sensor (an example of a three-dimensional sensor) 8c Distance sensor (an example of a three-dimensional sensor) 9 Data acquisition device 32 Reception unit (an example of a reception means) 33 Display control unit (an example of a display control means) 35 Evaluation target data generation unit (an example of evaluation target data generation means) 36 Detection unit (an example of a detection means) 38 Report generation unit (an example of a report generation means) 51 Communication unit (an example of a receiving means) 60 Mobile Systems [Prior art documents] [Patent documents]

[0112] [Patent Document 1] JP 2019-33478 A

Claims

1. A generating means for generating a photographed image of the deformation position on a slope, a two-dimensional image showing the slope in two dimensions, and a cross-sectional image showing a cross section of the slope, which is used in a report including the two-dimensional image and the cross-sectional image, The generating means Generate the two-dimensional image including a deformation position image indicating a deformation position corresponding to the photographed image, generating the cross-sectional image including a deformation position cross-sectional image showing a position in a cross section of the slope corresponding to the photographed image and the deformation position based on the three-dimensional data of the slope; Information processing device.

2. The generating means A two-dimensional image including a first deformation position image indicating a first deformation position corresponding to a first photographed image among a plurality of photographed images of a plurality of deformation positions on the slope is generated, and generating the cross-sectional image including the first photographed image and a first deformation position cross-sectional image indicating a position in a cross section of the slope corresponding to the first deformation position based on the three-dimensional data of the slope; The information processing device according to claim 1 .

3. The generating means The two-dimensional image is generated including the first deformation position image and a second deformation position image indicating a second deformation position corresponding to a second photographed image among the plurality of photographed images on the slope; generating the cross-sectional image including the first deformed position cross-sectional image that is not associated with the second deformed position image but is associated with the first deformed position image; The information processing device according to claim 2 .

4. Further comprising a report generating means for generating a report including the photographed image, the two-dimensional image, and the cross-sectional image.

4. The information processing device according to claim 1.

5. A condition inspection system comprising an information processing device and a data acquisition device, The data acquisition device The method includes: an imaging device that acquires image data of a slope; and a three-dimensional sensor that acquires three-dimensional data of the slope; The information processing device includes: A generation unit is provided for generating the two-dimensional image and the cross-sectional image used in a report including a photographed image of the deformation position on the slope, a two-dimensional image showing the slope in two dimensions, and a cross-sectional image showing a cross-section of the slope, The generating means Generate the two-dimensional image including a deformation position image indicating a deformation position on the slope corresponding to the photographed image, generating the cross-sectional image including a deformation position cross-sectional image showing a position in a cross section of the slope corresponding to the photographed image and the deformation position based on the three-dimensional data of the slope; Condition inspection system.

6. 6. The status inspection system according to claim 5, further comprising: a mobile body equipped with the data acquisition device, The data acquisition device is a condition inspection system that acquires the captured image data and the three-dimensional data as the moving object travels.

7. An information processing method executed by an information processing device, comprising: A generation step of generating a photographed image of the deformation position on the slope, a two-dimensional image showing the slope in two dimensions, and a cross-sectional image showing a cross section of the slope, which are used in a report, and generating the two-dimensional image and the cross-sectional image, In the generating step, Generate the two-dimensional image including a deformation image indicating a deformation position on the slope corresponding to the photographed image, generating the cross-sectional image including a deformation position cross-sectional image showing a position in a cross section of the slope corresponding to the photographed image and the deformation position based on the three-dimensional data of the slope; Information processing methods.

8. An information processing device, A generation function is realized to generate a photographed image of the deformation position on the slope, a two-dimensional image showing the slope in two dimensions, and a cross-sectional image showing the cross section of the slope, which are used in a report, and the two-dimensional image and the cross-sectional image are generated. In the generating function, Generate the two-dimensional image including a deformation image indicating a deformation position on the slope corresponding to the photographed image, generating the cross-sectional image including a deformation position cross-sectional image showing a position in a cross section of the slope corresponding to the photographed image and the deformation position based on the three-dimensional data of the slope; program.

Images