Information processing system, method, and program

Abstract

The present invention provides an information processing system, method, and program capable of including information on a specific region that can be a mark in drawing data representing a structure. In an information processing system (100) provided with a processor (110), the processor (110) acquires a first image and a second image obtained by capturing an image of a structure such as a conduit at different time points, performs alignment between the first image and the second image, detects, on the basis of the aligned first image and second image, a specific region including a repaired or reinforced part of the structure, and includes information on the specific region in drawing data representing the structure.

Description

Information processing systems, methods, and programs

[0001] This invention relates to an information processing system, method, and program, and more particularly to a technique for creating drawing data representing structures.

[0002] Inspections of structures such as water conduits are conducted every 3 to 5 years to assess the extent of damage. If necessary, wall surfaces are repaired, or reinforcing structures such as steel frames are installed to prevent collapse and ensure long-term use. This inspection requires accurate assessment of the damage to the structure's walls.

[0003] Traditional water conduit inspections involved many personnel entering the conduits to perform visual inspections. In recent years, to reduce the burden on inspectors, digital transformation (DX) services have been introduced that photograph the inside of water conduits and allow inspections to be conducted from offices or other locations using the captured images.

[0004] Patent Document 1 proposes a damage information processing device that performs alignment based on feature points between captured images taken at different inspection times, extracts difference information of damage information from each captured image after alignment, detects first to third classification locations based on the extracted difference information and damage information, and displays the damage information corresponding to the first to third classification locations in an identifiable manner, or by switching between displaying and hiding it.

[0005] Here, the first classification area is one where damage information exists only in the earlier image captured chronologically, or where the damage information in the earlier image captured chronologically is greater than the damage information in the later image captured chronologically.

[0006] Furthermore, the second classification area is where damage information exists only in later images chronologically, or where the damage information in later images chronologically is greater than the damage information in earlier images chronologically. The third classification area is where the damage information in each image overlaps or is equal.

[0007] Furthermore, Patent Document 2 proposes an information processing device that can distinguish between overlapping and non-overlapping crack areas (increased crack areas) between past and latest images, or displays only the increased crack areas, based on past and latest images.

[0008] Japanese Patent Publication No. 2006-23821 Japanese Patent Publication No. 2023-131267

[0009] One embodiment of the technology relating to the present disclosure provides an information processing system, method, and program that can include information about specific areas that can serve as markers in drawing data representing a structure.

[0010] The invention according to the first embodiment is an information processing system equipped with a processor, wherein the processor acquires a first image and a second image of a structure taken at different times, aligns the first image and the second image, detects a specific area of ​​the structure that has been repaired or reinforced based on the aligned first image and the second image, and includes information of the specific area in drawing data representing the structure.

[0011] In the second aspect of the present invention, the information processing system, in the first aspect, preferably has a second image taken at a later time than the first image, and the processor detects a damaged area of ​​the structure from the first image, and detects a damaged area in the second image where no damaged area is detected as a specific area of ​​the structure at the time the second image was taken.

[0012] In the third aspect of the present invention, in the first aspect, the second image is an image taken at a later time than the first image, and the processor preferably detects the changed region between the first image and the second image, determines whether the region corresponding to the changed region in the second image is a region other than a damaged region, and detects the changed region determined to be a region other than a damaged region as a specific region.

[0013] In the fourth aspect of the present invention, the information processing system includes a memory for storing a first learning model for aligning images in any of the first to third aspects, and preferably the processor uses the first learning model, inputs a first image and a second image to the first learning model, and obtains the first image and the second image aligned with each other from the first learning model.

[0014] In the fifth aspect of the present invention, in the third aspect, the processor preferably detects as a changed region a region that has changed by more than a threshold between the first image and the second image.

[0015] In the sixth aspect of the present invention, the information processing system, in the fifth aspect, includes a memory for storing a second learning model that detects change regions between aligned images, and preferably the processor uses the second learning model, inputs the first and second images aligned to the second learning model, and obtains change regions from the second learning model that have changed beyond a threshold.

[0016] In the seventh aspect of the present invention, the information processing system includes, in the third aspect, a third learning model for determining whether a changed region is a region other than a damaged region, and the processor preferably uses the third learning model and obtains a determination result from the third learning model indicating whether or not a changed region is a region other than a damaged region.

[0017] In the eighth aspect of the present invention, the information processing system, in the third or seventh aspect, is such that the area other than the damaged area is an area showing repair marks on the structure or an area of ​​a newly constructed structure on the structure.

[0018] In the information processing system according to the ninth aspect of the present invention, in any of the first to eighth aspects, it is preferable that the processor adds an image representing a specific region to the drawing data by projecting it.

[0019] In the information processing system according to the tenth aspect of the present invention, in any of the first to ninth aspects, it is preferable that the processor adds text information indicating the detection result of a specific area at a position corresponding to a specific area of ​​the drawing data.

[0020] The information processing system according to the eleventh aspect of the present invention, in any of the first to tenth aspects, is such that the drawing data is a design drawing of a structure, an unfolded drawing obtained by unfolding the design drawing of a structure in two dimensions, a plan view of a structure, a longitudinal section view, a cross section view, or a damaged drawing to which damage information has been added.

[0021] In the information processing system according to the twelfth aspect of the present invention, in any of the first to eleventh aspects, it is preferable that the structure is a structure having a total length exceeding a threshold.

[0022] In the 12th aspect of the information processing system according to the 13th aspect of the present invention, the structure is a water conduit, tunnel, elevated bridge, or bridge.

[0023] The invention according to the 14th aspect is an information processing method to be performed by an information processing system equipped with a processor, the method comprising: the step of the processor acquiring a first image and a second image of a structure taken at different times; the step of the processor aligning the first image and the second image; the step of the processor detecting a specific area of ​​the structure that has been repaired or reinforced based on the aligned first image and the second image; and the step of the processor including information of the specific area in drawing data representing the structure.

[0024] In the information processing method according to the 15th aspect of the present invention, in the 14th aspect, the second image is an image taken at a later time than the first image, and the step of detecting a specific region preferably involves detecting a damaged region of the structure from the first image and detecting a damaged region in the case where no damaged region is detected from the second image as a specific region of the structure at the time the second image was taken.

[0025] In the information processing method according to the 16th aspect of the present invention, in the 14th aspect, the second image is an image taken at a later time than the first image, and the step of detecting a specific region preferably involves detecting a changed region that has changed between the first image and the second image, determining whether the region corresponding to the changed region in the second image is a region other than a damaged region, and detecting the changed region determined to be a region other than a damaged region as a specific region.

[0026] The invention according to the 17th aspect is an information processing program that uses a computer to implement the following functions: acquiring a first image and a second image of a structure taken at different times; aligning the first image and the second image; detecting a specific area of ​​the structure that has been repaired or reinforced based on the aligned first and second images; and including information about the specific area in drawing data representing the structure.

[0027] Figure 1 is a block diagram showing an embodiment of the information processing system according to the present invention. Figure 2 is a diagram showing an example of information stored in memory. Figure 3 is a diagram showing an example of multiple databases constituting the water conduit management DB shown in Figure 1. Figure 4 is a diagram showing the relationship between the water conduit during installation and each inspection, the drawing data at the time of installation (design drawing (original drawing)) and the drawing data updated at each inspection (first design drawing, second design drawing, ...), and the group of images taken at the time of installation and each inspection. Figure 5 is a diagram showing an example of a two-dimensional unfolded drawing of the water conduit design drawing. Figure 6 is a drawing in which text information indicating the detection result of a specific area has been added to the unfolded drawing shown in Figure 5. Figure 7 is a diagram showing an example of a damage diagram in which an image indicating a specific area has been added to the damage diagram of the water conduit. Figure 8 is a diagram showing an example of a longitudinal section in which an image indicating a specific area has been added to the longitudinal section of the water conduit. Figure 9 is a flowchart showing the first embodiment of the information processing method according to the present invention. Figure 10 is a flowchart showing the second embodiment of the information processing method according to the present invention.

[0028] Preferred embodiments of the information processing system, method, and program according to the present invention will be described below with reference to the attached drawings.

[0029] [Information Processing System] Figure 1 is a block diagram showing an embodiment of the information processing system according to the present invention.

[0030] The information processing system 100 shown in Figure 1 can be composed of a computer, workstation, etc. Alternatively, the information processing system 100 may be composed of an information processing server, or user terminals connected to the information processing server via a communication network. The information processing server can be implemented using a cloud or distributed computing system.

[0031] The information processing system 100 shown in FIG. 1 includes a processor 110, a memory 120, a water channel management database (DB) 130, an input / output interface 140, a display 150, and an operation unit 160.

[0032] Note that all or part of the water channel management DB 130 may be provided in an external device. In this case, the information processing system 100 can communicate with the external device via the input / output interface 140 including a communication function and exchange necessary information.

[0033] The processor 110 is composed of a CPU (Central Processing Unit) or the like, and comprehensively controls each part of the information processing system 100. By executing an information processing program, the processor 110 performs a process of including information on a specific area where a structure (in this example, a water channel) is repaired or reinforced in drawing data representing the structure. The details of this process will be described later.

[0034] The memory 120 includes a flash memory, a ROM (Read-only Memory), a RAM (Random Access Memory), a hard disk device, and the like. The flash memory, the ROM, or the hard disk device is a non-volatile memory that stores various programs including an operation system and an information processing program for executing the information processing method according to the present invention.

[0035] The RAM functions as a work area for the processing by the processor 110. Also, it temporarily stores various programs stored in the flash memory or the like and data used for information processing. Note that the processor 110 may incorporate a part (RAM) of the memory 120.

[0036] FIG. 2 is a diagram showing an example of information stored in the memory.

[0037] The memory 120 shown in FIG. 2 stores an information processing program 122, a first learning model 124, a second learning model 126, a third learning model 128, etc. according to the present invention. Details of the first learning model 124, the second learning model 126, and the third learning model 128 will be described later.

[0038] FIG. 3 is a diagram showing an example of a plurality of DBs constituting the waterway management DB shown in FIG. 1.

[0039] As shown in FIG. 3, the waterway management DB 130 includes a construction information DB 132, a diagnosis element management DB 134, and a repair management DB 136.

[0040] The construction information DB 132 stores drawing data (including span information) such as design drawings of the waterway, information on structures (reinforcement members, other members) provided in the waterway, and position information of the distance plates when the distance plates are arranged in the waterway.

[0041] The diagnosis element management DB 134 includes an image management DB, a damage degree (element) evaluation DB, inspection date and time of the waterway, and position information.

[0042] At the time of laying the waterway and at the time of inspection every 3 to 5 years, the inspector moves a cart radially equipped with a plurality of cameras from one end to the other end of the waterway in order to photograph the entire circumference of the inner wall of the waterway, for example. It is preferable that the plurality of cameras are controlled to photograph the inner wall of the waterway every time the cart moves a set distance or at a set time interval.

[0043] The image management DB stores the image groups respectively photographed at the time of laying and inspection of the waterway as described above.

[0044] Further, the image management DB may store a composite image (for example, a composite image obtained by two-dimensionally developing the entire circumference of the wall surface of the waterway for each span at an interval of 10 m) obtained by combining a plurality of images of the wall surfaces of each span of the waterway among the image groups.

[0045] It is preferable that each image in the image management database for each inspection is linked to the date and time of the water conduit inspection and location information within the water conduit (for example, distance information from one end of the water conduit (water intake)). For example, a distance measuring device can be installed on the trolley to measure the distance traveled from the start of shooting at one end of the water conduit, and the information indicating the distance traveled measured by the distance measuring device for each camera shot can be used as location information indicating the shooting location within the water conduit.

[0046] Furthermore, when storing composite images of each span of the water conduit, it is preferable to store them in association with the inspection date and time of the water conduit and the span information (span number) of the water conduit.

[0047] The damage assessment database stores location information and inspection dates for various types of damage to the inner wall of the water conduit (cracks, free lime (efflorescence), water leakage, rust stains, peeling, delamination, budding, exposed rebar, honeycombing, etc.), along with a classification of the degree of damage for each type of damage (for example, a five-level classification from a to e).

[0048] The repair management DB136 stores repair information (date and time, construction method, materials (newly installed structures), location information, etc.).

[0049] Figure 4 is a diagram showing the relationship between the construction and inspection of the water conduit, the drawing data at the time of construction (design drawings (original drawings)), the drawing data updated at each inspection (first design drawing, second design drawing, ...), and the group of images taken at the time of construction and each inspection.

[0050] As mentioned above, the drawing data (first design drawing, second design drawing, etc.) is managed by the construction information DB 132, and the image data taken during installation and each inspection is managed by the image management DB included in the diagnostic element management DB 134. However, the DBs that manage these are not limited to the above embodiment. The method for updating the drawing data, which is updated after each inspection, will be described later.

[0051] Returning to Figure 1, the input / output interface 140 includes a communication unit that can connect to a communication network, and a connection unit that can connect to external devices. The connection unit that can connect to external devices can utilize USB (Universal Serial Bus), HDMI (High-Definition Multimedia Interface) (HDMI is a registered trademark), etc.

[0052] In this example, the processor 110 acquires a group of images taken during inspections of the water conduit from external devices (camera, user terminal, etc.) via the input / output interface 140 and stores them in the image management DB within the diagnostic element management DB 134 included in the water conduit management DB 130.

[0053] The display 150 displays drawing data such as design drawings of the water conduit managed in the construction information DB 132, as well as images taken inside the water conduit managed in the image management DB, and also displays an operation screen.

[0054] The operation unit 160 includes a keyboard, mouse, touch panel, etc., and is the part that receives various operation inputs from the user, while the display 150 displays a screen for operation on the operation unit 160. In other words, the display 150 and the operation unit 160 can function as a user interface.

[0055] When the information processing system 100 is configured as an information processing server, the display 150 and the operation unit 160 are unnecessary and will be provided by the user terminal that communicates with the information processing server.

[0056] [Processing details of the information processing system] <First embodiment> The information processing system 100 shown in Figure 1 functions as an image acquisition unit, an image alignment unit, a specific area detection unit, and a drawing update unit by executing various programs including the information processing program according to the present invention.

[0057] The image acquisition unit acquires two images, Image A and Image B, taken at different times (inspection times) of the water conduit from the image management database included in the water conduit management database 130. Here, Image B is an image taken at a later time (inspection time) than Image A. The image acquisition unit acquires, for example, a group of images I taken during the nth inspection as shown in Figure 4 and stored in the image management database. n A first image A is obtained from there, and a group of images I taken during the (n+1)th inspection and stored in the image management DB are also obtained. n+1 From there, one or more second images B are obtained, each capturing the same wall surface as the first image A. When the image acquisition unit acquires the first image A and the second image B, which captures the same location as the first image A, it can use the location information associated with each image.

[0058] The image alignment unit performs image processing to align the positions of the acquired first image A and second image B. The processor 110 uses a first learning model 124 (Figure 2) for alignment to perform alignment between images, and by inputting the first image A and second image B into the first learning model 124, it can obtain the first image A and second image B that have been aligned with each other from the first learning model 124. Alternatively, instead of using the first learning model 124, the processor 110 may extract a number of corresponding feature points from the first image A and second image B, and projectively transform one of the images (for example, the second image B) so that these number of corresponding feature points match.

[0059] The specific area detection unit detects a specific area where the inner wall of the water conduit has been repaired or reinforced, based on the aligned first image A and second image B.

[0060] The processor 110 uses a second learning model 126 (Figure 2) for detecting change regions between aligned images. By inputting the aligned first image A and second image B into the second learning model 126, the processor can obtain the change region between the aligned first image A and second image B from the second learning model 126. Preferably, the second learning model 126 is machine-trained to output the change region as the region that has changed by more than a threshold between the first image A and second image B.

[0061] Next, the processor 110 uses a third learning model 128 for determination and obtains a determination result from the third learning model 128 indicating whether the changed region acquired by the second learning model 126 is a region other than a damaged region.

[0062] In other words, by using the second learning model 126 and the third learning model 128, the processor 110 can acquire areas of change exceeding a threshold between the first image A and the second image B, excluding damaged areas, as specific areas where the inner wall of the water conduit has been repaired or reinforced.

[0063] Furthermore, instead of using the second learning model 126 and the third learning model 128, the processor 110 can detect a specific region by performing the following processing.

[0064] The processor 110 detects the difference in pixel values ​​at the same position between the aligned first image A and second image B, and detects the area of ​​the detected difference as a change area. Since the first image A and second image B were captured at different times, it is preferable to normalize the luminance values ​​of the first image A and second image B so that they become reference values, and to set the difference in pixel values ​​to be detected as exceeding a threshold.

[0065] Next, the processor 110 extracts the change regions that have changed significantly beyond a threshold from among the detected change regions, and then detects the specific region by clustering the extracted change regions into damaged regions and regions other than the damaged regions (specific regions).

[0066] The drawing update unit updates the drawing data by including information about a specific area in the drawing data showing the water conduit.

[0067] In other words, as shown in Figure 4, the drawing update unit adds the drawing data (nth design drawing) at the time of the nth inspection to the image group I taken at the time of the nth inspection. n and a group of images I taken during the (n+1)th inspection. n+1The (n+1)th design drawing is created by adding images that show specific areas of the water conduit that have been repaired or reinforced (i.e., areas that were repaired or reinforced during the period between the nth inspection and the (n+1)th inspection), based on the detection of these areas. When the drawing update unit adds images of specific areas to the drawing data, it is preferable to add the images after applying a projection transformation.

[0068] The drawing data updated by the drawing update unit is stored in the construction information DB 132. The drawing data stored in the construction information DB 132 can be read according to a read command from the user and displayed on the display 150, and can also be read according to a read command from the system and used for updating the next drawing data, etc.

[0069] Figure 5 shows an example of a two-dimensional unfolded diagram of a water conduit design.

[0070] As shown in Figure 5, images 10 and 20 indicating specific regions have been added to the unfolded diagram. In addition, span information (S001, S002, ..., S100) indicating each span of the water conduit has been added to the unfolded diagram.

[0071] Image 10 shows, for example, a repair area for cracks in the wall of a water conduit. Typical crack repair methods include crack filling, which involves cutting a U-shape or V-shape along the crack and filling it with a repair material such as epoxy resin, and crack injection, which involves injecting an injection material such as epoxy resin or fine particle cement into the crack.

[0072] Image 20 shows the area of ​​a newly constructed structure in the water conduit. The newly constructed structure can prevent peeling of the wall surface or reinforce damaged areas. Reinforcement methods using structures include, for example, the backing plate method, which involves attaching backing plates such as steel frames and joining them with high-strength bolts, and the netting method, which uses FRP mesh, vinylon mesh, etc.

[0073] Furthermore, the processor 110 can add images 10 and 20 to the unfolded view that indicate specific areas of the water conduit that have been repaired or reinforced, based on repair information (date and time, construction method, newly constructed structure, location information, etc.) managed in the repair management DB 136. In addition, the newly constructed structure is not limited to one that reinforces the wall surface of the water conduit, but can be any structure (including members) that is newly constructed afterwards.

[0074] According to the first embodiment described above, the drawing data (unfolded view) of the water conduit shown in Figure 5 can be updated by adding images 10 and 20 that indicate specific areas where the inner wall of the water conduit has been repaired or reinforced, thereby adding marker information to the drawing data.

[0075] Incidentally, as inspections and repairs are carried out on water conduits over many years, the number of repaired areas and newly constructed structures that were not there when the conduits were laid increases, so the inside of the conduit gradually changes.

[0076] On the other hand, while the design drawings for the water conduits are created at the time of their construction, they become outdated as the conditions within the conduits change over time.

[0077] When inspecting a water conduit, it is not easy to determine or detect the actual location of damaged areas that have been visually identified based on the images taken inside the conduit, or damaged areas that have been detected using a learning model or software for damage detection.

[0078] Furthermore, it is difficult to identify the target (cracks) to be checked in this inspection from damage diagrams of the water conduit (for example, diagrams in which damage information indicating damaged areas such as cracks is added to a two-dimensional unfolded drawing of the design drawing), and this is especially difficult for structures like water conduits that are long and have few landmarks.

[0079] According to the first embodiment, by adding landmark information (images 10 and 20 showing specific areas where the inner wall of the water conduit has been repaired or reinforced) to the drawing data, it becomes easier to identify the object to be checked.

[0080] Furthermore, the images 10 and 20 showing the specific area can be, for example, images in which the specific area is colored, and it is preferable to use a color that corresponds to the type of damage and / or the type of repair method for the specific area.

[0081] <Second Embodiment> In the second embodiment of the information processing system, the type of information in a specific area to be included in the drawing data representing the structure differs from that of the first embodiment.

[0082] Figure 6 is a diagram in which text information indicating the detection results of a specific region has been added to the unfolded diagram shown in Figure 5.

[0083] The repair management database DB136 manages repair information for each repair location in the water conduit (date and time, construction method, newly constructed structures, location information, etc.).

[0084] The processor 110 identifies the repair method for the detected specific area (repair location) based on the repair information for each repair location managed by the repair management DB 136, and adds text information indicating the repair method (detection result) for the specific area to the location corresponding to the specific area.

[0085] In the example shown in Figure 6, text information 12 indicating "crack filling work" is added to the position corresponding to image 10 showing the crack repair area, and text information 22 indicating "reinforcement plate work" is added to the position corresponding to image 20 showing the area of ​​the newly constructed structure.

[0086] According to the second embodiment, the visibility of a specific area that has been repaired or reinforced can be improved, and it is possible to easily confirm what kind of repair method has been applied.

[0087] <Third Embodiment> In the third embodiment of the information processing system, the drawing data representing the structure differs from that of the first embodiment.

[0088] Figure 7 shows an example of a damage diagram in which an image indicating a specific area has been added to the damage diagram of the water conduit.

[0089] The damage diagram shown in Figure 7 is a two-dimensional unfolded view of the water conduit design drawing, with damage information indicating cracks and other damaged areas added. Images 10 and 20, which indicate specific areas, are added to this damage diagram, similar to the diagram shown in Figure 5.

[0090] In Figure 7, reference numeral 30 indicates a damaged area of ​​"cracks," and reference numeral 40 indicates a damaged area of ​​"free lime."

[0091] The damage assessment database included in the diagnostic element management DB 134 shown in Figure 13 stores damage information of the water conduit (types of damage to the inner wall of the water conduit and location information of the damage) associated with the inspection date and time. The processor 110 can create a damage diagram by adding the damage information stored in the damage assessment database to the unfolded diagram of the water conduit managed by the construction information DB 132, and can store the created damage diagram in the construction information DB 132.

[0092] The processor 110 can create the drawing shown in Figure 7 by using the damage diagram created during the most recent inspection and adding images 10 and 20 to the damage diagram that show specific areas where the inner wall of the water conduit has been repaired or reinforced, which were not present at the time of installation.

[0093] According to the third embodiment, images 10 and 20, which show specific regions added to the damage diagram shown in Figure 7, can be used as markers to identify the target (crack) to be checked in this inspection, while damage information on the damage diagram (information on the damaged regions indicated by reference numerals 30 and 40) can also be used to identify the target.

[0094] <Fourth Embodiment> In the fourth embodiment of the information processing system, the drawing data representing the structure differs from that of the first and third embodiments.

[0095] Figure 8 shows an example of a longitudinal section of a water conduit with an image indicating a specific area added to the longitudinal section.

[0096] The longitudinal section shown in Figure 8 illustrates the water conduit 1, upper reservoir 2, lower reservoir 3, intake 4, and intake gate 5.

[0097] The upper reservoir 2 stores water flowing in from rivers, etc. The water conduit 1 is a channel that guides the water stored in the upper reservoir 2 to the lower reservoir 3, or to other rivers, spillways, etc. The intake gate 5 is a gate that adjusts the amount of water taken in from the intake port 4. When the water conduit 1 is being inspected, the intake port 4 is completely closed and the water in the water conduit 1 is drained. In addition, control gates, intake valves, etc. (not shown) may be installed in the intake facilities including the intake gate 5 and along the water conduit 1.

[0098] In Figure 8, the longitudinal section of the water conduit 1 includes images 10 and 20, which show specific areas where the inner wall of the water conduit 1 has been repaired or reinforced, similar to the diagram shown in Figure 5.

[0099] According to the fourth embodiment, images 10 and 20, which show specific regions added to the longitudinal section of the water conduit 1 shown in Figure 8, can be used as markers to identify the object to be checked in this inspection.

[0100] Furthermore, the drawing data for the water conduit to which information for a specific area is added is not limited to the unfolded drawings, damage drawings, and longitudinal sections obtained by unfolding the aforementioned water conduit design drawings in two dimensions, but may also be other drawing data such as unfolded drawings and damage drawings obtained by unfolding the water conduit design drawings, plan drawings, and cross-sectional drawings in two dimensions.

[0101] [Information Processing Method] <First Embodiment> Figure 9 is a flowchart showing the first embodiment of the information processing method according to the present invention.

[0102] Each step of the information processing method shown in Figure 9 corresponds to the processing performed by the processor 110 of the information processing system 100 shown in Figure 1.

[0103] In Figure 9, the processor 110 obtains the first image A and the second image B, which were taken at different times, from the image management DB in the water conduit management DB 130 (step S100). For example, the processor 110 obtains the image group I from the image management DB, which is managed for each water conduit inspection, and which was taken at the previous inspection (the nth inspection). n A first image A is obtained from this, and a group of images I taken during this inspection (the (n+1)th inspection) n+1 From there, one or more second images B are obtained, which are photographs of the wall surface at approximately the same position as the first image A.

[0104] For example, when each image in the image group is captured by a camera mounted on a carriage moving in the water conduit at set time intervals or at every set moving distance, the image group I n can obtain the second image B captured in the same or almost the same order as the first image A in the image group I n+1 from it.

[0105] Further, when position information in the water conduit (for example, distance information from the water intake of the water conduit) is attached to each image in the image group, the position information attached to the image is used, and the second image B with position information indicating the same position or a position in the vicinity of the position information attached to the first image A is obtained from the image group I n+1 from it. Furthermore, when obtaining the second image B from the image group I n+1 if span information indicating the span of the water conduit is attached to each image in the image group, the span information can be used. Furthermore, when the first image A includes characteristic parts of the water conduit (for example, distance plates, repair marks, characteristic structures arranged in the water conduit), the second image B including the same characteristic parts can be obtained from the image group I n+1 from it.

[0106] Note that the method for obtaining the first image A and one or more second images B that capture the wall surface at substantially the same position as the first image A is not limited to the above example, and not only when obtaining the second image B corresponding to the first image A based on the first image A, but also when obtaining the first image A corresponding to the second image B based on the second image B.

[0107] Subsequently, the processor 110 performs alignment between the first image A and the second image B (step S110). The processor 110 uses the first learning model 124 (FIG. 2) for alignment, inputs the first image A and the second image B into the first learning model 124, and obtains the first image A and the second image B that are mutually aligned from the first learning model 124. Here, the first learning model 124 can be, for example, one that is learned to match the first image A by translating, rotating, and scaling the second image B, or one that is learned to convert the second image B in pixel units to match the first image A.

[0108] Next, the processor 110 detects a specific area of ​​the water conduit that has been repaired or reinforced based on the aligned first image A and second image B (steps S120, S130).

[0109] Specifically, in step S120, a second learning model 126 (Figure 2) for detecting change regions is used, and the first image A and the second image B, aligned to the second learning model 126, are input. The second learning model 126 then acquires change regions that have changed beyond a threshold. Preferably, the second learning model 126 is trained not to detect slight change regions caused by the shooting conditions of the first image A and the second image B, or slight change regions of damage such as cracks (change regions below the threshold).

[0110] In step S130, a third learning model 128 (Figure 2) for specific region determination is used. Images of the changed region that have changed beyond a threshold are input to the third learning model 128, and a determination result (classification result) indicating whether the changed region is a region other than damaged (specific region) is obtained from the third learning model 128.

[0111] The processor 110 acquires the change region that has changed beyond a threshold from the second learning model 126, and if the change region is determined by the third learning model 128 to be a region other than a damaged region, it acquires the change region as a specific region in which the water conduit has been repaired or reinforced.

[0112] Next, the processor 110 aligns a specific region with a drawing of the water conduit (for example, the damage diagram shown in Figure 7) (step S140). This alignment can be performed using a learning model for alignment that performs the alignment between the drawing and the specific region, but it may also be performed by matching feature points.

[0113] The processor 110 updates the drawing by including information indicating a specific area aligned with the drawing of the water conduit (step S150). The information indicating the specific area can be, for example, an image in which the specific area is colored. In the damage diagram shown in Figure 7, images 10 and 20 are images indicating the specific area, respectively.

[0114] By performing the processes from step S100 to step S150, if repairs to the water conduit were carried out between the previous inspection and the current inspection, images 10 and 20 will be added to the drawing, indicating specific areas of repaired sections or newly constructed structures that were not present in the previous inspection.

[0115] Furthermore, the processing from step S100 to step S150 is performed using the image group I taken during the previous inspection. n A certain first image A obtained from [source] and a group of images I taken during this inspection. n+1 The process involves obtaining the second image B corresponding to the first image A, and updating the drawing, as shown in the image group I. n All first images that make up and group of images I n+1 It goes without saying that the process from step S100 to step S150 is repeated with respect to all the second images that make up the set.

[0116] According to the first embodiment of the information processing method, landmark information (images 10 and 20 showing specific areas where the inner wall of the water conduit has been repaired or reinforced) is added to the drawing of the water conduit, making it easier to identify the object to be checked (damage such as cracks).

[0117] <Second Embodiment> Figure 10 is a flowchart showing a second embodiment of the information processing method according to the present invention.

[0118] Each step of the information processing method shown in Figure 10 corresponds to the processing performed by the processor 110 of the information processing system 100 shown in Figure 1.

[0119] Furthermore, the first embodiment of the information processing method shown in Figure 9 uses the first learning model 124, the second learning model 126, and the third learning model 128, but the second embodiment of the information processing method shown in Figure 10 differs from the first embodiment in that it does not use the first learning model 124, the second learning model 126, and the third learning model 128.

[0120] In Figure 10, the processor 110 obtains a first image A and a second image B from the image management DB in the water conduit management DB 130, which were taken at different times (step S200). This step S200 can be performed in the same way as the step S100 shown in Figure 9.

[0121] Next, the processor 110 aligns the first image A and the second image B (step S210). The processor 110 obtains a plurality of first feature points from the first image A and a plurality of second feature points from the second image B (a plurality of second feature points corresponding to a plurality of first feature points) from the first image A and the second image B, respectively, and aligns the first image A and the second image B by transforming the second image B (for example, by a projection transformation) so that the plurality of first feature points and the plurality of second feature points match.

[0122] The processor 110 detects the changed region between the aligned first image A and second image B (step S220). The changed region can be detected by detecting the difference in pixel values ​​of pixels at the same position in the first image A and second image B. For example, the position of a pixel where the difference value exceeds a threshold can be the position of a pixel included in the changed region. That is, the changed region can be a set of pixels where the difference value exceeds a threshold, or a region within a contour when a closed region (contour) is formed by pixels where the difference value exceeds a threshold.

[0123] Next, the processor 110 extracts the larger areas (areas exceeding the threshold) from the change areas detected in step S220. This is to eliminate slight change areas caused by the shooting conditions of the first image A and the second image B, or slight change areas of damage such as cracks.

[0124] The processor 110 clusters the extracted change regions into damaged regions and regions other than damaged regions (specific regions) (step S240). Clustering of the extracted change regions can be performed by using a learning model that analyzes images within the extracted change regions to determine whether or not they are damaged, or to estimate whether or not a change region is a damaged region.

[0125] Next, the processor 110 aligns the second image B to the drawing and acquires alignment information for the clustered specific region in step S240 (step S250). This alignment information can be used, for example, as transformation parameters when projecting the image so that the feature points on the drawing coincide with the feature points of the second image B containing the specific region.

[0126] The processor 110 uses alignment information to project a specific area, aligns the specific area to the water conduit drawing, and updates the drawing by including information indicating the specific area in the drawing (step S260). The information indicating the specific area can be, for example, an image in which the specific area is colored.

[0127] According to the second embodiment of the information processing method, landmark information (an image showing a specific area where the inner wall of the water conduit has been repaired or reinforced) is added to the drawing of the water conduit, making it easier to identify the object to be checked (damage such as cracks).

[0128] [Other] In this embodiment, the first and second images, which were taken at different times when the water conduit was photographed, are the first image obtained from a group of images taken during one inspection of the water conduit and the second image obtained from a group of images taken during another inspection, and are one or more second images that photograph the wall surface at approximately the same position as the first image. However, this is not limited to this, for example, the first image may be one of the first composite images obtained by combining multiple images of the wall surface of each span of the water conduit taken during one inspection of the water conduit, and the second image may be one of the second composite images obtained by combining multiple images of the wall surface of each span of the water conduit taken during another inspection (a second composite image of the same span as the first composite image).

[0129] Furthermore, it is preferable that the composite images for each span of the water conduit be orthophotos that correspond to each span on the unfolded drawing obtained by unfolding the water conduit design drawing in two dimensions. In this case, the alignment of the first composite image and the second composite image corresponding to the same span of the water conduit can be easily performed, and similarly, information for a specific area can be easily added to the drawing data of the water conduit.

[0130] In this embodiment, the first and second images, which were taken at different times and depict a water conduit, are the subject of information processing. However, the structures depicted in the first and second images are not limited to water conduits; they may also be tunnels, elevated bridges, or other structures, and are particularly preferably structures with a total length exceeding a threshold. According to the present invention, even if the drawing data represents a structure with a total length exceeding a threshold, including information on a specific area that serves as a landmark in the drawing data makes it easier to identify the object to be checked (damage such as cracks).

[0131] Furthermore, in this embodiment, the hardware structure of a processing unit that performs various processes, such as the processor of an information processing system, is a variety of processors as shown below. These various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as a processing unit; a Programmable Logic Device (PLD), which is a processor whose circuit configuration can be changed after manufacturing, such as an FPGA (Field Programmable Gate Array); and a dedicated electrical circuit, which is a processor with a circuit configuration specifically designed to perform a particular process, such as an ASIC (Application Specific Integrated Circuit).

[0132] A single processing unit may be composed of one of these various processors, or it may be composed of two or more processors of the same or different type (for example, multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, multiple processing units may be composed of a single processor. Examples of composing multiple processing units with a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as multiple processing units, as is the case with computers such as clients and servers. Secondly, a configuration using a processor that realizes the functions of the entire system, including multiple processing units, on a single IC (Integrated Circuit) chip, as is the case with System-on-a-Chip (SoC). Thus, various processing units are configured, in terms of hardware structure, using one or more of the above-mentioned various processors.

[0133] Furthermore, the hardware structure of these various processors is, more specifically, an electrical circuit composed of circuit elements such as semiconductor devices.

[0134] Furthermore, the present invention includes an information processing program that, when installed on a computer, causes the computer to function as an information processing system according to the present invention or to execute an information processing method according to the present invention, and a non-volatile storage medium on which this information processing program is recorded.

[0135] Furthermore, it goes without saying that the present invention is not limited to the embodiments described above, and various modifications are possible without departing from the spirit of the invention.

[0136] 1...Water conduit 2...Upper reservoir 3...Lower reservoir 4...Water intake 5...Water intake gate 10, 20...Images 12, 22...Text information 100...Information processing system 110...Processor 120...Memory 122...Information processing program 124...First learning model 126...Second learning model 128...Third learning model 140...Input / output interface 150...Display 160...Operation unit A...First image B...Second image 130...Water conduit management DB 132...Construction information DB 134...Diagnostic element management DB 136...Repair management DB I n , I n+1 ...image group

Claims

1. An information processing system equipped with a processor, wherein the processor acquires a first image and a second image of a structure taken at different times, aligns the first image and the second image, detects a specific area of ​​the structure that has been repaired or reinforced based on the aligned first image and the second image, and includes information of the specific area in drawing data representing the structure.

2. The information processing system according to claim 1, wherein the second image is an image taken at a later time than the first image, and the processor detects a damaged area of ​​the structure from the first image, and detects the damaged area in the case where no damaged area is detected from the second image as a specific area of ​​the structure at the time the second image was taken.

3. The information processing system according to claim 1, wherein the second image is an image taken at a later time than the first image, the processor detects a changed region between the first image and the second image, determines whether the region in the second image corresponding to the changed region is a region other than a damaged region, and detects the changed region determined to be a region other than a damaged region as the specific region.

4. The information processing system according to claim 1, comprising a memory for storing a first learning model for performing alignment between images, wherein the processor uses the first learning model, inputs the first image and the second image to the first learning model, and obtains the first image and the second image aligned with each other from the first learning model.

5. The information processing system according to claim 3, wherein the processor detects a region that has changed by more than a threshold between the first image and the second image as the changed region.

6. The information processing system according to claim 5, comprising a memory for storing a second learning model for detecting change regions between aligned images, wherein the processor uses the second learning model, inputs the aligned first image and second image to the second learning model, and obtains the change region that has changed beyond the threshold from the second learning model.

7. The information processing system according to claim 3, comprising a third learning model for determining whether a region of change is a region other than a damaged region, wherein the processor uses the third learning model to obtain a determination result from the third learning model indicating whether the region of change is a region other than a damaged region.

8. The information processing system according to claim 3, wherein the area other than the damaged area is an area showing repair marks on the structure, or an area of ​​a newly constructed structure on the structure.

9. The information processing system according to claim 1, wherein the processor adds an image representing the specific region to the drawing data by projecting it.

10. The information processing system according to any one of claims 1 to 9, wherein the processor adds text information indicating the detection result of the specific region at a position corresponding to the specific region of the drawing data.

11. The information processing system according to any one of claims 1 to 9, wherein the drawing data is a design drawing of the structure, an unfolded drawing obtained by unfolding the design drawing of the structure in two dimensions, a plan view, a longitudinal section, a cross section of the structure, or a damage drawing to which damage information has been added.

12. The information processing system according to any one of claims 1 to 9, wherein the structure has an overall length exceeding a threshold.

13. The information processing system according to claim 12, wherein the structure is a water conduit, tunnel, elevated bridge, or bridge.

14. An information processing method to be performed by an information processing system equipped with a processor, comprising: the step of the processor acquiring a first image and a second image of a structure taken at different times; the step of the processor aligning the first image and the second image; the step of the processor detecting a specific area of ​​the structure that has been repaired or reinforced based on the aligned first image and the second image; and the step of the processor including information of the specific area in drawing data representing the structure.

15. The information processing method according to claim 14, wherein the second image is an image taken at a later time than the first image, and the step of detecting the specific region involves detecting the damaged region of the structure from the first image, and detecting the damaged region in cases where the damaged region is not detected from the second image as the specific region of the structure at the time the second image was taken.

16. The information processing method according to claim 14, wherein the second image is an image taken at a later time than the first image, and the step of detecting the specific region involves detecting a changed region that has changed between the first image and the second image, determining whether the region in the second image corresponding to the changed region is a region other than a damaged region, and detecting the changed region determined to be a region other than a damaged region as the specific region.

17. An information processing program that enables the following functions by computer: a function to acquire a first image and a second image of a structure taken at different times; a function to align the first image and the second image; a function to detect a specific area of ​​the structure that has been repaired or reinforced based on the aligned first image and the second image; and a function to include information of the specific area in drawing data representing the structure.

18. A non-temporary and computer-readable recording medium on which the program described in claim 17 is recorded.

Images