Information processing device, server device, control method for information processing device, control method for server device, and program

Abstract

To reduce the possibility that an operator overlooks an area of developing deformation.SOLUTION: An information processing apparatus acquires deformation data extracted from each image of a plurality of images photographed in different periods, and based on a difference in deformation data between the images in the difference periods, detects developing deformation. The information processing apparatus calculates the priority for the deformation based on the difference in deformation data, specifies areas covering the developing deformation, and displays the specified areas according to the priority.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to an information processing technique for detecting deformation of a structure or the like based on captured images. [Background technology]

[0002] In the inspection of infrastructure structures such as bridges and tunnels using images, a method is used in which multiple high-resolution images are taken and synthesized to generate a single composite image in order to detect defects such as cracks and exposed rebar on the structure's walls. Furthermore, to determine the soundness of the structure's components, it is necessary to detect defects from images of the structure's walls taken at different times and understand how much the defects have progressed (i.e., changes over time). In this case, for example, the degree of progress of each defect can be calculated by finding the difference between the detected defects. Furthermore, Patent Document 1 discloses a technology in which images taken at different times are input, positional deviations between the deformations are corrected, and then the progression of the deformation is calculated. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-20220 Summary of the Invention [Problem to be solved by the invention]

[0004] However, even if it is possible to calculate the progression of deformation using the technology disclosed in Patent Document 1, an image obtained by combining multiple high-resolution images as described above may calculate areas that include many progressing deformations. In this case, the inspector must check these many progressing areas, which may result in the worker overlooking areas of deformation that should be checked.

[0005] Therefore, an object of the present invention is to reduce the possibility that an operator will overlook an abnormality that should be checked. [Means for solving the problem]

[0006] The information processing device includes: an acquisition means for acquiring a plurality of deformation data relating to the deformation of the structure, each of which is generated from a plurality of images of the structure taken at different times; and a detection means for detecting changes over time in the deformation of the structure based on the plurality of deformation data. an identification means for identifying an area including a portion showing the deformation of the structure and the progress of the aging change of the deformation based on the detection result by the detection means; Detection results by the detection means and the area identified by the identifying means. Based on this, the deformation data acquired by the acquisition means and the image of the structure are acquired so that the part showing the deformation of the structure and the part showing the progress of the aging change of the deformation can be distinguished. In order according to the degree of progress of the deterioration of the structure over time and a display means for displaying the superimposed image. [Effects of the Invention]

[0007] According to the present invention, the possibility that an operator will overlook an abnormality that should be checked is reduced. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2 illustrates an example of a hardware configuration of an information processing device. [Figure 2] FIG. 10 is a diagram showing an example of the relationship between a drawing and an inspection target image. [Figure 3] FIG. 10 is a diagram illustrating an example of a list of detected abnormalities. [Figure 4] 4 is a flowchart of information processing according to the first embodiment. [Figure 5] FIG. 10 is a diagram showing an example of a list of evolving defects. [Figure 6] 10 is a flowchart of a priority calculation process. [Figure 7] 10 is a flowchart of an area identification process. [Figure 8] FIG. 10 is a diagram illustrating an example of specifying an area. [Figure 9] FIG. 10 is a diagram illustrating an example of an area identification table. [Figure 10] FIG. 10 is a diagram illustrating an example of a UI screen related to priority change. [Figure 11] FIG. 10 is a diagram showing an example of a UI screen relating to area selection. [Figure 12] FIG. 10 is a diagram illustrating an example of priority calculation based on the degree of progress (progress rate). [Figure 13] FIG. 10 is a diagram illustrating an example of a system configuration according to a second embodiment. [Figure 14] 10 is an information processing flowchart of a server device according to a second embodiment. [Figure 15] 10 is an information processing flowchart of a client terminal according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The following embodiments do not limit the present invention, and not all of the combinations of features described in the embodiments are necessarily essential to the solution of the present invention. The configurations of the embodiments may be modified or changed as appropriate depending on the specifications of the device to which the present invention is applied and various conditions (such as usage conditions and usage environment). In addition, the present invention may be configured by appropriately combining parts of each of the embodiments described below. In the following embodiments, the same components are described with the same reference numerals.

[0010] First Embodiment FIG. 1 is a block diagram showing an example of a schematic configuration of an information processing device 100 according to this embodiment. The information processing device 100 includes a CPU 101 , a ROM 102 , a RAM 103 , a HDD 104 , a display unit 105 , an operation unit 106 , and a communication unit 107 . The CPU 101 is a central processing unit that performs calculations and logical decisions for various processes and controls each component connected to a system bus 110. The ROM (Read-Only Memory) 102 is a program memory that stores programs including control procedures and information processing procedures executed by the CPU 101. The programs stored in the ROM 102 include programs for various information processing procedures according to the present embodiment, which will be described later. The RAM (Random Access Memory) 103 is used as a temporary storage area such as the main memory and work area of ​​the CPU 101. Note that the program memory may be realized by loading a program into the RAM 103 from an external storage device connected to the information processing device 100.

[0011] The HDD 104 is a device equipped with a hard disk for storing electronic data such as image data and programs according to this embodiment. An external storage device may also be used to perform a similar function. Here, the external storage device can be realized, for example, by media (recording media) and an external storage drive for realizing access to the media. Known examples of such media include flexible disks (FDs), CD-ROMs, DVDs, USB memories, MOs, and flash memories. The external storage device may also be a server device connected via a network.

[0012] The display unit 105 includes, for example, a liquid crystal display (LCD) or an organic light emitting diode (OLED) display, and generates images and a graphical user interface (GUI) to display them on the screen of the display. The display unit 105 may be included in an external device connected to the information processing device 100 by wire or wirelessly. The operation unit 106 has a keyboard and a mouse, and accepts various operations by a user, for example, a worker who performs a deformation investigation and inspection, which will be described later. The communication unit 107 performs wired or wireless two-way communication with other information processing devices, communication devices, external storage devices, and the like using known communication techniques.

[0013] The information processing device 100 of this embodiment detects deformations, such as cracks and exposed rebar, in infrastructure structures such as bridges and tunnels from high-resolution images of those structures and determines the extent of the progression of the deformation over time. Furthermore, the information processing device 100 of this embodiment calculates a priority based on the importance of the progressing deformation and changes the display method for the area of ​​the progressing deformation according to the priority. That is, the information processing device 100 of this embodiment acquires deformation data extracted from multiple images taken at different times, calculates the difference between the deformation data extracted from the images taken at different times, and detects progressing deformations based on the difference. The information processing device 100 of this embodiment then calculates a priority for the deformation based on the difference in the deformation data, identifies the area covering the progressing deformation, and displays the identified area according to the priority.

[0014] Here, in providing a detailed explanation of the information processing device 100 of this embodiment, we will explain the relationship between images and deformation data representing deformations, and structural information related to infrastructure structures, etc. In addition, in inspections using images, it is preferable to manage images of structure walls, etc. in association with drawings.

[0015] FIG. 2(a) is a diagram showing an image 211 of a bridge wall, as an example of an infrastructure structure, pasted onto an image of a drawing 200. The image of drawing 200 has drawing coordinates 201 with point 202 as the origin. The position of the image on the drawing is defined by the coordinates of the vertex at the top left of the image. For example, the coordinates of image 211 are the position of vertex 212 (X212, Y212). The image data of drawing 200 and the data of image 211 are stored in storage units such as RAM 103 and HDD 104, along with their respective coordinate information.

[0016] In this embodiment, the images used in the image inspection of infrastructure structures are large in size because they are captured at high resolution, with each pixel corresponding to 1 mm on the structure, allowing for the detection of minute cracks and other defects. For example, assume that image 211 in FIG. 2(a) is an image of a 20 m x 10 m bridge deck. If the image resolution is 1.0 mm per pixel (1.0 mm / pixel), the image size of image 211 is 20,000 pixels x 10,000 pixels. Note that if image 211 captured at high resolution contains, for example, more than 1,000 cracks or rebar exposures, it would be difficult to display all of these defects on paper. Therefore, only a portion of the defects is displayed on the page in FIG. 2. In the following description, even in diagrams displaying wide-area images or defect data, only a portion of the defects is displayed on the page.

[0017] The deformation data is information on the results of automatic deformation detection, in which deformations such as cracks on concrete walls are detected (extracted) from photographed images using image analysis processing, or information on the results of a human input that determines a deformation from a photographed image. The information processing device 100 of this embodiment acquires the deformation data extracted from the photographed image in this way. In explaining this embodiment, it is assumed that the input image and deformation data are managed in association with drawing coordinates.

[0018] Figure 2(b) shows the state in which the deformation data 221 corresponding to the image 211 is pasted onto the image of the drawing 200 in the same position as the image 211 in Figure 2(a). The deformation data 221 contains a large number of deformations (for example, more than 1000), including deformations that are not displayed on the paper. The position on the drawing of each deformation data in the deformation data 221 is defined by the pixel coordinates that make up the deformation data.

[0019] Fig. 3(a) is a diagram showing an example of a deformation data table 301 showing the data structure of deformation data. The deformation data table 302 shown in Fig. 3(b) will be described later. The deformation data table 301 is composed of the following items: deformation ID, deformation type, coordinates, line width, maximum width, line length, and area. The deformation ID represents the identification information (ID) of the detected deformation, and the deformation type represents the type of deformation, such as cracks or exposed rebar. The coordinates represent multiple coordinate information that make up the deformation data. If the deformation type is a crack, the line width is an attribute value that represents the width of the deformation at that coordinate. If the deformation type is a crack, the maximum width represents the maximum numerical value of the line width, and the line length represents the total length of the crack. If the deformation type is an area, such as exposed rebar, the area represents the area of ​​that area. For example, deformation ID Ca001 indicates that the deformation type is a crack, and that the coordinates are represented by n consecutive pixels consisting of n points, from (Xca001_1, Yca001_1) to (Xca001_n, Yca001_n).

[0020] As described above, in this embodiment, the deformation data is represented by pixels. The deformation data may be represented by vector data such as polylines or curves consisting of multiple points. When the deformation data is represented by vector data, the data volume is reduced and the representation is simpler. As an example of deformation data for an area shape other than a crack, the deformation ID Ta001 has a deformation type of exposed rebar. When an area shape such as exposed rebar is represented by coordinate information, it becomes a deformation having an area surrounded by a polyline. Note that the information contained in the deformation data is not limited to the information shown in the deformation data table 301, and may also hold information of other attributes.

[0021] Other attribute information may include structural information related to the structure being inspected, such as the structure type, basic structure, various dimensions of the structure, component information, and year of completion. Furthermore, other attribute information may include information on maintenance-related repair history, such as the repair year, repair locations, and repair method. In this embodiment, structural information related to specific positions of the structure, such as component information and repair information, is stored along with position information on the drawings. That is, the positions of each component and repair locations on the drawings are stored as part of the structural information. Therefore, the correspondence between the structural information and images and deformation data can be determined via the drawings. The structural information, along with images and deformation data, can be stored and retrieved in the RAM 103 and HDD 104, which are storage units. Note that the information included in the structural information is not limited to the above information, and other information may also be stored. Furthermore, information specific to each type of structure may be stored.

[0022] FIG. 4 is a main flowchart showing the flow of the aging priority display process, which is information processing related to the information processing device 100 in this embodiment. The information processing device 100 in this embodiment calculates the degree of progress of a deformation occurring on the wall surface of a structure using two images (a first input image and a second input image) taken at different times. The time when the second input image was taken is assumed to be several years (for example, five years) after the time when the first input image was taken. Hereinafter, each process (step) in FIG. 4 will be explained with an "S" added to the beginning of the reference numeral.

[0023] In S401, the CPU 101 acquires deformation data corresponding to the first input image (hereinafter referred to as first deformation data) from the deformation data stored in the storage unit. The first deformation data is assumed to be data acquired from the deformation data table 301 shown in FIG. 3(a). Here, crack data will be used as an example of deformation data. The crack data is assumed to be information input by manually tracing cracks on an image, or information automatically detected using a model previously trained by machine learning, and is stored in the storage unit.

[0024] In S402, the CPU 101 acquires deformation data corresponding to the second input image (hereinafter referred to as "second deformation data") in the same manner as in S401. Here, it is assumed that the deformation data corresponding to the second input image is described in, for example, the deformation data table 302 shown in FIG. 3(b). The deformation data table 302 in FIG. 3(b) is a table similar to the deformation data table 301 in FIG. 3(a) described above. Like the deformation data table 301, the deformation data table 302 in FIG. 3(b) is assumed to be information manually entered by tracing the second input image or data automatically detected using a learning model based on machine learning and stored in advance in a storage unit. That is, the CPU 101 acquires the second deformation data corresponding to the second input image from the deformation data table 302 stored in the storage unit.

[0025] Next, in S403, the CPU 101 calculates a difference value from the first deformation data and the second deformation data, and creates an aging data table 501 shown in FIG. 5 as information indicating the degree of progression of the deformation.

[0026] Fig. 5(a) shows an aging data table 501 obtained by calculating a difference value from the first deformation data and the second deformation data. The aging data table 501 shown in Fig. 5(b) will be described later. The secular change data table 501 consists of a progression ID, a corresponding ID, line length change, line width change, area change, intersection change, connection ID, and priority. The progression ID represents the ID of the progressing deformation, and the corresponding ID represents the deformation ID of the object to be compared. The deformation ID of the first deformation data and the deformation ID of the second deformation data are recorded. The first deformation data and the second deformation data may be associated using existing technology. For example, the center of gravity and feature values ​​may be calculated from the contours of the deformations detected from the first input image and the second input image, and the alignment may be performed by comparing the distance between the calculated feature values. The line length change, line width change, area change, and intersection change are recorded as the difference value between the first deformation data and the second deformation data for each item. Specifically, the line length change is calculated as the difference in line length between the deformation IDs listed in the corresponding IDs of the first deformation data and the second deformation data. Similarly, the line width change is calculated as the difference in the maximum line width between the first deformation data and the second deformation data, and the area change is calculated as the difference in area between the first deformation data and the second deformation data. The intersection change is calculated from the difference between the number of intersections of the deformation in the first deformation data and the number of intersections of the deformation in the second deformation data. The connection ID contains the deformation ID of the second deformation data that was newly detected as a deformation compared to the deformation in the first deformation data.

[0027] Next, in S404, the CPU 101 performs a process of calculating a priority for the worker to check based on the importance of the progression of the abnormality, and records the priority in the priority item of the aging data table. Details of this process will be described later using the flowchart in FIG.

[0028] Next, in S405, the CPU 101 specifies an area (referred to as an attention area) to be displayed as a location where the worker should pay attention to the progression of the abnormality. Details of this processing will be described later with reference to the flowchart in FIG.

[0029] Next, in S406, CPU 101 displays the areas of interest identified in S405 on display unit 105 in the order of priority calculated in S404 for the anomalies contained within those areas. After the worker has finished checking the areas displayed on display unit 105, the worker clicks the mouse on operation unit 106 or presses a specific key on the keyboard. This allows CPU 101 to recognize that the worker has finished checking. Then, CPU 101 displays on display unit 105 the area with the next highest priority relative to the currently displayed area.

[0030] FIG. 6 is a flowchart showing the detailed flow of the priority calculation process in S404 of FIG. In S601, the CPU 101 processes the progression IDs in the aging data table 501 in order and determines the type of deformation for the progression ID being processed. That is, in the priority calculation process, the priority calculation criteria differ depending on the type of deformation, so the CPU 101 first determines the type of deformation for the progression ID being processed in S601. Here, the CPU 101 determines whether the type of deformation for the progression ID being processed is a crack. If the type of deformation is a crack (Yes), the CPU 101 proceeds to S602, and if it is not a crack (No), the CPU 101 proceeds to S605.

[0031] When proceeding to S602, the CPU 101 multiplies the value of the line length change amount of the target progress ID in the aging data table 501 by a coefficient A. In this embodiment, the coefficient A is set to 20. In the case of the aging data table 501, the CPU 101 multiplies the line length change amount of the progress ID C001_D by 0.3 to calculate a multiplication value of 6.

[0032] Next, in S603, the CPU 101 multiplies the line width change amount value of the target progress ID in the aging data table 501 by a coefficient B. In this embodiment, the coefficient B is set to 10. In the case of table 501, the CPU 101 multiplies the line width change amount of the progress ID C001_D by 0.3 to calculate a multiplication value of 3.

[0033] Next, in S604, the CPU 101 multiplies the intersection change amount value of the target progress ID in the aging data table 501 by a coefficient C. In this embodiment, the coefficient C is set to 1. In the case of table 501, the CPU 101 multiplies the intersection change amount of the progress ID C001_D by 0 to calculate a multiplied value of 0. After S604, the CPU 101 proceeds to processing in S606.

[0034] On the other hand, when proceeding to S605, the CPU 101 multiplies the area change amount value of the target progress ID in the aging data table 501 by a coefficient D. In this embodiment, the coefficient D is set to 100. In the case of table 501, the CPU 101 multiplies the area change amount of the progress ID T001_D by 0.05 to calculate a multiplication value of 5. After S605, the CPU 101 proceeds to processing in S606.

[0035] The values ​​of coefficients A to D are weighted so that the order of priority is line length change > line width change > area change > intersection change, and in this example, coefficient A is set to 20, coefficient B to 10, coefficient C to 1, and coefficient D to 100. These may be changed depending on the value of the aging data table 501, and the coefficient values ​​may be set by normalizing them using units such as m (meters) or mm (millimeters).

[0036] Next, in S606, the CPU 101 adds up the values ​​multiplied in S602 to S604 or in S605, and enters the result in the priority item of the aging data table 501, as shown in FIG. 5(b).

[0037] Thereafter, in S607, the CPU 101 determines whether processing has been completed for all progress IDs. If processing has been completed for all progress IDs (Yes), the CPU 101 terminates the processing of the flowchart in Fig. 6, and if there are unprocessed progress IDs (No), the CPU 101 sets the unprocessed progress IDs as the IDs to be processed and proceeds to S601.

[0038] FIG. 7 is a flowchart showing the detailed flow of the attention area identification process in S405 of FIG. In S701, the CPU 101 obtains information about the display resolution of the display of the display unit 105 of the information processing device 100. Here, a liquid crystal display is used as an example, and the display resolution is assumed to be full HD. The display resolution of full HD is assumed to be 1920 x 1080.

[0039] In S702, the CPU 101 acquires coordinate information of the deformation data for the deformation ID to be processed from the second deformation data table 302 shown in Fig. 3(b) and calculates the coordinates of the center. For example, a calculation method can be used in which the average value of the maximum and minimum values ​​of the horizontal X coordinate and the maximum and minimum values ​​of the vertical Y coordinate from the coordinates in the deformation data table is used as the center coordinate (Xm, Ym).

[0040] The processing of S702 will be further explained with reference to FIGS. 8(a) to 8(i). FIG. 8( a ) is a diagram showing an example of a progression deformation superimposed image 800 and a crack 801 . The progressing deformation superimposed image 800 is an example of an image in which a detected crack is superimposed on the second input image. In the example shown, the crack 801 is composed of solid and dotted lines, with the solid lines representing the crack obtained from the first deformation data and the dotted lines representing the crack obtained from the second deformation data, i.e., the part that has progressed due to aging. For convenience of explanation, the existing part is represented by a solid line and the progressing part by a dotted line, but as an alternative method, the progressing part may be emphasized by changing the color or thickening the line.

[0041] 8(b) is a diagram showing the center 812 of the crack 801, and the coordinates of the center 812 are (Xm, Ym). In this example, the average value is calculated for the center 812, but a method of calculating a center of gravity value or other methods may also be used.

[0042] Next, in S703, CPU 101 sets a rectangular area covering the deformation based on the coordinates (center coordinates) of center 812 calculated in S702 and the display resolution acquired in S701, and saves information about the rectangular area in an area table, an example of which is shown in Fig. 9(a), which will be described later. For example, if the display resolution acquired in S701 is 1920 x 1080 and the center coordinates acquired in S702 are (Xm, Ym), the rectangular area is represented by an area whose upper left coordinates are (Xm-960, Ym-540) and whose lower right coordinates are (Xm+960, Ym+540).

[0043] FIG. 9A is a diagram showing an example of the region table 901. As shown in FIG. The region table consists of a region ID, a progress ID, rectangular coordinates, and a priority. The progress ID and priority represent the progress ID and priority in the secular change data table in Figure 5, and region IDs are assigned in order of priority. The rectangular coordinates represent the top left and bottom right coordinates of the rectangular region mentioned above.

[0044] FIG. 8( c ) is a diagram showing an example in which a rectangular area represented by an outer frame 831 is calculated from the center 812 of the crack 801 . 9(b) is a diagram showing an example of a confirmation screen 951 displayed on the display unit 105. The confirmation screen 951 is a screen in which a rectangular area is cut out from the progression deformation superimposed image 800. Also, as in the case of crack 802 in Figure 8(a), there are cases where the center coordinates are close to the edge (top, bottom, left or right edge) of the progression deformation superimposed image 800, making it impossible to ensure a rectangular area of ​​the display resolution. In such cases, a rectangular area indicated by an outer frame 832 in which the area outside the image is filled in black may be generated, as in Figure 8(d). As another method, a rectangular area indicated by an outer frame 833 may be set so that areas outside the edge are not selected, as in Figure 8(e), so that the center coordinates of the deformation do not have to be the center of the rectangular area.

[0045] In this embodiment, an example is given in which the area is displayed at actual size on the display screen. However, the magnification is not limited to actual size, and the worker may set the magnification as long as the worker can confirm the developing deformation. For example, as shown in the example of FIG. 8(f), if the crack 803, including the developing portion, does not fit within the rectangular area of ​​the outer frame 835, the display magnification may be changed so that the entire deformation is contained within the rectangular area, as shown in the example of FIG. 8(g). In this case, the user interface (UI) as indicated by the zoom button 871 may be used to change the display magnification to a value that allows the worker to confirm the deformation. Furthermore, as shown in the example of FIG. 8(h), the crack 803 may be rotated by any angle, such as 90 degrees, so that the entire crack 803 is displayed at actual size within the rectangular area. Furthermore, as shown in the example of FIG. 8(i), the display position may be changed vertically so that the developing portion of the crack 803 is displayed at actual size within the rectangular area.

[0046] Next, in S704, CPU 101 determines whether processing of all IDs to be processed has been completed. If processing of all IDs has been completed (Yes), CPU 101 ends the processing of the flowchart in Fig. 7, and if other IDs remain to be processed (No), CPU 101 returns to S702 and changes the processing target to the next ID.

[0047] As described above, in this embodiment, the information processing device 100 calculates the degree of progression of a deformation detected from two images taken at different times, identifies the area covering the deformation based on the display resolution, and displays the area of ​​the deformation according to the degree of progression. As a result, the information processing device 100 of this embodiment can display, from among multiple deformations, deformations that have a high degree of progression and should be checked with priority, according to the degree of progression, at a resolution that can be confirmed by the worker.

[0048] In this embodiment, the process of S401 for acquiring deformation data for the first input image is an example of a process using previously detected deformation data. However, deformation data may also be re-detected using, for example, newly established current standards. For example, the performance of the processing algorithm for detecting deformations may improve since generating the first deformation data, which is previously processed data, making it possible to detect more deformations than in the previous first deformation data. In such cases, detecting deformation data using a new processing algorithm makes it possible to compare deformations that were not detectable when the first input image was captured with the current second deformation, enabling a more accurate understanding of developing deformations. Similarly, previously detected deformation data may also be used as input data in the process of S402 for acquiring deformation data for the second input image.

[0049] Furthermore, in this embodiment, the first input image and the second input image are images that are aligned on the drawing, and no alignment process between the images is required. Therefore, data on aging can be created simply by calculating the difference. On the other hand, for example, if the first input image and the second input image are not aligned, alignment process between the images may be performed, and then the difference between the images may be calculated to create aging data. The alignment process in this example may use existing technology. For example, an information processing device may calculate feature points from each image and align the images based on corresponding feature points. Alternatively, for example, an operator may manually specify corresponding points between the images to perform alignment.

[0050] Furthermore, in this embodiment, the degree of progress is calculated based on the difference between the first deformation data and the second deformation data. However, the information processing device may also calculate the degree of progress by finding the difference using a third deformation data set obtained at a different time. Using the differences between three or more deformation data sets makes it possible to calculate an accelerated degree of progress. The value of the coefficient by which the value of the accelerated degree of progress is multiplied may also be changed. For example, if the degree of progress is large, the corresponding coefficient value in the flowchart of FIG. 6 can be increased (made larger) to give it a higher priority, allowing workers to check it first.

[0051] Furthermore, although this embodiment provides an example of calculating the priority based on the degree of progression of the deformation, the priority may also be calculated using information representing the attributes of the structure. For example, in the case of a bridge deck, even if the degree of progression of cracks is the same, the items that the survey and inspection workers prioritize will be different between the area close to the pier and the area intermediate between the piers. For this reason, the information processing device uses, for example, information representing the attributes of the structure, information on the area close to the pier and the area intermediate between the piers, and changes the value of the coefficient when calculating the priority. Similarly, because the items that are prioritized differ between the upper and lower parts of the pier, the information processing device may use these as attribute information of the structure to change the value of the coefficient used in calculating the priority.

[0052] In addition, in this embodiment, an example is given in which the deformation progresses over time, but if the deformation is repaired between the capture of the first input image and the capture of the second input image, the crack line representing the difference may become shorter or the area may decrease. In this case, the difference amount will be negative, but since the deformation has not progressed, the difference amount may be set to 0 and removed from the display.

[0053] In the description of this embodiment, the processing executed by the CPU 101 of the information processing device 100 is represented as each process (step) in the flowchart of Fig. 4, but each process of each process can also be represented as a functional block diagram consisting of each function formed by the CPU 101. Each function in the functional block is generally similar to each process in Fig. 4, so illustration and description thereof will be omitted.

[0054] <Modification 1 of the First Embodiment> In the priority calculation process of the first embodiment, priorities are calculated by multiplying a predetermined coefficient. In Modification 1 of the first embodiment, an example will be described in which the order in which priorities are presented to an inspection and investigation worker can be changed by allowing the worker to change the priority calculation criteria (calculation rules) on a UI.

[0055] FIG. 10(a) is a diagram showing an example of a UI screen 1000 presented to an investigation and inspection worker who is a user. The UI screen 1000 allows the worker to select an item to be emphasized when calculating priority, and is composed of radio buttons 1010 and important items 1021 to 1024. The radio button 1010 is a button that is indicated when the worker selects one of the important items 1021 to 1024. The important items 1021 to 1024 that can be selected using the radio button 1010 correspond to coefficients A to D described in the flowchart of FIG. 6. In Modification 1 of the first embodiment, the coefficient value of the important item selected using the radio button 1010 is made relatively high, and the coefficients of the items that are not selected are made relatively low.

[0056] The above is the UI screen that allows the operator to change the priority calculation criteria (priority calculation rules) in Modification 1 of the first embodiment. This allows the operator to change the priority calculation rules, and thus the operator can change the items that the operator wants to emphasize.

[0057] <Modification 2 of the First Embodiment> In the first modification of the first embodiment, an example of a UI screen 1000 using radio buttons is given, but in the second modification of the first embodiment, an example of a UI for changing the priority order of important items will be described.

[0058] FIG. 10(b) is a diagram illustrating an example of a UI screen 1050 according to the second modification of the first embodiment. The UI screen 1050 includes priority levels 1061 to 1064 and important items 1071 to 1074. On the UI screen 1050, each of the important items 1071 to 1074 is selected in the order listed in the priority levels 1061 to 1064, allowing the items to be moved or changed. Based on the changed priority level, the strength or magnitude of the coefficient value of the important item is changed. In the example of FIG. 10(b), coefficient B, which represents line width, is set as the first priority, coefficient D, which represents area, is set as the second priority, coefficient C, which represents intersections, is set as the third priority, and coefficient A, which represents line length, is set as the fourth priority. In this case, the information processing device sets the magnitude of the value to be set for each coefficient such that coefficient B > coefficient D > coefficient C > coefficient A. This allows the priority to be changed according to the operator's intention.

[0059] <Modification 3 of the First Embodiment> In Modifications 1 and 2 of the first embodiment, the worker checks (visually confirms) the screen of the developing deformation by area. In Modification 3 of the first embodiment, an example will be described in which the developing deformation is displayed at a display resolution that allows the worker to view (confirm) it, and the entire image is also displayed.

[0060] FIG. 11(a) is an example of a confirmation screen 1101. The confirmation screen 1101 displays the same area as the confirmation screen 951 in FIG. 9(b), but a reduced image 1111 is displayed in FIG. 11(a). The reduced image 1111 displays a rectangular area 1121 displayed on the confirmation screen 1101 and a rectangular area 1131 displayed on a confirmation screen 1151 in FIG. 11(b), which will be described later. Note that the rectangular area 1121 corresponds to the area of ​​the outer frame 831 in FIG. 8(a) of the first embodiment, and the rectangular area 1131 corresponds to the area of ​​the outer frame 833 in FIG. 8(a) of the first embodiment. In FIG. 11(a), the rectangular area 1121 is highlighted. Similarly, in the confirmation screen 1151 in FIG. 11(b), when the rectangular area 1131 is displayed on the display unit 105, the rectangular area 1131 is highlighted.

[0061] 11(a) and 11(b) show an example in which two rectangular areas are displayed in the reduced image 1111, but three or more rectangular areas including developing deformations may be displayed. Also, when the survey and inspection worker selects the area to be checked next in the reduced image 1111, for example by clicking on it, the selected area may be specified. Also, the information processing device may change the way in which the areas to be checked by the worker are presented by changing the color, color density, or color saturation of the outer frame of the rectangular areas in order of priority.

[0062] As described above, according to the third variant of the first embodiment, the area to be checked can be selected at the discretion of the worker. For example, while checking an area with a high priority, the worker can continue to check an area with a low priority that has abnormalities in the surrounding area. Alternatively, instead of displaying a thumbnail image, it is also possible to display the items in a list format in order of priority, as shown in list 1175 in FIG. 11(c), and make the items selectable.

[0063] <Fourth Modification of the First Embodiment> In Modifications 1, 2, and 3 of the first embodiment, the priority is calculated by changing the coefficient depending on the progression of the deformation, such as an increase in the crack line length or width, or an increase in the area of ​​exposed rebar, as shown in the flowchart of Fig. 6. In Modification 4 of the first embodiment, an example will be described in which the priority is calculated based on the rate at which the deformation is progressing (degree of progression, progression rate).

[0064] Figures 12(a) and 12(b) show examples of cracks present in the same inspection target image. Crack 1200 in Figure 12(a) and crack 1250 in Figure 12(b) are examples of cracks present in the same inspection target image and represent second deformation data. Crack 1200 in Figure 12(a) is composed of, for example, a 4 mm new crack portion 1211 (dotted line) and a 10 mm existing crack portion 1212 (solid line). Crack 1250 in Figure 12(b) is composed of, for example, a 4 mm new crack portion 1261 (dotted line) and a 5 mm existing crack portion (solid line). The line lengths of the new crack portions of crack 1200 and crack 1250 are the same, but the proportion of the new crack portion relative to crack 1200 is approximately 29% (= 4 / 14), while the proportion of the new crack portion relative to crack 1250 is approximately 44% (= 4 / 9). In this case, crack 1250 in Figure 12(b), which has a high rate of crack progression, has a high degree of progression (progression rate), so the information processing device changes the coefficient of the crack with the higher degree of progression (progression rate) so that it has a higher priority.

[0065] In this way, in variant example 4, even if the amount of progression of the deformation (in the above example, the line length of the new crack) is the same, if the degree of progression of the deformation is different, the index for calculating the priority is changed based on the degree of progression. Although the above example uses line segment length as an example, the information processing device may calculate the degree of progression (progression rate) of a deformation using the difference between the existing width and the expanded width, the existing area and the expanded area, or other difference values. Furthermore, instead of using the degree of progression (proportion, rate) of a single deformation, multiple degrees of progression may be used. Furthermore, the amount of progression of the deformation and the degree of progression of the deformation may be used in combination.

[0066] <Second embodiment> In the first embodiment, a screen showing the developing deformation is displayed, and one survey and inspection worker checks (visually confirms) each area. In the second embodiment, information on the developing deformation is stored in a server device and distributed to multiple client terminals, which are information processing devices, so that multiple workers can perform the checking process.

[0067] Fig. 13 is a diagram showing an example of a system configuration in the second embodiment. The system configuration shown in Fig. 13 is configured with a server device 1301 and multiple information processing devices 1311 to 1313, which are connected to each other so as to be able to communicate via a network 1305 or the like. Note that Fig. 13 shows an example in which there are three information processing devices, but the number is not limited to three. The hardware configurations of the server device 1301 and the information processing devices 1311 to 1313 are assumed to be the same as the hardware configurations described in Fig. 1.

[0068] 14 is a flowchart showing the flow of information processing in the server device 1301. The same processes as those in the information processing device 100 of the first embodiment are assigned the same numbers, and the description thereof will be omitted. In S1401, the CPU 101 of the server device 1301 divides the region table 901, as shown in FIG. 9A, identified and generated in S405, according to the number of information processing devices, and distributes the divided regions to each information processing device via the communication unit 107. When dividing the region table 901, the region IDs in the region table 901 are divided according to the number of information processing devices. For example, as shown in FIG. 13, if there are three information processing devices on the client terminal side and the region table 901 has 90 region IDs, the CPU 101 of the server device 1301 divides the region table 901 into three regions of 30 region IDs each. The server device 1301 then sends the region table consisting of 30 region IDs to the information processing device 1311, the region table consisting of the next 30 region IDs to the information processing device 1312, and the region table consisting of the remaining 30 region IDs to the information processing device 1313.

[0069] 15 is a flowchart showing the flow of information processing in the information processing device 1311. The other information processing devices 1312 and 1313 also perform similar information processing. In step S1501 , the CPU 101 of the information processing device 1311 acquires the area table information sent from the server device 1301 via the communication unit 107 . Next, in S1502, the CPU 101 of the information processing device 1311 displays the area to be checked based on the area table acquired in S1501, similar to S406 in Fig. 4. This allows the operator of the information processing device 1311 to check.

[0070] In the second embodiment, the processing that was performed within the information processing device in the first embodiment is divided between the server device and the client terminal, so that the second embodiment allows multiple survey and inspection workers to share the work of checking many areas of progressing abnormalities, thereby reducing the time required for the checking work.

[0071] The first deformation data and the second deformation data may be registered on the server device, or may be registered on the server device from any client terminal. In addition, in S1401, an example was given in which the area table was simply divided into equal parts by a predetermined number, but the allocation at the time of division may be changed depending on the priority of the progression of the abnormality. For example, if there is a difference in the level of expertise among the inspectors, the highly skilled inspectors may be assigned area IDs containing high-priority abnormalities, and the less skilled inspectors may be assigned area IDs with low priority. This makes it possible to assign inspectors with different levels of expertise according to priority, enabling efficient confirmation of progressing abnormalities.

[0072] As explained above, the information processing devices of the first and second embodiments calculate a priority according to the importance of an evolving deformation, and can change the display method for the area of ​​the evolving deformation. That is, according to the information processing devices of the first and second embodiments, the priority of the area including the evolving deformation is calculated, and the area including the deformation to be displayed is changed based on the calculated priority. This enables the survey and inspection worker to efficiently inspect (confirm) the deformation that should be checked with priority.

[0073] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. The above-described embodiments are merely examples of specific implementations of the present invention, and should not be construed as limiting the technical scope of the present invention. In other words, the present invention can be implemented in various forms without departing from its technical concept or main features. [Explanation of symbols]

[0074] 100: Information processing device, 101: CPU, 102: ROM, 103: RAM, 105: Display unit

Claims

1. An acquisition means for acquiring a plurality of deformation data relating to deformation of the structure, each of which is generated from a plurality of images of the structure taken at different times; A detection means for detecting changes over time in the deformation of the structure based on the plurality of deformation data; an identification means for identifying an area including a portion showing the deformation of the structure and the progress of the aging change of the deformation based on the detection result by the detection means; a display means for displaying the deformation data acquired by the acquisition means and an image of the structure in superimposed order according to the degree of progress of the aging of the structure's deformation, so that the part showing the deformation of the structure and the part showing the progress of the aging of the deformation can be distinguished based on the detection result by the detection means and the area identified by the identification means; An information processing device comprising:

2. The information processing device described in claim 1, characterized in that when the deformation is a crack, the part indicating the deformation of the structure and the part indicating the progression of the deformation over time are displayed with different line colors, line types, or line thicknesses.

3. The information processing device described in claim 1 or 2, characterized in that the display means first displays the area identified by the identification means for the deformation of the structure that has the greatest degree of progression over time and the part that shows the progression of the deformation over time.

4. 4. The information processing apparatus according to claim 1, wherein the display unit displays the area identified by the identifying unit based on a display resolution that can be visually recognized by the worker.

5. 5. The information processing apparatus according to claim 1, wherein the display unit displays the area specified by the specifying unit and also displays an entire image including the area in a superimposed manner.

6. further comprising an operation means for accepting a user operation; The information processing device described in any one of claims 1 to 5, characterized in that, in response to the completion of work on the displayed area by the user operation, the display means displays, based on the detection results of the detection means, the deformation of the structure that has the next greatest degree of progression in the deterioration over time among the deformations included in the displayed area, and a part showing the progression of the deterioration over time of the deformation.

7. 7. The information processing apparatus according to claim 6, wherein the display means displays the area identified by the identification means further based on the priority of the inspection items for the abnormality set by the operation means.

8. An acquisition means for acquiring a plurality of deformation data relating to deformation of the structure, each of which is generated from a plurality of images of the structure taken at different times; A detection means for detecting changes over time in the deformation of the structure based on the plurality of deformation data; an identification means for identifying an area including a portion showing the deformation of the structure and the progress of the aging change of the deformation based on the detection result by the detection means; a transmitting means for transmitting to a client device data for superimposing and displaying the deformation data acquired by the acquiring means and an image of the structure in an order according to the degree of progress of the aging of the deformation of the structure, so that the part showing the deformation of the structure and the part showing the progress of the aging of the deformation can be distinguished based on the detection result by the detecting means; A server device comprising:

9. 9. The server device according to claim 8, further comprising a connection means for connecting to the client device via a network.

10. The server device described in claim 8 or 9, characterized in that the transmitting means first transmits the data to the client device so that the area identified by the identifying means is displayed, regarding the deformation of the structure that has the greatest degree of progression over time and the part that shows the progression of the deformation over time.

11. An acquisition step of acquiring a plurality of deformation data related to deformation of the structure, each of which is generated from a plurality of images of the structure taken at different times; a detection step of detecting changes over time in the deformation of the structure based on the plurality of deformation data; a step of identifying an area including a portion showing the deformation of the structure and the progress of the aging change of the deformation based on the detection result of the detection step; a display step of superimposing and displaying the deformation data acquired in the acquisition step and an image of the structure in an order according to the degree of progress of the aging of the deformation of the structure, so that the part showing the deformation of the structure and the part showing the progress of the aging of the deformation can be distinguished based on the detection result of the detection step and the area identified in the identification step; 1. A method for controlling an information processing device, comprising:

12. An acquisition step of acquiring a plurality of deformation data related to deformation of the structure, each of which is generated from a plurality of images of the structure taken at different times; a detection step of detecting changes over time in the deformation of the structure based on the plurality of deformation data; a step of identifying an area including a portion showing the deformation of the structure and the progress of the aging change of the deformation based on the detection result of the detection step; a display step of superimposing and displaying the deformation data acquired in the acquisition step and an image of the structure in an order according to the degree of progress of the aging of the deformation of the structure, so that the part showing the deformation of the structure and the part showing the progress of the aging of the deformation can be distinguished based on the detection result of the detection step and the area identified in the identification step; A program that causes a computer to execute the following.

13. An acquisition step of acquiring a plurality of deformation data related to deformation of the structure, each of which is generated from a plurality of images of the structure taken at different times; a detection step of detecting changes over time in the deformation of the structure based on the plurality of deformation data; a step of identifying an area including a portion showing the deformation of the structure and the progress of the aging change of the deformation based on the detection result of the detection step; a transmission step of transmitting to a client device data for superimposing and displaying the deformation data acquired by the acquisition step and an image of the structure in an order according to the degree of progress of the aging of the deformation of the structure, so that the part showing the deformation of the structure and the part showing the progress of the aging of the deformation can be distinguished based on the detection result by the detection step; 10. A method for controlling a server device, comprising:

14. An acquisition step of acquiring a plurality of deformation data related to deformation of the structure, each of which is generated from a plurality of images of the structure taken at different times; a detection step of detecting changes over time in the deformation of the structure based on the plurality of deformation data; a step of identifying an area including a portion showing the deformation of the structure and the progress of the aging change of the deformation based on the detection result of the detection step; a transmission step of transmitting to a client device data for superimposing and displaying the deformation data acquired by the acquisition step and an image of the structure in an order according to the degree of progress of the aging of the deformation of the structure, so that the part showing the deformation of the structure and the part showing the progress of the aging of the deformation can be distinguished based on the detection result by the detection step; A program that causes a computer to execute the following.

Images