Information processing device, information processing method, and program

Abstract

To achieve a technique capable of managing all deformation information detected from images obtained by photographing an inspection object by using the unique identification information given to each defect information.SOLUTION: An information processing device includes: input means for inputting one or more images belonging to a predetermined group; detection processing means for executing processing in which a deformed shape is detected for each image inputted by the input means; setting means for setting unique identification information in the predetermined group to deformation information detected by the detection processing means; and output means for outputting a detection result including the deformation information to which the identification information is given by the setting means.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a technique for detecting anomalies in an image of an inspection object. [Background technology]

[0002] When inspecting infrastructure structures, images of the object to be inspected are used to detect abnormalities, and the detection results are managed in correspondence with the images of the object to be inspected.

[0003] Patent Document 1 describes a method for displaying an image of an inspection target in association with drawing information of the inspection target structure, and inputting the abnormality. In this case, a high-definition image is required to detect the abnormality from the image of the inspection target, but since the image of the entire inspection target structure is very large, it requires a lot of effort to detect and input the abnormality.

[0004] Patent Document 2 describes a method of detecting abnormalities such as cracks by performing noise removal processing using wavelet transform on an image of an inspection target. Patent Document 3 describes a method of dividing an image of the entire structure to be inspected into images of a size suitable for image processing for abnormality detection, and performing abnormality detection on the divided images. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2005-310044 A [Patent Document 2] Patent No. 6099479 [Patent Document 3] Patent No. 5645730 Summary of the Invention [Problem to be solved by the invention]

[0006] However, when the deformation information detected for each divided image, as in Patent Documents 2 and 3, is managed in association with drawing information of the structure being inspected, as in Patent Document 1, if the same identification information is assigned to different deformation information, it becomes difficult to collectively manage the deformation information of the entire structure being inspected.

[0007] The present invention has been made in consideration of the above-mentioned problems, and its purpose is to realize a technology that can manage all abnormality information detected from an image of an inspection object using unique identification information assigned to each abnormality information. [Means for solving the problem]

[0008] In order to solve the above problems and achieve the object, the information processing device of the present invention has an input means for inputting one or more images belonging to a specified group, a detection processing means for executing a process to detect abnormalities for each image input by the input means, a setting means for setting unique identification information within the specified group for the abnormality information detected by the detection processing means, and an output means for outputting detection results including the abnormality information to which the identification information has been assigned by the setting means. Effect of the Invention

[0009] According to the present invention, it becomes possible to manage all abnormality information detected from an image of an inspection object using unique identification information assigned to each abnormality information. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic explanatory diagram of a first embodiment. [Diagram 2] FIG. 4 is a diagram for explaining a deformation information list according to the first embodiment. [Diagram 3] 1A is a block diagram showing a hardware configuration of an image processing apparatus according to a first embodiment, and FIG. 1B is a functional block diagram. [Figure 4] 4 is a flowchart for explaining a control process according to the first embodiment. [Diagram 5]4A to 4C are diagrams for explaining a process for detecting a change in a detection target. [Figure 6] 5A and 5B are diagrams for explaining outer shape information of partial images. [Figure 7] 13A and 13B are diagrams illustrating an example of offset arrangement of deformation information for each partial image. [Figure 8] FIG. 4 is a diagram illustrating a selection screen for an offset pattern. [Figure 9] FIG. 11 is a functional block diagram of an image processing apparatus according to a second embodiment. [Figure 10] 10 is a flowchart illustrating a control process according to the second embodiment. [Figure 11] 10A to 10C are views for explaining coordinate conversion processing according to the second embodiment. [Figure 12] 13A to 13C are diagrams for explaining an individual viewer and a drawing viewer according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, the embodiments will be described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicated descriptions are omitted.

[0012] [Embodiment 1] In the following, an embodiment will be described in which an information processing device of the present invention is applied to a computer device used for inspecting infrastructure such as a concrete structure, which is an example of an inspection target.

[0013] In embodiment 1, an example is described in which a computer device operates as an information processing device, performs an abnormality detection process on multiple partial images obtained by dividing the entire structure to be inspected into multiple parts and photographing them, assigns unique, mutually distinguishable identification information to each detected abnormality, and manages the information in association with the drawing information of the structure to be inspected.

[0014] The main terms used in the description of this embodiment are defined as follows.

[0015] The "inspection target" refers to a concrete structure that is the target of infrastructure inspection, such as a highway, a bridge, a tunnel, a dam, etc. The information processing device performs an anomaly detection process that detects the presence or absence and state of anomalies such as cracks using an image of the inspection target captured by a user.

[0016] "Deformation" refers to, for example, cracks, floating, or spalling of concrete in the case of concrete structures. It also includes efflorescence, exposed rebar, rust, water leakage, dripping, corrosion, damage (missing parts), cold joints, deposits, junk, etc.

[0017] "Deformation information" refers to unique identification information assigned to each deformation and coordinate information indicating the position and shape of the deformation.

[0018] The "deformation information list" is information that compiles a list of deformation information to which identification information has been assigned.

[0019] The "partial images" are individual images (detection images) obtained by photographing an entire structure to be inspected after dividing it into a number of parts.

[0020] The "image coordinate system" is a planar Cartesian coordinate system that represents coordinate positions in a partial image.

[0021] The "drawing coordinate system" is a plane Cartesian coordinate system that represents the coordinate position in the drawing information of the structure to be inspected.

[0022] <Summary> First, an overview of this embodiment will be described with reference to FIGS. 1 and 2. FIG.

[0023] When inspecting infrastructure such as concrete structures, inspectors visually inspect the walls of the structure being inspected and record any abnormalities they find. When inspecting using images of the inspection target (detection images), inspectors record information about the abnormalities, such as the position and shape of the abnormalities found in the detection images, as the inspection results. The abnormality information is managed in association with the drawing information of the inspection target structure together with the detection images. In this case, it is very laborious for inspectors to find and record all abnormalities in the detection images, so it is desirable to automatically detect and record the abnormalities using image processing using a computer device.

[0024] Furthermore, to detect abnormalities from detection images, high-resolution images are required, but it is difficult to capture the entire structure to be inspected in one image while maintaining the resolution required for image processing. A single image of the entire structure can be generated by synthesizing multiple images taken at different shooting positions relative to the structure to be inspected, but the image size becomes very large, and due to restrictions such as the memory capacity of the information processing device, it is difficult to execute abnormality detection processing on a large-sized image at once.

[0025] Therefore, in the anomaly detection process of this embodiment, the anomaly detection process is performed on a partial image obtained by photographing a part of the structure to be inspected. The anomaly information obtained as a detection result is managed in association with the drawing information of the structure to be inspected, and identification information for distinguishing the anomaly information from one another, as well as information on the shape, position, and the like, are output as a list. Furthermore, in this embodiment, to facilitate the task of accurately aligning the anomaly information with the drawing information of the structure to be inspected and superimposing it on the drawing information, external shape information of the partial image is added to the anomaly information list.

[0026] Fig. 1(a) illustrates a drawing 100 depicting a bridge deck 101 as an example of an inspection target. Fig. 1(b) illustrates a plurality of partial images 111-114 captured by dividing the deck 101 into a plurality of portions (four regions).

[0027] The user selects one partial image for which anomaly detection processing is to be performed from the partial images 111 to 114. There is no particular restriction on the order of selection. For example, when the user selects the partial image 113 located at the bottom left, the information processing device performs anomaly detection processing on the selected partial image 113 and acquires anomaly information.

[0028] Fig. 1(c) illustrates an example of a state in which an abnormality detection process is performed on the partial image 113, and the acquired abnormality information 121 is superimposed and displayed on the partial image 113. Fig. 1(d) illustrates an example of a state in which an abnormality detection process is performed on the partial image 111, and the acquired abnormality information 122 is superimposed and displayed on the partial image 111.

[0029] FIG. 2(a) shows an example of a deformation information list obtained by performing deformation detection processing on partial image 113. One piece of deformation information is treated as a one-line record, and each record includes unique identification information (ImageFileA_001, ImageFileA_002, ...) and a set of coordinate information indicating the position and shape of the deformation. A unique character string that is not duplicated within the entire structure being inspected is assigned as the identification information for each deformation. A method for generating identification information for each deformation will be described later.

[0030] The abnormality information obtained by executing the abnormality detection process for each partial image is managed in association with the drawing information (hereinafter, drawing information) of the structure to be inspected, which is created by a design support tool such as CAD. Here, in order to accurately align the abnormality information with the drawing information and display it superimposed, it is necessary to adjust the position and scale of the abnormality information. FIG. 1(e) illustrates an example of a state in which the abnormality information 121 and the abnormality information 122 are aligned and superimposed on the drawing 100 of the floorboard 101. In this case, various inconveniences occur when the identification information of the abnormality information is duplicated. For example, abnormality information with the same identification information is mixed in one drawing information, and the identification information no longer plays a role of uniquely identifying the abnormality information. Or, one abnormality information that is different from each other but has the same identification information is overwritten by the other abnormality information.

[0031] In view of this background, in this embodiment, as shown in Fig. 2(a), unique identification information that does not overlap with the entire structure to be inspected is added to the abnormality information detected from the partial image. This makes it possible to reliably associate the abnormality information with the drawing information and manage it with little effort.

[0032] Fig. 2(b) illustrates an example of a state in which the external shape information (FRAME1) of the partial image 113 has been added to the abnormality information list obtained by executing the abnormality detection process on the partial image 113 shown in Fig. 2(a). Also, Fig. 2(c) illustrates an example of a state in which the external shape information (FRAME2) of the partial image 111 has been added to the abnormality information list obtained by executing the abnormality detection process on the partial image 111. As with the identification information of the abnormality information, the external shape information is given identification information that is unique for the entire structure to be inspected.

[0033] As described above, in order to accurately align the deformation information with the drawing information and display it superimposed, it is necessary to adjust the position and scale of the deformation information, which requires the user to perform time-consuming work. Specifically, the user must repeat trial and error by manually aligning the vector data of the deformation information loaded into the drawing information or changing the magnification ratio of the vector data so that the vector data overlaps with the deformation reflected in the partial image. Therefore, in this embodiment, the external shape information of the partial image, which is useful when aligning the deformation information with the drawing information, is added to the deformation information list.

[0034] FIG. 1(f) illustrates an example of a state in which the deformation information 121 and the deformation information 122 are pasted and aligned together with the contour information of the partial image on the drawing 100 of the floorboard 101. When the contour information of the partial image is available, the position and scale of the contour information of the partial image can be adjusted to match the partial image in the drawing information, so that the deformation information can be accurately superimposed on the drawing information with little work. For example, the user can accurately superimpose the vector data of the deformation information read into the drawing information on the deformation reflected in the partial image by simply aligning the vertices of the contour information of the partial image with the partial image in the drawing information and changing the scale so that the diagonal vertices overlap. FIG. 2(d) illustrates an example of an deformation information list that integrates the deformation information detected from the partial image 113 and the partial image 111.

[0035] On the other hand, if the contour information is not available, as described above, the user must repeatedly perform time-consuming tasks such as aligning the position of the deformation with the drawing information and changing the magnification ratio of the vector data of the deformation information.

[0036] In this way, by assigning unique identification information to the abnormality information detected from the partial images, different abnormality information can be managed in an integrated manner for the entire structure to be inspected, without duplication or overwriting. Also, by including the external shape information of the partial images in the abnormality information list, it becomes easier to align the abnormality information with the drawing information.

[0037] <Hardware configuration> Next, the hardware configuration of the information processing apparatus according to the first embodiment will be described with reference to FIG.

[0038] FIG. 3A is a block diagram showing the hardware configuration of an information processing apparatus 300 according to the first embodiment.

[0039] In the first embodiment, a computer device operates as the information processing device 300. The processing of the information processing device of this embodiment may be realized by a single computer device, or may be realized by distributing each function among a plurality of computer devices as necessary. The plurality of computer devices are connected to each other so as to be able to communicate with each other.

[0040] The information processing device 300 includes a control unit 301 , a non-volatile memory 302 , a work memory 303 , a storage device 304 , an input device 305 , an output device 306 , a network interface 307 , and a system bus 308 .

[0041] The control unit 301 includes an arithmetic processor such as a CPU or MPU that controls the entire information processing device 300. The non-volatile memory 302 is a ROM that stores programs and parameters executed by the processor of the control unit 301. Here, the programs refer to programs for executing the processes of the first and second embodiments described below. The work memory 303 is a RAM that temporarily stores programs and data supplied from an external device or the like. The work memory 303 holds data obtained by executing the control process of FIG. 4 described below.

[0042] The storage device 304 is an internal device such as a hard disk or a memory card built into the information processing device 300, an external device such as a hard disk or a memory card detachably connected to the information processing device 300, or a server device connected via a network. The storage device 304 includes a memory card or a hard disk constituted by a semiconductor memory or a magnetic disk. The storage device 304 also includes a storage medium constituted by a disk drive that reads / writes data from / to optical disks such as DVDs and Blue-ray Discs.

[0043] The input device 305 is an operating member such as a mouse, keyboard, or touch panel that accepts user operations and outputs operation instructions to the control unit 301. The output device 306 is a display device such as a monitor or a display configured with an LCD or an organic EL, and displays an abnormality information list created by the information processing device 300 or a server device. The network interface 307 is communicatively connected to a network such as the Internet or a LAN (Local Area Network). The system bus 308 includes an address bus, a data bus, and a control bus that connect the components 301 to 307 of the information processing device 300 to be able to transmit and receive data.

[0044] The non-volatile memory 302 stores an OS (operating system), which is basic software executed by the control unit 301, and applications that cooperate with the OS to realize applied functions. In this embodiment, the non-volatile memory 302 also stores applications that allow the information processing device 300 to realize control processing, which will be described later.

[0045] The control process of the information processing device 300 of this embodiment is realized by reading software provided by an application. It is assumed that the application has software for utilizing basic functions of the OS installed in the information processing device 300. It is also possible for the OS of the information processing device 300 to have software for realizing the control process of this embodiment.

[0046] <Functional configuration> Next, functional blocks of the information processing device 300 according to the first embodiment will be described with reference to FIG.

[0047] FIG. 3B is a functional block diagram of the information processing apparatus 300 according to the first embodiment.

[0048] The information processing device 300 includes a storage unit 321 , a management unit 322 , an image input unit 323 , a detection processing unit 324 , an identification information setting unit 325 , an information adding unit 326 , and an output unit 327 .

[0049] Each function of the information processing device 300 is configured with hardware and / or software. Each functional unit may be configured with one or more computer devices or server devices, and may be configured as a system connected via a network. In addition, when each functional unit shown in Fig. 3(b) is configured with hardware instead of being realized with software, it is sufficient to have a circuit configuration corresponding to each functional unit in Fig. 3(b).

[0050] The storage unit 321 corresponds to the storage device 304, and the management unit 322 can write and read partial images and deformation information thereto.

[0051] The management unit 322 manages the registration, deletion, updating, etc. of partial images, abnormality information, and the like stored in the storage unit 321.

[0052] The image input unit 323 inputs a partial image that is read out from the storage unit 321 by the management unit 322 and is a processing target for executing the abnormality detection process.

[0053] The detection processing unit 324 executes an abnormality detection process on the partial image to be processed input by the image input unit 323, and creates an abnormality information list that summarizes the abnormality information as the detection result.

[0054] The identification information setting unit 325 sets unique identification information for the abnormality information acquired by the abnormality detection process.

[0055] The information adding unit 326 adds the external shape information of the partial image to the abnormality information list.

[0056] The output unit 327 displays the abnormality information list on the output device 306 , and the management unit 322 stores the abnormality information list in the memory unit 321 .

[0057] <Control processing> Next, the control process of the information processing device 300 in this embodiment will be described with reference to FIG.

[0058] The process of FIG. 4 is realized by the control unit 301 of the information processing apparatus 300 shown in FIG. 3(a) expanding and executing the program stored in the non-volatile memory 302 in the work memory 303, thereby controlling each component shown in FIG. 3(a) to operate as each functional unit shown in FIG. 3(b). Further, the process of FIG. 4 is started when the information processing apparatus 300 receives an instruction to start the deformation detection process from the input device 305.

[0059] <S401: Input of partial image to be processed> The image input unit 323 inputs one or more partial images to be processed read from the storage unit 321 by the management unit 322. The partial images to be processed are, for example, one or more partial images belonging to a predetermined group specified by the user. The predetermined group is, for example, a group of images taken of the same inspection target, a group of images taken in the same time period, or a group of images taken at the same location. In the present embodiment, a group of images taken of the same inspection target is treated as images of the same group. Methods for the user to manage images by group include, for example, a method of classifying and storing them in folders / directories of the file system, a method of classifying them according to naming rules such as adding a prefix to the file name, a method of classifying them by attaching information indicating the group to the attributes of the images, and a method of managing them in a database in association with the group information.

[0060] The user designates one or more partial images belonging to a predetermined group at the start of the process. The image file may be directly designated, or it may be designated in units of groups according to the file classification method described above. When designating in units of groups, the image files belonging to the designated group become the partial images to be processed. For example, when the partial images are classified and stored in folders / directories of the file system, by designating the folder of the desired inspection target, a plurality of partial images taken of the same inspection target and stored in the same folder can be designated.

[0061] In this embodiment, an example in which a user designates a partial image belonging to one group will be described, but the user may be able to input partial images of a plurality of groups. In this case, information indicating the relationship between the groups and the partial images belonging to the groups is stored in advance in the storage unit 321 by the management unit 322.

[0062] When the user designates a plurality of partial images, the order of the partial images input by the image input unit 323 in S401 is not particularly limited, but the same partial image is not input repeatedly. In this embodiment, the partial image is described as a single image generated by one shot, but the partial image may be an image obtained by synthesizing a plurality of shot images or dividing an image into a plurality of parts.

[0063] <S402: Deformation Detection Process> The detection processing unit 324 performs deformation detection processing on the partial image input by the image input unit 323 in S401.

[0064] FIG. 5 is a diagram for explaining a process of detecting a crack as an example of the deformation to be detected. In FIG. 5, for the sake of simplicity of explanation, a state in which only one crack is shown in the partial image is illustrated.

[0065] FIG. 5(a) illustrates a partial image 500 in which a crack 511 is shown. FIG. 5(b) illustrates the deformation information detected from the partial image 500. In the example of FIG. 5(b), a state in which the deformation information 512 of the crack 511 detected from the partial image 500 is superimposed on the partial image 500 shown in FIG. 5(a) is illustrated. The deformation information 512 is configured as vector data associated with an image coordinate system 501 with the top vertex 510 at the upper left of the partial image 500 as the origin, and includes positions P1 to Pm. The positions P1 to Pm each have position coordinates in the image coordinate system, and the crack is represented by connecting each position with a straight line. FIG. 5(c) illustrates the data configuration of the vector data of the deformation information 512. Note that the deformation information 512 may be configured as raster data. In this case, the crack is represented by a set of positions in the image coordinate system.

[0066] The anomaly detection process can be performed using a trained model and parameters that have been trained using AI (artificial intelligence) machine learning or deep learning, which is a type of machine learning. The trained model can be configured, for example, as a neural network model. For example, a trained model that has been trained using different parameters for each type of crack as a detection target deformation may be prepared, and a trained model may be used for each crack to be detected, or a general-purpose trained model that can detect various types of cracks may be used. Trained models may also be used based on texture information of a partial image. For example, a method of determining texture information from a partial image is a method of determining the texture information based on spatial frequency information of an image obtained by FFT. The learning process may be performed by a GPU (Graphics Processing Unit). A GPU is a processor that can perform processing specialized for computer graphic calculations, and has a calculation processing capacity that performs matrix calculations and the like required for the learning process in a short time. The learning process is not limited to a GPU, and may be performed by any circuit configuration that performs matrix calculations and the like required for a neural network.

[0067] The trained model and parameters used in the deformation detection process may be obtained from a cloud server connected to the network via the network interface 307. The detection image and parameters may be transmitted to the cloud server, and the cloud server may execute the deformation detection process using the trained model, and the results obtained may be obtained via the network interface 307.

[0068] The anomaly detection process is not limited to a method using a trained model, and may be realized, for example, by performing image processing using wavelet transform or other image analysis or image recognition processing on the detection image as in Patent Document 2. Also, in this case, the detection results of anomalies such as cracks are not limited to vector data, and may be raster data.

[0069] Further, the deformation detection process may be executed in parallel for a plurality of partial images. In this case, in S401, the image input unit 323 inputs a plurality of partial images, and for each partial image, the detection processing unit 324 executes the deformation detection process in parallel, and obtains the detection results of the respective partial images. The obtained detection results are output as vector data in the image coordinate system associated with each partial image.

[0070] <S403: Assign Identification Information to Deformation Information> The identification information setting unit 325 generates and assigns unique identification information for each piece of deformation information detected by the detection processing unit 324 in S402. "Unique" means that there is no duplicate identification information among the pieces of deformation information detected from the partial images belonging to the same group. In the present embodiment, partial images that capture the same inspection target are regarded as partial images of the same group.

[0071] One method of generating identification information is, for example, a method in which a character string that uniquely indicates a partial image is used as a prefix, and a character string obtained by concatenating that and a serial number is used as the identification information. Specifically, for example, a character string that uniquely indicates a partial image (ImageFileA, ImageFileB,...) is used as a prefix, an underscore (_) or a hyphen (-) is used as a concatenation string, and a serial number incremented by one each time a deformation is detected is concatenated at the end (ImageFileA_001, ImageFileA_002,..., ImageFileB_001, ImageFileB_002,...). As the character string that uniquely indicates a partial image, a file path or a file name that is guaranteed to be unique in the file system may be used, or an internal serial number may be assigned at the time of image input. Further, a known identification information generation method such as UUID may be used, or a non-duplicate character string may be generated for each image using a known hash function.

[0072] Other methods of generating identification information include, for example, a method in which the identification information setting unit 325 accepts the issuance of identification information in a single queue and sequentially generates a serial number while incrementing a numerical value. Specifically, each time the detection processing unit 324 detects a change, it requests the identification information setting unit 325 to issue identification information, and returns identification information with a serial number such as 1, 2, 3 in the order of request reception. In this case, a specific character string may be concatenated to the serial number. Since the identification information setting unit 325 issues identification information different from the identification information already issued, the uniqueness of the identification information is maintained. Instead of the serial number, a known identification information generation method such as UUID may be used, or a unique hash value using a known hash function may be used.

[0073] Furthermore, when the change detection process is executed in parallel, it is possible to prevent the duplication of identification information among the execution jobs by concatenating a character string indicating the execution job (such as a job ID) as a prefix.

[0074] <S404: Acquisition of Outline Information of Sub - Image> The information addition unit 326 acquires the outline information of the sub - image input by the image input unit 323 and adds it to the change information list. FIG. 6 illustrates the outline information of the sub - image 601 associated with the image coordinate system 600. In the example of FIG. 6, the upper - left vertex 610 of the sub - image is set as the origin of the image coordinate system 600. The outline information 611 is configured as vector data and includes positions P1 to Pm. The positions P1 to Pm each have position coordinates in the image coordinate system 600, and the outline information is represented by connecting each position with a straight line. Note that the outline information 611 may be configured as raster data. In this case, the outline information is represented by a set of positions in the image coordinate system. Also, the sub - image does not necessarily have to be rectangular. It is conceivable that the sub - image has transparency information and the displayed image has a shape other than a rectangle. In such a case, the outline of the displayed image may be acquired.

[0075] <S405: Assignment of Identification Information to Image Outline> The identification information setting unit 325 generates and assigns identification information to the external shape information of the partial image acquired by the information assignment unit 326 in S404. The identification information is a unique character string that does not overlap within the same inspection target range, similar to the case where the identification information is assigned to the abnormality information in S403. Since the external shape information is added to the abnormality information list, the identification information is set to be unique among abnormality information detected from partial images belonging to different groups. However, the identification information is set to be capable of distinguishing the external shape information from abnormality information of other groups.

[0076] One method for adding identification information to external shape information is to use character strings or symbols indicating the shape of the image (such as "FRAME" or "@") as reserved words or characters and not use them in the identification information of the abnormality information, but to combine them as prefixes or suffixes to create character strings.

[0077] By including the external shape information of the partial image to be processed in the abnormality information list in this way, the user can easily manage the abnormality information by associating it with the drawing information. On the other hand, in order to accurately align and superimpose the abnormality information list that does not include the external shape information on the drawing information, the abnormality information detected from the partial image must first be loaded into a design support tool such as CAD that can view the drawing information. Then, the user must manually align the position and scale of the abnormality information with the partial image in the drawing information.

[0078] When the outline information of the partial image is included in the deformation information list, the user only needs to adjust the position and scale of the outline information to match the range of the partial image, and at the same time, the position and scale of the deformation information can also be adjusted. For example, by aligning the top left vertex of a rectangular partial image with the drawing information and enlarging or reducing the vector data so that the bottom right vertex overlaps, the alignment of the deformation information is possible. In this case, it is desirable to load the partial image into a design support tool such as CAD where the drawing information can be viewed and align the partial image with the drawing information. Furthermore, it is desirable that the partial image is in a range where it is easy to distinguish in the structure to be inspected. Specifically, the partial image may correspond to the slab of one span of a bridge or one span of a tunnel. Furthermore, when the software for viewing the drawing information brings the deformation information including the outline information closer to the range of the partial image displayed in conjunction with the drawing, the software may automatically move or deform the deformation information so that the deformation information fits within the range of the partial image based on the outline information.

[0079] <S406: Determine whether the processing of all partial images is completed> The control unit 301 determines whether the processing of S402 to S405 has been executed for all the partial images input by the image input unit 323 in S401. Then, when the processing is completed, the control unit 301 proceeds to S406, and when the processing is not completed, it returns to S402 and repeats the processing from S402 to S405.

[0080] <S407: Output of the deformation information list> The output unit 327 outputs, as a deformation information list, the information obtained by adding the outline information of the partial image to the deformation information detected from the partial image. The deformation information list is, for example, in a tabular format such as CSV or general CAD data. The deformation information list includes a file in the image coordinate system for each partial image (individual deformation information list) and a file in the drawing coordinate system obtained by integrating the files for each partial image over the entire structure to be inspected (integrated deformation information list), and either one may be output depending on the application.

[0081] The identification information in the integrated abnormality information list may be modified according to a specific rule. For example, a character string indicating integration (such as "MERGED_") may be added as a prefix to the beginning of the identification information. The rule for modifying the identification information is one that allows the identification information in the individual abnormality information list and the integrated abnormality information list to be kept in a state in which it is possible to associate the identification information. In other words, the rule is one that allows the identification information indicating the same abnormality to be derived from one identification information that does not overlap with the identification information in the abnormality information lists of different groups.

[0082] As described above, according to the first embodiment, the abnormality information detected for each partial image can be associated with the drawing information, and the abnormality information can be integrated and managed for the entire structure to be inspected. In addition, the operation of aligning the abnormality information with the drawing information can be simplified.

[0083] [Variations] In the above-mentioned embodiment 1, a deformation information list was created in which unique identification information was given to the deformation information detected from the partial image and the external shape information of the partial image was added. In addition, in order to further facilitate the task of matching the position and scale of the deformation information to the partial image in the drawing information, a deformation information list in which the deformation information is offset in advance at a certain interval may be output. Since the deformation information detected from the partial image is expressed, for example, by the coordinate information of the image coordinate system with the upper left corner of the partial image as the origin, when the deformation information is read into the drawing information by a design support tool such as CAD, it is read in the same position relatively. However, the partial images are often generated by photographing (or dividing) the structure to be inspected in a row or a grid at a certain interval. For this reason, the deformation information list may be created in which the coordinate information of the deformation information is offset at a certain interval as if the partial images were arranged adjacent to each other.

[0084] 7 illustrates an example of a state in which abnormality information 711 detected from partial image 701 and external shape information 722 of partial image 702 are arranged with an offset in the same image coordinate system. The abnormality information 711 detected from partial image 701 is expressed as coordinate information of an image coordinate system 700 with an upper left vertex 750 of partial image 701 as the origin. Similarly, external shape information 721 of partial image 701 is also expressed as coordinate information of the image coordinate system 700. In contrast, the abnormality information 712 and external shape information 722 detected from partial image 702 are expressed as coordinate information with the origin offset to the upper left vertex 751 of partial image 702, rather than in an image coordinate system with the upper left vertex 750 of partial image 701 as the origin. In this way, by offsetting the deformation information of partial images at regular intervals (offsetting the origin of the image coordinate system), when displaying deformation information of multiple partial images in the same image coordinate system, the deformation information for each partial image can be displayed with the partial images offset at regular intervals, as if the partial images were arranged adjacent to each other.

[0085] In the example of FIG. 7, the offset distance is the width of partial image 701 (the length from point P1 to point P4), but the offset distance may be multiplied by a coefficient such as 1.2 or may be increased by a constant value.

[0086] The offset method may be selectable by the user. Fig. 8 illustrates an example of a UI screen that allows the user to select an offset pattern. In the example of Fig. 8, the user can select one horizontal row, n horizontal rows (n is input by the user), one vertical column, or m vertical columns (m is input by the user), and a deformation information list is output in which the deformation information and coordinate information of the external shape information for each partial image are offset at regular intervals according to the pattern specified by the user.

[0087] As described above, according to the modification of the first embodiment, in addition to the effects of the first embodiment, the operation of aligning the deformation information with the drawing information becomes easier.

[0088] [Embodiment 2] In the second embodiment, an example will be described in which the task of aligning the deformation information with the drawing information is further simplified by converting the coordinate information of the deformation information from the image coordinate system of the partial image to the drawing coordinate system of the structure to be inspected. Specifically, a deformation information list is output in which the coordinate information of the deformation information is converted to coordinate information of the drawing information according to the information required for coordinate conversion input by the user (position and resolution of the partial image). This eliminates the need for the user to align the deformation information with the drawing information by simply inputting the information required for coordinate conversion and loading the deformation information list in the drawing coordinate system into the drawing information.

[0089] Furthermore, in the second embodiment, problems and solutions that arise when abnormality detection is performed again on a portion of a partial image for which abnormality detection has already been performed will also be described.

[0090] For example, the following cases may be considered when performing abnormality detection again on a portion where abnormality detection has already been performed.

[0091] In order to investigate and record changes in deterioration over time, partial images of the same inspection target are taken again at some later time, and abnormalities are detected using these images.

[0092] Anomaly detection is performed on partial images of the inspection object that are re-photographed under improved conditions such as weather and resolution.

[0093] Anomaly detection is performed again on the same partial image using more optimal parameters and a trained model.

[0094] In such a case, if the identification information of the re-detected abnormality information overlaps with the identification information of another abnormality information detected in the past, it becomes difficult to manage the abnormality information in association with the drawing information. Therefore, in this embodiment, the re-detected abnormality information is assigned an identification information that does not overlap with the identification information of another abnormality information detected in the past.

[0095] The following description will focus on the differences from the first embodiment.

[0096] The hardware configuration of the information processing device 300 of the second embodiment is similar to the configuration shown in FIG.

[0097] FIG. 9 is a functional block diagram of an information processing apparatus 300 according to the second embodiment.

[0098] In the second embodiment, a coordinate conversion unit 328 is added to the configuration shown in Fig. 3(b). The coordinate conversion unit 328 performs a process of converting the coordinate information of the deformation information from the image coordinate system of the partial image to the drawing coordinate system of the drawing information.

[0099] Each function of the information processing device 300 is configured by hardware and / or software. Each functional unit may be configured as one or more computer devices or server devices, and may be configured as a system connected by a network. In addition, when each functional unit shown in FIG. 9 is configured by hardware instead of being realized by software, it is sufficient to have a circuit configuration corresponding to each functional unit in FIG. 9.

[0100] <Control processing> Next, the control process of the information processing device 300 in this embodiment will be described with reference to FIG.

[0101] The process in Fig. 10 is realized by the control unit 301 of the information processing device 300 shown in Fig. 3(a) expanding a program stored in the non-volatile memory 302 into the work memory 303 and executing it, thereby controlling each component shown in Fig. 9 to operate as each functional unit shown in Fig. 9. The process in Fig. 10 is started when the information processing device 300 receives an instruction to start the abnormality detection process from the input device 305.

[0102] In S401 to S405, the same processes as in S401 to S405 in FIG. 4 are performed.

[0103] In S1001, the coordinate conversion unit 328 converts the coordinate information of the deformation information and the outline information of the partial image acquired in the processes of S401 to S405 from the image coordinate system to the drawing coordinate system. In the first embodiment, as described in FIG. 5(b), the coordinate information of the deformation information is expressed in the image coordinate system with the upper left vertex of the partial image as the origin. In contrast, in the second embodiment, the coordinate information of the image coordinate system is converted to coordinate information of the drawing coordinate system. Information required for the coordinate conversion (partial image information) is input by the user when the image input unit 323 inputs the partial image. The partial image information is specifically the position and resolution (scale) of the partial image in the drawing information.

[0104] Fig. 11(a) illustrates a drawing 1100 depicting a bridge deck 1101 as an example of an infrastructure structure, and a drawing coordinate system 1103 with a position 1102 as the origin. Fig. 11(b) illustrates a partial image 1111 obtained by photographing a part of the deck 1101, and an image coordinate system 1113 with its upper left vertex 1112 as the origin. In this embodiment, the unit of the image coordinate system 1113 is pixels, and the unit of the drawing coordinate system 1103 is meters. Deformation information detected from the partial image 1111 is expressed by coordinate information of the image coordinate system 1113. This is converted into coordinate information of the drawing coordinate system 1103. The coordinate conversion process can be calculated from the position of the origin 1112 of the image coordinate system 1113 expressed in coordinate information of the drawing coordinate system 1103, and the ratio of units between the image coordinate system 1113 and the drawing coordinate system 1103, i.e., information on the resolution (how many meters in the drawing coordinate system one pixel in the image coordinate system corresponds to).

[0105] By performing coordinate conversion using the partial image information input by the user, a deformation information list can be generated in which the deformation information detected for each partial image is expressed in the drawing coordinate system. Then, by loading the deformation information list in the drawing coordinate system into the drawing information, the user can superimpose the deformation information on the drawing information without having to adjust the position or scale of the deformation information relative to the drawing information. As a result, the effort required to load the deformation information into the drawing information is reduced, and it becomes easier to manage the deformation information in association with the drawing information.

[0106] In addition, a deformation information list in the image coordinate system may be output together with the deformation information list expressed in the drawing coordinate system. The deformation information list may be divided for each partial image and output, or may be integrated in group units and output.

[0107] Next, we will explain the process of assigning identification information to the re-detected abnormality information that does not overlap with other abnormality information detected in the past when abnormality detection is performed again on a part of a partial image where abnormality detection has already been performed.

[0108] In the first embodiment, a method for generating unique identification information in which no duplicate identification information exists among abnormality information detected from partial images belonging to the same group has been described. In the second embodiment, in addition to the first embodiment, each time identification information is generated, identification information already issued for each group is stored, and when new identification information is generated, identification information that does not duplicate existing identification information of the same group is generated.

[0109] For example, in a generation method using serial numbers, numbers that have already been used are stored for each group, and a new numerical value is always used when generating new identification information, thereby generating identification information that does not overlap with existing identification information of the same group. In the example of Fig. 9, each time the identification information setting unit 325 generates identification information, the management unit 322 reads information about the group to which the partial image belongs and the identification information already issued from the storage unit 321, generates new identification information, and stores the identification information already issued for the group.

[0110] In addition, if the re-detected deformation can be considered to be the same as the deformation detected in the past, that is, if the state of the re-detected deformation can be determined to be equivalent to the state of the deformation detected in the past, the same identification information as the existing identification information may be assigned. A publicly known technique is used to compare the new and old deformation information, and a judgment threshold is set. For example, the coordinate information of the new and old deformation information is compared to determine whether the difference is within a predetermined range, or the same crack is determined to have progressed if the length or angle of the line segment representing the crack extends beyond a predetermined range in consideration of the aging of the deformation.

[0111] If the same identification information is assigned to abnormality information that can be considered equivalent, the information associated with the identification information of existing abnormality information can be used as is, and changes in abnormality over time can be managed to estimate the rate of deterioration of the structure, thereby reducing the effort required to manage abnormalities in structures being inspected.

[0112] As described above, according to embodiment 2, the task of aligning the redetected abnormality information for each partial image with the drawing information is simplified, and the redetected abnormality information can be managed in association with the drawing information.

[0113] [Embodiment 3] In the above-mentioned first and second embodiments, it has been explained that unique identification information is assigned to the deformation information, and that a deformation information list in an image coordinate system and a drawing coordinate system that can be superimposed on each of the partial image and the drawing information can be output. Therefore, in the third embodiment, an example will be explained in which an individual viewer that can edit the deformation information for each partial image and a drawing viewer that can edit the deformation information in the drawing information of the entire structure to be inspected can be used.

[0114] When inspectors check the deformation information on-site while inspecting infrastructure, it is difficult to carry a high-performance information processing device or a large-screen display device from the viewpoint of portability. Therefore, in this embodiment, the deformation information of the entire structure to be inspected is consolidated and managed on a server, and the deformation information can be confirmed and edited on-site using an individual viewer. Since the individual viewer does not require high information processing capabilities or large screen output, the inspection work can be performed using a mobile terminal such as a tablet. In addition, when checking and editing the deformation of the entire structure to be inspected, such as when creating an inspection work report, a drawing viewer is used. Since the editing work in each viewer is consolidated on the server, simultaneous editing by multiple people during inspection work can also be supported.

[0115] The following describes the third embodiment, focusing on the differences from the first and second embodiments.

[0116] The hardware configuration of the information processing device 300 of the third embodiment is similar to that of the information processing device 300 of the first embodiment shown in Fig. 3(a). Moreover, the functional configuration of the information processing device 300 of the third embodiment is similar to that of the information processing device 300 of the first or second embodiment shown in Fig. 3(b) or Fig. 9. However, in this embodiment, three different information processing devices, an individual viewer, a drawing viewer, and a server, are used.

[0117] 12 illustrates individual viewers 1201, 1202, 1203, and a drawing viewer 1211. The individual viewer 1201 displays a partial image 1206, superimposed deformation information 1205, and a deformation information list 1204. The drawing viewer 1211 displays a drawing 1212 of the inspected structure, superimposed deformation information 1214, 1215, and a deformation information list 1213. The individual viewers display an editing screen based on the deformation information list expressed in the image coordinate system for each partial image. The drawing viewer displays an editing screen based on the deformation information list expressed in the drawing coordinate system of the inspected structure.

[0118] The information of the individual viewers 1201, 1202, 1203 and the drawing viewer 1211 is synchronized with each other. That is, the edited contents in the individual viewers are reflected in the deformation information list of the drawing viewer, and the edited contents in the drawing viewer are reflected in the deformation information list of the individual viewer.

[0119] First, a process will be described in which a user confirms and edits abnormality information using an individual viewer.

[0120] The user specifies a partial image to be displayed in the individual viewer. The individual viewer requests the server for a list of abnormality information associated with the partial image specified by the user, and the server transmits the partial image to be transmitted and the list of abnormality information to the individual viewer.

[0121] The individual viewer displays the partial image acquired from the server and the abnormality information in a superimposed manner. To accommodate various editing tasks performed by the user, the abnormality information list may be displayed in a table format, and the display format is not particularly limited. The user checks the abnormality information in the individual viewer and performs an inspection task, such as comparing it with the abnormality of the structure at the site. Through the inspection task, the user performs an editing task of the abnormality information, such as adding a crack that was not detected or deleting an abnormality that was falsely detected.

[0122] The individual viewer records these edited contents. When editing of the abnormality information is finished, the individual viewer sends the edited contents to the server. The server judges whether there is a conflict between the edited contents and the drawing viewer, and sends the judgment result to the individual viewer. If there is a conflict between edited contents between viewers, for example, the user re-edits the contents in the individual viewer to resolve the conflict, and the re-edited contents are sent to the server. Any known technology may be used to resolve the conflict in edited contents. In addition, conflicts may be prevented by using an exclusive control mechanism such as checkout lock, without using a method of detecting conflicts in edited contents and prompting the user to resolve them.

[0123] Next, a process will be described in which a user uses the drawing viewer to check the latest defect information and create an inspection report.

[0124] The user specifies the inspection target structure that they wish to display in the drawing viewer. The drawing viewer requests a list of abnormality information for the entire inspection target structure from the server, and the server sends a partial image of the inspection target structure and the abnormality information list in response to the request to the drawing viewer. The drawing viewer superimposes the partial image and abnormality information obtained from the server. The user uses the drawing viewer to check the abnormality information and input the necessary information into the inspection report. The drawing viewer sends the edited contents of the abnormality information to the server, and similarly to the individual viewers described above, it performs processes such as checking the edited contents, determining conflicts, and resolving conflicts. The drawing viewer outputs an inspection report that reflects the edited contents, and ends the process.

[0125] As described above, according to the third embodiment, a deformation information list for each partial image, in which each deformation information is assigned unique identification information, and a deformation information list in which the deformation information for each partial image is integrated with drawing information of the entire structure to be inspected, can be edited with different viewers depending on the purpose. This makes it possible to facilitate inspection work and the creation of inspection reports.

[0126] In addition, because the defect information can be uniquely distinguished by the identification information, editing work in different viewers can be managed in an integrated manner for the entire structure. Also, by using a defect information list in a coordinate system that matches the viewer, defect information can be displayed as it is, either on a partial image basis or on a structure drawing basis.

[0127] In the above-mentioned embodiments, the information processing device is described as one device, but the functions may be divided between a server and a client and implemented by multiple information processing devices. For example, a user inputs a partial image from a client, the server detects abnormality, and transmits a abnormality information list to the client. The abnormality information list may be selected by the user from among an image coordinate system abnormality information list, a drawing coordinate system abnormality information list, whether divided for each partial image, or integrated for each group, or all combinations may be output.

[0128] [Other embodiments] The present invention can also be realized by supplying a program that realizes one or more functions of each embodiment to a system or device via a network or a storage medium, and having one or more processors of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0129] The disclosure of this specification includes the following information processing device, information processing method, and program. [Configuration 1] an input means for inputting one or more images belonging to a predetermined group; A detection processing means for executing a process of detecting an abnormality for each image input by the input means; A setting means for setting unique identification information within the predetermined group for the abnormality information detected by the detection processing means; and an output means for outputting a detection result including the abnormality information to which the identification information has been assigned by the setting means. [Configuration 2] 2. The information processing device according to configuration 1, further comprising an information adding unit that adds image information for arranging the abnormality information in the image to the detection result. [Configuration 3] 3. The information processing device according to configuration 2, wherein the image information is external shape information of the image. [Configuration 4] 4. The information processing device according to configuration 3, wherein the information providing means provides unique identification information to the external shape information of the image. [Configuration 5] 5. The information processing device according to any one of configurations 1 to 4, further comprising a coordinate conversion means for converting first coordinate information representing a position of the deformation information in the image into second coordinate information. [Configuration 6] The information processing device according to configuration 5, wherein the coordinate conversion means offsets the coordinate information of the deformation information for each image in a predetermined pattern. [Configuration 7] 7. The information processing device according to configuration 6, wherein the predetermined pattern is selectable by a user. [Configuration 8] The image is an image of a structure to be inspected, 8. The information processing device according to any one of configurations 5 to 7, wherein the second coordinate information is coordinate information representing a position of the image in drawing information of a structure to be inspected. [Configuration 9] 9. The information processing device according to any one of configurations 1 to 8, wherein the setting means assigns new identification information when detecting abnormality information to which identification information has already been assigned again. [Configuration 10] The information processing device according to any one of configurations 1 to 9, characterized in that the setting means assigns the same identification information to abnormality information that can be considered equivalent to abnormality information to which identification information has already been assigned. [Configuration 11] The information processing device according to any one of configurations 1 to 8, wherein the output means outputs a first detection result for each of the images and a second detection result obtained by integrating the detection results for each of the images by the predetermined group. [Configuration 12] The deformation information of the first detection result is expressed by coordinate information representing a position of the deformation information in the image, 12. The information processing device according to configuration 11, wherein the deformation information of the second detection result is expressed by coordinate information indicating the position of the image in drawing information of the structure to be inspected. [Configuration 13] The information processing device according to configuration 12, characterized in that it includes a first viewer that displays the first detection result in an editable manner superimposed on the image, and a second viewer that displays the second detection result in an editable manner superimposed on drawing information of the structure to be inspected. [Configuration 14] 14. The information processing device according to configuration 13, wherein editing contents in the first viewer are reflected in the second detection result, and editing contents in the second viewer are reflected in the first detection result. [Configuration 15] The image is a partial image obtained by photographing a part of a structure to be inspected, The first detection result includes information on a defect detected for each of the partial images, The information processing device according to configuration 11, characterized in that the second detection result includes deformation information that integrates deformation information detected for each partial image for the entire structure being inspected. [Configuration 16] An input means inputs one or more images belonging to a predetermined group; A step in which a detection processing means executes a process of detecting an abnormality for each of the input images; A step of setting unique identification information within the predetermined group for the detected abnormality information by a setting means; An information processing method comprising: a step in which an output means outputs a detection result including the abnormality information to which the identification information is added. [Configuration 17] A program for causing a computer to function as each of the means of an information processing device according to any one of configurations 1 to 15.

[0130] The invention is not limited to the above-described embodiments, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0131] 300: information processing device, 301: control unit, 321: storage unit, 322: management unit, 323: image input unit, 324: detection processing unit, 325: identification information setting unit, 326: information adding unit, 327: output unit, 328: coordinate conversion unit

Claims

1. Input means for inputting one or more images belonging to a predetermined group; Detection processing means for executing a process of detecting deformation for each image input by the input means; Setting means for setting identification information unique among the predetermined group with respect to the deformation information detected by the detection processing means; An information processing apparatus comprising: output means for outputting a detection result including the deformation information to which the identification information has been given by the setting means.

2. The information processing apparatus according to claim 1, further comprising information adding means for adding, to the detection result, image information for arranging the deformation information on the image.

3. The information processing apparatus according to claim 2, wherein the image information is outer shape information of the image.

4. The information processing apparatus according to claim 3, wherein the information adding means gives identification information unique to the outer shape information of the image.

5. The information processing apparatus according to claim 1, further comprising coordinate conversion means for converting first coordinate information representing a position of the deformation information in the image into second coordinate information.

6. The information processing apparatus according to claim 5, wherein the coordinate conversion means offsets the coordinate information of the deformation information for each image in a predetermined pattern.

7. The information processing apparatus according to claim 6, wherein the user can select the predetermined pattern.

8. The image is an image obtained by photographing a structure to be inspected, The information processing apparatus according to claim 5, wherein the second coordinate information is coordinate information representing a position of the image in drawing information of the structure to be inspected.

9. The information processing apparatus according to claim 1, wherein the setting means assigns new identification information when redetecting variation information to which identification information has already been assigned.

10. The information processing apparatus according to claim 1, wherein the setting means assigns the same identification information to variation information that can be regarded as equivalent to variation information to which identification information has already been assigned.

11. The information processing apparatus according to claim 1, wherein the output means outputs a first detection result for each image and a second detection result obtained by integrating the detection results for each image in the predetermined group.

12. The variation information of the first detection result is represented by coordinate information indicating the position of the variation information in the image, The information processing apparatus according to claim 11, wherein the variation information of the second detection result is represented by coordinate information indicating the position of the image in the drawing information of the structure to be inspected.

13. The information processing apparatus according to claim 12, comprising: a first viewer that superimposes and displays the first detection result on the image in an editable manner; and a second viewer that superimposes and displays the second detection result on the drawing information of the structure to be inspected in an editable manner.

14. The information processing apparatus according to claim 13, wherein the editing content in the first viewer is reflected in the second detection result, and the editing content in the second viewer is reflected in the first detection result.

15. The image is a partial image obtained by photographing a part of the structure to be inspected, The first detection result includes variation information detected for each partial image, The information processing apparatus according to claim 11, wherein the second detection result includes variation information detected for each partial image and variation information integrated over the entire structure to be inspected.

16. A step in which an input means inputs one or more images belonging to a predetermined group; A step in which a detection processing means executes a process of detecting a change for each of the input images; A step in which a setting means sets identification information unique within the predetermined group for the detected change information; An information processing method, comprising: a step in which an output means outputs a detection result including the change information to which the identification information is attached.

17. A program for causing a computer to function as each means of the information processing apparatus according to any one of claims 1 to 15.

Images