Information Processing Apparatus, Information Processing Method, and Program

Abstract

To increase the efficiency of a work using an image by determining the range of the image properly.SOLUTION: The information processor includes: image acquisition means for acquiring an image; first determination means for determining the range of a first image used to make a determination related to an inspection of an inspection target in the image on the basis of a result of detection of the inspection target from the image; and second determination means for determining the range of a second image used to record the result of the inspection of the inspection target, the range of the second image being wider than the range of the first image.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Conventionally, in the inspection work of infrastructure, visual determination of the damage degree of a portion with deformation such as cracks on the concrete surface, and the determination result and the image of the corresponding portion are manually compiled as a document as a record of the inspection result. For the purpose of improving the efficiency of such inspection work, in Patent Document 1, a method has been proposed for detecting deformation from an image obtained by photographing an inspection target and automatically determining the damage degree based on the deformation.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the inspection work of infrastructure as described above, not only the determination result but also the image of the corresponding portion needs to be left in the record. However, the method of Patent Document 1 described above supports the determination work of the damage degree, and the image area extracted as the detection result of the deformation used for the determination of the damage degree may not be suitable as an image for use in recording in a document or the like. For example, the image range used for the determination is a local area such as deformation or a specific member. Further, when inspecting the inner wall of a bridge or a tunnel, it may be difficult to grasp the overall state such as the position of the corresponding portion in the structure and the periphery of the target from a document in which such a local image range is recorded. Thus, for example, in the inspection work of infrastructure, further improvement in efficiency is desired regarding the method for determining the image range for recording the inspection result.

[0005] This invention has been made in view of these problems, and aims to improve the efficiency of image-based work by appropriately determining the image range. [Means for solving the problem]

[0006] The information processing apparatus of the present invention is characterized by comprising: an image acquisition means for acquiring an image; a first determination means for determining a first image range for use in determining the inspection of an inspection target included in the image, based on the detection result of the inspection target from the image; and a second determination means for determining a second image range which represents an image range wider than the range indicated by the first image range, and is used for recording the inspection result of the inspection target. [Effects of the Invention]

[0007] According to the present invention, the efficiency of image-based work can be improved by appropriately determining the image range. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows the hardware configuration of the information processing device according to Embodiment 1. [Figure 2] This is a block diagram showing the functional configuration of the information processing device according to Embodiment 1. [Figure 3] This is a diagram illustrating the processing of the information processing device according to Embodiment 1. [Figure 4] This is a flowchart showing the processing of the information processing device according to Embodiment 1. [Figure 5] This is a diagram illustrating the display control process of Embodiment 1. [Figure 6] This is a diagram illustrating the modification process of Embodiment 1. [Figure 7] This is a diagram illustrating examples of subjects to be tested. [Figure 8] This is a diagram illustrating the processing of the information processing device according to Embodiment 2. [Figure 9]This is a flowchart showing the processing of the information processing device according to Embodiment 2. [Figure 10] This is a diagram illustrating the processing of the information processing device according to Embodiment 2. [Figure 11] This is a block diagram showing the functional configuration of the information processing device according to Embodiment 3. [Figure 12] This is a flowchart showing the processing of the information processing device according to Embodiment 3. [Figure 13] This is a diagram illustrating the processing of the information processing device according to Embodiment 3. [Figure 14] This is a diagram illustrating the processing of the information processing device according to Embodiment 4. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. [Embodiment 1] This embodiment describes an example of its application to inspection work on infrastructure structures. However, the present invention is not limited to inspection work on infrastructure structures, but can be applied to various tasks performed using images. First, let's explain the inspection work for infrastructure structures. Infrastructure structures subject to inspection include, for example, bridges, tunnels, or buildings. Hereafter, infrastructure structures will simply be referred to as "structures." Structures are damaged over time due to various causes such as earthquakes and salt damage. As damage progresses, various deformations such as cracks and deposits appear on the surface of the structure, allowing for the confirmation of structural damage from surface deformation information. Furthermore, in infrastructure structure inspection work, the degree of damage to the relevant area or component is determined from the state of these deformations. Therefore, in conventional inspections, inspectors take photographs of areas with deformations or specific components, visually assess the degree of damage, and record the results of the assessment in a report along with the photographs. In determining the degree of damage, ranks such as "A," "B," or "C" are used according to evaluation categories based on the characteristics of the deformation. Note that while three levels of damage are explained as examples, they are not limited to these.

[0010] Hereinafter, the surface deformation of the structure and specific members that are the targets of damage degree determination as inspections in the inspection work are described as inspection targets. The inspection target is an example of an object. In the present embodiment, a method for appropriately determining, for an image obtained by photographing the surface of the structure to be inspected, an image range for use in determining the damage degree for the inspection target and an image range for recording the inspection target in a document will be described. Note that the items to be determined are not limited to the damage degree, and other items such as the state of the object and the object-likeness may be used.

[0011] [[ID=�]] In the following description, the image range for use in determining the damage degree is described as the determination range (corresponding to the first image range), and the image range for recording the inspection target in a document is described as the document range (corresponding to the second image range). The information processing apparatus 100 of Embodiment 1 determines the determination range and the document range such that the same inspection target is included but the sizes are different. Specifically, the information processing apparatus 100 may determine an image range suitable for damage degree determination as the determination range, and determine the document range so as to be wider than the determination range. The determination range may be, for example, a range including the inspection target centered on the inspection target. This makes it easy to grasp the position of the inspection target and the state of its surroundings when referring to the document later. Further, the information processing apparatus 100 may determine the document range based on a predetermined feature portion (hereinafter referred to as a landmark) on the structure of the document range that can specify or estimate the position. Specifically, the information processing apparatus 100 makes the landmark included in the document range. This makes it even easier to grasp the position of the inspection target. Hereinafter, the details of Embodiment 1 will be described with reference to FIGS. 1 to 7.

[0012] First, the configuration of the information processing apparatus 100 according to Embodiment 1 will be described with reference to FIGS. 1 and 2. FIG. 1 is a hardware configuration diagram of the information processing apparatus 100 according to Embodiment 1. As shown in FIG. 1, the information processing apparatus 100 includes a CPU 101, a ROM 102, a RAM 103, an HDD 104, a display unit 105, an operation unit 106, a communication unit 107, and a system bus 108 that connects these components. The CPU 101 is a Central Processing Unit that controls the entire information processing device 100. The ROM (Read-Only Memory) 102 is a program memory that stores programs for control by the CPU 101. The RAM (Random Access Memory) 103 is used as a temporary storage area such as the main memory and work area of the CPU 101.

[0013] The HDD 104 is a hard disk for storing data and programs necessary for the processes described later. The information processing device 100 may have an external storage device instead of or together with the HDD 104. Here, the external storage device can be realized, for example, by a medium (recording medium) and an external storage drive for accessing the medium. Examples of such a medium include a flexible disk (FD), CD-ROM, DVD, USB memory, MO, and flash memory. Also, the external storage device may be a server device connected by a network or the like.

[0014] The display unit 105 is, for example, a CRT display or a liquid crystal display, and is a device that outputs an image to the display screen. Note that the display unit 105 may also be an external device connected to the information processing device 100 by wire or wirelessly. The operation unit 106 has a keyboard and a mouse and accepts various operations by the user. The communication unit 107 performs two-way communication, either wired or wireless, with external devices such as other information processing devices, communication devices, and server devices using known communication technologies.

[0015] Figure 2 is a block diagram showing the functional configuration of the information processing device 100 according to Embodiment 1. The information processing device 100 functions as the various functional units shown in Figure 2 by having the CPU 101 load a program stored in the ROM 102 into the RAM 103 and execute it. As another example, the various functions and processes of the information processing device 100 may be realized by the CPU 101 loading a program from an external storage device connected to the information processing device 100. As yet another example, some of the functional configuration of the information processing device 100 may be realized using hardware circuits.

[0016] As shown in Figure 2, the information processing device 100 includes an image acquisition unit 200, an inspection target data acquisition unit 210, a determination range determination unit 220, a determination unit 230, a landmark data acquisition unit 240, and a record range determination unit 250. Furthermore, the information processing device 100 includes a display control unit 260, a modification unit 270, and a storage unit 280. The following describes each process performed by each functional unit in Figure 2 using Figure 3. Figure 3 is a diagram illustrating a series of processes performed by the information processing device 100 according to Embodiment 1.

[0017] The image acquisition unit 200 acquires images of the structure to be inspected. Here, we will explain the images acquired by the image acquisition unit 200. In the inspection of a structure, even fine deformations on the concrete wall surface are inspected, so a high-resolution image of the entire surface of the structure is required. For this reason, in this embodiment, the image acquisition unit 200 divides the surface of the structure into multiple sections and acquires multiple section images taken for each section. These section images correspond to the design drawings of the structure. Below, we will explain how the information processing device 100 sequentially acquires the section images and performs the series of processing described thereafter. In this embodiment, we will also explain an example of inspecting the deck of a bridge. For this reason, the image acquisition unit 200 acquires images of the surface of the bridge deck. The object of inspection is cracks on the structure.

[0018] In Figure 3, rectangle 310 is a design drawing of the bridge deck, and rectangle 311 is an image showing a specific section of the deck. Rectangle 311 is divided into a grid, and section images correspond to the divided areas. Image 320 is a section image corresponding to divided area 312 of rectangle 311. Curves 322 and 323 in image 320 represent cracks. Chalk lines 321 represent chalk lines drawn on the surface of the deck by inspectors in the past, and are numbers indicating specific areas of the deck.

[0019] The inspection target data acquisition unit 210 acquires inspection target data from the image. The inspection target data refers to the position data of the inspection target and is referenced when determining the range for judgment and the range for reporting. In Embodiment 1, the inspection target data acquisition unit 210 acquires inspection target data using a crack detection model that has been trained to detect detection targets (in this case, cracks) from images. This crack detection model is stored in an HDD 104 or the like and is a trained model generated by training using a large amount of training data consisting of pairs of concrete wall images and ground truth images indicating the location of cracks in the image. The ground truth images are the same size as the corresponding images, and pixels corresponding to the detection target on the image are stored as 1, while other pixels are stored as 0. Any machine learning algorithm can be used to train the model, including the model described later, for example, an algorithm such as a neural network can be used. When the inspection target data acquisition unit 210 applies the crack detection model to an image, a likelihood map is obtained in which values ​​close to 1 are stored for areas on the image that are likely to be detection targets, and values ​​close to 0 are stored for areas that are not likely to be detection targets. The image obtained by binarizing the likelihood map by a predetermined value is called the detection result. That is, based on the output of the crack detection model in response to the image input, the deformation or specific member that is the inspection target is detected. As described above, the inspection target data acquisition unit 210 acquires the crack detection result from the image.

[0020] Image 330 in Figure 3 shows the crack detection results obtained by applying a crack detection model to image 320. Regions 331 and 332 are contiguous regions of pixels detected as cracks. The data obtained by the model in this way is called detection data. The crack detection data corresponds to the inspection target data mentioned above. Region 331 in image 330 corresponds to curve 322 in image 320. Region 332 in image 330 corresponds to curve 323 in image 320.

[0021] The determination range determination unit 220 determines an image range as the determination range for performing damage degree determination processing on the inspection target in the image. The position of the determination range is determined based on the inspection target data. That is, the first image range in the image to be used for determination related to the inspection of the inspection target included in the image is determined based on the detection result of the inspection target from the image. In Embodiment 1, the determination range determination unit 220 determines the image range based on the image conditions of the image used for training the damage degree determination model, which will be used by the determination unit 230 later. Specifically, the determination range is determined by making the conditions regarding the image size and the position of the object to be inspected in the image the same as the image used for training. For example, the size of the image range is set to the default size used for training the damage degree determination model, and the center position of the image range is set to coincide with the center of the object to be inspected data. In the example in Figure 3, the determination range determination unit 220 calculates the centroid of the region 331 representing the object to be inspected data, and determines the position of the default-sized image range so that the centroid is located at the center, thereby determining the determination range 351. Note that image 350 in Figure 3 is the same image as image 320.

[0022] The determination unit 230 performs a determination on the image cropped by the determination range. In Embodiment 1, the determination unit 230 performs damage determination using a model for determining the degree of damage to an image (damage determination model) that has been pre-trained and stored in the HDD 104 or the like. In Embodiment 1, since the object of inspection is a crack, the damage determination model is trained to output a damage level corresponding to the state of the crack in the image. For example, the damage level is set to four stages from "A" to "D", and the damage determination model is trained to output a rank of "A" or a rank close to it the more damaged the object of inspection in the image is. To generate such a model, a large number of images of a predetermined size are prepared, and a person visually assigns a damage level rank to each image.

[0023] For example, if the image shows dense cracks or cracks accompanied by water leakage, the damage is considered advanced, and the damage level is assigned a rank of "A". On the other hand, if the image shows thin cracks, the damage is considered not very advanced, and the damage level is assigned a rank of "D". The determination unit 230 then learns using a large number of pairs of images of a predetermined size and the damage levels assigned to those images. Any machine learning algorithm can be used for this learning; for example, a support vector machine can be used. Alternatively, as a method using deep learning, an architecture consisting of a CNN (Convolutional Neural Network) and FC layers can be used for the images. When the determination unit 230 performs damage level determination using the damage level determination model learned in this way, it inputs an image of the same size as the image used for training into the damage level determination model. That is, the determination unit 230 can simply input the image cropped according to the determination range determined by the determination range determination unit 220. This allows the damage level rank to be obtained.

[0024] The landmark data acquisition unit 240 acquires landmark information from the image. Here, a landmark is information that allows for the identification or estimation of a location within a structure, such as a characteristic structural part of an individual part of a structure or a mark that has been artificially added. In this case, the chalk line 321 is used as a landmark. In Embodiment 1, the landmark data acquisition unit 240 acquires location data of landmarks that are within a predetermined range in distance from the inspection target data.

[0025] First, the landmark data acquisition unit 240 sets a landmark search range to search for landmarks in the vicinity of the object to be inspected. For example, the landmark data acquisition unit 240 sets the landmark search range 333 such that the center of an image range of a predetermined size coincides with the centroid of region 331 (data to be inspected). Next, the landmark data acquisition unit 240 detects landmarks from the range of image 320 corresponding to the landmark search range 333. In Embodiment 1, the landmark data acquisition unit 240 acquires landmark data using a model that has been pre-trained to detect predetermined landmarks in a manner similar to the method for generating the crack detection model described above. Landmark data refers to location data of landmarks. In Embodiment 1, a model that detects chalk lines as predetermined landmarks is applied to the image of the landmark search range 333, and image 340 is obtained as a detection result. Region 341 of image 340 corresponds to the chalk lines 321 shown in image 320.

[0026] The recording range determination unit 250 determines an image range for the image that is different from the judgment range. Specifically, the recording range determination unit 250 determines the recording range based on landmark data. More specifically, the recording range determination unit 250 determines the recording range so as to include both the landmark data and the inspection target data. For example, the recording range determination unit 250 obtains the coordinate points of the left edge, right edge, top edge, and bottom edge from the data that combines the inspection target data and the landmark data, and determines the recording range so as to include these points. In Embodiment 1, the recording range determination unit 250 includes the entire landmark data, but it may also include only a part of the landmark data. For example, for landmarks with a large area, such as diagonal members, even if only a part of it is included in the recording range, it is useful for identifying its position on the structure.

[0027] Alternatively, the recording range determination unit 250 may determine a recording range that includes the object being inspected and is wider than the judgment range. In this case, the landmark data acquisition unit 240 is unnecessary. Also, if landmark data is not acquired by the landmark data acquisition unit 240, the recording range may be determined to be wider than the set landmark search range. In this case, if the size of the landmark search range is, for example, 1000 pixels x 1000 pixels, the recording range determination unit 250 will determine the size of the recording range to be wider, such as 2000 pixels x 2000 pixels. This will result in an image with a slightly wider field of view being extracted as the recording range, making it easier to understand what was photographed. The recording range determination unit 250 may also determine the size of the recording range according to the size of the judgment range. For example, the recording range determination unit 250 may determine the size of the recording range to be a predetermined size that expands the periphery of the judgment range by a predetermined amount.

[0028] The display control unit 260 controls the display unit 105 to create a confirmation screen (Figure 5) and output it to the display unit 105 in order to allow the user to confirm the determination result by the determination unit 230 and the record range determined by the record range determination unit 250. This confirmation screen displays the image (hereinafter referred to as the record image) that has been cut out from the image 320 according to the record range. The modification unit 270 accepts modifications from the user to the judgment result and the range for reporting via the operation unit 106. The storage unit 280 records the judgment result and the image for the report as a set for each object being inspected, and stores them in the HDD 104 as the inspection result.

[0029] Figure 4 is a flowchart showing the processing performed by the information processing device 100 according to Embodiment 1. In the following description, each process (step) will be preceded by an S, and the notation of the process (step) will be omitted. Figure 4(A) is a flowchart showing the overall flow of processing by the information processing device 100. Figure 4(B) is a detailed flowchart of the process (S450) for determining the range for recording in Figure 4(A). The processing in the flowcharts of Figures 4(A) and 4(B) is realized by the CPU 101 loading the program stored in ROM 102 into RAM 103 and executing it.

[0030] First, in S410, the image acquisition unit 200 acquires an image of the object to be inspected from an external device connected via the HDD 104 or the communication unit 107. Here, image 320 in Figure 3 is acquired. In Embodiment 1, the image acquisition unit 200 acquires an image corresponding to a design drawing, but it may also acquire an image that does not correspond to a design drawing. Next, in S420, the inspection target data acquisition unit 210 acquires inspection target data to be judged from the image acquired in S410. In Embodiment 1, the inspection target data acquisition unit 210 acquires inspection target data by performing detection processing using a model (here, a crack detection model) from the image 320. Regions 331 and 332 in Figure 3 correspond to the inspection target data. Alternatively, the inspection target data acquisition unit 210 may detect the inspection target by manual input on the image without performing detection processing. That is, the inspection target data acquisition unit 210 may acquire position information on the image that has been indicated as the inspection target by manual input from the user. The following describes the process of determining the judgment range and the record range for the inspection target data corresponding to region 331.

[0031] Next, in S430, the determination range determination unit 220 determines the determination range for the image acquired in S410. In Embodiment 1, the determination range determination unit 220 determines the determination range based on the inspection target data acquired in S420 and the image conditions of the image used by the determination unit 230 to train the model (in this case, the damage degree determination model). For example, the size of the image range is set to the default size used for training the damage degree determination model, so that the center position of the image range coincides with the center of the inspection target data. In this case, the determination range 351 shown in Figure 3 is determined for region 331.

[0032] Furthermore, if the object to be inspected does not fit within the image range of the predetermined size, information about the deformation that affects the judgment may be leaked from the image range, and the degree of damage may not be correctly determined. Therefore, the HDD 104 or the like may store multiple damage degree determination models with different default image sizes used for training, and the determination range determination unit 220 may select a default size according to the size of the object to be inspected. In this case, in the following S440, the determination unit 230 performs the determination using the damage degree determination model applicable to the selected default image size. Furthermore, if the data to be inspected is located near the edge of the image, the image range may extend beyond the image acquired in S410. In such cases, the image acquisition unit 200 may acquire adjacent section images on the design drawing, and the determination range determination unit 220 may determine the determination range so as to span between the section images. In Embodiment 1, the determination range determination unit 220 determines the determination range based on the image conditions of the image used by the determination unit 230 for model training. However, the method is not limited to this, and the determination range may be determined based on pre-set image size and aspect ratio.

[0033] Next, in S440, the determination unit 230 performs a determination on the determination range determined in S430. In Embodiment 1, the determination unit 230 obtains the rank of damage output as a result of inputting the image of the determination range determined by the determination range determination unit 220 to the damage level determination model. Here, it is assumed that the determination unit 230 inputs the image of the determination range 351 in Figure 3 to the damage level determination model, and "C" is output as the damage level.

[0034] Next, in S450, the landmark data acquisition unit 240 and the record range determination unit 250 perform a process to determine the record range for the image acquired in S410. Details of the process for determining the record range will be explained using Figure 4(B). Here, a landmark is an object that serves as a marker to identify the location of the corresponding structure to be inspected. Furthermore, a landmark is defined as a prominent area that has different characteristics from the surrounding area. First, in S451, the landmark data acquisition unit 240 sets a landmark search range for the inspection target data acquired in S420. For example, the landmark data acquisition unit 240 sets the landmark search range to a predetermined size by setting the image range to a predetermined size, so that the center position of the image range coincides with the center of the inspection target data. This makes it possible to search for landmarks that exist in the vicinity of the inspection target data. In the example shown in Figure 3, the landmark search range 333 is set for the region 331 which is the inspection target data.

[0035] The landmark data acquisition unit 240 sets the landmark search range based on the data to be inspected, but this method is not limited to this; the entire range of image 320 may also be set as the landmark search range. In this case, in S452, the landmark data acquisition unit 240 acquires landmark data from the entire range of image 320, and in S453, the record range determination unit 250 determines the record range so that the acquired landmark data and the data to be inspected are included. Furthermore, the record range determination unit 250 may select landmarks to be included in the record range using predetermined conditions. Note that if the locations of the data to be inspected and the landmarks are far apart, the record image may become extremely large. Therefore, the information processing device 100 may set the maximum size of the record range from the viewpoint of ease of confirmation of the data to be inspected and visual balance in the record image. The maximum size of the record range is set based on the size of the data to be inspected, the recording size of the record image, or the resolution of the image. Note that the size of the data to be inspected includes not only the overall size of the data to be inspected but also the size of the parts that make up the data to be inspected. The record range determination unit 250 limits the size of the record range so that it is smaller than this maximum size. Furthermore, if the data to be inspected is located near the edge of the image, the image acquisition unit 200 may acquire adjacent section images on the design drawing, and the landmark data acquisition unit 240 may set the landmark search range to span across the section images.

[0036] Next, in S452, the landmark data acquisition unit 240 detects landmarks from the image of the landmark search range set in S451. In Embodiment 1, the landmark data acquisition unit 240 acquires landmark data by detecting landmarks using a model. In the example shown in Figure 3, region 341 is detected as landmark data from the image of the landmark search range 333. That is, in the example shown in Figure 3, the chalk line 321 is a prominent region with features different from the surrounding region and is detected as a landmark. In Embodiment 1, the landmark data acquisition unit 240 acquires landmark data by detecting landmarks from an image using a model. However, the location of landmarks may be determined based on manually set location information without detection. Specifically, information indicating a part suitable for a landmark (for example, a distinctive part such as a number written in chalk) may be set. The landmark data acquisition unit 240 may then accept an input operation for a part suitable for a landmark on the image and acquire the location information of the input operation. This makes it possible to create a document image that suits the user's preferences through processing by the document range determination unit 250.

[0037] Next, in S453, the record range determination unit 250 determines the record range based on the inspection target data acquired in S420 and the landmark data acquired in S452. Here, the image range including regions 331 and 341 in Figure 3 is determined as the record range 352. In this example, the chalk line 321 in image 320, which corresponds to region 341, represents a number indicating a specific region of the floor slab, as described above. Therefore, if this information is included in the record range, it becomes easy to identify which part of the floor slab it is when referring to the record image later. Note that although the chalk line 321 represents a number indicating a specific region of the floor slab, the chalk line 321 does not have to be a number; it may be a curve or a mark. Furthermore, in S452, if no landmark is detected, the recording range determination unit 250 may determine a recording range that is wider than the determination range and of a predetermined size. For example, if no landmark is detected, the recording range determination unit 250 may determine the landmark search range as the recording range, or it may determine a range wider than the landmark search range as the recording range. Furthermore, while the description of the reporting range determination unit 250 has shown an example where the reporting range includes the inspection target by setting the range centered on the inspection target, the reporting range determination unit 250 may also determine a range that does not include the inspection target as the reporting range. In this case, the report may be configured to include both an image of the judgment range and an image of the reporting range. In this case, the reporting range may be the area around the inspection target and be a prominent area where these areas can be identified, or an area around the inspection target that contains a deformation of note. By doing so, the report will include information that can identify the location of the inspection target structure and information about the state of the area around the target, improving convenience. The processing steps S451 to S453 described above allow for the determination of the area for recording in the image. After S453, the process proceeds to S460. If multiple inspection target data are acquired in S420, the information processing device 100 executes the processing steps S430 to S453 for the next inspection target data.

[0038] Once the reporting range is determined by the processing in S450, in S460, the display control unit 260 generates and outputs a confirmation screen to allow the user to confirm the damage degree determination result in S440 and the reporting range determined in S450. In Embodiment 1, the display control unit 260 generates a confirmation screen in report format and displays it on the display unit 105. Figure 5 is a diagram illustrating the display control processing performed by the display control unit 260. Figure 5(A) shows an example of a confirmation screen. In the confirmation screen 510 of Figure 5(A), the left area 512 displays fields for items necessary for the report (for example, date and type of damage), and the right area 513 displays an image for the report. One of the items is a field 511 that shows the result of the damage assessment (in this case, "C").

[0039] In addition, one image for reporting is displayed in the right-hand area 513 of the confirmation screen 510, but multiple images for reporting may be displayed. In this case, the reporting range determination unit 250 determines the reporting range determined by the method described above as the first reporting range, and in addition to the first reporting range, it determines a second reporting range that is wider than the first reporting range. The display control unit 260 then outputs the first reporting range and the second reporting range together with the judgment result. In the confirmation screen 520 of Figure 5(B), along with the damage degree entry field 511, an image 523 of the first reporting range and an image 521 of the second reporting range are displayed. Image 521 is the same as image 320. Image 523 is an enlarged image of the local area including the inspection target 522 of image 521. Recording multiple reporting images of different sizes makes it even easier to identify locations.

[0040] Furthermore, the display control unit 260 may display the judgment range and the reporting range on the same screen for comparison, as shown in the confirmation screen 530 in Figure 5(C). In the confirmation screen 530 of Figure 5(C), the judgment range 534 of the inspection target 531 and the reporting range 533 of the inspection target 531 are displayed superimposed on the image 350. By displaying the judgment range and the reporting range on the same screen, the output can be checked from the perspective of whether the reporting range is appropriate in relation to the judgment range.

[0041] Let's return to the explanation of Figure 4(A). Next, in S470, the correction unit 270 corrects the damage assessment result and the reporting range output in S460 based on user operation. The specific correction method will be explained using Figure 6. Figure 6 is a diagram illustrating the correction process performed by the correction unit 270.

[0042] <How to correct the damage assessment results> First, we will explain how to correct the damage assessment results. When the correction button 515 displayed in the damage level entry field 511 on the confirmation screen 510 is selected, the display control unit 260 switches the display from the confirmation screen 510 to the correction screen 610 shown in Figure 6(A). The correction screen 610 displays the image obtained in S410 with the judgment range 351 superimposed, along with the damage level judgment result 613. If the user wishes to directly correct the judgment result, the correction unit 270 corrects the damage level judgment result by allowing the user to directly edit the damage level judgment result 613.

[0043] On the other hand, if the user wishes to modify the judgment range, the modification unit 270 modifies the judgment range 351 through an operation on the judgment range 351. In Embodiment 1, since the judgment range 351 is of a predetermined size, the modification unit 270 modifies the position of the judgment range 351 without changing its size. For example, the user moves the position of the judgment range 351 by hovering the mouse pointer 611 over the judgment range 351 and dragging it. When the user selects the judgment button 612, the judgment unit 230 re-performs the damage degree determination. Subsequently, the display control unit 260 receives the result of the damage degree determination obtained from the judgment unit 230 and updates the damage degree determination result 613. Next, when the user selects the confirm button 614, the modification is finalized, and the display control unit 260 switches the display from the modification screen 610 to the confirmation screen 510. The modified damage degree is reflected in the damage degree entry field 511 on the confirmation screen 510 after the switch. The above describes the method for modifying the result of the damage degree determination.

[0044] <How to modify the scope of the report> Next, we will explain how to modify the scope of the report. Here, we will explain how to directly edit the scope of the report and how to select from several candidate scopes. First, let's explain the first method: direct editing. When the user selects the edit button 516 for the report image on the confirmation screen 510, the display control unit 260 switches the display from the confirmation screen 510 to the edit screen 620 in Figure 6(B). The edit screen 620 displays an image in which the report area 352 is superimposed on the image 320. For example, when the user places the mouse pointer 611 over the endpoint of the report area 352 and drags in the direction of the arrow 622, the edit unit 270 changes the report area to the image area 621.

[0045] Next, as a second method, a method of selecting from multiple candidate document ranges will be described. In this method, the document range determination unit 250 stores multiple candidate document ranges in advance in the ROM 102 or the like (for example, at the time of S405), and the modification unit 270 modifies the document range to the selected candidate by having the user select an appropriate image range from among the candidate document ranges. On the confirmation screen 630 in Figure 6(C), an image range 631 where the coordinates of the left edge, right edge, top edge, and bottom edge of the inspection target data and landmark data are located at the edge of the image, and an image range 632 which is an extension of this image range 631 by a predetermined amount are displayed on the image 320 as candidate document ranges. When the user selects the image range 632 with the mouse pointer 611, the modification unit 270 determines the document range to be the image range 632. Note that the process of selecting from multiple candidate document ranges as described above may also be performed in the S405 process.

[0046] Once the document range is changed using the method described above, and the confirmation buttons 623 and 633 are selected, the changes are confirmed, and the display control unit 260 switches the display to the confirmation screen 510. The right-hand area 513 of the confirmation screen 510 after the switch displays the document image of the changed document range. This concludes the method for modifying the document range.

[0047] Finally, in S480, the storage unit 280 saves the damage assessment result and the reporting range as a set in the ROM 102 or the like. In Embodiment 1, when the accept button 517 on the confirmation screen 510 is selected, the storage unit 280 saves the data in the reporting format shown on the confirmation screen 510. After that, the processing of the series of flowcharts is completed. Furthermore, the data stored by the storage unit 280 is not limited to data in report format, but may also be file format data as shown in Figure 5(D). Figure 5(D) is a data table that stores location information for the judgment range and the report range. This data table stores coordinate information for the judgment range and coordinate information for the report range for each inspection object in the image. This coordinate information is location information on the image, but it may also be location information on the design drawing.

[0048] According to Embodiment 1 described above, when performing damage assessment on an object being inspected during structural inspection work, it is possible to determine appropriate image ranges for assessment and for documentation purposes. This ensures accuracy in assessment of the object being inspected, while also allowing for the recording of images that make it easier to later understand the location and condition of the object being inspected. In other words, it is possible to improve the efficiency of structural inspection work. Although the process flow has been explained using the flowchart in Figure 4, the order of the processes may be rearranged as appropriate. For example, the process of determining the range for judgment (S430) and the process of determining the degree of damage (S440) may be executed after the process of determining the range for reporting (S450) has been executed.

[0049] <About Landmarks> While chalk lines were used as landmarks above, landmarks are not limited to chalk lines; any distinctive feature (a designated distinctive feature) that allows for the identification or estimation of the location of the area being inspected is acceptable. In structural inspection work, variations of landmarks that can be used include man-made marks, structural members, structural material boundaries, cracks from construction, and artificial objects. Landmarks will be explained in detail below.

[0050] Artificial marks include not only chalk lines but also marks made on concrete surfaces with ink or other materials. Components include diagonal members, beams, bearings, steel plates, bolts, etc. If components are included in the document images, their locations can be determined by referring to the positions of the components indicated in the design drawings. Structural boundaries include the boundary between the sky and the concrete wall, and the boundary between the ground and the concrete wall. If the images used for documentation include the boundaries of structural materials, it indicates that the edge of the structure was photographed, which helps in understanding the location and condition of that area. Cracks during construction can be found in formwork, joints, etc. Since joints and formwork are often present at regular intervals, including joints and formwork in the images for documentation makes it easier to understand the scale of the object being inspected. Artificial structures include fences, stairs, catwalks, wire mesh, wire cords, pipes, lighting, electronic display boards, and panels. Larger artificial structures such as fences, stairs, catwalks, wire mesh, and wire cords have a relatively large surface area, so if a portion of such structures is included in the image used for documentation, it is easier to identify the location of the relevant area. Similarly, smaller artificial structures such as cords, pipes, lighting, electronic display boards, and panels are also useful in identifying the location of the relevant area. For example, if a document image showing a portion of a tunnel includes lighting, it can be determined that the object of inspection is located at a high position, such as the ceiling. Other landmarks such as road markings, manholes, and guardrails can also be used. Including these landmarks in the images for documentation makes it easier to understand the location and condition of the relevant areas.

[0051] <Regarding the grouping of subjects to be tested> In the above explanation, the inspection target was assumed to be a deformation (crack), and the inspection target data consisted of information from a single deformation. However, it may also consist of information from multiple deformations. Figures 7(A) and 7(B) show examples of inspection targets consisting of multiple deformations. Image 710 in Figure 7(A) shows an inspection target consisting of broken cracks. Broken cracks may extend and reconnect over time. Therefore, the information processing device 100 groups the broken cracks 711 and 712 together so that they can be recorded in a report. The grouping method involves first selecting one data point of interest from the deformation data detected by the inspection target data acquisition unit 210, and setting a deformation search range to search for deformations within a predetermined range centered on the data point of interest. If the inspection target data acquisition unit 210 detects other deformation data within the deformation search range, it groups them together as deformation data of the same group.

[0052] Image 720 in Figure 7(B) shows an inspection target consisting of multiple exposed rebars that are spaced apart but close together. Even with multiple such exposed rebars 721, it is possible to group the data into the same category using the method described above. When data on multiple abnormalities are grouped together as subjects for inspection as described above, the judgment range determination unit 220 and the record range determination unit 250 perform the process of determining the judgment range and the record range for the group.

[0053] In the examples in Figures 7(A) and 7(B), the inspection target consists of multiple deformations, but the inspection target is not limited to deformations. The inspection target may be a group consisting of one or more members, one or more deformations, or both. That is, the information processing device 100 may group objects of different types. For example, when determining the degree of damage to a certain part of a particular member, the determination may be made based on the deformations of the member and its surroundings. For example, if the corrosion area around the member is wide, it can be determined that the degree of damage is high. Therefore, in order to include deformations that affect the determination in the determination range, the information processing device 100 groups the member and the deformations around it, and determines the determination range for the group.

[0054] Image 730 in Figure 7(C) shows an example of an inspection target consisting of a member and deformation. Image 730 shows a bolt 731 and cracks 732 and 733. In this case, the inspection target data acquisition unit 210 acquires detection data for each using a crack detection model and a model trained to detect specific members. The inspection target data acquisition unit 210 then sets a deformation search range centered on the member detection data and groups the member detection data and the deformation detection data included in the deformation search range. As described above, the information processing device 100 may group multiple objects and determine a range for judgment and a range for recording for the grouped objects.

[0055] Furthermore, if the inspection target includes a component, the determination range determination unit 220 may determine the determination range so that the center of the image range coincides with the center of the component. In this case, the determination unit 230 can learn a damage degree determination model using an image in which the component is located at the center of the image, and apply this model to the determination range in which the component is located at the center, thereby making it easier to ensure the accuracy of the determination. Furthermore, the information processing device 100 may exclude unimportant data (for example, data that does not contribute to damage assessment) from the group. An example of data to exclude is data on deformations that are not significantly damaged (narrow cracks). Narrow cracks can be considered unimportant data because they indicate a lower degree of damage compared to wide cracks. Another example is when assessing the damage level of a specific component; data that is relatively far from that component can be considered unimportant data. By excluding unimportant data from the group, unnecessary information is removed from the assessment range, making it easier to ensure assessment accuracy.

[0056] [Embodiment 2] In Embodiment 1 described above, the information processing device 100 determined the judgment range based on the inspection target data and a predetermined size. However, in Embodiment 2, no predetermined size is set, and the judgment range is determined based on a certain range of the inspection target. Specifically, the information processing device 100 detects grid-like cracks from the image and determines the judgment range and the reporting range based on the range where the grid-like cracks are located. Grid-like cracks are a type of deformation that occurs in structural materials such as bridge decks. Grid-like cracks are a type of deformation caused by cracks occurring in one direction (perpendicular to the structural material axis) due to drying shrinkage, followed by repeated application of loads from vehicles, etc., to that area, resulting in the occurrence of cracks in a direction perpendicular to the original cracks (in the structural material axis direction). This deformation is particularly important because there is a possibility that the closed regions formed by the cracks may fall off. The information processing device 100 in Embodiment 2 has the same configuration as the information processing device 100 in Embodiment 1. Therefore, the same reference numerals are used for parts that are the same as in Embodiment 1, and further explanation is omitted. The details of Embodiment 2 will be described below with reference to Figures 8 to 10.

[0057] Figure 8 is a diagram illustrating the process of acquiring data on grid-like cracks. In Embodiment 2, the inspection target data acquisition unit 210 detects grid-like cracks in the image 810 and acquires the area containing the grid-like cracks as inspection target data. Specifically, the inspection target data acquisition unit 210 acquires polyline data, which is obtained by processing the crack detection data, and uses this polyline data to identify the area containing the grid-like cracks. The chalk lines 811 in image 810 are numbers written in chalk, similar to the example in Embodiment 1.

[0058] The processing of the information processing device 100 according to Embodiment 2 differs from the flowchart in Figure 4(A) mainly in the details of the processing at S420. Therefore, this processing will be explained in detail using Figure 9, and explanations of other processes will be omitted. Figure 9 is a flowchart of the processing performed by the inspection target data acquisition unit 210 of Embodiment 2. Once the processing in Figure 9 is completed, the area with grid-like cracks is designated as the inspection target data, and the processing from S430 onwards in Figure 4 is executed. The processing in Figure 9 is achieved by the CPU 101 loading the program stored in ROM 102 into RAM 103 and executing it.

[0059] First, when the image acquisition unit 200 acquires an image of the object to be inspected (S410), in S901, the inspection target data acquisition unit 210 acquires polyline data as crack detection data. This polyline data is data that contains the position information of the detected cracks, and is data that represents the region connecting adjacent pixels detected as cracks with line segments. In Embodiment 2, the inspection target data acquisition unit 210 acquires polyline data by applying image processing such as thinning and vectorization to the crack detection data. Image 820 in Figure 8 is the polyline data acquired for image 810.

[0060] Next, in S902, the inspection target data acquisition unit 210 uses the polyline data acquired in S901 to detect closed regions formed by cracks. Here, seed-fill, one of the image processing techniques, is used. Seed-fill is a method of filling in continuous regions (inside or outside of closed regions) in an image by repeatedly filling in a pixel on the image, starting with a certain pixel on the image, filling in the pixel at the starting point, and filling in any pixels adjacent to the filled pixel that are not contour pixels. In the example in Figure 8, by applying this technique to image 820, regions 831, 832, 833, 834, and 835 are detected as closed regions, as shown in image 830. In Embodiment 2, an image processing technique was used to detect closed regions, but it is also possible to use a method in which a model trained to detect closed regions is prepared in advance and detection is performed using the model.

[0061] Next, in S903, the inspection target data acquisition unit 210 groups adjacent closed regions from among the closed regions detected in S901. In the example in Figure 8, closed regions 883, 832, 833, and 834 are adjacent and are therefore grouped into one group 836, while closed region 835 has no adjacent closed regions and remains in one group. Note that not only adjacent closed regions but also the regions of polylines that include adjacent closed regions may be grouped. Subsequently, in S904, the inspection target data acquisition unit 210 selects one group from those obtained in S903, and in S905, it determines whether the selected group is a grid crack. Specifically, the determination is made based on the orientation of the closed regions and polylines included in the group. As one example, it is determined whether there are two or more closed regions in the group. As another example, it is determined whether there are two or more polylines extending perpendicularly to the structural material and two or more polylines extending horizontally among the polylines passing over the group. The orientation of a polyline is detected, for example, from the direction of the vector connecting the start and end points of the polyline.

[0062] If the determination process in S905 determines that the group has a grid-like crack pattern (Yes in S906), the process moves to S907, and the inspection target data acquisition unit 210 adds the group to the inspection target data. On the other hand, if the group is determined not to have a grid-like crack pattern (No in S906), the process moves to S908. In the example in Figure 8, group 836 is determined to be an area with a grid-like crack pattern. On the other hand, the group of closed region 835 is determined not to have a grid-like crack pattern. In S908, the inspection target data acquisition unit 210 checks whether the grid crack detection process has been performed for all groups grouped in S903. If the process has been performed for all groups, the flowchart in Figure 9 is completed, and the process returns to Figure 4. On the other hand, if there are any groups for which the grid crack detection process has not been performed, the next group is selected, and the process returns to S904.

[0063] According to the flowchart in Figure 9 described above, the area containing the grid-like cracks (specific object) can be acquired as inspection target data. Subsequent processing can be carried out in the same manner as in Embodiment 1. First, the determination range determination unit 220 determines the determination range 842 for image 840 (same as image 810) so as to include group 836 (inspection target data). Next, the determination unit 230 performs a damage degree determination on the determination range 842 and acquires the result of the damage degree determination. Next, the record range determination unit 250 determines the record range based on the inspection target data and landmark data, similar to Embodiment 1. Here, it is assumed that detection data (landmark data) for area 843 has been obtained as a result of the landmark data acquisition unit 240 detecting the chalk line 811, and the record range 841 is determined so as to include group 836 and area 843.

[0064] In Embodiment 2, the determination unit 230 performs damage determination using a damage determination model learned using images for determining grid-like cracks. Alternatively, the determination unit 230 may calculate the inter-grid distance and perform damage determination based on the inter-grid distance without using the damage determination model. Specifically, first, the determination unit 230 calculates the crack spacing of each closed region of the grid-like crack. The crack spacing is derived by detecting it from the number of pixels, etc., and converting it to the actual size. Furthermore, the determination unit 230 obtains the average, minimum, and maximum values ​​of the crack spacing of each closed region and uses these as the inter-grid distance of the grid-like crack. The determination unit 230 compares this inter-grid distance with a predetermined standard and performs damage determination. For example, if the distance between grids is 20 cm or less, the damage rank is A.

[0065] According to the above-described embodiment 2, in the inspection work of a structure, it is possible to determine image ranges suitable for judgment and for documentation purposes for the area containing grid-like cracks that are the target of inspection. This ensures the accuracy of judgment regarding the inspection target, while also allowing for the recording of images that make it easy to understand the location and condition of the inspection target later. In other words, it is possible to improve the efficiency of the inspection work of structures.

[0066] Although the above describes the inspection target as grid-like cracks, the inspection target may also include tortoise-shell-like cracks and closed crack regions (closed cracks). Tortoise-shell-like cracks are cracks in which multiple cracks propagate and intersect with each other, forming closed regions in a tortoise-shell pattern. Grid-like cracks, tortoise-shell-like cracks, and closed cracks are examples of specific objects. When detecting a range containing grid-like cracks, tortoise-shell-like cracks, and closed cracks together, the inspection target data acquisition unit 210 detects the intersection points where the vectors constituting all polylines intersect and acquires the inspection target data based on the density of these intersection points.

[0067] Figure 10 is a diagram illustrating the process of acquiring inspection target data using the intersections of polyline vectors. Image 1010 is the same image as Image 820 and represents polyline data. The black circles indicate the vector intersections. For example, the inspection target data acquisition unit 210 divides Image 1010 into multiple sections, calculates the density of intersections for each section, and determines that these deformations exist in sections with high density. Although this method reduces the positional accuracy of the inspection target data compared to the flowchart in Figure 9, it requires less processing and thus shortens execution time. In addition to the method that uses the density of intersections, there is also a method that uses intersections and vectors. For example, as shown by the dashed line 1011 in Image 1010, the inspection target data acquisition unit 210 traces the polyline vector from an intersection and determines that there is a closed region (deformation) if it returns to the original intersection. This method that uses intersections and vectors can improve the positional accuracy of the inspection target data compared to the method that uses the density of intersections.

[0068] [Embodiment 3] In Embodiment 3, the information processing device 100 determines the range for recording in a different manner than in the above embodiments. In the above embodiments, the information processing device 100 detected chalk lines as landmarks, but in Embodiment 3, the information processing device 100 detects a plurality of pre-set types of landmarks. When a plurality of types of landmarks are detected, the information processing device 100 selects one of the landmarks to include in the range for recording. For example, by selecting a landmark that is easier to connect to when viewing the recording image later, it becomes easier to understand where the inspection target is located. In Embodiment 3, the landmark data acquired by the landmark data acquisition unit 240 is called candidate data, and the landmark data selected from the candidate data to be included in the range for recording is called selected data. The details of Embodiment 3 will be described below with reference to Figures 11 to 13.

[0069] Figure 11 is an example of a block diagram showing the functional configuration of the information processing device 100 according to Embodiment 3. The information processing device 100 functions as the functional units shown in Figure 11 by having the CPU 101 load the program stored in the ROM 102 into the RAM 103 and execute it. The information processing device 100 according to Embodiment 3 further includes a priority setting unit 1110 in addition to the functional units of the information processing device 100 according to Embodiment 1. The priority setting unit 1110 sets a priority for the candidate data acquired by the landmark data acquisition unit 240. The record range determination unit 250 determines selected data from the candidate data based on this priority and determines the record range based on the selected data.

[0070] The processing of the information processing device 100 according to Embodiment 3 is the same as the flowchart in Figure 4(A), but the process (S450) in which the record range determination unit 250 determines the record range is different. Therefore, the details of the process for determining the record range will be explained in detail using the flowchart in Figure 12, with reference to Figure 13. The processing in the flowchart in Figure 12 is realized by the CPU 101 loading the program stored in the ROM 102 into the RAM 103 and executing it.

[0071] Figure 13 is a diagram illustrating the process of determining the range for reporting using selected data. Embodiment 3 describes a method in which the information processing device 100 acquires the image 1310 in Figure 13(A) and performs the series of processing described below on the image 1310. In addition to the crack 1311, the image 1310 includes the chalk line 1312 and the wire mesh 1313. Here, the information processing device 100 acquires the detection data of the crack 1311 as the inspection target data, and determines the judgment range and the reporting range based on the inspection target data.

[0072] First, in S1201, similar to S451 in Figure 4(B), the record range determination unit 250 sets the landmark search range for the inspection target data of image 1310. Figure 13(A) shows the landmark search range 1314 set for the detection data of crack 1311. Next, in S1202, the landmark data acquisition unit 240 acquires candidate landmark data from the landmark search area 1314. Here, the landmark data acquisition unit 240 applies a model for detecting chalk lines and a model for detecting artificial objects such as wire mesh and cords to the image of the landmark search area 1314. As a result, the landmark data acquisition unit 240 acquires detection data corresponding to the chalk lines 1312 and detection data corresponding to the wire mesh 1313, respectively. This detection data is the candidate landmark data.

[0073] Next, in S1203, it is determined whether or not there is any candidate data to be acquired. If it is determined that there is candidate data (Yes in S1203), the process moves to S1204, where the priority setting unit 1110 sets a priority for each candidate data. Then, in S1205, the priority setting unit 1110 determines the selected data from the candidate data based on the set priority. On the other hand, if it is determined that there is no candidate data (No in S1203), it is determined that there are no landmarks in the landmark search range 1314 that are eligible to have a priority set, and the process moves to S1206.

[0074] The process in S1204 will now be explained in detail. The priority setting unit 1110 stores priorities associated with the type of landmark and sets the priority corresponding to the type of candidate data based on that type. Alternatively, the priority setting unit 1110 may set the priority of the candidate data based on the results of a priority questionnaire previously conducted with the user. For example, the priority setting unit 1110 displays a screen on the display unit 105 that allows the user to specify priorities for various landmarks such as chalk lines, members, block boundaries, and artificial objects, and sets the priority through user operation. If the priority of chalk lines is set to be high, then among the candidate data (detection data of chalk lines 1312 and detection data of wire mesh 1313), the priority of the detection data of chalk lines 1312 will be set higher than the priority of the detection data of wire mesh 1313. In this case, in S1205, since the priority of chalk lines is high, the detection data of chalk lines 1312 becomes the selected data. Note that the selected data is not limited to one; multiple candidate data with higher priorities may be selected. Furthermore, if only one candidate data point is obtained, that candidate data point may be used as the selected data point.

[0075] Next, in S1206, similar to S453 in Figure 4(B), the recording range determination unit 250 determines the recording range based on the inspection target data and selected data. Figure 13(B) shows the recording range 1321 determined for the same image 1320 as image 1310, including the detection data for cracks 1311 and the detection data for chalk lines 1312. After S1206, the process proceeds to S460.

[0076] According to the above embodiment 3, specific and easily connectable landmarks in a location can be preferentially included in the image range for documentation. This makes it easier to understand the location of the inspection target when reviewing the documentation later. In other words, it is possible to improve the efficiency of structural inspection work.

[0077] <How to set priorities> The method for setting priorities is not limited to the method described in the flowchart in Figure 12. Other methods include one based on the positional relationship between the data to be examined and the candidate data, one based on the rarity of the candidate data, and one based on the significance of the candidate data. These methods are described below. First, we will explain the method based on the positional relationship between the data to be inspected and the candidate data. The information processing device 100 calculates the distance between each candidate data and the data to be inspected. Then, the priority setting unit 1110 sets a higher priority for candidate data that is closer to the data to be inspected.

[0078] Next, we will explain a method based on the rarity of candidate data. In this method, the priority setting unit 1110 sets a higher priority for rare landmarks, based on the assumption that using fewer landmarks on a structure makes it easier to identify a location. Specifically, the information processing device 100 first applies a model suitable for each type of landmark to each of the multiple segmented images and calculates the total number of detection data for each model. The smaller this total number, the higher the rarity. Therefore, the priority setting unit 1110 sets a higher priority for candidate data of types with a small total number. For example, cracks (joints and formwork) present on a wall surface during construction are more numerous than diagonal braces, making it difficult to identify their location later. Therefore, by lowering the priority of joints and formwork and increasing the priority of diagonal braces, landmarks that are easier to identify can be preferentially included in the image range for documentation.

[0079] Next, a method based on the prominence of candidate data will be described. In this method, more prominent landmarks are included in the recording area. Specifically, the information processing device 100 first calculates the brightness difference inside and outside the candidate data for each candidate data, based on the pixel values ​​within the candidate data and the pixel values ​​surrounding the candidate data. The larger this brightness difference, the higher the prominence. Therefore, the priority setting unit 1110 sets a higher priority for candidate data with a large brightness difference. Note that the method of determining prominence is not limited to using brightness difference; for example, a method using the size of the area may also be used.

[0080] In Figure 12, at S1202, the landmark data acquisition unit 240 acquires candidate landmark data by applying a model suitable for each type of landmark. However, the method of acquiring candidate data is not limited to this. For example, a method of acquiring candidate data using a saliency map may be used. A saliency map is a map that estimates and visualizes areas in an image that are likely to attract human attention. For example, it can be obtained by creating a map that detects characteristic areas for each item such as brightness information and color information from the image, and then calculating the linear sum of these for each pixel. In the saliency map, pixels below a predetermined value are converted to 0, and areas where non-zero pixels are continuous are designated as candidate data. The landmark data acquisition unit 240 then sets priorities based on the pixel values ​​that constitute the candidate data. For example, statistical values ​​such as the average and maximum value of the pixel values ​​for each candidate data can be acquired, and priorities can be set based on these statistical values.

[0081] [Embodiment 4] In each of the embodiments described above, the information processing device 100 determined the judgment range and the recording range from a single image. In contrast, in Embodiment 4, the information processing device 100 uses two images containing the same inspection target to determine the judgment range from one image and the recording range from the other. This makes it possible to capture the image to which the damage degree judgment is applied and the image to which it is recorded for documentation purposes in a manner suitable for each. Embodiment 4 can be implemented with the same configuration as Embodiment 1, so a further explanation will be omitted. However, in Embodiment 4, the inspection target is a grid-like crack. Below, with reference to Figure 14, the differences between Embodiment 1 and Embodiment 4 will be described regarding the image acquisition unit 200, the judgment range determination unit 220, and the recording range determination unit 250.

[0082] The image acquisition unit 200 acquires multiple images that include the same inspection target. Specifically, as the image to which the degree of damage determination is applied, the image acquisition unit 200 acquires image 1410 in Figure 14, which is an image of the wall surface of the structure taken from directly in front of the inspection target with the inspection target magnified. On the other hand, as the image to be used to extract the area for documentation, the image acquisition unit 200 acquires image 1420 in Figure 14, which is an image of the wall surface of the structure including the area surrounding the inspection target. Images 1410 and 1420 include the same inspection target, and the areas 1421 and 1422 in image 1420 include the girder. In Embodiment 4, the girder is used as a landmark.

[0083] Next, the processing of the determination range determination unit 220 and the record range determination unit 250 will be described. First, the determination range determination unit 220 determines a determination range 1431 for the image 1410 so as to include the object to be inspected. Next, the record range determination unit 250 determines a record range 1441 for the image 1420 so as to include the object to be inspected and a part of the landmark (digit). Since both processes can be carried out by the method described in Embodiment 1, a detailed explanation will be omitted.

[0084] Although the above describes a method using two images containing the same inspection target, the information processing device 100 may use a viewpoint-shifted image for at least one of the two images. In this case, the image acquisition unit 200 generates a viewpoint-shifted image of the inspection target from a virtual viewpoint based on images of the same inspection target taken from multiple positions. Then, at least one of the determination range unit 220 and the record range unit 250 processes using the viewpoint-shifted image.

[0085] According to the embodiment 4 described above, by using multiple images that include the same inspection target to determine the range for judgment and the range for recording, it is possible to extract images suitable for judgment and recording, respectively. This ensures the accuracy of judgment regarding the inspection target, while also allowing for the recording of images that make it easier to understand the location and condition of the inspection target later. In other words, it is possible to improve the efficiency of structural inspection work.

[0086] Although the present invention has been described above along with its embodiments, these embodiments are merely examples of how the present invention can be implemented, and the technical scope of the present invention should not be interpreted as being limited by them. In other words, the present invention can be implemented in various forms without departing from its technical concept or its main features.

[0087] (Other embodiments) The present invention can also be realized by supplying a program that implements one or more of the functions of the above-described embodiments to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be realized by a circuit (e.g., an ASIC) that implements one or more functions. [Explanation of symbols]

[0088] 100: Information processing unit, 101: CPU, 102: ROM, 103: RAM, 104: HDD, 105: Display unit, 106: Operation unit, 107: Communication unit

Claims

1. image acquisition means for acquiring an image; a first determination means for determining a first image range to be used for determining the inspection of an inspection object included in the image based on a detection result of the inspection object from the image; a second determination means for determining a second image range, which is an image range that is wider than the range indicated by the first image range, and which is to be used for recording the inspection results of the inspection object; An information processing device comprising:

2. The information processing apparatus according to claim 1 , wherein the second image range includes the object to be inspected.

3. The method further includes a data acquisition means for acquiring data indicating a characteristic portion in the image, 3. The information processing apparatus according to claim 1, wherein the second determining means determines the second image range based on the data.

4. the data acquisition means acquires data indicating the characteristic portion whose distance based on the position of the inspection object is within a predetermined range, 4. The information processing apparatus according to claim 3, wherein the second determining means includes at least a part of the characteristic portion in the second image range.

5. 5. The information processing device according to claim 4, wherein, when there is no characteristic portion within the predetermined range, the second determination means determines, as the second image range, a range whose distance from the position of the inspection object as a reference is wider than the predetermined range.

6. When a plurality of characteristic parts are indicated by the data acquired by the data acquisition means, the device further comprises a setting means for setting a priority for each of the plurality of characteristic parts; The information processing device according to any one of claims 3 to 5, characterized in that the second determination means determines the second image range based on at least one of the plurality of characteristic parts selected based on the priority for each of the plurality of characteristic parts.

7. The information processing device according to claim 6, characterized in that the setting means sets the priority for each of the plurality of feature parts based on at least one of the positional relationship between the inspection object and the plurality of feature parts, the rarity of each of the plurality of feature parts, and the prominence of each of the plurality of feature parts.

8. The information processing device according to any one of claims 1 to 7, characterized in that the first determination means determines the first image range based on image conditions of an image used in training of a trained model used for the judgment.

9. The information processing device according to any one of claims 1 to 7, further comprising a determination means for selecting from a plurality of trained models the determination based on the size of the first image range determined by the first determination means, and making the determination using the selected trained model.

10. The information processing device according to any one of claims 1 to 9, characterized in that the second determination means determines the second image range so as not to exceed a maximum size based on at least one of the size of the object to be inspected, the resolution of the image, and the size of the image to be recorded.

11. further comprising a detection means for detecting an object from the image; 11. The information processing apparatus according to claim 1, wherein the inspection target is based on a detection result by the detection means.

12. 12. The information processing apparatus according to claim 11, wherein the object to be inspected is a group including a first object detected by the detection means and a second object detected by the detection means, the second object being an object within a predetermined range of distance from the position of the first object.

13. The information processing device according to any one of claims 1 to 12, characterized in that the second determination means determines a plurality of image ranges of different sizes, and determines an image range selected from the plurality of image ranges based on a user operation as the second image range.

14. 14. The information processing device according to claim 1, further comprising a display control means for controlling to display the result of the determination using the first image range and the image cut out by the second image range.

15. 15. The information processing device according to claim 1, further comprising a correction unit that corrects the result of the determination using the first image range or the second image range based on a user operation.

16. 16. The information processing device according to claim 1, further comprising a storage means for storing the result of the determination using the first image range and the image cut out using the second image range in association with each other.

17. the first determining means determines the first image range in the image; 17. The information processing apparatus according to claim 1, wherein the second determining means determines the second image range in an image different from the image.

18. 18. The information processing apparatus according to claim 1, wherein the image acquired by the image acquisition means is a viewpoint converted image.

19. 19. The information processing device according to claim 1, wherein the inspection target is a lattice crack, a tortoiseshell crack, or a closed crack.

20. 20. The information processing apparatus according to claim 1, wherein the determination is a determination of a degree of damage to the inspection object.

21. 21. The information processing apparatus according to claim 1, further comprising output means for outputting coordinate information of each of the first image range and the second image range.

22. image acquisition means for acquiring an image; a detection means for detecting landmarks, which are characteristic portions different in characteristics from surrounding areas, from the image; a determining means for determining an image range including an object included in the image that is an inspection target and the landmark detected by the detecting means as an image range to be used for recording an inspection result of the inspection target; An information processing device comprising:

23. determining a first image range to be used for determining an inspection of an inspection object included in an image based on a detection result of the inspection object from the image; determining a second image range that indicates a range wider than the range indicated by the first image range and that is to be used for recording the inspection results of the inspection object; 2. An information processing method according to claim 1, wherein:

24. A program for causing a computer to function as the information processing device according to any one of claims 1 to 22.

Images