Information processing apparatus, information processing method, and program
By creating and selecting images with varying resolutions based on deformation types, the method addresses the challenge of large data volumes in infrastructure inspections, enhancing efficiency and reducing storage and processing loads.
Patent Information
- Application Number
- JP2023070796
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-24
- Publication Date
- 2024-11-06
AI Technical Summary
Conventional image inspection methods for infrastructure structures result in large amounts of image data, leading to increased reading and storage loads, as high-resolution images are necessary for detecting deformations, which can be difficult to confirm and manage efficiently.
The method involves creating multiple processed images with different resolutions based on the type of deformation, selecting an appropriate resolution for display and detection, and reducing image sizes accordingly to minimize data volume while maintaining visibility of relevant deformations.
This approach reduces the data load for image reading and communication, allowing efficient deformation detection and management by selecting images with appropriate resolutions for confirmation and learning, thereby optimizing storage and processing requirements.
Smart Images

Figure 2024156382000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]
[0002] In recent years, image inspection using images of the wall surfaces of structures has been performed as an example of an inspection method for concrete structures such as bridges and tunnels. In this image inspection, a method has been proposed in which image recognition processing is performed on the images of the inspection target to detect abnormalities such as cracks (see Non-Patent Document 1), and it is said that the use of such technology makes it possible to efficiently detect abnormalities.
[0003] On the other hand, the anomaly detection results created using image recognition processing or image analysis processing may include false positives or non-detections. Therefore, the inspector confirms and corrects the anomaly detection results while visually checking the image of the inspection target by performing display operations such as zooming in and out. In addition, in order to reduce false positives and non-detections, the anomaly detection results may be learned. Patent Document 1 discloses a method of adjusting the anomaly detection results so that they are easily identifiable according to the image reduction magnification and superimposing them on the inspection image. It is said that by using the technology described in Patent Document 1, the anomaly can be adjusted to match the display magnification. [Prior art documents] [Patent documents]
[0004] [Patent Document 1] JP 2008-46065 A [Non-patent literature]
[0005] [Non-Patent Document 1] Dang Ji et al., "Simultaneous Detection of Multiple Bridge Damage Types Using YOLO," Proceedings of AI and Data Science, Vol. 2, No. J2, 2021, pp. 447-456 Summary of the Invention [Problem to be solved by the invention]
[0006] There are various types of anomalies that can be detected, and some types can only be confirmed with high-resolution images. In conventional technology, images are taken so that anomalies that can only be confirmed with high resolution can be detected, and the capacity of the image data becomes large. If the capacity of the image data is large, the read load increases and the memory capacity is strained when the image data is displayed for confirmation work or stored as learning data.
[0007] In view of the above-mentioned problems, an object of the present invention is to make it possible to reduce the amount of image data when acquiring an image that can identify an object in a processing target. [Means for solving the problem]
[0008] The information processing device of the present invention is characterized in that it comprises a processing means for creating multiple processed images with different resolutions for an image to be processed, a setting means for setting the type of object contained in the image to be processed, and a selection means for selecting, from the multiple processed images created by the processing means, a processed image with a resolution corresponding to the type of object set by the setting means as an image for identifying an object of the set type. Effect of the Invention
[0009] According to the present invention, it is possible to reduce the amount of data when managing the detection results of an object in an image. [Brief description of the drawings]
[0010] [Figure 1] FIG. 1 is a diagram for explaining an overview of a first embodiment. [Diagram 2] 1 is a block diagram showing an example of the configuration of an information processing device according to a first embodiment. [Diagram 3]11 is a diagram for explaining the association of images and deformation data to be processed with drawings. FIG. [Figure 4] 5 is a flowchart showing an example of a processing procedure according to the first embodiment. [Diagram 5] FIG. 13 is a diagram for explaining a process for creating images with different resolutions. [Figure 6] FIG. 11 is a diagram for explaining a process for setting a label type. [Figure 7] FIG. 11 is a diagram for explaining a process for acquiring label data. [Figure 8] 11A and 11B are diagrams for explaining a process of selecting a processed image according to a label type. [Figure 9] 11A and 11B are diagrams for explaining a process of creating display data and displaying a display screen. [Figure 10] FIG. 11 is a block diagram showing an example of a functional configuration of an information processing device according to a second embodiment. [Figure 11] 10 is a flowchart showing an example of a processing procedure according to the second embodiment. [Figure 12] FIG. 13 is a diagram for explaining a process of acquiring label data of a learning target. [Figure 13] 11A and 11B are diagrams for explaining a process of selecting a processed image according to a label type. [Figure 14] FIG. 11 is a diagram for explaining a process of collecting learning data. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] Hereinafter, the embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the present invention. Although the embodiments describe a number of features, not all of these features are essential to the invention, and the features may be combined in any manner.
[0012] (First embodiment) Hereinafter, an information processing device used for inspecting infrastructure structures such as concrete structures will be described. In this embodiment, an example will be described in which, when displaying an image of an inspection target and a defect detected from the image, a light-weight image can be selected and displayed according to the type of defect.
[0013] In this embodiment, the "inspection target" refers to a concrete structure to be inspected, such as a highway, a bridge, a tunnel, or a dam. The information processing device performs an abnormality detection process to detect the presence or absence and state of abnormalities such as cracks, using an image of the inspection target captured by the user. In addition, in the case of a concrete structure, "deformations" refer to cracks, lifting, or spalling of concrete. Other abnormalities include efflorescence (whitening phenomenon), exposed rebar, rust, water leakage, dripping, corrosion, damage (missing parts), cold joints, deposits, junk, and the like.
[0014] First, an overview of this embodiment will be described. FIG. 1(a) is a diagram showing an example of an image 101 obtained by photographing a wall surface of a bridge as an example of an inspection target. The image 101 obtained by photographing the inspection target (image to be processed) shows cracks 102, corrosion 103, and water leakage 104. In the deformation detection process, deformation data is created by digitalizing detection results such as the position and shape of deformation from the image 101 to be processed, and is recorded in association with the image 101 to be processed. Examples of methods for creating the deformation data include a method of acquiring the deformation data by detecting deformation from an image using a trained model created by machine learning or deep learning of AI (artificial intelligence).
[0015] After that, the inspector checks the image to see if the position and shape of the created abnormality data are correct. At this time, the larger the image size, the greater the load on reading the image. Also, if the information processing device for checking the image and the information processing device for managing the image and abnormality data are separate devices, the larger the image size, the greater the load on data communication. In particular, for infrastructure structures, large images taken with high resolution are used to detect abnormalities such as minute cracks that occur in structures of several tens of meters in size. Therefore, the load on reading the images and data communication to check for abnormalities becomes very large.
[0016] Therefore, in this embodiment, an image with an appropriate resolution is selected and displayed according to the type of deformation to be confirmed. The information processing device of this embodiment prepares multiple types of images with different resolutions based on the original image, and selects and displays an image with an appropriate resolution according to the type of deformation to be confirmed. For example, a method of creating multiple images with different resolutions includes a method of repeatedly reducing the original image at a fixed ratio.
[0017] 1(b) is a diagram showing an example of images with different resolutions created from the processing target image 101, and shows images with different resolutions created by repeatedly reducing the image size by half. Here, the highest resolution image 111 corresponds to resolution level 5, and the lowest resolution image 113 corresponds to resolution level 1.
[0018] 1(c) is a diagram showing an example of deformation data 121 detected from the processing target image 101. This deformation data 121 includes three types of deformation data: crack data 122, corrosion data 123, and water leakage data 124.
[0019] In this embodiment, an appropriate image is selected from the images shown in Fig. 1(b) based on a list of resolution levels for each type of abnormality as shown in Fig. 1(d). For example, when "water leakage" is selected as the type of abnormality to be confirmed, an image 112 with a resolution level of 3 is selected based on the list of resolution levels 131. Then, display data is created using the water leakage data 124 to be confirmed and the selected image 112.
[0020] FIG. 1(e) is a diagram showing an example of a display screen 141. A display image 142 on the display screen 141 is a display image created based on the image 112 and the water leakage data 124. In this display image 142, the crack 102 that existed in the image 101 to be processed cannot be identified by visual inspection, but the water leakage 104 to be confirmed can be identified by visual inspection. In this way, by selecting an image with an appropriate image resolution according to the type of deformation, it is possible to reduce the load of reading the image while maintaining a state in which the deformation to be confirmed can be visually confirmed. Also, even when the information processing device for confirming the image and the information processing device for managing the image and deformation data are separate, the load of data communication can be reduced.
[0021] <Hardware configuration> 2(a) is a block diagram showing an example of a hardware configuration of an information processing device 200 according to this embodiment. In this embodiment, the processing of the information processing device 200 may be realized by a single computer device, or may be realized by distributing each function among multiple computer devices as necessary. In this case, the multiple computer devices are connected to each other so that they can communicate with each other.
[0022] The information processing device 200 includes a control unit 201 , a non-volatile memory 202 , a work memory 203 , a storage device 204 , an input device 205 , an output device 206 , a communication interface 207 , and a system bus 208 .
[0023] The control unit 201 includes an arithmetic processor such as a CPU or MPU that controls the entire information processing device 200. The non-volatile memory 202 is a ROM that stores programs and parameters executed by the processor of the control unit 201. Here, the programs refer to programs for executing the processes of each embodiment described later. The non-volatile memory 202 also stores an OS (operating system), which is basic software executed by the control unit 201, and applications that work with the OS to realize applied functions. The work memory 203 is a RAM that temporarily stores programs and data supplied from an external device, etc. The work memory 203 also holds data obtained by executing the control process of FIG. 4 described later.
[0024] The storage device 204 is an internal device such as a hard disk or a memory card built into the information processing device 200, or an external device such as a hard disk or a memory card detachably connected to the information processing device 200. The storage device 204 includes a memory card or a hard disk constituted by a semiconductor memory, a magnetic disk, etc. The storage device 204 also includes a storage medium constituted by a disk drive that reads / writes data from / to optical disks such as DVDs and Blue-ray Discs.
[0025] The input device 205 is an operating member such as a mouse, keyboard, or touch panel that accepts user operations and outputs operation instructions to the control unit 201. The output device 206 is a display device such as a monitor or a display configured with an LCD or an organic EL, and displays data held by the information processing device 200 or data supplied from an external device. The communication interface 207 is communicatively connected to a network such as the Internet or a LAN (Local Area Network). The system bus 208 includes an address bus, a data bus, and a control bus that connect the components of the information processing device 200 to transmit and receive data.
[0026] In this embodiment, the non-volatile memory 202 stores an application for implementing a control process described below. The control process of the information processing device 200 of this embodiment is implemented by reading software provided by the application. The application has software for utilizing basic functions of the OS installed in the information processing device 200. Note that the OS of the information processing device 200 may have software for implementing the control process in this embodiment.
[0027] <Function block> FIG. 2(b) is a block diagram showing an example of a functional configuration of the information processing device 200 according to this embodiment. The information processing device 200 has a storage unit 221, a management unit 222, an image processing unit 223, a label type setting unit 224, a label data acquisition unit 225, an image selection unit 226, and a display control unit 227. Each function of the information processing device 200 is configured by hardware and / or software. Each functional unit may be configured as a system that is configured by one or more computer devices or server devices and connected via a network. In addition, when each functional unit shown in FIG. 2(b) is configured by hardware instead of by software, it is sufficient to have a circuit configuration corresponding to each functional unit in FIG. 2(b).
[0028] The management unit 222 manages the registration, deletion, acquisition, update, etc. of image data and deformation data to be processed stored in the storage unit 221. The image processing unit 223 processes the image data to create a processed image. The label type setting unit 224 sets the label type. The label data acquisition unit 225 acquires the label data to be displayed. The image selection unit 226 selects an image from the processed images based on the label type. The display control unit 227 creates display data using the selected image and the acquired label data, outputs the display data to the output device 206, and displays the display data on the output device 206. Details of the processes and functions of the image processing unit 223, the label type setting unit 224, the label data acquisition unit 225, the image selection unit 226, and the display control unit 227 will be described later with reference to FIG. 4.
[0029] <Explanation of images and deformation data> Next, the images and deformation data used in the control process of the information processing device 200 of this embodiment will be described with reference to FIG. 3. In image inspection, an image of a bridge wall as a structure is managed in association with a drawing. FIG. 3(a) is a diagram for explaining a state in which an image 311 of a bridge wall as an example of an infrastructure structure is pasted onto a drawing 300. The drawing 300 has drawing coordinates 301 with a point 302 as the origin. The position of the image on the drawing is defined by the vertex coordinates of the upper left corner of the image. For example, the coordinates of the image 311 are the position (x312, y312) of the vertex 312. The image is stored in the storage unit 221 together with the coordinate information.
[0030] In this embodiment, the images used in the image inspection of infrastructure structures are taken at a high resolution (e.g., 1 mm per pixel) so that minute cracks and the like can be confirmed, and therefore the image size is large. For example, image 311 in FIG. 3(a) is an image taken of a bridge deck measuring 20 m×10 m. When the resolution is 1.0 mm per pixel (1.0 mm / pixel), the image size of image 311 is 20,000 pixels×10,000 pixels. The image 311 taken at a high resolution includes various large and small abnormalities such as cracks, exposed rebar, corrosion, water leakage, and efflorescence. The abnormality data is information on the automatic detection result of abnormalities such as cracks occurring on a concrete wall surface, or the detection result updated by human operation. In this embodiment, the abnormality data is managed in association with a drawing.
[0031] Fig. 3(b) is a diagram for explaining a state in which deformation data 321 corresponding to image 311 in Fig. 3(a) is pasted onto drawing 300 at the same position as image 311. The deformation data 321 includes a large number of deformations, and the position of each deformation data in the deformation data 321 on the drawing is defined by pixel coordinates constituting the deformation data.
[0032] FIG. 3(c) is a diagram showing an example of the deformation data table 331. In the deformation data table 331, the coordinate column is an attribute value representing a plurality of coordinates constituting the deformation data. For example, the crack with ID "C001" indicates that it is represented by continuous pixels consisting of n points from (Xc001_1, Yc001_1) to (Xc001_n, Yc001_n). Thus, in this embodiment, it is assumed that the deformation data is represented by pixels. Note that the deformation data may be represented by vector data such as a polyline or a curve composed of a plurality of points. When the deformation data is represented by vector data, the data volume decreases and a simpler representation is obtained. An example of deformation data other than cracks is the corrosion with ID "F001". When the corrosion is represented by a polyline, it becomes a deformation having a region surrounded by the polyline. Note that the attribute information possessed by the deformation data is not limited to the attribute information shown in the deformation data table 331, and other attribute information may be retained.
[0033] In the control process of the information processing apparatus 200 of this embodiment, by associating the image and the deformation data, the positional relationship between the deformation data and the image can be obtained without referring to the drawing.
[0034] <Control Process> FIG. 4 is a flowchart showing an example of the processing procedure by the information processing apparatus 200 in this embodiment. The processing in FIG. 4 is realized by the control unit 201 expanding and executing the program stored in the non-volatile memory 202 in the work memory 203 and operating as each functional unit shown in FIG. 2(b). Hereinafter, the processing of this embodiment will be described according to the flowchart of FIG. 4.
[0035] <S401: Image Processing> First, the image processing unit 223 performs a process of processing the image to be processed to create a processed image. In this embodiment, as a method of processing the image, a method of converting the resolution and creating images having different resolutions as processed images will be described.
[0036] Image 501 in Fig. 5(a) is an example of an image captured of a concrete wall surface of a bridge, and image 501 is an image with a very large image size of 20,000 pixels x 10,000 pixels. It is assumed that cracks 502, water leakage 503, exposed rebar 504, and efflorescence 505 exist on this image 501. In this embodiment, a method of repeating a resolution conversion process that reduces the image size at a fixed ratio is used as a method for creating an image with a different resolution from this image 501.
[0037] Fig. 5(b) is a diagram showing an example of images with different resolutions created by repeatedly performing the same reduction process on image 501. Note that, in the example shown in Fig. 5(b), images with five different resolutions are created, but images with six or more different resolutions may be created, or images with fewer than five resolutions may be created. As another method for converting the resolution, a method using real values of the resolution may be used.
[0038] First, the image processing unit 223 acquires the image to be processed and information on the resolution of the image (e.g., 0.5 mm / pixel). Next, the image to be processed is subjected to a resolution conversion process so as to match the image to a predetermined target resolution 521 as shown in Fig. 5(c). This allows multiple images having the specified resolution to be created as processed images.
[0039] As another method of processing an image, a method combining resolution conversion processing and image segmentation processing may be used. FIG. 5(d) is a diagram for explaining the result of performing image segmentation processing on a processed image after repeatedly performing reduction processing on the image to be processed, with a predetermined segmentation size. Note that the dotted lines in each image shown in FIG. 5(d) indicate the boundaries of the divisions by segmentation. Also, as a method of image segmentation processing, the file size may be used. For example, the segmentation size is determined so that the file size per single segmented image is less than a fixed reference value (e.g., 100 KB), and image segmentation processing is performed. At this time, if the file size of the image file before segmentation is less than the reference value, the segmentation processing is omitted. In this way, image segmentation processing can be performed by using the file size of each segmented image.
[0040] <S402: Label Type Setting Process> Next, the label type setting unit 224 performs a process of setting the label type. In the present embodiment, among the detected abnormalities, the abnormality type to be confirmed is set as the label type. Examples of methods for setting the label type include, for example, a method of setting the abnormality type of the detection processing target as the label type.
[0041] FIG. 6(a) is a diagram showing an example of the abnormality type obtained by performing detection processing on the image to be processed. When the abnormality type obtained by performing detection processing is the result shown in FIG. 6(a), the label type setting unit 224 sets this "water leakage" and "efflorescence" as the label type. Note that in the present embodiment, the information on the abnormality type obtained by performing detection processing is stored in the storage unit 221, and the label type setting unit 224 can acquire the information on the abnormality type obtained by performing detection processing via the management unit 222.
[0042] Also, as another method of setting the label type, first, the display control unit 227 may display on the output device 206 a screen for receiving a user operation, and set the label type according to the user's operation instruction. FIG. 6(b) is a diagram showing an example of a screen 611 for receiving an input of the label type. On the screen 611, a list 612 of deformation types is displayed, and "water leakage" is selected by a user operation among them. Then, according to the operation of the OK completion button 613, the selected "water leakage" is set as the label type. In this way, the label type can also be set according to the user's operation instruction.
[0043] <S403: Label Data Acquisition Process> Next, the label data acquisition unit 225 performs a process of acquiring label data. In the present embodiment, the label data acquisition unit 225 acquires, as label data, part or all of the deformation data related to the image to be processed stored in the storage unit 221 via the management unit 222.
[0044] FIG. 7(a) shows an example of deformation data 701. This deformation data 701 includes crack data 702, rebar exposure data 703, water leakage data 704, and efflorescence data 705. The label data acquisition unit 225 acquires, as label data, the deformation data of the detection result related to the label type from among the deformation data 701. For example, when "water leakage" is selected as the label type as shown in FIG. 7(b), the label data acquisition unit 225 acquires the water leakage data 704 from among the deformation data 701 as the label data. Note that FIG. 7(c) shows an example of the acquisition result.
[0045] In this label data acquisition process, a method may be used in which a screen for receiving a user's operation instruction is displayed to allow the user to select the label data to be acquired. As a method for allowing the user to select, the user may be allowed to select a label type, or the user may be allowed to select some of the label data. Further, all the change data of the detection results related to the image to be processed can be acquired as label data. Note that if there is no change detection result related to the image to be processed, or if the change detection result is not to be displayed, this step may be skipped. If this step is skipped, display data may be created based on the image selected by the image selection unit 226, and the display data may be output to the output device 206.
[0046] <S404: Image Selection Process> Next, the image selection unit 226 performs a process of selecting an image from the processed images according to the label type. In the present embodiment, a method of selecting an image with an appropriate resolution according to the label type from the images with different resolutions created in S401 will be described.
[0047] FIG. 8(a) is a diagram showing an example of the processed image 801 created in S401. The image selection unit 226 selects an image with an appropriate resolution from these processed images 801 using the resolution level list 811 for each label type shown in FIG. 8(b). This resolution level list 811 defines one of the resolution levels from 5 to 1 for each label type, and the resolution levels 5 to 1 and the processed images 802 to 806 correspond to each other. Here, when "water leakage" is set as the label type in S402 as shown in FIG. 8(c), since "water leakage" is associated with the resolution level 3 in the resolution level list 811, the image selection unit 226 selects the processed image 804 corresponding to the resolution level 3. In this way, an image with an appropriate resolution is selected according to the label type.
[0048] On the other hand, even when multiple label types are set in S402, an image with an appropriate resolution can be selected. For example, when "rebar exposure" and "leakage" are set as label types as shown in FIG. 8(d) in S402, in the resolution level list 811, "rebar exposure" is associated with resolution level 4, and "leakage" is associated with resolution level 3. The image selection unit 226 selects the processed image 803 as the image corresponding to the highest resolution level 4 among the set label types. In this way, when multiple label types are set, an image with a resolution corresponding to the maximum resolution level is selected.
[0049] Also, when the image segmentation process is also performed in S401, a method of combining the label type and the display range may be used. FIG. 8(e) is a diagram showing an example of the processed image 821 in which the image segmentation process is also performed in S401. First, the resolution level corresponding to the label type is selected. For example, when "leakage" is set as the label type, the image selection unit 226 selects a partial image (e.g., partial image 823) corresponding to the display range of the deformation from among the processed images 822 corresponding to resolution level 3. In this way, based on the label type and the display range of the deformation, a partial image in the processed image can be selected.
[0050] <S405: Display Process> Next, the display control unit 227 creates display data using the processed image selected in S404 and the label data acquired in S403, and performs a process of outputting the display data to the output device 206. Hereinafter, the process of the display control unit 227 will be described with reference to FIG. 9.
[0051] FIG. 9(a) is a diagram showing an example of the processed image 901 selected by the image selection unit 226 in S404. This processed image 901 is an image created by repeatedly performing a reduction process on the image of the inspection target (processing target) in S401. FIG. 9(b) is a diagram showing an example of the label data 911 acquired by the label data acquisition unit 225 in S403. This label data 911 is label data corresponding to the image of the processing target and includes leakage data 912. The display control unit 227 performs a process of creating superimposed data by superimposing this processed image 901 and the label data 911. In the present embodiment, as shown in FIG. 3, the image of the processing target and the label data each have coordinate values in the drawing coordinate system. Therefore, since the coordinate values of the processed image 901 created based on the image of the processing target can also be obtained indirectly, it is possible to create superimposed data by adjusting the scale and positional relationship between the processed image 901 and the label data 911 and then superimposing them. FIG. 9(c) shows an example of the superimposed data 921 created using the processed image 901 and the label data 911.
[0052] After creating the superimposed data, the display control unit 227 creates display data for display on the output device 206 using the superimposed data and performs a process of outputting it to the output device 206. FIG. 9(d) is a diagram showing an example of a screen 931 related to the display data output to the output device 206. This screen 931 includes partial data 933 obtained by enlarging a part of the superimposed data 921. Further, the screen 931 also includes the label type 934 of the display target, the resolution level 935 of the image related to the partial data 933, and the display magnification 936 of the partial data. Thereby, it is possible for the user to easily grasp the label type and resolution during display. Also, as a method of displaying the resolution of the image, a display method other than the resolution level may be used. For example, as shown in FIG. 9(e), the resolution numerical value 941 may be displayed, or as shown in FIG. 9(f), a two-stage display mode 951 of high resolution / low resolution may be displayed. Note that when displaying the resolution numerical value 941, it can be handled by acquiring the numerical value of the resolution of the image used when creating the display data.
[0053] <S406: End determination> Subsequently, the control unit 201 determines whether all the processes in the present embodiment have been completed. If all the processes in the present embodiment have been completed, the process ends. On the other hand, when the control unit 201 receives a user operation to change the display settings of the display data via the input device 205, it is necessary to continue the process. Therefore, in such a case, the process proceeds to the process of S407.
[0054] <S407: Change of Display Settings of Display Data> In S407, the control unit 201 receives an operation instruction from the user to change the display settings related to the display data via the input device 205. Examples of methods for changing the display settings include changing the display range, changing the display magnification, changing the label type of the display target, changing the resolution, and the like.
[0055] For example, in S407, on the screen 931 of FIG. 9(d), the control unit 201 receives an operation instruction from the user to change the selection state of the label type 934 and select, for example, two types of "rebar exposure" and "leakage". In this case, in the present embodiment, after receiving the operation instruction from the user, the process returns to S403, and the label data acquisition unit 225 acquires new label data. Then, after the process of S404, in S405, the display control unit 227 re-creates the display data and outputs it to the output device 206.
[0056] Also, in the above example, when the control unit 201 receives an operation instruction from the user to change the resolution level on the screen 931 of FIG. 9(d), since it is not necessary to acquire new label data, the process may return to S404. Then, in S404, the image selection unit 226 selects a processed image corresponding to the resolution level selected by the user's operation instruction, and in S405, the display control unit 227 re-creates the display data and outputs it to the output device 206.
[0057] 9(d), since there is no need to acquire new label data or reselect an image, the process may be returned to S405. Then, in S405, the display control unit 227 changes the display range and the display magnification of the superimposition data to generate display data, and outputs the display data to the output device 206. In this way, the display data can be updated by performing a process to update the display data in response to receiving an operation to change the display settings related to the display data.
[0058] <Modification of the first embodiment> In this embodiment, the display control unit that performs the creation process and display process of the display data may be configured as another information processing device via a network. Specifically, the information processing device having the display control unit acquires the processed image selected in S404 and the label data acquired in S403 by data communication via the network. Then, the display data is created using the acquired processed image and label data, and output to the display device. In this case as well, an image with a suitable resolution can be selected according to the type of abnormality to be confirmed, so that the load of data communication can be reduced.
[0059] In this embodiment, a method for selecting an image with a resolution according to the type of deformation when checking the deformation detection result has been described, but an image of the processing target may be selected as the detection image used in the deformation detection process. Specifically, a plurality of processed images with different resolutions are created from an image of the inspection target. Then, an image with a resolution appropriate for the type of deformation to be detected is selected, and the deformation detection process is performed. In this way, the detection process can be performed using an image with a resolution appropriate for the type of deformation, so that the detection process can be performed efficiently.
[0060] In addition, in the present embodiment, an example of confirming the detection result of anomalies in an image of an inspection target in infrastructure inspection as an object in an image has been described, but the present invention can also be applied to objects other than anomalies. For example, the present invention can also be applied to a medical diagnosis confirmation work in a hospital or the like in which a lesion (object) is confirmed from a photographed image of a human body (inspection target). Specifically, multiple lesions are detected from a photographed image of a human body using AI (artificial intelligence, etc.). Then, multiple processed images with different resolutions are created based on the photographed image, and an image with an appropriate resolution is selected according to the type of lesion to be confirmed. Then, the detected lesion data and the selected image are superimposed, and the display data is displayed. In this way, a lightweight image is selected according to the type of lesion, thereby making it possible to perform lightweight display processing. Furthermore, the image is not necessarily limited to an optically photographed image, and may be, for example, vector data.
[0061] As described above, according to this embodiment, an image with an appropriate resolution is selected depending on the type of label to be checked, and the label data and the processed image are superimposed to create display data, which is then displayed on the display unit. This reduces the data read load and communication load.
[0062] Second embodiment In the first embodiment, an example was described in which an image with an appropriate resolution according to the type of deformation was selected and displayed when displaying the deformation detection result. Here, by collecting and using a combination of label data such as cracks and images as learning data used in machine learning and deep learning, it is expected that the deformation detection performance can be improved. In collecting this learning data, it is desirable to collect high-resolution images when learning fine deformations such as cracks. On the other hand, since deformations that occur over a wide area such as water leakage can be visually recognized on an image reduced to a low resolution, it is sufficient to collect low-resolution images. In this way, by collecting images with an appropriate resolution according to the type of deformation to be learned as learning data, it is possible to suppress the size of the learning data from increasing. Therefore, in this embodiment, a method of selecting and collecting images with an appropriate resolution according to the type of deformation in collecting learning data for improving the detection performance of a detector that detects deformations such as cracks from wall images of concrete structures is described.
[0063] The hardware configuration of the information processing device according to this embodiment is the same as that of the first embodiment shown in Fig. 2(a), and therefore the description will be omitted. Fig. 10 is a block diagram showing an example of the functional configuration of the information processing device 1000 according to this embodiment. The functional configuration of Fig. 10 differs from the functional configuration shown in Fig. 2(b) described in the first embodiment in that a learning data collection unit 1001 is added instead of the display control unit 227.
[0064] The learning data collection unit 1001 is a functional unit of the control unit 201, and performs a process of collecting combinations of label data and processed images as learning data. In this embodiment, the label type setting unit 224 performs a process of setting the label type of the learning target, and the label data acquisition unit 225 performs a process of acquiring the label data of the learning target.
[0065] FIG. 11 is a flowchart showing an example of a processing procedure by the information processing apparatus 1000 according to the present embodiment. In the flowchart of FIG. 11, in steps with the same numbers as those in the flowchart of FIG. 4 described in the first embodiment, the same processing as in the first embodiment is executed, and thus the description thereof is omitted. In the case of the present embodiment, after the label type setting unit 224 sets the label type of the learning target in S402, the information processing apparatus 1000 proceeds to the processing of S1101. In S1101, the label data acquisition unit 225 performs a process of acquiring the label data of the learning target. Thereafter, in the process of S1102, the image selection unit 226 selects an image of the learning target from the processed images. In the subsequent S1103, the learning data collection unit 1001 collects the combination of the selected label data and the processed image as learning data. Hereinafter, the outlines of the processes of S1101, S1102, and S1103 will be described.
[0066] <S1101: Label Data Acquisition Process> In the process of S1101, the label data acquisition unit 225 performs a process of acquiring the label data of the learning target. In the present embodiment, the deformation data stored in the storage unit 221 is acquired as label data via the management unit 222.
[0067] FIG. 12(a) is a diagram showing an example of the deformation data 1201 related to the image to be processed. It is assumed that this deformation data 1201 includes crack data 1202, rebar exposure data 1203, water leakage data 1204, and efflorescence data 1205. The label data acquisition unit 225 acquires, as label data, the deformation data of the detection result related to the label type of the learning target among these. For example, when "water leakage" is selected as the label type of the learning target as shown in FIG. 12(b), the water leakage data 1204 is acquired as label data. The result acquired as the label data of the learning target is shown in FIG. 12(c).
[0068] Deformation data created by detection processing using AI etc. may contain false positives or non-detections. Therefore, the user must correct these false positives or non-detections. Deformation data corrected by this process is highly likely to be deformation data with a high learning effect, so it is desirable to select it as label data for learning with priority. The method of selecting deformation data with a high learning effect will be explained using Fig. 12(d) and Fig. 12(e).
[0069] FIG. 12(d) is a diagram showing an example of deformation data 1231 created using AI or the like, and FIG. 12(e) is a diagram showing an example of deformation data 1241 in which a user has made corrections to the deformation data 1231. In this deformation data 1241, a hatched area 1242 is an area in which the updated portion of the deformation data is visualized. The deformation data 1231 created in advance by a user using AI or the like is updated to deformation data 1241 including the hatched area 1242 and stored in the storage unit 221. The label data acquisition unit 225 preferentially acquires this corrected deformation data as the label data to be learned. Here, the update history of the deformation data is assumed to be stored in the storage unit 221 together with the deformation data. As a result, the label data acquisition unit 225 can acquire the deformation data and the update history of the deformation data via the management unit 222.
[0070] When acquiring label data using the update history of deformation data, if the amount of updated deformation data is large, it is desirable to narrow down the label data to be acquired. For example, based on the update history of the deformation data, the update amount (e.g., update length, update area, number of label changes, number of update operations, update work time) for each deformation data is calculated. Then, multiple deformation data are acquired as label data in descending order of the calculated update amount. In this way, deformation data with the largest amount of update can be given priority as the learning target. Note that when narrowing down the label data to be acquired based on the update history of the deformation data, the priority order of the label data to be acquired may be determined by combining multiple types of update history.
[0071] As another method of obtaining the label data to be learned, a method may be used in which a screen for receiving a user's operation instruction is displayed and the user is allowed to select the label data to be obtained. For example, the variation data related to the label type of the learning target is displayed on the screen, and the label data acquisition unit 225 acquires the variation data selected on the screen as the label data of the learning target based on the user's operation instruction. When it is desired to learn some of the large amount of variation data, it is preferable to use the method of receiving the user's operation instruction in this way.
[0072] <S1102: Image Selection Process> In the process of S1102, the image selection unit 226 performs a process of acquiring the processed image to be learned from the processed images. In the present embodiment, a method of selecting an image with an appropriate resolution according to the label type of the learning target from the images with different resolutions created in S401 will be described.
[0073] FIG. 13(a) is a diagram showing an example of the processed image 1301 created in S401. The image selection unit 226 selects an image with an appropriate resolution from these processed images 1302 to 1306 using the resolution level list 1311 for each label type shown in FIG. 13(b). This resolution level list 1311 is the same as the resolution level list 811 shown in FIG. 8(b). That is, when "water leakage" is set as the label type of the learning target as shown in FIG. 13(c), the image selection unit 226 selects the processed image 1304 corresponding to the resolution level 3.
[0074] Also, in the process of the image selection unit 226, instead of the resolution level list 1311 shown in FIG. 13(b), the actual resolution numerical values may be used. In this case, the actual resolution value (e.g., 1.0 mm / pixel) is set as the resolution for each label type, and at the same time, the resolution of the processed image is also acquired. Then, based on the actual resolution numerical value corresponding to the label type, an image with an appropriate resolution can be selected from the processed images.
[0075] In the process of the image selection unit 226, by combining the label type of the learning target and the label data of the learning target, a partial image with a resolution corresponding to the label type may be selected. For example, assume that the label data 1321 of the learning target shown in FIG. 13(d) in S1101 is acquired. This label data 1321 is the label data when "water leakage" at resolution level 3 is selected as the label type of the learning target. FIG. 13(e) is a diagram showing an example of the processed image 1331 for which image segmentation processing has also been performed in S401. At this time, the image selection unit 226 selects a partial image (e.g., partial image 1333) including a range corresponding to the water leakage data on the label data 1321 from among the images 1332 corresponding to resolution level 3. In this way, by selecting a partial image of the area where the label data of the learning target exists, in the learning data collection process described later, the learning data where the label data exists can be intensively collected.
[0076] <S1103: Learning Data Collection Process> Subsequently, the learning data collection unit 1001 collects the combination of the selected label data and the processed image as learning data. Hereinafter, when explaining the learning data collection process in the present embodiment, as shown in FIG. 14(a), it will be explained assuming that "water leakage" is set as the label type of the learning target.
[0077] FIG. 14(b) is a diagram showing an example of the processed image 1411 selected by the image selection unit 226. This processed image 1411 is an image obtained by performing reduction processing on the original image, and is an image in which water leakage 1412 and efflorescence 1413 can be confirmed. Further, FIG. 14(c) is a diagram showing an example of the label data 1421 acquired by the label data acquisition unit 225, and water leakage data 1422 exists. The learning data collection unit 1001 performs a process of collecting the combination of this processed image 1411 and the label data 1421 as learning data. Note that, as shown in FIG. 13(e), when the partial image 1333 is selected by the image selection unit 226, the combination of the partial image and the label data is collected as learning data.
[0078] The collected label data and processed images each hold coordinate values in the drawing coordinate system, as explained in Fig. 3. Therefore, the coordinate values for the processed image 1411 can be indirectly obtained, and the scale and positional relationship between the processed image 1411 and the label data 1421 can be adjusted before use as learning data.
[0079] <Modification of the second embodiment> In this embodiment, images collected together with label data as learning data may be images that have already been created. For example, a plurality of processed images with different resolutions are created in advance for use in the abnormality detection process and stored in the storage unit 221. Thereafter, in the learning data collection process of this embodiment, instead of the image processing unit 223 creating a processed image in S401, the created processed image stored in the storage unit 221 may be acquired via the management unit 222. This makes it possible to omit the image processing process for collecting learning data, thereby making the process more efficient.
[0080] As described above, according to this embodiment, combinations of images with appropriate resolutions and label data can be collected as training data according to the label type, thereby preventing the volume of training data from expanding.
[0081] (Other embodiments) The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.
[0082] The disclosure of this embodiment includes the following configuration, method, and program.
[0083] (Configuration 1) A processing means for generating a plurality of processed images having different resolutions from the image to be processed; A setting means for setting a type of an object included in the image to be processed; a selection means for selecting, from among a plurality of processed images created by the processing means, a processed image having a resolution corresponding to the type of object set by the setting means as an image for identifying the object of the set type; An information processing device comprising:
[0084] (Configuration 2) The information processing device according to configuration 1, further comprising a display control means for causing a display means to display display data for confirming the object of the set type by using the processed image selected by the selection means. (Configuration 3) The method further includes the steps of: acquiring a detection result of the object of the set type included in the image to be processed; 3. The information processing apparatus according to claim 2, wherein the display control means creates and displays the display data based on the processed image selected by the selection means and the detection result acquired by the acquisition means.
[0085] (Configuration 4) the processing means creates the plurality of processed images by combining a resolution conversion process and an image division process; The selection means selects a partial image based on the division into sections divided by the division process from the selected processed image, 4. The information processing device according to configuration 2 or 3, wherein the display control means creates the display data using the partial image and displays the display data. (Configuration 5) The information processing device according to any one of configurations 2 to 4, characterized in that when a plurality of types are set by the setting means, the selection means selects an edited image having a resolution corresponding to the type corresponding to the highest resolution among the plurality of types set.
[0086] (Configuration 6) 6. The information processing device according to any one of configurations 2 to 5, wherein the display data further includes at least the resolution of the processed image selected by the selection means, or the type of the object set by the setting means. (Configuration 7) The image processing device further includes an input unit for receiving an operation to change the resolution of the processed image displayed on the display unit or the type of the object, When an operation to change the resolution is accepted by the input means, the selection means selects a processed image having the changed resolution, 7. The information processing device according to configuration 6, wherein when an operation to change the type of the object is accepted by the input means, the setting means sets the changed type.
[0087] (Configuration 8) an acquisition means for acquiring a detection result of the object of the set type included in the image to be processed; a collection means for collecting a combination of the processed image selected by the selection means and the detection result acquired by the acquisition means as learning data; 2. The information processing device according to configuration 1, further comprising: (Configuration 9) 9. The information processing device according to configuration 8, wherein the acquiring means selects and acquires the detection result of the object based on an update history of the detection result. (Configuration 10) the processing means creates the plurality of processed images by combining a resolution conversion process and an image division process; The selection means selects a partial image based on the division into sections divided by the division process from the selected processed image, 10. The information processing device according to configuration 8 or 9, wherein the collection means collects combinations of the partial images and the detection results acquired by the acquisition means as learning data. (Configuration 11) 2. The information processing apparatus according to configuration 1, wherein the selection means selects an image for detecting an object contained in the image to be processed.
[0088] (method) An information processing method executed by an information processing device, A processing step of creating a plurality of processed images having different resolutions from the image to be processed; A setting step of setting a type of an object included in the image to be processed; a selection step of selecting, from among the plurality of processed images created in the processing step, a processed image having a resolution corresponding to the type of object set in the setting step as an image for identifying the object of the set type; An information processing method comprising:
[0089] (program) A processing step of creating a plurality of processed images having different resolutions from the image to be processed; A setting step of setting a type of an object included in the image to be processed; a selection step of selecting, from among the plurality of processed images created in the processing step, a processed image having a resolution corresponding to the type of object set in the setting step as an image for identifying the object of the set type; A program for causing a computer to execute the following. [Explanation of symbols]
[0090] 223 image processing section, 224 label type setting section, 226 image selection section
Claims
1. A processing means for generating a plurality of processed images having different resolutions from the image to be processed; A setting means for setting a type of an object included in the image to be processed; a selection means for selecting, from among a plurality of processed images created by the processing means, a processed image having a resolution corresponding to the type of object set by the setting means as an image for identifying the object of the set type; An information processing device comprising:
2. 2. The information processing apparatus according to claim 1, further comprising a display control means for causing a display means to display display data for confirming the object of the set type by using the processed image selected by the selection means.
3. The method further includes the steps of: acquiring a detection result of the object of the set type included in the image to be processed; 3. The information processing apparatus according to claim 2, wherein the display control means creates and displays the display data based on the processed image selected by the selection means and the detection result acquired by the acquisition means.
4. the processing means creates the plurality of processed images by combining a resolution conversion process and an image division process; The selection means selects a partial image based on the division into sections divided by the division process from the selected processed image, 3. The information processing apparatus according to claim 2, wherein the display control means creates the display data using the partial image and displays the display data.
5. 3. The information processing device according to claim 2, wherein, when a plurality of types are set by the setting means, the selection means selects a processed image having a resolution corresponding to the type corresponding to the highest resolution among the plurality of types set.
6. 3. The information processing apparatus according to claim 2, wherein the display data further includes at least a resolution of the processed image selected by the selection means or a type of the object set by the setting means.
7. The image processing device further includes an input unit for receiving an operation to change the resolution of the processed image displayed on the display unit or the type of the object, When an operation to change the resolution is accepted by the input means, the selection means selects a processed image having the changed resolution, 7. The information processing apparatus according to claim 6, wherein, when an operation to change the type of the object is accepted by the input means, the setting means sets the changed type.
8. an acquisition means for acquiring a detection result of the object of the set type included in the image to be processed; a collection means for collecting a combination of the processed image selected by the selection means and the detection result acquired by the acquisition means as learning data; The information processing apparatus according to claim 1 , further comprising:
9. The information processing apparatus according to claim 8 , wherein the acquiring means selects and acquires the detection result of the object based on an update history of the detection result.
10. the processing means creates the plurality of processed images by combining a resolution conversion process and an image division process; The selection means selects a partial image based on the division into sections divided by the division process from the selected processed image, The information processing apparatus according to claim 8 , wherein the collection means collects, as learning data, a combination of the partial image and the detection result acquired by the acquisition means.
11. 2. The information processing apparatus according to claim 1, wherein the selection means selects an image for detecting an object contained in the image to be processed.
12. An information processing method executed by an information processing device, A processing step of creating a plurality of processed images having different resolutions from the image to be processed; A setting step of setting a type of an object included in the image to be processed; a selection step of selecting, from among the plurality of processed images created in the processing step, a processed image having a resolution corresponding to the type of object set in the setting step as an image for identifying the object of the set type; An information processing method comprising:
13. A processing step of creating a plurality of processed images having different resolutions from the image to be processed; A setting step of setting a type of an object included in the image to be processed; a selection step of selecting, from among the plurality of processed images created in the processing step, a processed image having a resolution corresponding to the type of object set in the setting step as an image for identifying the object of the set type; A program for causing a computer to execute the following.
Citation Information
Patent Citations
Image analyzing method, image generating method, learning model generating method, annotation assigning device, and annotation assigning program
WO2021125305A1
Road surface image creating method and road surface image producing device
JP2008046065A
