Image processing device, control method, and program

Abstract

To realize a technique for making it easier to comprehend an image in which an inspection target is captured and a defect detection result.SOLUTION: The information processor includes display control means for displaying defect information detected from an image in which an inspection target is captured on display means in association with the image. The display control means controls the method for displaying the defect information on the basis of the display size of the image or the defect information.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present invention relates to a technique for detecting and displaying abnormalities from an image obtained by photographing an inspection target.

Background Art

[0002] There is a method of detecting abnormalities such as cracks by performing image analysis on an image obtained by photographing an inspection target such as the wall surface of a concrete structure. By the way, when there are a large number of images obtained by photographing an inspection target, it becomes difficult to grasp the images and the abnormality detection results.

[0003] In Patent Document 1, there is described a method of detecting crack information from a road surface image obtained by photographing a road surface and arranging images obtained by reducing and overlapping the road surface image and the crack information side by side on one screen for display.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, depending on the distribution form of the abnormalities, the abnormality detection results may be difficult to observe in Patent Document 1. For example, when the detected abnormalities are close to each other, they may be drawn so as to overlap, making it difficult to understand the shape of the abnormalities. Further, when the abnormalities are color-coded and displayed according to the attributes of the abnormalities (for example, the width of the crack), the appearance of the color may change depending on the reduction ratio and the color of the adjacent abnormalities.

[0006] The present invention has been made in view of the above problems, and an object thereof is to realize a technique that can more easily grasp an image obtained by photographing an inspection target and an abnormality detection result than in the prior art.

Means for Solving the Problems

[0007] To solve the above problems and achieve the objective, the image processing apparatus of the present invention has a display control means that displays abnormality information detected from an image of an object to be inspected in association with the image on a display means, and the display control means is The larger the display size of the image or deformation information, the more detailed the deformation information will be displayed. The smaller the display size of the image or deformation information, the more the display of the deformation information will be limited by omitting some of the deformation information. do. [Effects of the Invention]

[0008] According to the present invention, images of the object being inspected and the results of the deformation detection can be obtained more easily than before. [Brief explanation of the drawing]

[0009] [Figure 1] A block diagram showing the hardware configuration of the image processing device of this embodiment. [Figure 2] Functional block diagram of the image processing apparatus of this embodiment. [Figure 3] A diagram illustrating the display method of deformation information in this embodiment. [Figure 4] A diagram illustrating how the display of deformation information changes depending on the image display size. [Figure 5] Data structure diagram of the image information table. [Figure 6] Data structure diagram of the deformation information table. [Figure 7] A diagram illustrating the GUI for the deformation detection results of this embodiment. [Figure 8] A flowchart illustrating the display control process for thumbnail images and detailed images. [Figure 9] A flowchart illustrating the display image generation process shown in Figure 8. [Modes for carrying out the invention]

[0010] The embodiments will be described in detail below with reference to the attached drawings. Note that the following embodiments do not limit the invention as defined in the claims. While the embodiments describe multiple features, not all of these features are essential to the invention, and the features may be combined in any way. Furthermore, in the attached drawings, identical or similar configurations are given the same reference numerals, and redundant descriptions are omitted.

[0011] The following describes an embodiment in which the image processing device of the present invention is applied to a computer device used for inspecting infrastructure structures such as concrete structures.

[0012] In this embodiment, an example is described in which a computer device operates as an image processing device and displays an image of the object to be inspected (detection image) in association with the deformation information detected from the detection image, and the method of displaying the deformation information is controlled according to the display size of the detection image.

[0013] In this embodiment, the "inspection target" refers to concrete structures such as expressways, bridges, tunnels, and dams that are subject to inspection. The image processing device performs deformation detection processing to detect the presence and condition of deformations such as cracks using images captured by the user of the inspection target. "Deformation" refers to, for example, cracks, delamination, and spalling of concrete in the case of concrete structures. Other examples include efflorescence, exposed rebar, rust, water leakage, water dripping, corrosion, damage (defects), cold joints, precipitates, and honeycombing.

[0014] <Hardware Configuration> First, the hardware configuration of the image processing device 100 of this embodiment will be described with reference to Figure 1.

[0015] Figure 1 is a block diagram showing the hardware configuration of the image processing device 100 of this embodiment.

[0016] Note that the processing of the embodiments described below may be implemented by a single computer device, or each function may be distributed and implemented by a plurality of computer devices as necessary. The plurality of computer devices are communicably connected to each other.

[0017] The image processing apparatus 100 includes a control unit 101, a non-volatile memory 102, a work memory 103, a storage device 104, an input device 105, an output device 106, a network interface 107, and a system bus 108.

[0018] The control unit 101 includes arithmetic processing processors such as a CPU and an MPU that overall control the entire image processing apparatus 100. The non-volatile memory 102 is a ROM that stores programs and parameters executed by the processor of the control unit 101. Here, the program is a program for executing display control processing described later. The work memory 103 is a RAM that temporarily stores programs and data supplied from an external device or the like. The storage device 104 is an internal device such as a hard disk or a memory card built in the image processing apparatus 100 or an external device such as a hard disk or a memory card detachably connected to the image processing apparatus 100. The storage device 104 includes a memory card, a hard disk, etc. composed of a semiconductor memory, a magnetic disk, etc. Further, the storage device 104 includes a storage medium composed of a disk drive that reads / writes data to / from optical disks such as CD, DVD, Bl u- ray (Registered trademark) Disc and the like.

[0019] The input device 105 is an operating component such as a mouse, keyboard, or touch panel that accepts user input and outputs operation instructions to the control unit 101. The output device 106 is a display device such as a display or monitor made of an LCD or organic EL, which displays data held by the image processing device 100 or data supplied from external devices. The network interface 107 is connected to a network such as the Internet or a LAN (Local Area Network) for communication. The system bus 108 connects the components 101 to 107 of the image processing device 100 so that data can be exchanged.

[0020] The non-volatile memory 102 or storage device 104 stores the operating system (OS), which is the basic software executed by the control unit 101, and applications that work in cooperation with the OS to realize advanced functions. In this embodiment, the non-volatile memory 102 or storage device 104 also stores applications that enable the image processing device 100 to perform image analysis processing to detect abnormalities from images of the object to be inspected, as described later.

[0021] The processing of the image processing device 100 in this embodiment is achieved by loading software provided by an application. The application is assumed to have software for utilizing the basic functions of the OS installed on the image processing device 100. The OS of the image processing device 100 may also have software for implementing the processing in this embodiment.

[0022] <Functional Configuration> Next, with reference to Figure 2, the functional blocks of the image processing apparatus 100 of this embodiment will be described.

[0023] Figure 2 is a functional block diagram of the image processing apparatus 100 of this embodiment.

[0024] The image processing device 100 comprises an input unit 201, a management unit 202, a deformation detection unit 203, a display image generation unit 204, and an image display unit 205. Each function of the image processing device 100 is composed of hardware and software. Note that each functional unit may be composed of one or more computer devices or server devices, and the system may be configured as a network-connected system.

[0025] The input unit 201 acquires images from external devices such as cameras via the network interface 107 and stores them in the storage device 104 as images for abnormality detection (hereinafter referred to as detection images), which will be described later. The image processing device 100 also generates image information for managing the detection images and stores it in the image information table 500, which will be described later in Figure 5. The management unit 202 manages the registration, deletion, and updating of image information and deformation information. Image information is managed in the image information table 500, which will be described later in Figure 5. Deformation information is managed in the deformation information table 600, which will be described later in Figure 6. The management unit 202 also manages numerical values ​​that are referenced in the deformation detection model and the display control processing, which will be described later in Figures 8 and 9.

[0026] The deformation detection unit 203 performs deformation detection processing on the detection image and acquires deformation information. The deformation information includes the type of deformation, the shape of the deformation, and other attributes of the deformation. The deformation detection unit 203 also acquires detection results for each detection image. The detection results for each detection image include the type of deformation distribution pattern (e.g., tortoise shell cracks, closed cracks), the degree of damage, etc. If there is no deformation distribution pattern in the detection image, information such as the type of deformation distribution pattern cannot be obtained. The deformation detection processing performed by the deformation detection unit 203 uses a machine learning-trained model. In this case, for example, multiple combinations of input data and output data for the deformation detection processing are prepared as training data, knowledge is acquired from them by machine learning, and a trained model is generated that outputs output data for the input data as a result based on the acquired knowledge. The trained model can be configured as, for example, a neural network model. The trained model may be updated after certain processing as needed.

[0027] The display image generation unit 204 generates a display image for display on the image display unit 205 according to the display size of the detection image. The display image is an image in which the deformation information detected from the detection image is superimposed on the detection image and resized to fit the size of the display area.

[0028] The image display unit 205 displays the display image generated by the display image generation unit 204.

[0029] Figure 3 illustrates the display method of deformation information in this embodiment. When the display size of the detection image is small, the deformation information superimposed on the detection image is simplified and displayed in a simplified manner. Examples of simplified display include thinning out the deformation, simplifying the deformation shape, limiting the display method such as the drawing color, line width, and line type of the deformation, displaying icons or text strings for the deformation type, and displaying a heat map.

[0030] Thinning out deformations is a display method that reduces the difficulty in seeing deformations due to their density. However, thinning out deformations makes it impossible to identify the areas where deformations existed before thinning, so the areas where deformations existed are also visualized. Figure 3(a2) illustrates the state in which deformations are thinned out compared to Figure 3(a1). Some of the polylines indicating cracks have been removed, and the areas containing the removed polylines are visualized.

[0031] Simplification of deformation shape is a display method that makes it easier to grasp the general shape of deformation by simplifying the deformation shape. Figure 3(b2) illustrates a simplified deformation shape compared to Figure 3(b1). The vertices of the polyline indicating the crack shape have been reduced.

[0032] Limiting the drawing color, line width, and line type of deformations is a display method that makes it easier to understand the location and shape of deformations by limiting the drawing color, line width, and line type when at least one of these is changed according to the attributes of the deformation. Figure 3(c2) illustrates a state in which the drawing color of deformations is limited compared to Figure 3(c1). The drawing color of the polyline representing the crack is limited, and the polyline is drawn with a single drawing color.

[0033] The icons and text displays for deformation types are a way of displaying deformation characteristics at a glance by representing the distribution pattern of the deformation with icons and text. Figure 3(d2) is an example of how deformation types are represented with icons and text in relation to Figure 3(d1). Since the cracks have a distribution pattern of "tortoise shell cracks," the icon representing "tortoise shell cracks" is displayed.

[0034] Heatmap display is a method of displaying deformations that reduces the difficulty in seeing densely packed deformations by visualizing the spatial density of deformations through changes in hue and color intensity. Figure 3(e2) illustrates the deformations displayed as a heatmap in relation to Figure 3(e1). The density of cracks is represented by the intensity of the color.

[0035] If the display size of the detection image is large, the deformation information will be displayed in detail. For example, one method of displaying deformation information in detail is to visualize the deformation information according to its attributes. Visualization of deformation information according to its attributes is a display method that visualizes the attributes of the deformation information using at least one of the following: drawing color, line width, line type, etc.

[0036] Figure 4 illustrates how the display of deformation information changes depending on the display size of the detection image. In the example in Figure 4, the larger the display size of the detection image, the more detailed the shape of the deformation is displayed, and the smaller the display size of the detection image, the more simplified the shape of the deformation is displayed.

[0037] <Display control processing for detection images and deformation information> Next, with reference to Figures 5 to 9, the display control process for the detection image and deformation information of this embodiment will be described.

[0038] Figure 5 illustrates an image information table 500 for managing image information. In the example in Figure 5, each row of the image information table 500 represents one image.

[0039] Image information ID 501 contains identification information that uniquely identifies the image information stored in image information table 500.

[0040] The file path 502 contains a string representing the storage location of the detection image in the storage device 104.

[0041] The date and time field 503 contains a string representing the date and time the detection image was taken.

[0042] Detection result 504 describes the deformation detection processing results for each detection image. The deformation detection processing results include the type of deformation distribution pattern, the degree of damage, etc.

[0043] Figure 6 illustrates a deformation information table 600 for managing deformation information. In the example in Figure 6, each row of the deformation information table 600 represents one piece of deformation information.

[0044] The deformation information ID 601 contains identification information that uniquely identifies the deformation information stored in the deformation information table 600.

[0045] Image information ID 602 ​​contains identification information for the image information corresponding to the detection image in which deformation information was detected.

[0046] Type 603 contains a string representing the type of deformation. Examples of type 603 include cracking, efflorescence, delamination, honeycombing, and cold joints.

[0047] Shape 604 contains information describing the shape of the deformation. For example, if deformation information of type 603 is "crack," the shape 604 is a polyline representing the shape of the crack. Similarly, if deformation information of type 603 is "efflorescence," the shape 604 is a polygon representing the extent of the efflorescence.

[0048] Attribute 605 contains information representing the attributes of the deformation. The possible values ​​of attribute 605 vary depending on the type 603. For example, if type 603 is "crack," attribute 605 will include the confidence level of the deformation detection result and the width of the crack. If type 603 is "spray," attribute 605 will include the confidence level of the deformation detection result and whether or not the rebar is exposed.

[0049] Figure 7 illustrates the graphical user interface (GUI) of the deformation detection results provided by the application of this embodiment.

[0050] The thumbnail display area 701 is an area that simplifies and displays deformation information in an easy-to-understand manner. The thumbnail display area 701 displays a list of thumbnail images 702, which will be described later, that can be selected.

[0051] The thumbnail image 702 is a simplified image created by reducing the resolution and compression of the detection image to reduce its file size and display it in a reduced size, allowing for a quick overview of the detection image and the deformation information detected from it. When the user operates the input device 105 and selects a thumbnail image 702 in the GUI, the selected thumbnail image 702 is highlighted. In Figure 7, the selected thumbnail image 702 is highlighted with a thick border.

[0052] The detailed display area 703 is an area for displaying detailed information about the deformation. The detailed display area 703 displays the detailed image 704, which will be described later. The user can zoom in and scroll the detailed image 704 using the GUI by operating the input device 105.

[0053] Detailed image 704 is an image that allows for detailed observation of the detection image and the deformation information detected from the detection image by displaying the detection image at its original file size.

[0054] The date and time selection box 705 is an input element for specifying the date and time to narrow down the thumbnail images 702 displayed in the thumbnail display area 701. The user can specify a specific date and time or a range of dates and times via the GUI by operating the input device 105.

[0055] The folder selection box 706 is an input element for specifying a folder path to narrow down the thumbnail images 702 displayed in the thumbnail display area 701. The user can input this information via the GUI by operating the input device 105. When the user selects a thumbnail image via the GUI by operating the input device 105, the file path of the original detection image corresponding to the selected thumbnail image is automatically entered.

[0056] The update button 707 is a button for inputting the timing to execute the process of displaying the thumbnail image 702 in the thumbnail display area 701, which will be described later in Figure 8(a). When the user operates the input device 105 to change the input information in the GUI, for example, in the date and time specification box 705 or the folder specification box 706, and operates the update button 707, the list display of thumbnail images 702 in the thumbnail display area 701 is updated.

[0057] Furthermore, it may be possible to change the ratio of the sizes of the thumbnail display area 701 and the detail display area 703 in the GUI. For example, by enlarging the thumbnail display area 701, it is possible to increase the number of thumbnail images 702 displayed in the thumbnail display area 701 and improve visibility.

[0058] Figure 8 is a flowchart showing the display control process for detection images and deformation information in this embodiment.

[0059] The process shown in Figure 8 is realized when the control unit 101 of the image processing device 100 shown in Figure 1 loads the program stored in the non-volatile memory 102 into the work memory 103, executes it to control each component, and then performs the function shown in Figure 2.

[0060] In the following, it is assumed that, before starting the process shown in Figure 8, a detection image is input via the input unit 201, and the image information corresponding to the detection image is stored in the image information table 500.

[0061] Figure 8(a) is a flowchart showing the process of displaying a thumbnail image 702 in the thumbnail display area 701 of the GUI shown in Figure 7. The process in Figure 8(a) is started when the image processing device 100 is started or when the user operates the update button 707 on the GUI by operating the input device 105.

[0062] In S801, the management unit 202 retrieves image information for all detection images from the image information table 500 shown in Figure 5. If a date and time or a date and time range is specified in the date and time specification box 705 of the GUI shown in Figure 7, the image information to be retrieved may be limited to the specified date and time or date and time range. Also, if a folder path is specified in the folder specification box 706 of the GUI shown in Figure 7, the image information to be retrieved may be limited based on the specified folder path. For example, the specified folder path and the file path of the image information are compared using prefix matching, and only the matching image information is retrieved.

[0063] In S802, the display image generation unit 204 obtains the display size of the thumbnail image 702. The display size may be a predetermined size, or it may be determined based on the size of the thumbnail display area 701.

[0064] In S803, the display image generation unit 204 repeatedly performs the display image generation process in S804, which will be described later, for each image information acquired in S801.

[0065] In S804, the display image generation unit 204 generates a thumbnail image 702. Details of this process will be described later in Figure 9.

[0066] In S805, the image display unit 205 displays the thumbnail images 702 generated in S804 in a list in the thumbnail display area 701 of the GUI shown in Figure 7. The thumbnail images 702 are displayed in the thumbnail display area 701 in a list based on the size of the thumbnail display area 701 and the display size of the thumbnail images 702.

[0067] Figure 8(b) is a flowchart showing the process of displaying a detailed image 704 in the detailed display area 703 of the GUI shown in Figure 7. The process in Figure 8(b) is triggered when the user operates the input device 105 and selects one of the thumbnail images 702 displayed in the thumbnail display area 701 on the GUI. In this case, the management unit 202 obtains the image information corresponding to the selected thumbnail image 702 from the image information table 500 shown in Figure 5 and assigns it to the display image generation unit 204.

[0068] In S811, the display image generation unit 204 obtains the display size of the detail image 704. The display size may be a predetermined size, or it may be determined based on the size of the detail display area 703.

[0069] In step S812, the display image generation unit 204 generates a detailed image 704. Details will be described later in Figure 9.

[0070] In S813, the image display unit 205 displays the detailed image 704 generated in S812 in the detailed display area 703 of the GUI shown in Figure 7.

[0071] Figure 9 is a flowchart showing the display image generation process in S804 of Figure 8(a) and S812 of Figure 8(b).

[0072] In the process shown in Figure 9, the display image generation unit 204 is pre-assigned image information and display size to the detection image.

[0073] Furthermore, the process shown in Figure 9 illustrates simplified display of deformation information, including deformity reduction, simplification of deformation shape, limitation of display modes such as deformation drawing color, and display of deformation type icons and text. Additionally, a detailed display of deformation information is described, illustrating how deformation information can be visualized according to its attributes.

[0074] In S901, the deformation detection unit 203 refers to the file path of the image information given at the start of processing and acquires the detection image from the storage device 104. The deformation detection unit 203 also performs deformation detection processing on the detection image acquired from the storage device 104. The deformation detection unit 203 then transmits the deformation detection result for each detection image to the management unit 202. The management unit 202 stores the deformation information as the deformation detection result for each detection image in the deformation information table 600 shown in Figure 6. The management unit 202 also stores the image information for each detection image in the image information table 500 shown in Figure 5. Note that the deformation detection processing may be performed all at once when the image is input.

[0075] In S902, the display image generation unit 204 determines the distribution region of the deformation. More specifically, it finds a convex hull that encompasses all the deformation information. Alternatively, the deformation information may be clustered, and a convex hull may be found for each cluster. Known methods such as the k-means method can be used for clustering.

[0076] In S903, the display image generation unit 204 determines whether the display size of the detection image is below a threshold. If the display size of the detection image is below the threshold, the process proceeds to S904; otherwise, it proceeds to S906.

[0077] In S904, the display image generation unit 204 determines whether the detection result 504 of the image information table 500 shown in Figure 5 includes the "type of deformation distribution pattern". If the detection result 504 of the image information shown in Figure 5 includes the "type of deformation distribution pattern", the process proceeds to S905; otherwise, it proceeds to S906.

[0078] In S905, the display image generation unit 204 acquires an icon or string representing the distribution pattern of the deformation. For example, if the detection result of the image information table 500 shown in Figure 5 includes the deformation distribution pattern type "tortoise shell crack," an icon or string representing tortoise shell crack is acquired. Note that both an icon and a string may be acquired. Alternatively, either an icon or a string may be selected and acquired depending on the display size of the detection image.

[0079] In S906, the display image generation unit 204 thins out deformation information based on the display size of the detection image. The number or percentage of information to be thinned increases as the display size of the detection image decreases. In this case, the number or percentage of information to be thinned out may be adjusted for each type of deformation information. Also, if clustering of deformation information is performed in S902, deformation information may be thinned out for each cluster. To determine which deformation information to thin out, for example, the circumscribing rectangle is obtained from the shape of each deformation information and selected in order of increasing size. Alternatively, the deformation information to be thinned out may be selected based on the confidence level of the deformation detection result, which is one of the attributes of the deformation information.

[0080] In S907, the display image generation unit 204 simplifies the shape of each deformation information based on the display size of the detection image. The degree of simplification increases as the display size of the detection image decreases. Known methods such as the Douglas-Peucker method can be used for simplifying the shape.

[0081] In S908, the display image generation unit 204 determines the drawing color for each deformation information based on the display size of the detection image. Here, the method for determining the drawing color will be explained using deformation information representing cracks (deformation information of type "crack") as an example. First, a lookup table is prepared in which the range of possible values ​​for "crack width" is divided into multiple sections, and a different drawing color is assigned to each section. In this case, the number of sections is determined so that the smaller the display size of the detection image, the fewer sections there are. Next, the lookup table is referenced using "crack width" to obtain the drawing color. In this way, if the display size of the detection image is large, different drawing colors can be obtained according to the crack width, and if the display size of the detection image is small, the same drawing color can be obtained regardless of the crack width. Note that as the number of sections in the lookup table decreases, the drawing color assigned to each section may be selected from colors with high contrast relative to the representative color of the image (such as the average value of all pixels). In addition, the line width, line style, etc., of each deformation information may be determined based on the display size of the detection image.

[0082] In S909, the display image generation unit 204 generates a composite image by superimposing the deformation information and the deformation distribution area obtained in S902 onto the detection image. In this case, each piece of deformation information is drawn with the drawing color determined in S908. The deformation distribution area is composited only when the area ratio of the deformation distribution area to the entire detection image is below a predetermined threshold. This prevents the deformation distribution area from being drawn across the entire detection image if the deformation is distributed across the entire detection image.

[0083] In S910, the display image generation unit 204 resizes the composite image generated in S909 according to the display size of the detection image. Furthermore, it composites the icon or text acquired in S905 onto the resized composite image. The icon or text may be composited to be drawn at a predetermined position on the detection image (for example, the lower right), or it may be composited to be drawn near the deformation information.

[0084] Furthermore, the displayed image may be highlighted based on the degree of simplification of the distortion information in the displayed image. A greater degree of simplification means a greater discrepancy in appearance compared to when the image is not simplified, and this serves as a guideline for deciding whether to check the details, especially when the displayed image is listed as a thumbnail image 702. Indicators that represent the degree of simplification include the display size of the detection image. Examples of highlighting include enlargement, borders, and animations such as blinking.

[0085] In the process shown in Figure 9, deformation information is combined with the detection image in S909, and the combined image is resized in S910. However, the order of processing can be reversed. The display image is obtained by resizing the detection image, deformation information, and the deformation distribution area obtained in S902, and then combining them.

[0086] Furthermore, in the process shown in Figure 9, an icon or string representing the distribution pattern of the deformation information was composited onto the detection image. However, if the image information detection result includes "degree of damage," an icon or string representing the degree of damage may be obtained based on the degree of damage and composited onto the detection image.

[0087] Furthermore, in the process shown in Figure 9, the method of displaying deformation information was controlled based on the display size of the detection image. However, the method of displaying deformation information may also be controlled based on the display size of the deformation information itself. For example, if the display size of the deformation information is small, it becomes difficult for the user to check the deformation information, so the deformation information is simplified and displayed in an easy-to-understand manner. On the other hand, if the display size of the deformation information is large, the deformation information is displayed in detail so that the user can check the deformation information in detail (such as attributes). When controlling the method of displaying deformation information based on the display size of the deformation information, the display size of the deformation information should be determined after S902 in Figure 9, and in S903 and S906 to S908, the display size of the deformation information should be referred to instead of the display size of the detection image. The display size of the deformation information is determined as follows.

[0088] First, the display image generation unit 204 determines the resizing ratio when the detection image is resized to match the display size of the detection image. Next, it resizes the distribution area of ​​the deformation information using the resizing ratio. Then, it finds the bounding rectangle from the resized distribution area of ​​the deformation information and obtains its size. If the deformation information is clustered and the distribution area of ​​the deformation information is determined for each cluster, in order to determine the display size of the deformation information, the distribution area of ​​the deformation information is resized for each cluster, the size of the bounding rectangle is determined, and representative values ​​(such as the mean) are obtained.

[0089] Furthermore, in the process shown in Figure 9, the method of displaying deformation information was controlled based on the display size of the detection image. However, the method of displaying deformation information may also be controlled based on the resolution of the detection image. In this case, the higher the resolution of the detection image, the more detailed the deformation information will be displayed, and the lower the resolution of the detection image, the simpler the deformation information will be displayed.

[0090] As described above, according to this embodiment, a display image is generated from the detection image and the display size of the detection image. In this case, the method of displaying the deformation information superimposed on the detection image is controlled based on the display size of the detection image. The larger the display size of the detection image, the more detailed the deformation information is displayed, and the smaller the display size of the detection image, the simpler the deformation information is displayed. As a result, the deformation information is displayed with appropriate coarseness according to the display size of the detection image, so that the user can easily grasp the content of the detection image and the deformation information detected from the detection image.

[0091] In this embodiment, the display image generation process shown in Figure 9 was executed in the process of displaying the thumbnail image 702 in Figure 8(a) and the process of displaying the detailed image 704 in Figure 8(b). However, the display images may be generated at different timings. For example, the display images may be generated when the input unit 201 inputs the detection image prior to the process in Figure 8(a). In the processes of Figure 8(a) and Figure 8(b), the pre-generated display images are acquired and displayed. This method is effective when the display sizes of the thumbnail image 702 and the detailed image 704 are predetermined. Alternatively, the display images may be regenerated when the zoom magnification of the detailed image 704 displayed in the detailed display area 703 is changed.

[0092] In this embodiment, an example was described in which the deformation detection process and the display image generation process are performed on the same device, but these processes may be performed on separate devices. By doing so, it becomes possible to use a device with low computing performance as a device for displaying the display image.

[0093] In this embodiment, a thumbnail image 702 was generated by superimposing simplified deformation information onto the detection image and then reducing its size. However, the thumbnail image 702 may also be generated by reducing only the detection image without superimposing the deformation information. In this case, the pixels after reduction are determined by known methods such as nearest neighbor interpolation. In addition, each pixel may be weighted by an evaluation value that each pixel possesses. The evaluation value of a pixel can be, for example, the probability that a deformation exists in that pixel (the probability that a deformation exists in a pixel is determined in advance during the deformation detection process). Furthermore, the thumbnail image 702 described in this embodiment and the thumbnail image 702 obtained by reducing only the detection image may be configured to be switchable.

[0094] In this embodiment, the thumbnail images 702 are displayed in a list in the thumbnail display area 701 of the GUI shown in Figure 7. However, it may be possible to sort or filter based on the image information corresponding to the thumbnail images 702 or the deformation information contained in the thumbnail images 702. For example, thumbnail images 702 with the same value can be sorted together based on the type of deformation information or the detection result of the image information (e.g., the type of deformation distribution pattern). Alternatively, the image information can be sorted based on the detection result of an ordinal scale or higher (e.g., degree of damage).

[0095] [Other embodiments] The present invention can also be realized by supplying a program that implements one or more functions of each embodiment to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. Furthermore, the present invention can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0096] The disclosures herein include the following image processing apparatus, control methods, and programs. [Configuration 1] The system includes a display control means that displays abnormality information detected from an image of the object being inspected, in association with the image, on a display means. The image processing apparatus is characterized in that the display control means controls the method of displaying the deformation information based on the display size of the image or the deformation information. [Configuration 2] A detection means for detecting deformation information from the aforementioned image, The display means and, The image processing apparatus according to configuration 1, characterized in that the display control means displays the position of the deformation information superimposed on the image in accordance with the image. [Configuration 3] The display control means is The larger the display size of the image or the deformation information, the more detailed the deformation information will be displayed. The image processing apparatus according to configuration 1 or 2, characterized in that the smaller the display size of the image or the deformation information, the simpler the deformation information is displayed. [Structure 4] The image processing apparatus according to configuration 3, characterized in that the display control means limits the display mode of the deformation information as the display size of the image or deformation information decreases. [Composition 5] The image processing apparatus according to configuration 4, characterized in that the display control means limits the display mode by decimating the number of deformation information items as the display size of the image or deformation information decreases. [Composition 6] The image processing apparatus according to configuration 4, characterized in that the display control means limits the display mode by simplifying the shape of the deformation information as the display size of the image or deformation information decreases. [Composition 7] The image processing apparatus according to configuration 4, characterized in that the display control means limits the display mode by limiting at least one of the color, line width, and line type used to draw the deformation information as the display size of the image or deformation information decreases. [Structure 8] The image processing apparatus according to configuration 3, characterized in that the display control means displays the type of deformation information as an icon and / or a string of characters when the display size of the image or deformation information is less than or equal to a threshold. [Composition 9] The image processing apparatus according to configuration 8, characterized in that the type of deformation information includes the distribution form of the deformation information. [Configuration 10] The image processing apparatus according to configuration 3, characterized in that the display control means displays the deformation information as a heat map indicating the density of the deformation information, the smaller the display size of the image or the deformation information. [Composition 11] The image processing apparatus according to configuration 3, characterized in that the display control means visualizes the deformation information when displaying the deformation information in detail. [Composition 12] The image processing apparatus according to configuration 11, characterized in that the visualization of the deformation information includes at least one of the color, line width, and line type used to draw the deformation information. [Composition 13] The display control means generates a display image for displaying the image and the deformation information on the display means based on the display size of the image or the deformation information. The image processing apparatus according to any one of configurations 3 to 12, characterized in that the displayed image is an image obtained by resizing the image and the deformation information to match the size of the display area of ​​the display means. [Composition 14] The display means displays a list of first display images that simplify the deformation information in a selectable manner in the first display area. The image processing apparatus according to configuration 3, characterized in that when the first display image is selected, a second display image, which displays the deformation information in detail relative to the first display image, is displayed in the second display area. [Composition 15] The image processing apparatus according to any one of configurations 1 to 14, characterized in that the deformation information includes at least one of cracks, peeling, efflorescence, cold joints, and honeycombing. [Composition 16] A method for controlling an image processing device, The system includes a display control step that displays abnormality information detected from an image of the object being inspected, in association with the image, on a display means. The control method is characterized in that the display control step controls the method of displaying the deformation information based on the display size of the image or the deformation information. [Composition 17] A program for causing a computer to function as a display control means for an image processing apparatus described in any one of configurations 1 to 15.

[0097] The invention is not limited to the embodiments described above, and various modifications and variations are possible without departing from the spirit and scope of the invention. Accordingly, claims are attached to disclose the scope of the invention. [Explanation of Symbols]

[0098] 100...Image processing device, 101...Control unit, 102...Non-volatile memory, 104...Storage device, 201...Input unit, 202...Management unit, 203...Anomaly detection unit, 204...Display image generation unit, 205...Image display unit

Claims

1. The system includes a display control means that displays abnormality information detected from an image of the object being inspected, in association with the image, on a display means. The display control means displays the deformation information in more detail the larger the display size of the image or deformation information. An image processing apparatus characterized in that the smaller the display size of the image or the deformation information, the more the display of the deformation information is limited by thinning out multiple pieces of deformation information.

2. A display control means that displays deformation information detected from an image of an object to be inspected in association with the image on a display means, The display control means displays the deformation information in more detail the larger the display size of the image or deformation information. An image processing apparatus characterized in that the smaller the display size of the image or the deformation information, the simpler the shape of the deformation information is, thereby limiting the display mode of the deformation information.

3. A display control means that displays deformation information detected from an image of an object to be inspected in association with the image on a display means, The display control means displays the deformation information in more detail the larger the display size of the image or deformation information. An image processing apparatus characterized in that, as the display size of the image or the deformation information decreases, the display mode of the deformation information is limited by limiting at least one of the color, line width, or line type used to draw the deformation information.

4. A detection means for detecting deformation information from the aforementioned image, The display means and, The image processing apparatus according to any one of claims 1 to 3, characterized in that the display control means displays the position of the deformation information superimposed on the image in accordance with the image.

5. The image processing apparatus according to any one of claims 1 to 3, characterized in that the display control means displays the type of deformation information as an icon and / or a string of characters when the display size of the image or deformation information is less than or equal to a threshold.

6. The image processing apparatus according to claim 5, characterized in that the type of deformation information includes the distribution form of the deformation information.

7. The image processing apparatus according to any one of claims 1 to 3, characterized in that the display control means displays the deformation information as a heat map indicating the density of the deformation information, the smaller the display size of the image or the deformation information.

8. The image processing apparatus according to any one of claims 1 to 3, characterized in that the display control means visualizes the deformation information when displaying the deformation information in detail.

9. The image processing apparatus according to claim 8, characterized in that the visualization of the deformation information includes at least one of the color, line width, and line type used to draw the deformation information.

10. The display control means generates a display image for displaying the image and the deformation information on the display means based on the display size of the image or the deformation information. The image processing apparatus according to any one of claims 1 to 3, characterized in that the displayed image is an image obtained by resizing the image and the deformation information to match the size of the display area of ​​the display means.

11. The display means displays a list of first display images that simplify the deformation information in a selectable manner in the first display area. The image processing apparatus according to any one of claims 1 to 3, characterized in that when the first display image is selected, a second display image, which displays the deformation information in detail relative to the first display image, is displayed in the second display area.

12. The image processing apparatus according to any one of claims 1 to 3, characterized in that the deformation information includes at least one of cracks, peeling, efflorescence, cold joints, and honeycombing.

13. A control method performed by an image processing device, The system includes a display control step that displays abnormality information detected from an image of the object being inspected, in association with the image, on a display means. In the display control step, the larger the display size of the image or the deformation information, the more detailed the deformation information is displayed. A control method characterized in that the smaller the display size of the image or the deformation information, the more the display of the deformation information is limited by thinning out multiple pieces of deformation information.

14. A control method performed by an image processing device, The system includes a display control step that displays abnormality information detected from an image of the object being inspected, in association with the image, on a display means. In the display control step, the larger the display size of the image or the deformation information, the more detailed the deformation information is displayed. A control method characterized in that the smaller the display size of the image or the deformation information, the more the shape of the deformation information is simplified, thereby limiting the display mode of the deformation information.

15. A control method performed by an image processing device, The system includes a display control step that displays abnormality information detected from an image of the object being inspected, in association with the image, on a display means. In the display control step, the larger the display size of the image or the deformation information, the more detailed the deformation information is displayed. A control method characterized in that the smaller the display size of the image or the deformation information, the more the display of the deformation information is limited by limiting at least one of the color, line width, or line type used to draw the deformation information.

16. A program for causing a computer to function as a display control means for an image processing apparatus described in any one of claims 1 to 3.

Images