Image processing system and control method thereof

Abstract

To determine a parameter that enables high-precision image processing with lower processing load.SOLUTION: An image processing system that performs analysis processing on an image using an image analysis unit comprises determination means for determining whether an image is of a predetermined size or larger, trial means for performing multiple times of analysis processing on the image by changing a parameter used by the image analysis unit, and display control means for displaying multiple analysis results corresponding to the multiple times of analysis processing on a display. The trial means controls the execution of multiple times of analysis processing on a partial region image within the image when the determination means determines that the image is of a specified size or larger.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to image analysis technology.

Background Art

[0002] There is a known technique for detecting deformation of a structure by performing image analysis on a photographed image of the structure. In Patent Document 1, a method is disclosed in which a pre-trained artificial intelligence (AI) model is prepared and image analysis is performed using the AI model. Further, in image analysis using an AI model, in order to obtain a suitable output result for each image according to various input images, the parameters used in the AI model are made variable, and image analysis may be tried for each of a plurality of parameters.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In images used for inspecting social infrastructure structures such as roads and bridges, the types of target structures and the conditions (such as sunlight conditions) when photographing the structures are likely to vary for each image. Therefore, in order to improve the detection accuracy of deformation (reduce false detection and non-detection), it is necessary to appropriately adjust the parameters of the AI model for each image.

[0005] However, when performing image analysis on a large number of parameters (and combinations of parameters), the number of trials increases and the processing load increases. In addition, the images used for inspecting structures may be generated by synthesizing a plurality of images taken with a digital camera, and may be images of extremely large sizes (for example, tens of thousands × tens of thousands of pixels). Therefore, when performing image analysis on a large number of parameters to determine appropriate parameters, there is a problem that the processing load increases significantly.

[0006] The present invention has been made in view of such problems, and an object thereof is to provide a technique for determining image processing parameters that enable highly accurate deformation detection with a lower processing load.

Means for Solving the Problems

[0007] In order to solve the above problems, an image processing apparatus according to the present invention has the following configuration. That is, an image processing apparatus that executes analysis processing on an image using an image analysis unit Determination means for determining whether the image is of a predetermined size or more, Trial means for performing a plurality of analysis processes with the parameters used by the image analysis unit changed on the image, Display control means for displaying a plurality of analysis results corresponding to the plurality of analysis processes on a display unit, and When the determination means determines that the image is of the predetermined size or more, the trial means controls to perform the plurality of analysis processes on a partial region image within the image.

Effects of the Invention

[0008] According to the present invention, it is possible to provide a technique for determining image processing parameters that enable highly accurate deformation detection with a lower processing load.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Embodiments for Carrying Out the Invention

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the invention according to the claims. Although a plurality of features are described in the embodiments, not all of these plurality of features are essential to the invention, and the plurality of features may be arbitrarily combined. Further, in the accompanying drawings, the same or similar configurations are denoted by the same reference numerals, and redundant descriptions are omitted.

[0011] (First Embodiment) As a first embodiment of the image processing apparatus according to the present invention, an image processing apparatus that detects deformation in a structure by executing image analysis processing using an AI model on an image obtained by photographing the structure will be described below as an example.

[0012] <Summary> In general, a large amount of computer resources are required to execute analysis processing using an AI model. In addition, using hardware such as a high-performance graphics processing unit (GPU) directly leads to an increase in the running cost of image analysis processing. On the other hand, in the use case of detecting deformation of infrastructure, it is necessary to appropriately adjust parameters for each image. In particular, since the images of the infrastructure to be processed may be extremely large in size (for example, 1 billion pixels), the processing load will increase significantly.

[0013] Therefore, in the first embodiment, in the parameter trial process, when the size of the image to be processed is larger than a predetermined size, control is performed to process a partial region of the image. As a result, parameters enabling highly accurate deformation detection can be determined with a lower processing load.

[0014] <Device Configuration> FIG. 1 is a diagram showing the hardware configuration of the image processing apparatus 100. The image processing apparatus 100 includes a network interface (I / F) 101, a read only memory (ROM) 102, a random access memory (RAM) 103, and a secondary storage device 104. In addition, it includes a central processing unit (CPU) 105, a GPU 106, a user interface (UI) 107, and a bus 108.

[0015] The network I / F 101 connects to a network 150 such as a local area network (LAN) and communicates with other computers and network devices. The communication method can be either wired or wireless. The ROM 102 records embedded programs and data. The RAM 103 is a temporary memory area. The secondary storage device 104 is a large-capacity storage device typified by a hard disk drive (HDD) or a flash memory. The CPU 105 executes programs read from the ROM 102, the RAM 103, the secondary storage device 104, etc., and performs various processes. The GPU 106 performs graphics processing and processing (learning / inference) using an AI model. The UI 107 is a general term for an output device, a display (display unit), and input devices such as a keyboard, a mouse, buttons, and a touch panel. Note that it may be configured to be connected and operated from another computer via a remote desktop or a remote shell. The above-described respective units are communicably connected to each other via a bus 108.

[0016] Figure 2 is a diagram showing the functional configuration of the image processing apparatus 100. The image processing apparatus 100 generally includes an application program section (application 200), a storage program section (storage 210), and an image analysis program section (image analysis section 220).

[0017] Application 200 is an application that executes image analysis processing. It may be configured as a native application running on an operating system (OS), or as a web application running on a server. As a function of Application 200, there is UI201. When Application 200 is configured as a native application, a graphical user interface (GUI) screen including operation controls and the like is displayed as UI201, accepting input from the user and outputting results to the user. When Application 200 is configured as a web application, a GUI screen including operation controls and the like is displayed on a browser (not shown), accepting input from the user and outputting results to the user. In this case, the browser renders GUI screen data described by HTML, JavaScript, etc. to display UI201. In any configuration case, UI201 operates as the output part (such as a screen or data) and the input part (accepting operations from the user) of Application 200.

[0018] Storage 210 is responsible for data storage, search, reading, etc. As Storage 210, there are mainly a database for storing data schemas, values, and files, a file storage for storing image files, and the like. Image storage unit 211 stores, for example, the image file to be processed that has been received as input via Application 200. Analysis result storage unit 212 stores the image analysis results described later. Model management unit 213 stores and manages pre-trained AI models. It is assumed that Model management unit 213 can manage one or more AI models. Parameter management unit 214 stores and manages, for each AI model, the types of parameters, the range and options of available parameter values, and the like.

[0019] The image analysis unit 220 reads images from the image storage unit 211 using an AI model and executes image analysis processing using the AI model. The analysis results of the images are stored in the analysis result storage unit 212. The above-described units are connected to each other via a communication path 250 so as to be able to communicate with each other. These software programs may be deployed on a single computer 100 or may be distributed and deployed on a plurality of computers 100.

[0020] In this embodiment, as the image analysis processing, a process of detecting a deformation of a structure from a captured image of the structure (or a composite image obtained by combining a plurality of captured images) will be described as an example. For example, an exterior photo image of a concrete structure is input, image analysis processing using an AI model is executed, and the detection result of cracks is stored. The user can display and confirm the deformation detection result via the UI 201 and use it for inspecting the structure. The AI model prepares a pre-trained model for each type of deformation to be detected so that it can detect a plurality of types of deformations from the image. By pre-learning about other deformations such as efflorescence in addition to cracks as the deformation types, it is assumed that the AI model can detect a plurality of types of deformations from a single image.

[0021] <Graphical User Interface (GUI) Screen> FIG. 3 is a diagram showing a GUI screen 300 of an image list. The GUI screen 300 of the image list is a form of the UI 201, and various GUI components are arranged.

[0022] The component 301 is a tree view control that enables display and selection of a folder list in the image storage unit 211. The selected folder is highlighted, and the image files in that folder are listed and displayed on the right side.

[0023] The folder name 302 indicates the selected folder name. The resolution 303 is the set value of the actual size per pixel of the image. For example, when 0.5 mm / pixel is set, 1 pixel in the image corresponds to a 0.5 mm square area on the surface of the structure. By using this set value, the actual size values in the vertical and horizontal directions of the image can be calculated from the number of pixels in the image, and the actual size values of the length and width of the deformation in the deformation detection result can be calculated.

[0024] The component 304 is a checkbox for selecting an image to be processed. For each image, a thumbnail 305, a file name 306, a vertical and horizontal size 307, and a registration date and time 308 are displayed. The component 309 displays information on the number of detection executions for the image file and functions as a link button to the GUI screen 500 (Figure 5 described later) for the detection result. The component 310 displays information on the number of detection attempts for the image file and functions as a link button to the GUI screen 600 (Figure 6 described later) for the detection attempt result.

[0025] The component 311 is an add button that selects an image file and uploads it to the image storage unit 211. The uploaded file is displayed as an image file under the folder on the image list screen 300. Note that the upload date and time will be displayed as the registration date and time 308. The component 312 is a delete button that deletes the file selected by the checkbox 304 from the image storage unit 211. The component 313 is a detection execution button that displays the GUI screen 400 (Figure 4 described later) for detection execution to execute image analysis processing on the image selected by the checkbox 304.

[0026] Figure 4 is a diagram showing the GUI screen 400 for detection execution. Here, "crack" and "efflorescence" are assumed as the deformations to be detected, but it may be configured to detect other deformations. That is, it means that multiple deformation types can be detected from a single image.

[0027] Label 401 is a label that displays the name of the change in the object to be detected, and here it is displayed as "crack". Check box 402 is a check box for specifying whether or not to make the change displayed on label 401 the object to be detected. When check box 402 is checked, the change displayed on label 401 is detected. Drop-down 403 is a drop-down for selecting an AI model. For example, when performing the change detection of a structure using an AI model, according to the type of the structure to be inspected (such as the concrete surface of a bridge pier of a bridge, the asphalt surface of a road, the wall surface of a tunnel, etc.), the corresponding pre-trained AI model can be selected. Note that it is also possible to select the "standard model", which is a pre-trained AI model that is not specialized for a specific structure.

[0028] In addition, for the selected model, it is possible to specify the crack detection sensitivity 404, short crack removal 405, and crack width correction 406. These are examples of parameters that can be specified when using an AI model. These parameters are managed by the AI model parameter management unit 214 for the values that can be used for each AI model, and the available parameters are presented on the GUI screen 400.

[0029] The crack detection sensitivity 404 is a threshold parameter for estimating a pixel as a crack location regarding the score (likelihood of a crack) estimated by the AI model for each pixel. Specifically, when the score is equal to or higher than the threshold, it is determined as a crack location. "Medium" is the default setting (standard). In "Weak", a relatively high threshold is set. Since pixels equal to or higher than the threshold are determined as crack locations, in "Weak", the number of crack locations is relatively small. On the other hand, in "Strong", a relatively low threshold is set. Since pixels equal to or higher than the threshold are determined as crack locations, in "Strong", the number of crack locations is relatively large.

[0030] The short crack removal 405 is a threshold parameter for estimating a cracked location in terms of length. Specifically, when the length of a crack is below the threshold, it is determined that it is not a cracked location. "Medium" is the default setting (standard). In the case of "weak", a relatively short threshold is set. Since only short cracks below the threshold are excluded from the cracked locations, the number of cracked locations becomes relatively large in the case of "weak". On the other hand, in the case of "strong", a relatively long threshold is set. Since only short cracks below the threshold are excluded from the cracked locations, the number of cracked locations becomes relatively small in the case of "strong".

[0031] The crack width correction 406 is a parameter used for correcting the width of a line segment when calculating individual crack line segments from the set of cracked locations and detected pixels. "Medium" is the default setting (standard). In the case of "weak", the correction is relatively thin. In the case of "strong", the correction is relatively thick.

[0032] When inspecting a structure by AI image analysis, it cannot be expected that 100% correct results can be obtained in the inference of the AI. Therefore, ultimately, it is necessary for a human to confirm and correct the AI image analysis results. However, by adjusting and setting the above parameters to more appropriate values, it is possible to output detection results with higher accuracy. As a result, it becomes possible to minimize the confirmation and correction work by humans in the subsequent process.

[0033] The component 407 is a checkbox for setting on / off of the parameter trial mode. When the parameter trial mode is off, the deformation detection is executed once with the parameters set in the settings 404 to 406. When the parameter trial mode is on, the radio buttons in the settings 404 to 406 are disabled and unselectable, and an exhaustive trial is performed with the combinations of parameters that can be set in each of the settings 404 to 406.

[0034] Label 411 is a label that displays other names of abnormalities to be detected, and here it is displayed as "efflorescence". Check box 412 is a check box for specifying whether or not to detect the abnormality displayed on label 411. When check box 412 is checked, the abnormality displayed on label 411 is detected. Drop-down 413 is a drop-down for selecting an AI model. For example, when performing abnormality detection of a structure using an AI model, a corresponding pre-trained AI model can be selected according to the type of the structure to be inspected (such as the concrete surface of a bridge pier of a bridge, the asphalt surface of a road, the wall surface of a tunnel, etc.). In addition, it is also possible to select a "standard model", which is a pre-trained AI model that is not specialized for a specific structure.

[0035] In display areas 421 and 422, the estimated execution cost and the estimated time when performing the detection process using the selected AI models 403 and 413 are displayed. These estimated values enable the cost and time of AI detection execution to be grasped in advance. Component 430 is an execution button for image analysis. When pressed (such as mouse click) by the user, application 200 instructs image analysis unit 220 to execute image analysis processing.

[0036] Figure 5 is a diagram showing the GUI screen 500 of the detection result. After the execution of the detection process is completed, the GUI screen 500 can be displayed from link 309.

[0037] Folder name 501 is the name of the folder where the image file displaying the result is stored. File name 502 is the name of the image file displaying the result. Execution count 503 indicates which detection execution result is being displayed when there are multiple detections. In addition, the detection results can be switched and displayed by the switch button arranged on the right side.

[0038] In the display areas 504 and 505, the set values of the parameters used at the time of each detection execution are displayed. The detection result image 510 is an image obtained by overlay-drawing the detected deformation on the original image and displaying it as a superimposed image. For example, in order for the user to easily grasp each type of deformation and the degree of deformation, it is displayed discriminately according to each type of deformation and the degree of deformation (for example, different pattern images, different colors). In the display area 520, a legend of the display corresponding to each type of deformation and the degree of deformation is displayed. Also, check boxes may be provided for each type of deformation or crack width so that only the display corresponding to the specified type of deformation and degree of deformation can be overlaid and displayed on the detection result image 510. The component 521 is a check box for the background image and can be used to specify whether to display the original image as the background image. The component 530 is a download button and downloads the file of the detection result image 510. Also, as another option, it may be possible to download the CAD data file together with the original image file. This makes it possible to draw a desired drawing by combining the original image and the CAD data with separate CAD software.

[0039] Figure 6 is a diagram showing a GUI screen 600 of the parameter comparison result. After the execution of the parameter trial detection is completed, the GUI screen 600 can be opened from the link 310.

[0040] The folder name 601 is the folder name of the storage destination of the image file displaying the result. The file name 602 is the file name of the image file displaying the result. The size 603 is the size (number of pixels in the vertical and horizontal directions) of the image displaying the result.

[0041] In the display areas 604 and 605, the set values of the parameters used at the time of each detection execution are displayed. The detection result image 610 is an image obtained by overlay-drawing the detected deformation on the original image and displaying it as a superimposed image.

[0042] On the GUI screen 600 for parameter comparison results, by changing each parameter displayed in the display areas 604 and 605 (such as switching between weak, medium, and strong using the up and down buttons), the differences in detection results can be compared. That is, each detection result is displayed in association with the parameters used. By observing the changes in the detection result image 610 displayed when the user changes the parameters, it becomes possible to easily determine appropriate parameters for anomaly detection. Component 620 is a parameter determination button that determines the parameter set adjusted and selected in the display areas 604 and 605.

[0043] Figure 7 is a diagram showing the GUI screen 700 for parameter trials. The GUI screen 700 when the image to be processed is a large-size image is shown. Here, an image with 1 billion ( = 1 gigapixel) or more pixels is called a large-size image. For example, an image of 50,000 × 50,000 pixels is 2.5 billion pixels and thus a large-size image. However, the threshold of 1 billion pixels is merely an example, and other threshold pixel counts may also be used.

[0044] The folder name 701 is the name of the folder where the image file displaying the results is stored. The file name 702 is the name of the image file displaying the results. The size 703 is the size (number of pixels in the vertical and horizontal directions) of the image displaying the results.

[0045] In the display area 704, a message indicating that a large-size image has been input is displayed. The trial target anomaly 705 is the name of the anomaly type specified as the parameter trial target on the GUI screen 400. Here, the GUI screen 700 when the check boxes of components 407 and 402 are checked on the GUI screen 400 is shown.

[0046] In the display area 710, the original image to be processed is displayed. Since the input image is a large-size image here, there is a concern that if parameter trials are performed, the cost and processing waiting time of the image analysis process will increase significantly. Therefore, here, among the original images displayed in the display area 710, for example, a partial area where cracks often appear can be selected (area specified) by the selection control 711. The display area 712 displays the area size (number of pixels in the vertical and horizontal directions) selected by the selection control 711. The component 720 is an OK button that determines the area selected by the selection control 711.

[0047] FIG. 8 is a diagram showing another GUI screen 800 for parameter trials. It is almost the same as the GUI screen 700, but the area selection method is different. Specifically, the area size indicated by the selection control 810 is fixed to a certain size. The user moves the selection control 810 up, down, left, or right to select the area to be processed. The display area 821 displays the area size selected by the selection control 810, and also displays the actual size using the value of the resolution setting 303.

[0048] In the detection of deformation of a structure, there is a problem that if the image size is too large, it will cost time, while if it is too small, the deformation cannot be detected sufficiently. For example, if the type of deformation to be detected is a crack and it is desired to output the detection result up to cracks of 0.1 m to 2 m or more. In that case, in order to be able to output the deformation as a trial result sufficiently, the actual size (for example, 2 m × 2 m) is defined in advance, and the size of the selection control 810 is set to a fixed size corresponding to the actual size. Move the selection control 810 to a suitable location where cracks appear to select and determine the parameter trial target area. The advantage of the partial area selection in FIG. 8 is that it becomes easy to select an area based on the actual size of the structure shown in the image using the resolution setting value. That is, since the size of the selection control 710 is determined in advance, the user only needs to move the selection control 710 (by pressing the cursor key, etc.), which reduces the effort.

[0049] FIG. 9 is a diagram showing a GUI screen 900 of parameter comparison results. Specifically, it shows a GUI screen 900 for confirming the results of detection trials on the region image selected on the GUI screen 800.

[0050] The folder name 901 is the name of the folder where the image file displaying the results is stored. The file name 902 is the name of the image file displaying the results. The size 903 is the size (number of vertical and horizontal pixels) of the image displaying the results.

[0051] In the display areas 904 and 905, the set values of the parameters used during each detection execution are displayed. The detection result image 910 is an image obtained by overlay-drawing the detected changes on the original image and displaying it as a superimposed image.

[0052] On the GUI screen 900 for parameter comparison results, by changing each parameter displayed in the display areas 904 and 905 (such as switching between weak, medium, and strong using the up and down buttons), the differences in detection results can be compared. By observing the changes in the detection result image 610 displayed when the user changes the parameters, it becomes possible to easily determine appropriate parameters for change detection. The component 920 is a parameter determination button, which determines the parameter set adjusted and selected in the display areas 904 and 905.

[0053] <Operation of the device> FIG. 10 is a flowchart for explaining the overall process of image analysis. That is, it shows the process when the CPU 105 executes the application 200.

[0054] In S1001, the CPU 105 receives the upload of an image via the GUI screen 300 of the image list and saves the image in the image storage unit 211.

[0055] In S1002, the CPU 105 receives the selection of the image to be processed in the change detection process via the GUI screen 300 of the image list. Also, the CPU 105 acquires the available AI model and parameters from the image analysis AI model management unit 213 and the AI model parameter management unit 214, and displays the GUI screen 400 for detection execution.

[0056] In S1003, the CPU 105 receives a parameter trial instruction. For example, this is realized when the parameter trial mode is specified as on in the component 407 of the GUI screen 400 for detection execution, and the component 430 is pressed while the change type to be tried is selected by the components 402 and 412.

[0057] In S1004, the CPU 105 acquires the image size of the image to be processed. Here, the total number of pixels of the image is used as the image size, but alternatively, the image analysis processing cost or processing time estimated and calculated from the total number of pixels and the number of trials may be used.

[0058] In S1005, the CPU 105 determines whether the image to be processed is a large-size image based on the value acquired or calculated in S1004. For example, if the image size is equal to or larger than a predetermined size (for example, the total number of pixels is 1 billion pixels or more), it is determined to be a large-size image. If it is determined not to be a large-size image, the process proceeds to S1006, and if it is determined to be a large-size image, the process proceeds to S1021.

[0059] In S1006, the CPU 105 determines to use the original image of the input image for parameter trials.

[0060] In S1021, the CPU 105 receives the selection of a partial region (partial region image) within the image via the region selection control 711 (or 810) on the GUI screen 700 (or 800) for parameter trials.

[0061] In S1022, the CPU 105 determines to use the region image selected in S1021 for parameter trials.

[0062] In S1007, the CPU 105 causes the image analysis unit 220 to perform parameter trials (multiple image analysis processes with changed parameters). The image analysis unit 220 stores the used parameter set and the analysis result in the analysis result storage unit 212.

[0063] In S1008, the CPU 105 controls the display of the GUI screen 600 for the parameter comparison result, and provides the user with the used parameters and the change detection result which is the analysis result.

[0064] In S1009, the CPU 105 accepts the selection of parameters via the GUI screen 600. That is, the user compares the differences in the detection results corresponding to each parameter via the GUI screen 600 displayed in S1008, and selects the parameter with a good detection result.

[0065] In S1010, the CPU 105 determines whether the image to be processed is a large-size image from the value obtained or calculated in S1004. S1010 is the same process as S1005. Note that the determination result in S1005 may be used as it is.

[0066] In S1023, the CPU 105 causes the image analysis unit 220 to perform change detection processing on the original input image (that is, the entire area of the image) using the parameters selected in S1009.

[0067] In S1011, the CPU 105 determines the result of the detection execution. If the determination in S1010 is No, since the detection result by the parameter set selected in S1009 has already been obtained in the parameter trial (S1007), that is determined as the result of the change detection processing. If the determination in S1010 is Yes, the detection result of S1023 is determined as the result of the change detection processing by the parameter selected in S1009. Then, the detection result GUI screen 500 is displayed via the link 309 of the image list GUI screen 300.

[0068] According to the first embodiment as described above, in the image analysis process (deformation detection process) using the AI model, when the size (total number of pixels, etc.) of the image to be processed exceeds the threshold, the image analysis process is executed for a partial region of the image. As a result, it becomes possible to determine the parameters of the AI model that enables high-precision deformation detection with a lower processing load.

[0069] (Modification example) As a modification example, a form in which parameter trials are performed for a plurality of deformation types will be described. Since the hardware configuration and functional configuration are the same as those in the first embodiment, the description thereof will be omitted.

[0070] <GUI screen> FIG. 11 is a diagram showing a GUI screen 1100 for parameter trials. Specifically, it shows the GUI screen 1100 in the case of a large-size image and when parameter trials are performed for a plurality of parameters. Different from the first embodiment (GUI screen 700), it is configured such that the region images used for parameter trials can be individually selected for each deformation type.

[0071] The deformation to be tested 1104 is the name of the deformation type specified as the parameter trial target on the GUI screen 400. Here, it shows the GUI screen 1100 when the check boxes of components 407, 402, and 412 are checked on the GUI screen 400.

[0072] The display area 1111 displays the original image to be processed. The region selection controls 1112 and 1113 are for selecting a partial region from within the image. Components 1121 and 1122 are selection start / retry buttons for each deformation type. The user presses component 1121 or 1122, selects a region with the region selection controls 1121 and / or 1122, and then presses component 1123.

[0073] The advantage of the GUI screen 1100 is that the area images used for parameter trials can be individually selected for each type of deformation. When specifying a partial area of an image, it may not be appropriate to specify the same partial area for different types of deformation. For example, as shown in the display area 1111, if the same partial area is specified when the locations of cracks and efflorescence are different, appropriate parameter trials cannot be performed, and as a result, it may not be possible to determine appropriate parameters. With the GUI screen 1100, by individually selecting the area images used for parameter trials for each type of deformation, it becomes possible to determine more appropriate parameters.

[0074] FIG. 13 is a diagram showing a GUI screen 1300 for detection execution. A detection execution instruction is received with the parameter set determined on the GUI screen 1100 for parameter trials.

[0075] The label 1301 is a label that displays the name of the deformation to be detected, and here it is displayed as "crack". The left check box is a check box for specifying whether or not to make the deformation displayed on the label 1301 a detection target. The label 1301 is the name of the AI model used for detecting the deformation displayed on the label 1301.

[0076] In the display area 1303, the parameter set determined on the GUI screen 1100 is displayed.

[0077] The folder name 1311 is the name of the folder where the image file is stored. The file name 1312 is the file name of the image file. The thumbnail 1313 is the thumbnail of the image file. The component 1314 is a check box for file selection.

[0078] When it is desired to execute detection processing on other image files with the type of deformation and the parameter set set at the upper part of the GUI screen 1300, the target image file is selected via the component 1314 or the all-selection check box.

[0079] Component 1320 is a detection execution button, which executes a detection process on the selected image file with the deformation type and parameter set configured at the upper part of the GUI screen 1300.

[0080] The advantage of the GUI screen 1300 is that after determining a suitable parameter set through parameter trials, the parameter set can be easily applied to other image files (images other than the image processed in FIG. 10). For example, when taking photos of multiple different locations of bridge piers under the same shooting conditions and wanting to execute deformation detection with the same parameter set and align the detection results under the same conditions, it is suitable. With the above configuration, first, parameter trials are performed on one image to determine a suitable parameter set, and the parameter set can be easily applied to the detection process for other images. This saves the user's effort and improves convenience.

[0081] <Operation of the device> FIG. 12 is a flowchart for explaining the trial process of multiple deformation types. That is, it shows the process when the CPU 105 executes the application 200. Note that FIG. 12 shows the process after being determined as Yes in S1005.

[0082] In S1201, the CPU 105 acquires the number N of deformation types to be tried selected on the GUI screen 400. Here, it is assumed that detection of cracks and efflorescence is to be performed, and N = 2.

[0083] In S1202, the CPU 105 determines whether the number N of deformation types is greater than 1. If it is greater than 1, it proceeds to S1203; if it is 1, it proceeds to S1021. The process when proceeding to S1021 is the same as that in the first embodiment, so the description is omitted.

[0084] In S1203, the CPU 105 executes, for each type of deformation to be detected, detection processing on the entire image using default parameters for each type of deformation to be detected in the image analysis unit 220. Then, the detection result for each type of deformation [i] is stored in the analysis result storage unit 212. Here, the type of deformation [1] is a crack, and the type of deformation [2] is efflorescence.

[0085] In S1204, the CPU 105 reads the detection result for each type of deformation [i] and detects a region in the image where the type of deformation [i] can be suitably detected. For example, regarding cracks, a region where a plurality of cracks are detected and gathered, such as the partial region 1112, is detected. Also, regarding efflorescence, a region where a larger area of efflorescence is detected, such as the partial region 1113, is detected.

[0086] In S1205, the CPU 105 sets the region detected in S1204 as a candidate for the selection region for parameter trials of the type of deformation [i]. That is, automatic setting is performed instead of manual designation by the user.

[0087] In S1206, the CPU 105 displays (partial regions 1112, 1113) the candidate regions automatically set in S1205 for each type of deformation on the GUI screen 1100 and accepts movement and adjustment of the user with respect to the candidate regions. Then, it proceeds to S1007. The processing when proceeding to S1007 is the same as that in the first embodiment, so the description is omitted.

[0088] As described above, according to the modification example, when performing detection processing for a plurality of types of deformations on a large-size image, a partial region of the image to be the target of the detection processing is specified for each type of deformation. Thereby, it becomes possible to determine the parameters of the AI model that enables highly accurate deformation detection with a lower processing load.

[0089] The disclosure of this specification includes the following image processing apparatus, control method, and program. (Item 1) An image processing apparatus that executes analysis processing on an image using an image analysis unit, Determination means for determining whether the image is of a predetermined size or more, Trial means for performing a plurality of analysis processes with parameters changed by the image analysis unit on the image, Display control means for displaying a plurality of analysis results corresponding to the plurality of analysis processes on a display unit, and comprising, when the trial means determines that the image is of the predetermined size or more by the determination means, controls to perform the plurality of analysis processes on a partial region image within the image An image processing apparatus characterized by this. (Item 2) Reception means for receiving selection of one analysis result included in the plurality of analysis results, Control means for controlling to execute an analysis process on the image using the parameters used by the image analysis unit when the one analysis result is output, further comprising, the control means controls to perform the analysis process on the entire area of the image The image processing apparatus according to Item 1, characterized by this. (Item 3) the control means further controls to execute an analysis process on an image other than the image using the parameters used by the image analysis unit when the one analysis result is output The image processing apparatus according to Item 2, characterized by this. (Item 4) the display control means displays the plurality of analysis results on the display unit in association with the parameters used by the image analysis unit when each analysis result is output The image processing apparatus according to any one of Items 1 to 3, characterized by this. (Item 5) further comprising region reception means for receiving designation of the region of the partial region image The image processing apparatus according to any one of Items 1 to 4, characterized by this. (Item 6) The area reception means receives the area designation of the partial region image by a selection region having a fixed size set based on the actual size of the deformation. The image processing apparatus according to item 5, characterized in that. (Item 7) The area reception means presents a candidate region of the partial region image based on the plurality of analysis results and receives a correction to the candidate region. The image processing apparatus according to item 5, characterized in that. (Item 8) The image is a captured image of a structure. The analysis process is a process of detecting a deformation occurring in the structure. The image processing apparatus according to any one of items 1 to 7, characterized in that. (Item 9) The analysis process is a process of detecting deformations of a plurality of deformation types occurring in the structure. When the determination means determines that the image is equal to or larger than the predetermined size, the trial means individually designates the partial region image for each of the plurality of deformation types. The image processing apparatus according to item 8, characterized in that. (Item 10) The predetermined size is indicated by the number of pixels. The image processing apparatus according to any one of items 1 to 9, characterized in that. (Item 11) The image analysis unit is a pre-trained artificial intelligence (AI) model. The image processing apparatus according to any one of items 1 to 10, characterized in that. (Item 12) A control method for an image processing apparatus that executes an analysis process on an image using an image analysis unit, A determination step of determining whether the image is equal to or larger than a predetermined size, A trial step of performing a plurality of analysis processes on the image with parameters used by the image analysis unit changed, A display control step of displaying a plurality of analysis results corresponding to the plurality of analysis processes on a display unit. including In the trial process, when it is determined by the determination process that the image is equal to or larger than the predetermined size, the plurality of analysis processes are performed on the partial region image in the image. A control method characterized by the above. (Item 13) A program for causing a computer to execute the control method according to Item 12.

[0090] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiment to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. Further, it can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

[0091] The invention is not limited to the above-described embodiment, and various changes and modifications are possible without departing from the spirit and scope of the invention. Therefore, claims are attached to disclose the scope of the invention.

Explanation of reference numerals

[0092] 200 Application; 201 User interface; 210 Storage; 211 Image storage unit; 212 Analysis result storage unit; 213 Model management unit; 214 Parameter management unit; 220 Image analysis unit; 250 Communication path

Claims

1. An image processing apparatus that executes analysis processing on an image using an image analysis unit, comprising: determination means for determining whether the image is of a predetermined size or more; trial means for performing a plurality of analysis processes on the image in which parameters used by the image analysis unit are changed; display control means for displaying a plurality of analysis results corresponding to the plurality of analysis processes on a display unit; wherein when the determination means determines that the image is of the predetermined size or more, the trial means controls to perform the plurality of analysis processes on a partial region image within the image An image processing apparatus characterized by the above.

2. reception means for receiving selection of one analysis result included in the plurality of analysis results; control means for controlling to execute analysis processing on the image using the parameters used by the image analysis unit when the one analysis result is output; further comprising wherein the control means controls to perform the analysis processing on the entire area of the image The image processing apparatus according to claim 1, characterized by the above.

3. wherein the control means further controls to execute analysis processing on an image other than the image using the parameters used by the image analysis unit when the one analysis result is output The image processing apparatus according to claim 2, characterized by the above.

4. The display control means displays the plurality of analysis results on the display unit in association with the parameters used by the image analysis unit when each analysis result is output The image processing apparatus according to claim 1, characterized by the above.

5. further comprising area reception means for receiving area designation of the partial region image The image processing apparatus according to claim 1, characterized by the above.

6. The area reception means receives area designation of the partial region image by a selection area of a fixed size set based on the actual size of the deformation The image processing apparatus according to claim 5, characterized by the above.

7. The area reception means presents candidate areas of the partial region image based on the plurality of analysis results and receives corrections to the candidate areas The image processing apparatus according to claim 5, characterized by the above.

8. The image is a captured image of a structure, The analysis processing is processing for detecting a deformation occurring in the structure The image processing apparatus according to claim 1, characterized by the above.

9. The analysis process is a process of detecting deformations of a plurality of deformation types occurring in the structure. When it is determined by the determination means that the image is equal to or larger than the predetermined size, the trial means individually designates the partial region image for each of the plurality of deformation types. The image processing apparatus according to claim 8, characterized in that.

10. The predetermined size is indicated by the number of pixels. The image processing apparatus according to claim 1, characterized in that.

11. The image analysis unit is a pre-trained artificial intelligence (AI) model. The image processing apparatus according to claim 1, characterized in that.

12. A control method for an image processing apparatus that executes an analysis process on an image using an image analysis unit, A determination step of determining whether the image is equal to or larger than a predetermined size; A trial step of performing a plurality of analysis processes with parameters changed by the image analysis unit on the image; A display control step of displaying a plurality of analysis results corresponding to the plurality of analysis processes on a display unit; including In the trial step, when it is determined by the determination step that the image is equal to or larger than the predetermined size, the plurality of analysis processes are performed on a partial region image within the image. A control method characterized by that.

13. A program for causing a computer to execute the control method according to claim 12.

Images