Display processing device, display processing method, and display processing program

Abstract

Provided are a display processing device, a display processing method, and a display processing program for displaying an image with which correlations between a plurality of detection target regions are easily confirmed. The display processing device comprises: an acquiring unit for acquiring segmentation results by distinguishing between detection targets of a target object and other targets, for each pixel in a captured image in which the target object has been imaged, on the basis of the captured image; an extracting unit for extracting detection target regions from the segmentation results; a measuring unit for measuring a feature quantity for determining a correlation between the detection target regions if a plurality of discontinuous detection target regions have been extracted; a correlation assessing unit for assessing a correlation between the plurality of detection target regions, on the basis of the feature quantity; a region setting unit for setting a target region to be displayed in an integrated display format, among the plurality of detection target regions, on the basis of an evaluation result obtained by the assessment performed by the correlation assessing unit; and a rendering unit for rending the target region using the display format.

Description

[Technical Field]

[0001] The present invention relates to a display processing device, a display processing method, and a display processing program, and more particularly to a display processing device, a display processing method, and a display processing program that display a detection target using a segmentation image determined using segmentation. [Background technology]

[0002] One method of inspecting defects in an industrial product is non-destructive testing, which involves irradiating the industrial product with light or radiation. In non-destructive testing, an image obtained by irradiating the industrial product with light or radiation is divided into different regions by determining whether each pixel is defective or not, a process called segmentation.

[0003] At this time, in order to present the results to the inspector, the areas determined to be defects, i.e., the locations where defects have been detected, are displayed by filling in the defect areas with a color that has been previously assigned to the defect type. However, there is a problem in that saving and transferring all of the coordinate information for filling in the viewer used at the inspection site results in a large file size.

[0004] As a method for displaying defects, for example, Patent Document 1 listed below describes a defect detection method in which defects are appropriately combined and the smallest rectangle that covers the entire circumscribing rectangle of all detected defects after the combination is used. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2008-241298 Summary of the Invention [Problem to be solved by the invention]

[0006] To display the results of segmentation, a bounding box of a fixed size is defined for each continuous area detected by segmentation, with the bounding box centered on the center of gravity of the area, and each defect area is enclosed by a bounding box.

[0007] However, when the segmentation detection results are divided into multiple regions, capturing each region individually and enclosing them in a rectangle results in too many displays, which takes time to check, resulting in a decrease in inspection efficiency.

[0008] The present invention has been made in consideration of these circumstances, and aims to provide a display processing device, a display processing method, and a display processing program that display an image that makes it easy to confirm the relationship between multiple detection target areas. [Means for solving the problem]

[0009] In order to achieve the object of the present invention, the display processing device of the present invention comprises an acquisition unit that distinguishes between detection targets and other targets for each pixel of a captured image of an object based on the captured image and acquires the segmentation results; an extraction unit that extracts the detection target area from the segmentation results; a measurement unit that measures features for determining the relevance between the detection target areas when multiple discontinuous detection target areas are extracted; a relevance determination unit that determines the relevance between the multiple detection target areas based on the features; a region determination unit that determines the target area that should be displayed in a unified display format from the multiple detection target areas based on the evaluation result determined by the relevance determination unit; and a drawing unit that draws the target area in a display format.

[0010] According to one aspect of the present invention, it is preferable to further include a display control unit that passes information required for displaying the target area to the display means and displays the target area.

[0011] According to one aspect of the present invention, the captured image is preferably a transmission image captured by transmitting radiation through the object.

[0012] According to one aspect of the present invention, the display format is preferably at least one of a frame surrounding the target area, a highlight display of the target area and other areas, and a mark indicating the target area.

[0013] According to one aspect of the present invention, the detection target is preferably a defect.

[0014] According to one aspect of the present invention, it is preferable that the relevance determining unit determines the relevance of areas of the detection target having defects of the same type.

[0015] According to one aspect of the present invention, the feature amount is preferably a distance between regions of the detection target.

[0016] According to one aspect of the present invention, it is preferable that the relevance determination unit has a distance threshold determined based on the characteristics of the object, past detection history, pass / fail criteria for the detection object, and the type of detection object, and determines the relevance based on the threshold and the feature measured by the measurement unit.

[0017] According to one aspect of the present invention, the feature amount is preferably position information of the detection target.

[0018] According to one aspect of the present invention, the feature amount is preferably a regularity of the region to be detected.

[0019] According to one embodiment of the present invention, the area determination unit determines a target area and a sub-target area within the target area in which the occurrence density distribution of the detection target differs from that of other areas inside the target area, and the drawing unit preferably draws the target area in a first display format and the sub-target area in a second display format.

[0020] According to one aspect of the present invention, the target area preferably includes defects of different types, and the drawing unit preferably provides information indicating that the target area includes defects of different types.

[0021] According to one aspect of the present invention, the information is preferably provided by at least one of a display color, a line type, and a mark of a frame surrounding the target region.

[0022] In order to achieve the object of the present invention, the display processing method of the present invention comprises an acquisition process for distinguishing between detection targets of the object and other targets for each pixel of the captured image based on a captured image of the object and acquiring a segmentation result; an extraction process for extracting detection target regions from the segmentation result; a measurement process for measuring features for determining the relevance between the detection target regions when multiple discontinuous detection target regions are extracted; a relevance determination process for determining the relevance between the multiple detection target regions based on the features; a region determination process for determining a target region in which the multiple detection target regions should be displayed in a unified display format based on the evaluation result determined in the relevance determination process; and a drawing process for drawing the target region in a display format.

[0023] In order to achieve the object of the present invention, the display processing method of the present invention comprises an acquisition process for acquiring information on the detection target area included in the segmentation result that distinguishes between the detection target and other objects for each pixel of the object; a measurement process for measuring features for determining the relevance between the detection target areas when there are multiple discontinuous detection target areas; a relevance determination process for determining the relevance between the multiple detection target areas based on the features; and a region determination process for determining the target area in which the multiple detection target areas should be displayed in an integrated display format based on the evaluation result determined in the relevance determination process.

[0024] In order to achieve the object of the present invention, the display processing program of the present invention has a computer realize: an acquisition function that distinguishes between detection targets and other targets for each pixel of a captured image of an object based on the captured image, and acquires the segmentation results; an extraction function that extracts the detection target area from the segmentation results; a measurement function that measures features for determining the relevance between the detection target areas when multiple discontinuous detection target areas are extracted; a relevance determination function that determines the relevance between the multiple detection target areas based on the features; an area determination function that determines the target area in which the multiple detection target areas should be displayed in a unified display format based on the evaluation result determined by the relevance determination function; and a drawing function that draws the target area in a display format. [Effects of the Invention]

[0025] According to the present invention, it is possible to easily check the correlation between a plurality of detection target regions. [Brief explanation of the drawings]

[0026] [Figure 1] FIG. 1 is a block diagram showing a defect display processing device according to one embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing an example of the image processing unit 24. As shown in FIG. [Figure 3] FIG. 3 is a block diagram showing an example of object photographing data. [Figure 4] FIG. 4 is a block diagram showing an example of product data. [Figure 5] FIG. 5 is a block diagram showing an example of an imaging system. [Figure 6] FIG. 6 is a flowchart showing a defect display processing method according to one embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing an example of a display process according to an embodiment of the present invention. [Figure 8] FIG. 8 is a diagram showing another example of the display process according to one embodiment of the present invention. [Figure 9]FIG. 9 is a diagram showing yet another example of the display process according to one embodiment of the present invention. [Figure 10] FIG. 10 is a diagram showing yet another example of the display process according to one embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing yet another example of the display process according to one embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0027] A display processing device, a display processing method, and a display processing program according to the present invention will be described below with reference to the accompanying drawings. Note that, as one embodiment of the present invention, a defect display processing device, a defect display processing method, and a defect display processing program in which defects are detected as a detection target will be described below.

[0028] [Configuration of defect display device] FIG. 1 is a block diagram showing a defect display processing device (display processing device) 10 according to one embodiment of the present invention.

[0029] The defect display processing device 10 according to this embodiment is a device that performs defect display processing from photographed images of an industrial product as an object, and is a device for assisting an inspector in diagnosing defects in the object. As shown in Fig. 1, the defect display processing device 10 according to this embodiment includes a control unit 12, an operation unit 14, an input / output interface (hereinafter referred to as I / F (interface)) 16, a display unit 18, a buffer memory 20, an image recognition unit 22, an image processing unit 24, and a recording unit 26.

[0030] The control unit 12 (display control unit) includes a CPU (Central Processing Unit) that controls the operation of each unit of the defect display processing device 10. The control unit 12 receives an operation input from an inspector via the operation unit 14, and transmits a control signal corresponding to this operation input to each unit of the defect display processing device 10 to control the operation of each unit.

[0031] The operation unit 14 is an input device that accepts operation input from the inspector, and includes a keyboard for character input, and a pointing device (mouse, trackball, etc.) for operating a pointer, icons, etc. displayed on the display unit 18. Note that the operation unit 14 may be provided with a touch panel on the surface of the display unit 18 instead of or in addition to the means listed above.

[0032] The I / F 16 is a means for communicating with an external device via a network NW. Data can be transmitted and received between the defect display processing device 10 and the external device via wired communication (e.g., LAN (Local Area Network), WAN (Wide Area Network), Internet connection, etc.) or wireless communication (e.g., LAN, WAN, Internet connection, etc.).

[0033] The defect display processing device 10 is capable of receiving input of object photography data D100 including photographed image data of the object OBJ photographed by the photography system 100 via the I / F 16. Note that the method of inputting the object photography data D100 from the photography system 100 to the defect display processing device 10 is not limited to the communication via the network NW listed above. For example, the defect display processing device 10 and the photography system 100 may be connected via a USB (Universal Serial Bus) cable, Bluetooth (registered trademark), infrared communication, or the like, or the object photography data D100 may be stored in a memory card that is detachable and readable from the defect display processing device 10, and the photographed image data may be input to the defect display processing device 10 via the memory card.

[0034] Furthermore, the defect display processing device 10 is capable of communicating with a product database (product DB (database)) 200 via the network NW. The product DB 200 stores product data D200 for each industrial product of the object. The control unit 12 is capable of searching for and reading out object identification information for identifying the object OBJ from the photographed image data of the object OBJ obtained from the photographing system 100, and of obtaining product data D200 corresponding to the read object identification information from the product DB 200. Using this product data D200 makes it possible to detect defects according to the type or characteristics of the object OBJ.

[0035] The product DB 200 may be installed on the network NW as in this embodiment so that the manufacturer or the like can update the product data D200, or may be provided in the defect display processing device 10.

[0036] The display unit (display means) 18 is a device for displaying the captured image and the segmentation image (segmentation result). As the display unit 18, for example, a liquid crystal monitor can be used.

[0037] The buffer memory 20 is used as a work area for the control unit 12 and as an area for temporarily storing image data to be output to the display unit 18 .

[0038] The recording unit 26 is a means for storing data including a control program used by the control unit 12. For example, a device including a magnetic disk such as an HDD (Hard Disk Drive), or a device including a flash memory such as an eMMC (Embedded Multi Media Card) or an SSD (Solid State Drive) can be used as the recording unit 26. The recording unit 26 stores object photography data D100 and product data D200.

[0039] The image recognition unit 22 (acquisition unit) identifies the type of object in the captured image using a defect type identification model. The defect type identification model is a model that takes an object image as input data and outputs a segmentation image (segmentation result) that is the result of identifying the defect type. The obtained segmentation image is stored in the recording unit 26.

[0040] A segmentation image is an image in which the type of defect (e.g., foreign matter, cracks, scratches, air bubbles, gas defects, wear, rust, and welding defects (excess fill, poor penetration, spatter, undercut)) in the image of the target object is identified on a pixel-by-pixel basis, with different colors used to distinguish between them. This method of identifying the type of defect in an image on a pixel-by-pixel basis is called semantic segmentation. An example of a machine learning model that performs segmentation is a U-shaped convolutional neural network (U-Net).

[0041] The image processing unit 24 reads the segmentation image from the image recognition unit 22 or the recording unit 26 and extracts defect areas (areas to be detected). Then, it measures feature quantities for determining the relevance between the extracted defect areas and determines the relevance between the defect areas based on these feature quantities. Furthermore, based on the evaluation results of the relevance determination, it determines the target areas to be displayed in an integrated display format and draws a display format showing the target areas. The image processing unit 24 outputs these results and information to the buffer memory 20. The control unit 12 uses the data output to the buffer memory 20 to create a display image in which multiple defect areas are displayed in an integrated display format on the segmentation image, and displays this display image on the display unit 18. This allows an inspector to interpret the image displayed on the display unit 18 and inspect the object OBJ.

[0042] Fig. 2 is a block diagram showing an example of the image processing unit 24. As shown in Fig. 2, the image processing unit 24 includes an extraction unit 240, a measurement unit 242, a relevance determination unit 244, an area determination unit 246, and a drawing unit 248.

[0043] The extraction unit 240 extracts defects in the object OBJ by detecting different colors from the segmentation image, thereby identifying the position and shape of the defect.

[0044] When the extraction unit 240 extracts multiple discontinuous defective areas, the measurement unit 242 measures feature quantities to determine the relevance of these defective areas. The feature quantities include the distance (pixels) between the defective areas for determining the relevance, the positional information of the defective areas, and the regularity of the distribution of the defective areas.

[0045] The distance between defective areas is measured by measuring the distance between the centers of the defective areas, the distance between the centers of gravity, or the shortest distance between the edges. The center of a defective area refers to the center of a circle or ellipse circumscribing the defective area. Positional information between defects is measured by the presence or absence of a brightness step (edge) between the defective areas, or by measuring differences in texture around the defective areas. Regularity in the distribution of defective areas is measured by measuring whether two or more defective areas are distributed at equal intervals, whether the defective areas are distributed in a straight line or a curve, etc.

[0046] The relevance determination unit 244 determines the relevance between defect areas based on the feature quantities. When determining the relevance between defect areas based on the distance between defect areas, the relevance determination unit 244 stores a distance threshold determined based on the characteristics of the object, past detection history, defect pass / fail criteria, and defect type, and determines whether or not there is a relevance between the measured distance between defect areas based on the threshold. Furthermore, if a brightness step is observed between defects or if a difference in texture is observed, the unit determines that there is no relevance. Furthermore, if the shape of the object is known, such as a 3D model, and it is determined that multiple layers of components overlap in the depth direction near the defect based on the correspondence with the defect position information in the segmentation image, the unit determines that the defect is unrelated. Furthermore, regarding the regularity of the distribution of defect areas, if two or more defect areas are distributed at equal intervals and if the defect areas are distributed in a linear or curved direction, the unit determines that the defect is related. Furthermore, shape information of the defect area, for example, directionality considering the long and short sides, may be taken into account, and if two elliptical defects are aligned along the long axis direction, they can be determined to be a series of related linear defects, whereas if the two defects are aligned along the short axis direction, they can be determined to be separate, unrelated defects.

[0047] The area determination unit 246 determines a target area in which multiple defect areas should be displayed in an integrated display format based on the evaluation results determined by the relevance determination unit 244. Defect areas that are of the same type and that are determined to be related by the relevance determination unit 244 are determined as target areas to be displayed in an integrated display format. Furthermore, if defects of different types are displayed in a single display format, the displays may overlap, making it difficult to confirm the defect areas on the display unit 18. In this case, the different types of defects are determined as target areas to be displayed in an integrated display format.

[0048] The drawing unit 248 draws the target area determined by the area determination unit 246 in an integrated display format. Examples of the integrated display format include surrounding the target area with a frame, highlighting the target area by changing the brightness of the target area and areas other than the target area, and displaying the target area with a mark such as an arrow.

[0049] Fig. 3 is a block diagram showing an example of object shooting data. As shown in Fig. 3, object shooting data D100 includes object identification information, shot image data, shooting condition data, and lighting condition data.

[0050] The object identification information is information for identifying the object OBJ, and includes, for example, information indicating the product name, product number, ID (identification) information, manufacturer name, and technical classification of the object OBJ.

[0051] The captured image data is image data obtained by capturing an image of the object OBJ (for example, an X-ray transmission image or a visible light image), and includes information on the image resolution or resolving power.

[0052] The shooting condition data is stored for each captured image data of the object OBJ, and includes information indicating the shooting date and time of each captured image data, the location to be photographed, the distance between the object OBJ and the imaging device at the time of shooting, and the angle relative to the imaging device.

[0053] The illumination condition data includes information indicating the type of radiation used to photograph the object OBJ (e.g., X-rays, visible light, transmitted light, or reflected light), irradiation intensity, irradiation angle, and tube current and tube voltage parameters.

[0054] Fig. 4 is a block diagram showing an example of product data. As shown in Fig. 4, the product data D200 includes product identification information, product attribute information, and inspection area designation information. The product data D200 may be associated with the object photography data D100 via the object identification information and the product identification information and recorded in the recording unit 26, or may be obtained from the product DB200 each time a defect inspection is performed.

[0055] The product identification information is information for identifying a product, and includes, for example, information indicating the product name, product number, manufacturer name, and technical classification.

[0056] The product attribute information includes, for example, information indicating the material and dimensions of each part of the product and the product's intended use. The information indicating the product's intended use includes, for example, information regarding the name, type, processing state, and installation method (e.g., joints, welds, screwing, fitting, soldering) of the device to which the product is attached. The product attribute information also includes defect occurrence information. The defect occurrence information includes, for example, at least one of the following information: past inspection date and time, the material of the object OBJ, the type of defect that occurred in the past (e.g., foreign matter, cracks, scratches, air bubbles, welding gas defects, wear, rust, etc.), position information, shape, size, depth, the location of occurrence (location coordinates, material thickness, processing state (e.g., joints, welds, etc.)), frequency information regarding the defect occurrence frequency, and a captured image of the defect.

[0057] The inspection area designation information includes information indicating the inspection area designated by the manufacturer of each product, etc. (for example, information including the position of the inspection area, which is created based on defect occurrence information such as whether or not defects have occurred in the past and frequency information regarding the frequency of defect occurrence.) The inspection area designation information is created, for example, by identifying locations where defects are likely to occur statistically and structurally, based on information from when the manufacturer, etc., repaired the product in the past.

[0058] The defect occurrence information includes, for example, at least one of the following information: past inspection date and time, material of the object OBJ, type, shape, size, depth, occurrence location, and captured image of the defect that occurred in the past.

[0059] [Imaging system configuration] Next, an image capturing system 100 for capturing an image of an object OBJ will be described. Fig. 5 is a block diagram showing an example of the image capturing system.

[0060] The imaging system 100 is used to photograph an object OBJ placed in an imaging room 114, and as shown in FIG. 5, it comprises an imaging control unit 102, an imaging operation unit 104, an image recording unit 106, an imaging device 108, and radiation sources 110 and 112.

[0061] The photography control unit 102 includes a CPU (Central Processing Unit) that controls the operation of each unit of the photography system 100. The photography control unit 102 receives an operation input from an operator (photographer) via the photography operation unit 104, and transmits a control signal according to this operation input to each unit of the photography system 100 to control the operation of each unit.

[0062] The imaging operation unit 104 is an input device that accepts operation inputs from an operator, and includes a keyboard for entering characters, and a pointing device (mouse, trackball, etc.) for operating a pointer, icons, etc. displayed on the display unit 18. Through the imaging operation unit 104, the operator can input information about the object OBJ, input instructions to the imaging device 108 to perform imaging (including settings of imaging conditions such as exposure time, focal length, and aperture, imaging angle, imaging location, etc.), input instructions to the radiation sources 110 and 112 to irradiate radiation (including settings of irradiation start time, irradiation duration, irradiation angle, irradiation intensity, etc.), and input instructions to record the acquired image data in the image recording unit 106.

[0063] The image recording unit 106 records image data (received light image) of the object OBJ photographed by the imaging device 108. Information for identifying the object OBJ is recorded in the image recording unit 106 in association with the image data.

[0064] The imaging device 108 and radiation sources 110 and 112 are arranged inside an imaging room 114. The radiation sources 110 and 112 are, for example, X-ray sources, and the partition and entrance between the imaging room 114 and the outside are protected from X-rays by X-ray protective materials (for example, lead, concrete, etc.). Note that when imaging an object OBJ by irradiating it with visible light, it is not necessary to use a protected imaging room 114.

[0065] The radiation sources 110 and 112 irradiate radiation onto an object OBJ placed in an imaging room 114 in accordance with instructions from the imaging control unit 102 .

[0066] The imaging device 108 receives radiation irradiated from the radiation source 110 to the object OBJ and reflected by the object OBJ, or radiation irradiated from the radiation source 112 to the object OBJ and transmitted through the object OBJ, in accordance with an instruction to perform imaging from the imaging control unit 102, and images the object OBJ. OBJ When an X-ray source is irradiated onto the object, a light receiving panel can be used, and when visible light is irradiated, a camera can be used. The object OBJ is held in the imaging room 114 by a holding member (e.g., a manipulator, a mounting table, a movable mounting table) (not shown), and the distance and angle of the object OBJ relative to the image capturing device 108 and the radiation sources 110 and 112 can be adjusted. The operator can control the relative positions of the object OBJ, the image capturing device 108, and the radiation sources 110 and 112 via the imaging control unit 102, and can capture an image of a desired part of the object OBJ.

[0067] The radiation sources 110 and 112 finish irradiating the object OBJ with radiation in synchronization with the end of the imaging operation by the imaging device 108.

[0068] In the example shown in FIG. 5, the imaging device 108 is placed inside the photography room 114, but the imaging device 108 may be placed outside as long as it is capable of photographing the object OBJ inside the photography room 114.

[0069] 5, one imaging device 108 and two radiation sources 110 and 112 are provided, but the number of imaging devices and radiation sources is not limited to this. For example, there may be a plurality of imaging devices and a single radiation source, or there may be only one of each.

[0070] [Defect display processing method] FIG. 6 is a flowchart showing a defect display processing method according to one embodiment of the present invention.

[0071] First, the defect display processing device 10 acquires object photographic data D100 including photographic image data (photographed image) of the object OBJ from the photographing system 100 via the I / F 16. The image recognition unit 22 acquires a segmentation image (segmentation result) that is a defect type identification result from the acquired photographic image data using a defect type identification model (step S12: acquisition step).

[0072] Next, the extraction unit 240 of the image processing unit 24 extracts defect areas from the segmentation image (step S14: extraction step). The segmentation image shows different types of defects in different colors, and defect areas are extracted by detecting the different colors.

[0073] Next, the measurement unit 242 of the image processing unit 24 measures a feature amount for determining the relevance between the defect areas extracted in the extraction step (step S14) (step S16: measurement step). The feature amount to be measured is the distance (pixels) between the defect areas for determining the relevance. The distance between the defect areas is measured as the distance between the centers of the defect areas, the distance between the centers of gravity, or the shortest distance between the ends of the defect areas.

[0074] Next, the relevance determination unit 244 of the image processing unit 24 determines the relevance between the defect areas based on the feature amount (distance between the defect areas) measured in the measurement step (step S16) (step S18: relevance determination step). The relevance determination unit 244 holds a distance threshold determined from the characteristics of the object, past detection history, defect pass / fail criteria, etc. The characteristics of the object, past detection history, defect pass / fail criteria, etc. can be acquired from the object identification information of the object photography data D100 and the product attribute information of the product data D200.

[0075] The reference threshold can be determined depending on the type of defect. For example, considering that defects caused by air bubbles (called porosity) may occur in clusters over a wide area, the threshold for the distance between defect areas that are determined to be related can be set longer for defects caused by air bubbles.

[0076] Furthermore, the distance between defect areas can be a distance weighted by the size of the defect area. Fig. 7 is a diagram showing an example of display processing according to an embodiment of the present invention. 700A in Fig. 7 is a diagram showing two defects determined to be related, and 700B in Fig. 7 is a diagram showing two defects determined to be unrelated. 700A is a diagram showing defect areas 302 and 304 on segmentation image 300, which represent two circular defects with a diameter of 15 px, and which are 10 px apart from each other's edges. 700B is a diagram showing defect areas 306 and 308 on segmentation image 300, which represent two circular defects with a diameter of 2 px, and which are 10 px apart from each other's edges. Even if the distance between the defect areas is the same, the ratio of the distance to the size of the defect itself is different, so 7 It can be determined that the defective areas of 00A are related to each other. 7 It can be determined that the defective areas of 00B are not related to each other.

[0077] Returning to FIG. 6, the area determination unit 246 of the image processing unit 24 determines a target area in which multiple defect areas should be displayed in an integrated display format based on the evaluation results determined in the relevance determination step (step S18) (step S20: area determination step). The target area to be displayed in an integrated display format is an area including multiple defect areas determined to be related in the relevance determination step (step S18). In 700A of FIG. 7, defect areas 302 and 304 are determined to be related defects, and the area including defect areas 302 and 304 is determined to be target area 309. In 700B, defect area 306 and defect area 308 are determined to be unrelated defects, and the area including defect area 306 and the area including defect area 308 are determined to be target areas 311 and 313, respectively.

[0078] Next, drawing unit 248 of image processing unit 24 draws each of target areas 309, 311, and 313 determined in the area determination step (step S20) in an integrated display format (step S22: drawing step). In 700A of Fig. 7, target area 309 is drawn as a display format using rectangular frame 310 to indicate target area 309. In 700B, defective area 306 and defective area 308 are determined to be unrelated defects in the correlation determination step (step S18), and target areas 311 and 313 including defective area 306 and defective area 308 are drawn as rectangular frames 312 and 314 to indicate the target areas.

[0079] In FIG. 7, the target area is depicted by being surrounded by a rectangular frame as an integrated display format for depicting the target area, but this is not limiting. For example, the target area can be shown by maintaining the brightness of the target area and lowering the brightness of the area around the target area to highlight it. The target area can also be shown with an arrow. Furthermore, the shape of the frame is not limited to a rectangle, and can be other shapes such as a circle or an ellipse. The line type of the frame is also not limited, and can be a dashed line, for example.

[0080] The control unit 12 creates a display image (see FIG. 7) showing the target area on the segmentation image based on the segmentation image acquired in the acquisition step (step S12), the target area determined in the area determination step (step S20), and the position information of the rectangular frames 310, 312, and 314 drawn in the drawing step (step S22), and passes this information to the display unit 18. As a result, the inspection image showing the target area together with the associated defects is displayed on the display unit 18.

[0081] FIG. 8 illustrates another example of display processing according to an embodiment of the present invention, in which the relevance of defect regions is determined based on the regularity of the defect regions. 800A in FIG. 8 illustrates two defects determined to be relevant, while 800B in FIG. 8 illustrates two defects determined to be unrelevant. 800A illustrates segmentation image 300 in which defect regions 322 and 324, each representing two elliptical defects with a major axis of 10 px and a minor axis of 3 px, are arranged contiguously in the major axis direction with a center distance of 8 px. Similarly to 800A, 800B illustrates segmentation image 300 in which defect regions 326 and 328, each representing two elliptical defects with a major axis of 10 px and a minor axis of 3 px, are arranged contiguously with a center distance of 8 px, but with the minor axes of defect regions 326 and 328 facing each other. Even if the distance between the defect areas is the same, defect areas 322, 324 that are continuous along the long axis direction can be determined to be related defects, while defect areas 326, 328 that are facing each other along the short axis can be determined to be unrelated and separate defects.

[0082] Defect areas 322 and 324 are determined to be relevant defects, and the area including defect areas 322 and 324 is determined to be target area 329, and target area 329 is indicated by frame 330. Defect areas 326 and 328 are determined to be unrelated defects, and the area including defect area 326 and the area including defect area 328 are determined to be target areas 331 and 333, respectively, and the target areas are indicated by frames 332 and 334.

[0083] FIG. 9 is a diagram showing yet another example of display processing according to an embodiment of the present invention. The segmentation image 300 shown in FIG. 9 has defect regions 342 indicating 30 or more defects. Each of the defect regions 342 faces the same direction along the longitudinal axis and has curved continuity in the lateral direction. Therefore, these defect regions 342 can be determined to be a series of defects (flaws) and to be related defects. Therefore, the region including the series of defect regions 342 is determined as a target region 344, and the target region 344 is indicated by a frame 346.

[0084] Furthermore, defect position information can be used as a feature for determining the relevance of defect regions. When a brightness step is confirmed between defect regions on the segmentation image, the defect position information can be used to determine that the defect regions are separate defects separated in the depth direction of the captured image, and that the defects are unrelated. Also, when the textures around the defect regions are different, the defects are determined to have occurred in different regions and are separate defects, and that the defects are unrelated.

[0085] Furthermore, if the shape of the object is known in advance from a 3D model, etc., and it is determined that multiple layers of parts are overlapping in the depth direction near the defect based on the correspondence with the defect position information in the segmentation image, it can be determined that the defects are separate defects, and that the defects are unrelated.

[0086] 10 is a diagram showing yet another example of display processing according to an embodiment of the present invention, illustrating an image in which a cluster of porosity defects has occurred. Defective regions 362 indicating defects due to the inclusion of air bubbles may occur in a cluster over a wide area. The relevance determination process (step S18) determines that these defect regions 362 indicating defects due to the inclusion of air bubbles are related defects. Then, the region determination process (step S20) determines a region including these defect regions 362 as a target region 364. Furthermore, the region determination process (step S20) determines regions within the target region 364 where the defect density distribution differs from that of other regions within the target region 364 as sub-target regions 366 and 368.

[0087] In the example shown in FIG. 10, within the target area 364, an area with a high defect density distribution is designated as a first sub-target area 366, and an area with a low defect density distribution is designated as a second sub-target area 368.

[0088] In the drawing process (step S22), the target area 364 is drawn in a rectangular first frame (first display format) 370, and the first sub-target area 366 and the second sub-target area 368 are drawn in rectangular second frames (second display format) 372 and 374, respectively. The first and second display formats are not limited to frames, and can also be drawn by highlighting or marks, as described above.

[0089] In addition, while Figure 10 shows an example of a two-level hierarchical structure having a first sub-target area and a second sub-target area within a target area, it is also possible to have a three-level or more hierarchical structure having an additional sub-target area within a sub-target area.

[0090] 11 is a diagram showing yet another example of display processing according to an embodiment of the present invention, in which an area including different types of defects is determined as a target area.

[0091] In segmentation image 300, defect area 382 is distinguished from defect areas 384 and 386 by different colors, and they are different defect types. Furthermore, defect area 384 and defect area 386 are the same type of defect, but because defect area 382 exists between defect area 384 and defect area 386, they are unrelated defects.

[0092] In such a case, the display process determines the areas including each of the defective areas 382, ​​384, and 386 as target areas and displays the target areas by drawing each of them in a separate display format. However, in Figure 11, the defective areas 382, ​​384, and 386 are close to each other, so if they are drawn in separate display formats, they will overlap, which could make it difficult to see the defects.

[0093] Therefore, in the display processing of this embodiment, when the display formats are drawn overlapping, in the area determination process (step S20), defect areas 382, ​​384, 386 containing different types of defects and unrelated defects are determined as target area 388, and in the drawing process (step S22), target area 388 is drawn in a unified display format.

[0094] As an example of the display format, 1100A in Fig. 11 is a diagram showing a display format in which the target area 388 is displayed as a frame painted in a plurality of different display colors. In 1100A, a frame 390 surrounding the target area 388 is displayed with a side 390A given a first color and a side 390B given a second color. The first color and the second color correspond to the colors of the defect area 382 and the defect areas 384 and 386 used to distinguish the defect types in the segmentation image 300. This makes it possible to read the type of defect from the display of the frame 390.

[0095] 1100B in FIG. 11 is a diagram showing a display format in which the target area 388 is displayed with a double frame 391. In 1100B as well, the frame 391 surrounding the target area 388 is displayed with an inner frame 391A given a first color and an outer frame 391B given a second color. The first color and the second color correspond to the colors of the defect area 382 and the defect areas 384 and 386 used to distinguish the defect types in the segmentation image 300. In this way, the defect types can be distinguished by the frame. 391 This can be read from the display.

[0096] 11 shows a display format in which the target area 388 is displayed with a dashed frame 392. In 1100C, the type of defect cannot be read from the frame 392, but the dashed line makes it possible to read that different types of defects are included.

[0097] According to this embodiment, the detailed shape and distribution of the detection target can be confirmed from the results of discrimination between detailed detection targets and other targets for each pixel by segmentation, and information necessary for inspection can be extracted. Then, by determining the correlation between the inspection targets, multiple inspection targets can be grouped and displayed in a unified format, making it clear which inspection targets the inspector should pay particular attention to, thereby improving inspection efficiency.

[0098] In the above embodiment, the detection target is described as a defect, but the detection target is not limited to defects. For example, the detection target may be a minute flaw that passes the product standard but has an abnormality.

[0099] Furthermore, the present invention is not limited to use in inspection, but can also be used for purposes other than inspection, such as analyzing objects, classifying and evaluating products, and the like.

[0100] The present invention can also be realized as a program (display processing program) that causes a computer to perform the above processing, or as a non-transitory recording medium or program product that stores such a program. By applying such a display processing program to a computer, the computing means and recording means of the computer can function as the acquisition function, extraction function, measurement function, relevance determination function, area determination function, and drawing function of the display processing program. [Explanation of symbols]

[0101] 10 Defect display processing device 12 Control Unit 14 Control section 16 Input / Output Interface (I / F) 18 Display 20 Buffer Memory 22 Image Recognition Unit 24 Image processing section 26 Recording Section 100 Shooting System 102 Imaging control unit 104 Shooting operation section 106 Image recording unit 108 Imaging device 110 Radiation source 112 Radiation Source 114 Photo Studio 200 Product Database (Product DB) 240 Extraction part 242 Measurement Unit 244 Relevance Determination Unit 246 Area determination part 248 Drawing Department 300 segmentation images 302 Defective Area 304 Defective Area 306 Defective Area 308 Defective Area 309 Target Area 310 slots 311 Target Area 312 slots 313 Target Area 314 slots 322 Defective Area 324 Defective Area 326 Defective Area 328 Defective Area 329 Target Area 330 slots 331 Target Area 332 slots 333 Target Area 334 slots 342 Defective Area 344 Target Area 346 slots 362 Defective Area 364 Target Area 366 Sub-subject area (first sub-subject area) 368 Sub-subject area (second sub-subject area) 370 First Frame 372 Second Frame 374 Second Frame 382 Defective Area 384 Defective Area 386 Defective Area 388 Target Area 390 slots Around 390A Around 390B 391 slots 391A inner frame 391B outer frame 392 slots D100 object photography data D200 Product Data OBJ Object NW Network

Claims

1. an acquisition unit that, based on a captured image of an object, distinguishes between a detected object of the object and other objects for each pixel of the captured image and acquires a segmentation result; an extraction unit that extracts the detection target region from the segmentation result; a measurement unit that, when a plurality of discontinuous detection target regions are extracted, measures the distance between the detection target regions as a feature amount for determining the relevance between the detection target regions; a relevance determination unit that determines the relevance of the plurality of detection target regions based on the feature amount for the same type of detection target regions; an area determination unit that determines an object area to be displayed in a unified display format among the areas of the plurality of detection objects based on the evaluation result determined by the relevance determination unit; a drawing unit that draws the target region in the display format; Equipped with the relevance determination unit has a threshold value for the distance determined from at least one of the characteristics of the object, a past detection history, a pass / fail criterion for the detection object, and a type of the detection object, and determines the relevance based on the threshold value and the feature amount measured by the measurement unit. Display processing device.

2. a display control unit that transfers information required for displaying the target area to a display unit and displays the target area; The display processing device according to claim 1 .

3. The captured image is a transmission image captured by transmitting radiation through the object. The display processing device according to claim 1 or 2.

4. the display format is at least one of a frame surrounding the target area, a highlight display of the target area and other areas, and a mark indicating the target area; The display processing device according to claim 1 or 2.

5. the detection target is a defect; The display processing device according to claim 1 or 2.

6. the relevance determination unit has a threshold value for the distance determined based on characteristics of the object, a past detection history, a pass / fail criterion for the detection object, and a type of the detection object, and determines the relevance based on the threshold value and the feature amount measured by the measurement unit. The display processing device according to claim 1 or 2.

7. the region determination unit determines the target region and, within the target region, a sub-target region in which an occurrence density distribution of the detection target differs from that of other regions inside the target region; the drawing unit draws the target region in a first display format and the sub-target region in a second display format; The display processing device according to claim 1 or 2.

8. the target area includes different types of defects; the rendering unit provides information indicating that different types of defects are included in the target region; The display processing device according to claim 5 .

9. The information is provided by at least one of a display color, a line type, and a mark of a frame surrounding the target area. The display processing device according to claim 8 .

10. an acquisition unit that, based on a captured image of an object, distinguishes between a detected object of the object and other objects for each pixel of the captured image and acquires a segmentation result; an extraction unit that extracts the detection target region from the segmentation result; a measurement unit that measures a feature amount for determining a relationship between the detection target regions when a plurality of non-contiguous detection target regions are extracted; a relevance determination unit that determines the relevance of the plurality of detection target regions based on the feature amount for the same type of detection target regions; an area determination unit that determines an object area to be displayed in a unified display format among the areas of the plurality of detection objects based on the evaluation result determined by the relevance determination unit; a drawing unit that draws the target region in the display format; Equipped with the region determination unit determines the target region and, within the target region, a sub-target region in which an occurrence density distribution of the detection target differs from that of other regions inside the target region; the drawing unit draws the target region in a first display format and the sub-target region in a second display format; Display processing device.

11. an acquisition unit that, based on a captured image of an object, distinguishes between a detected object of the object and other objects for each pixel of the captured image and acquires a segmentation result; an extraction unit that extracts the detection target region from the segmentation result; a measurement unit that measures a feature amount for determining a relationship between the detection target regions when a plurality of non-contiguous detection target regions are extracted; a relevance determination unit that determines a relevance of the plurality of detection target regions based on the feature amount; an area determination unit that determines an object area to be displayed in a unified display format among the areas of the plurality of detection objects based on the evaluation result determined by the relevance determination unit; a drawing unit that draws the target region in the display format; Equipped with the detection target is a defect, the target area includes different types of defects; the rendering unit provides information indicating that different types of defects are included in the target region; Display processing device.

12. The information is provided by at least one of a display color, a line type, and a mark of a frame surrounding the target area. The display processing device according to claim 11 .

13. an acquisition unit that, based on a captured image of an object, distinguishes between a detected object of the object and other objects for each pixel of the captured image and acquires a segmentation result; an extraction unit that extracts the detection target region from the segmentation result; a measurement unit that measures a feature amount for determining a relationship between the detection target regions when a plurality of non-contiguous detection target regions are extracted; a relevance determination unit that determines a relevance of the plurality of detection target regions based on the feature amount; an area determination unit that determines an object area to be displayed in a unified display format among the areas of the plurality of detection objects based on the evaluation result determined by the relevance determination unit; a drawing unit that draws the target region in the display format; and the target area to be displayed in the integrated display format determined by the area determination unit includes at least two target areas extracted as different types from each other. Display processing device.

14. an acquisition step of distinguishing between a detected object and other objects for each pixel of a captured image of the object based on the captured image, and acquiring a segmentation result; an extraction step of extracting the detection target region from the segmentation result; a measuring step of measuring a distance between the detection target regions as a feature for determining a relationship between the detection target regions when a plurality of non-contiguous detection target regions are extracted; a relevance determination step of determining the relevance of the plurality of detection target regions based on the feature amount for the same type of detection target regions; an area determination step of determining an object area in which the areas of the plurality of detection objects should be displayed in an integrated display format based on the evaluation result determined in the relevance determination step; a rendering step of rendering the target region in the display format; Equipped with the relevance determination step has a distance threshold determined based on at least one of the characteristics of the object, a past detection history, a pass / fail criterion for the detection object, and a type of the detection object, and determines the relevance based on the threshold and the feature amount measured in the measurement step. Display processing method.

15. an acquisition step of acquiring information on the area of ​​the detection target included in the segmentation result obtained by distinguishing between the detection target and other objects for each pixel of the object; a measuring step of measuring a distance between the detection target regions as a feature for determining a relationship between the detection target regions when the detection target regions are discontinuous; a relevance determination step of determining the relevance of the plurality of detection target regions based on the feature amount for the same type of detection target regions; an area determination step of determining an object area in which the areas of the plurality of detection objects should be displayed in an integrated display format based on the evaluation result determined in the relevance determination step; Equipped with the relevance determination step has a distance threshold determined based on at least one of the characteristics of the object, a past detection history, a pass / fail criterion for the detection object, and a type of the detection object, and determines the relevance based on the threshold and the feature amount measured in the measurement step. Display processing method.

16. an acquisition function that, based on a captured image of an object, distinguishes between a detected object of the object and other objects for each pixel of the captured image and acquires a segmentation result; an extraction function for extracting the detection target region from the segmentation result; a measurement function that, when a plurality of discontinuous detection target regions are extracted, measures the distance between the detection target regions as a feature for determining the relevance between the detection target regions; a relevance determination function that determines the relevance of the plurality of detection target regions based on the feature amount for the same type of detection target regions; an area determination function that determines an object area in which the areas of the plurality of detection objects should be displayed in an integrated display format based on the evaluation result determined by the relevance determination function; a drawing function of drawing the target area in the display format, the relevance determination function has a distance threshold determined from at least one of the characteristics of the object, a past detection history, a pass / fail criterion for the detection object, and a type of the detection object, and determines the relevance based on the threshold and the feature amount measured by the measurement function; Display processing program.

17. A non-transitory computer-readable recording medium having the program according to claim 16 recorded thereon.

Images