Analysis device and analysis method

Abstract

An analysis device (10) is a device for visualizing an accuracy of a trained determination device (20) for outputting a non-defective product label or a defective product label and includes an acquisition unit (11) that acquires an image pair (DP) of a non-defective product image (G1) with a non-defective product as a subject and a defective product image (G2) with a defective product as a subject, an extraction unit (12) that extracts an image region (R) of a defective part of the defective product on the basis of the image pair (DP), a generation unit (13) that changes a feature quantity of the image region (R) of the defective part to generate a plurality of image regions (Pl to P4) of pseudo-defective parts, a compositing unit (14) that synthesizes each of the image regions (Pl to P4) of the plurality of pseudo-defective parts with the non-defective product image (G1) to generate a plurality of composite images (D1) having different feature quantities, an image output unit (15) that outputs the plurality of composite images (D1) to the determination device (20), a result acquisition unit (16) that acquires a label corresponding to each of the plurality of composite images from the determination device (20), and a display control unit (17) that displays an object indicating the label corresponding to each of the plurality of composite images (D1) in an array based on the feature quantities.

Description

Analysis Device and Analysis Method

[0001] The present disclosure relates to an analysis device and an analysis method.

[0002] Patent Document 1 discloses an image inspection device having means for creating a pseudo-defective image for training a neural network. This image inspection device extracts difference data between a non-defective product image and a defective product image, and creates a plurality of patterns of pseudo-defective images by synthesizing the difference data with the position and size transformed onto the non-defective product image. The image inspection device causes the neural network to learn using the created pseudo-defective images as defective product training data. The image inspection device determines whether the subject of the image is non-defective or defective based on the neural network.

[0003] Japanese Patent Application Laid-Open No. 2005-156334

[0004] By the way, the pseudo-defective images generated by the image inspection device described in Patent Document 1 may include images that may be determined as non-defective products. Since the image inspection device described in Patent Document 1 learns all pseudo-defective images as defective products, there is a possibility that the determination result of the image inspection device may differ from the assumption. Also, with the image inspection device described in Patent Document 1, it is not possible to verify up to what extent the defect represented by the difference data should be determined as a non-defective product.

[0005] An object of the present disclosure is to provide an analysis device and an analysis method capable of visualizing the accuracy of a determination device.

[0006] An analysis device relating to one aspect of this disclosure is an analysis device for visualizing the accuracy of a trained judgment device that outputs either a good product label indicating that an item is a good product or a defective product label indicating that an item is a defective product, using an image of an item as the subject, and comprises: an acquisition unit that acquires an image pair of a good product image with a good product as the subject and a defective product image with a defective product as the subject; an extraction unit that extracts an image region of a defective part of a defective product based on the image pair; a generation unit that generates multiple image regions of pseudo-defective parts by changing the feature quantities of the image regions of the defective parts; a synthesis unit that synthesizes each of the multiple image regions of pseudo-defective parts with a good product image to generate multiple composite images with different feature quantities of pseudo-defective parts; an image output unit that outputs multiple composite images to a judgment device; a result acquisition unit that acquires a label corresponding to each of the multiple composite images from the judgment device; and a display control unit that displays an object indicating the label corresponding to each of the multiple composite images in an array based on feature quantities.

[0007] According to this disclosure, the accuracy of the judgment device can be visualized.

[0008] This is a block diagram showing an example of the functions of the analysis apparatus and determination apparatus according to the embodiment. Figure 2(A) shows an example of the process of extracting image regions of defective areas in the extraction unit. Figure 2(B) shows an example of the process of generating image regions of pseudo-defective areas in the generation unit. Figure 2(C) shows an example of the process of generating a composite image in the synthesis unit. This is an example of displaying objects indicating labels corresponding to each of multiple composite images in an array based on feature quantities. This is another example of displaying objects indicating labels corresponding to each of multiple composite images in an array based on feature quantities. This is a block diagram showing an example of the hardware configuration of the apparatus shown in Figure 1. This is a flowchart of the analysis method. This is a block diagram showing other examples of the functions of the analysis apparatus and determination apparatus according to the embodiment.

[0009] Embodiments of the present disclosure will be described below with reference to the drawings. In the following description, the same or equivalent elements will be denoted by the same reference numerals, and redundant descriptions will not be repeated.

[0010] [Functional Configuration of the Analysis Device] Figure 1 is a block diagram showing an example of the functions of the analysis device 10 and the judgment device 20 according to the embodiment. The analysis device 10 shown in Figure 1 is a device that visualizes the accuracy of the judgment device 20. The judgment device 20 is pre-trained to determine the quality of an item using an image of the item as the subject. The trained judgment device 20 outputs either a good product label or a defective product label. A label is information that identifies a pre-set category. A good product label indicates that the item is a good product, and a defective product label indicates that the item is a defective product. A good product is an item that meets the appearance quality standards, and a defective product is an item that does not meet the appearance quality standards.

[0011] The determination device 20 has a model M2 that performs a determination function. In the determination device 20, a target image D2 is input to the model M2, and a label L2 is output from the model M2. The target image D2 is an image of an item that is the subject of determination by the determination device 20. The label L2 is either a good product label or a defective product label. The output label L2 may be attached to the target image D2. Attachment means associating them. For example, attachment may involve recording the relationship between the target image D2 and the label L2 in a table, changing the attribute information of the target image D2 to include the label L2, or embedding the label L2 into the target image D2 itself.

[0012] Model M2 includes a neural network and parameters. The neural network has a structure in which multiple neurons are connected. The neural network may also be a hierarchical multilayer neural network consisting of layers in which multiple neurons are grouped. The neural network is defined by the number of neurons and their connections. The strength of the connections between neurons or between layers is defined using parameters (such as weight coefficients). In the neural network, data is input, and the characteristics of the data are output as a solution based on the calculation results and parameters of multiple neurons. The parameters of Model M2 are learned using training images, etc., so that the desired ability can be acquired. Learning is the process of adjusting the parameters to their optimal values. Model M2 may be a copy of a model learned outside of the judgment device 20. In the following, we will explain, as an example, the case in which Model M2 takes a target image D2 of an electronic component as input and outputs a label L2 regarding the quality of the electronic component. In this case, the appearance quality standard is determined by the degree of chipping, discoloration, and scratches on the electronic component.

[0013] The analysis device 10 visualizes the accuracy with which the trained judgment device 20 determines good and defective products. First, the analysis device 10 generates multiple images of electronic components, each with a different predetermined feature size related to the defective area. Next, the analysis device 10 outputs the generated multiple images to the judgment device 20 and obtains a label corresponding to each of the multiple images. Then, the analysis device 10 displays objects representing the obtained labels in an array based on the feature size on a display device or the like. This allows the annotator (worker) to confirm the relationship between the size of the feature size of the defective area and the label. To realize this function, the analysis device 10 includes an acquisition unit 11, an extraction unit 12, a generation unit 13, a synthesis unit 14, an image output unit 15, a result acquisition unit 16, and a display control unit 17.

[0014] The acquisition unit 11 acquires an image pair DP. An image pair DP consists of two images of an electronic component. The subject of the images constituting the image pair DP is the same item as the target image D2 input to the determination device 20. Preferably, the two images in the image pair DP are images taken at the same field of view. The image pair DP consists of a good product image where the electronic component is a good product and a defective product image where the electronic component is a defective product. The image pair DP may be selected from a group of images that have been acquired in advance, including good product images and defective product images. The image pair DP may or may not have a label (tag) attached to it.

[0015] The extraction unit 12 extracts the image region of the defective part of the defective product from the defective product image based on the image pair DP. The defective part is the area where electronic components are missing, discolored, or scratched. The defective part is the part that differs between the good product image and the defective product image. The extraction unit 12 extracts the image region of the defective part based on the difference between the data of the good product image and the data of the defective product image.

[0016] Figure 2(A) shows an example of the process for extracting image regions of defective areas in the extraction unit 12. In Figure 2(A), an image pair DP is shown, consisting of a good product image G1 and a defective product image G2 that has defective areas. The electronic component in the defective product image G2 has a missing part (an example of a defective area). As shown in Figure 2(A), the extraction unit 12 extracts the image region R of the defective area based on the difference in pixel values ​​at each pixel position between the good product image G1 and the defective product image G2. The defective areas targeted by the extraction unit 12 are not limited to missing electronic components. Since the defective areas are extracted based on the difference between the good product image G1 and the defective product image G2, the extraction unit 12 can extract defective areas without considering the type of defective area (missing part, discoloration, scratch). Note that the defective product image G2 does not have to be an image obtained by imaging an electronic component that actually has a defective area. For example, an annotator may use a PC paint tool to manually mark defective areas on a good product image G1, and this image may be used as the defective product image G2. By having the extraction unit 12 extract the defective areas marked by the annotator, the annotator's own ideas can be reflected in the identification of the defective areas, thus achieving an effect.

[0017] The generation unit 13 generates multiple image regions of pseudo-defective areas by changing the feature quantities of the image region R of the defective area. A pseudo-defective area is a simulated defective area and is generated by calculations based on the image region R. The feature quantities of the image region R of a defective area depend on the type of defective area. For example, the feature quantities of chips and scratches on electronic components are width, length, position, and area. The feature quantities of discoloration on electronic components are width, length, position, hue, saturation, and brightness. Here, position is a concept that includes the XY direction (translation) and the θ direction (rotation direction) in the XY coordinate space. Below, we will explain as an example the case in which an image region of a pseudo-defective area is generated using the area (width and length) of a chip in an electronic component as a feature quantity.

[0018] Figure 2(B) shows an example of the process of generating image regions of pseudo-defects in the generation unit 13. As shown in Figure 2(B), the generation unit 13 enlarges or reduces the image of the missing electronic component extracted as the image region R of the defective area. This generates multiple image regions P1 to P4 of pseudo-defects. The generation unit 13 generates multiple image regions P1 to P4 of pseudo-defects with different areas of missing electronic components by changing the enlargement or reduction ratio. In the example shown in Figure 2(B), four image regions P1 to P4 were generated, but the number of image regions is not limited to four.

[0019] The synthesis unit 14 synthesizes each of the image regions P1 to P4 of the multiple pseudo-defective areas with the good product image G1 to generate multiple composite images with different feature quantities. The synthesis unit 14 synthesizes one image region of a pseudo-defective area with one good product image G1 to generate one image data. Synthesis is performed by changing the pixel values ​​of the pixels in the good product image G1 corresponding to the image region of the pseudo-defective area. Figure 2(C) is a diagram showing an example of the process of generating a composite image in the synthesis unit 14. The synthesis unit 14 synthesizes the image region P2 of the pseudo-defective area with the good product image G1 to generate a composite image D1. By performing synthesis for each of the multiple image regions P1 to P4 of the multiple pseudo-defective areas, the synthesis unit 14 obtains multiple composite images that have defects of various areas.

[0020] The generation unit 13 and the synthesis unit 14 may generate the composite image using a method different from the method described above. For example, the generation unit 13 can generate multiple masks using the image region R of the defective area. The generation unit 13 can generate multiple image regions P1 to P4 of pseudo-defective areas by extracting the defective area from the defective product image G2 using each of the multiple masks. In this case, each of the multiple image regions P1 to P4 of pseudo-defective areas will have a smaller area than the image region R of the defective area. The synthesis unit 14 can synthesize each of the multiple image regions P1 to P4 of pseudo-defective areas extracted from the defective product image G2 with the good product image G1.

[0021] The image output unit 15 outputs multiple composite images to the judgment device 20. The judgment device 20 inputs the composite images to the model M2 and outputs a label L1. Label L1, like label L2, is information that the model M2 determines to be either a good product or a defective product based on the area of ​​defects in the composite image. The judgment device 20 outputs a label L1 corresponding to each of the multiple composite images.

[0022] The result acquisition unit 16 acquires labels L1 corresponding to each of the multiple composite images from the judgment device 20. As a result, for each composite image, the feature quantities of the image region of the pseudo-defect area in the composite image are associated with the label L1 corresponding to the composite image.

[0023] The display control unit 17 causes the display device to display objects representing the label L1 corresponding to each of the multiple composite images in an array based on feature quantities. The objects can be figures, icons, symbols, etc. The display control unit 17 may also display the multiple composite images and objects on the display device as they are, without changing the representation of the multiple composite images. Furthermore, the display control unit 17 may also display the multiple composite images and objects on the display device that have been enlarged or reduced by a desired ratio, without changing the representation of the multiple composite images. The display device is a display, etc.

[0024] Figure 3 shows an example of displaying objects representing labels corresponding to multiple composite images in an array based on feature quantities. The horizontal axis represents the feature quantity, indicating the area of ​​a pseudo-defect in an image of an electronic component taken to a predetermined size. The area of ​​the pseudo-defect is expressed in pixels. The objects are arranged linearly for each image pair based on the magnitude of the feature quantity. Here, objects representing labels corresponding to multiple composite images generated from eight types of image pairs DP are shown for pairs 1 to 8. If label L1 is a good product label, the object is displayed as a white circle. If label L1 is a defective product label, the object is displayed as a black circle. For example, in the array for pair 1, composite image D1 corresponding to the first four objects from the left is assigned a good product label. Composite image D1 corresponding to the fifth object from the left onward is assigned a defective product label. The composite image D1 corresponding to the object displayed on the left side of Figure 3 has a small area of ​​missing electronic component and is close to a good product image. The composite image D1, which corresponds to the object displayed on the right side of Figure 3, has a large area of ​​defects and is similar to the image of a defective product.

[0025] In Figure 3, the threshold for determining the label L1 of the judgment device 20 is visualized with respect to the feature quantity of the area of ​​the pseudo-defective area. For example, it can be seen that the threshold for pair 1 is located around the area of ​​100 pixels. Since the threshold for determining the label L1 of the judgment device 20 is visualized for each pair, it is also possible to compare the thresholds of different pairs. For example, the threshold for pair 8 is located approximately between 600 and 700 pixels. The threshold for pair 8 is significantly larger than the thresholds of pairs 1 through 7. When a pair with a significantly larger threshold than others is visualized, the annotator can display the composite image D1 corresponding to the objects before and after the threshold of pair 8 to visually confirm whether the threshold is accurate.

[0026] Figure 4 shows another example of displaying objects representing labels corresponding to multiple composite images in a feature-based array. The horizontal axis represents the first feature, indicating the area of ​​a pseudo-defect in an image of an electronic component taken to a predetermined size. The vertical axis represents the second feature, indicating the width of a pseudo-defect in an image of an electronic component taken to a predetermined size. The area and width of the pseudo-defect are expressed in pixels. For each image pair, the objects are arranged on a plane with the first and second feature axes as coordinate axes. In the example in Figure 4, objects representing labels corresponding to composite images D1 generated from six types of image pairs DP are shown as an array of pairs 11 to 16. Similar to Figure 3, if label L1 is a good product label, the object is displayed as a white object. If label L1 is a defective product label, the object is displayed as a black object.

[0027] In Figure 4, the threshold values ​​for determining the label L1 of the judgment device 20 are visualized for two feature quantities: the area and width of the pseudo-defective area. In Figure 4, by connecting the threshold values ​​of each pair with lines, the region separating good products from defective products is visualized in two dimensions. For example, the threshold value of pair 15 is small compared to other threshold values ​​when focusing only on the area. However, by visualizing the threshold values ​​as a two-dimensional region, it becomes clear that the threshold value of pair 15 is not an outlier when focusing on the width. If a threshold value is observed that protrudes from the two-dimensional region separating good products from defective products, the annotator can display a composite image D1 corresponding to the objects before and after the protruding threshold to visually confirm whether the threshold value is accurate.

[0028] [Hardware Configuration of the Analytical Device] Figure 5 is a block diagram showing the hardware configuration of the device shown in Figure 1. As shown in Figure 5, the analytical device 10 is configured as a normal computer system, including a CPU (Central Processing Unit) 101, RAM (Random Access Memory) 102, ROM 103 (Read Only Memory), a graphics controller 104, an auxiliary storage device 105, an external connection interface 106 (hereinafter referred to as "I / F"), a network I / F 107, and a bus 108.

[0029] The CPU 101 consists of arithmetic circuits and provides overall control of the analysis device 10. The CPU 101 reads programs stored in the ROM 103 or auxiliary storage device 105 into the RAM 102. The CPU 101 executes various processes in the programs read into the RAM 102. The ROM 103 stores system programs used for controlling the analysis device 10. The graphics controller 104 generates the screen to be displayed by the display control unit 17. The auxiliary storage device 105 has the function of a storage device. The auxiliary storage device 105 stores application programs that execute various processes. The auxiliary storage device 105 is composed of, for example, an HDD (Hard Disk Drive) or an SSD (Solid State Drive). The external connection I / F 106 is an interface for connecting various devices to the analysis device 10. The external connection I / F 106 is used to connect, for example, a judgment device 20, a display, a keyboard, a mouse, etc. The network interface 107 communicates with the determination device 20 and other components via the network, based on the control of the CPU 101. Each of the above-mentioned components is connected via the bus 108 to enable communication.

[0030] The analysis device 10 may have hardware other than those described above. For example, the analysis device 10 may include a GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), etc. The analysis device 10 does not need to be housed in a single enclosure as hardware, and may be separated into several devices.

[0031] The functions of the analysis device 10 shown in Figure 1 are realized by the hardware shown in Figure 5. The acquisition unit 11, extraction unit 12, generation unit 13, synthesis unit 14, image output unit 15, result acquisition unit 16, and display control unit 17 are realized by the CPU 101 executing a program stored in RAM 102, ROM 103, or auxiliary storage device 105, and processing the data stored in RAM 102, ROM 103, or auxiliary storage device 105, or the data acquired via the external connection I / F 106 or network I / F. For example, the display connected to the external connection I / F 106 displays the screen that the display control unit 17 will display. The determination device 20 shown in Figure 1 is also composed of some or all of the hardware shown in Figure 5.

[0032] [Operation of the Analytical Device] Figure 6 is a flowchart of the analysis method. The analysis method by the analytical device 10 includes an acquisition process (step S10), an extraction process (step S20), a generation process (step S30), a synthesis process (step S40), an image output process (step S50), a result acquisition process (step S60), and a display control process (step S70). The flowchart shown in Figure 6 is started, for example, based on a start instruction from the annotator.

[0033] First, the acquisition unit 11 of the analysis device 10 acquires an image pair DP consisting of a good product image G1 and a defective product image G2 as an acquisition process (step S10).

[0034] The extraction unit 12 of the analysis device 10 extracts the image region R of the defective part of the defective product contained in the defective product image G2 as an extraction process (step S20). As an example, the extraction unit 12 extracts the image region R of the defective part based on the difference between the data of the good product image G1 and the data of the defective product image G2.

[0035] The generation unit 13 of the analysis device 10 generates multiple image regions of pseudo-defective areas by changing the feature quantities of the image region R of the defective area as a generation process (step S30). As an example, the generation unit 13 generates image regions of pseudo-defective areas by enlarging or shrinking the image region of the defective area of ​​the electronic component extracted as the image region R of the defective area.

[0036] The synthesis unit 14 of the analysis device 10 performs a synthesis process (step S40) by combining the image region of the suspected defective area with the image of a good product G1 to generate multiple composite images D1 with different feature quantities. As an example, the synthesis unit 14 generates multiple composite images having suspected defective areas of various sizes that have been enlarged or reduced.

[0037] The image output unit 15 of the analysis device 10 outputs multiple composite images to the determination device 20 as part of the image output processing (step S50).

[0038] The result acquisition unit 16 of the analysis device 10 acquires labels L1 corresponding to each of the multiple composite images D1 from the judgment device 20 as a result acquisition process (step S60). As a result, for each composite image, the feature quantities of the image region of the pseudo-defect area in the composite image and the label L1 corresponding to the composite image are associated.

[0039] Finally, the display control unit 17 of the analysis device 10 performs a display control process (step S70) in which it displays objects representing the labels L1 corresponding to each of the multiple composite images D1 in an array based on feature quantities. As an example, the display control unit 17 displays objects representing at least one of the good product label and the defective product label in an array based on the area of ​​the pseudo-defective area, which is a feature quantity of the composite image D1. When the display control process (step S70) is completed, the flowchart shown in Figure 6 is completed.

[0040] [Modification] Figure 7 is a block diagram showing an example of the functions of the analysis device 10 and determination device 20 according to the embodiment. The analysis device 10 may include a tag unit 18 and a learning unit 19.

[0041] The tagging unit 18 assigns tags corresponding to the composite image D1 based on user operation. A user is a user of the analysis device 10, including an annotator. A tag, like labels L1 and L2, is information that indicates either that the composite image is good or that the composite image is defective. The tagging unit 18 has a function to accept user operation. Operation refers to the input and output of information via a user interface such as a mouse, keyboard, and display. Operation is implemented via an external connection I / F 106. Specifically, user operation refers to an instruction by the user to assign a tag, which is implemented by selection or input. The tagging unit 18 assigns a tag associated with the composite image D1. Alternatively, it may change a good product tag associated with the composite image D1 to a defective product tag, or change a defective product tag to a good product tag. The tagging unit 18 may assign multiple tags at once based on a user operation that changes a threshold. When generating a new training image without going through the determination device 20, the tagging unit 18 may newly assign a tag to the composite image D1 based on user operation.

[0042] The following example illustrates the process of tagging composite images D1 associated with the eight objects in pair 8 of Figure 3. In the pair 8 array, the composite image D1 associated with the first object from the right is labeled as defective. Composite images D1 associated with the second and subsequent objects from the right are labeled as good. The threshold for pair 8 is approximately between 600 and 700 pixels. If the annotator checks the composite image D1 associated with the fifth object from the right and determines it to be defective, the annotator tags the composite image D1 as defective. Once the composite image D1 corresponding to the fifth object from the right in pair 8 is tagged as defective, the composite images D1 corresponding to the second through fourth objects from the right in pair 8 are also tagged as defective.

[0043] The learning unit 19 trains the determination device 20 based on the composite image D1 to which tags are assigned by the tag unit 18. As an example, the learning unit 19 trains the determination device 20 by training a model M1 that becomes the master (original) of the model M2.

[0044] The model M1 has a structure including the same neural network and parameters as the model M2. The learning unit 19 trains the model M1 using the composite image D1 to which tags are assigned by the tag unit 18 as the teacher image. The teacher image is data that is the target for the determination device 20 to determine, with the correct tag associated with the subject. The teacher image is the composite image D1 to which the correct tag is assigned by the annotator.

[0045] Hereinafter, a case will be described as an example where a label associated with the object of the pair 8 in FIG. 3 is assigned, and the composite image D1 corresponding to the pair 8 becomes the teacher image. The model M2 of the determination device 20 outputs a non-defective product label for the composite image D1 associated with the fifth object counted from the right of the pair 8. The annotator assigns a defective product tag to the composite image D1. The learning unit 19 trains the model M1 using the composite image D1 to which the defective product tag is assigned by the annotator as the teacher image. The model M1 learns that the composite image D1 associated with the fifth object counted from the right of the pair 8 is determined to be defective. The learning result of the model M1, which is the master, is reflected in the model M2 of the determination device 20.

[0046] The analysis device 10 may select the composite image D1 as the teacher image based on the effect of training the model M1. The effect of training the model M1 is the accuracy with which the improved model M1 outputs the correct label L1 by the learning unit 19. The effect of training the model M1 is determined based on a predetermined evaluation value.

[0047] A predetermined evaluation value is calculated based on the overall features of the good product image G1 that constitutes the image pair DP, the overall features of the defective product image G2 that constitutes the image pair DP, and the overall features of the composite image D1 generated from the image pair DP. The overall features may be the result of a calculation at the layer one level before the output layer, after inputting an image to model M1 or model M2. Here, we will illustrate the case where a good product label is assigned to the composite image D1. In this case, the predetermined evaluation value is determined by considering the degree to which the overall features of the composite image D1 with the good product label are similar to the overall features of the good product image G1 (first evaluation value) and the degree to which the overall features of the composite image D1 with the good product label are not similar to the overall features of the defective product image G2 (second evaluation value). The degree of similarity may be evaluated by the distance between the two in an n-dimensional feature space on which the overall features can be plotted. The first evaluation value can be made larger, for example, the shorter the distance between the overall features of the composite image D1 with the good product label and the overall features of the good product image G1. The second evaluation value can be increased, for example, as the distance between the overall feature quantities of the composite image D1 with the good product label and the overall feature quantities of the defective product image G2 increases. The predetermined evaluation value can be determined by a combination (e.g., a ratio) of the first evaluation value and the second evaluation value. The predetermined evaluation value for the composite image D1 with the defective product label can also be calculated by determining the first evaluation value and the second evaluation value, and then using a combination of the first and second evaluation values, similar to the example of the good product label.

[0048] Hereinafter, a case where the acquisition unit 11 selects an image pair DP in combination from an image group including a good product image and a defective product image acquired in advance will be described as an example. The learning unit 19 learns the model M1 using the composite image D1 to which a tag is assigned by the tag unit 18 as a teacher image. The learning unit 19 inputs the composite image D1 and the image pair DP which is the original data of the composite image D1 into the model M1 and calculates a predetermined evaluation value. If it is determined that the learning of the model M1 is sufficient based on the predetermined evaluation value, the learning unit 19 ends the learning. The determination as to whether the learning of the model M1 is sufficient is made, for example, based on the magnitude relationship between the predetermined evaluation value and a predetermined threshold value. If it is not determined that the learning of the model M1 is sufficient, the learning unit 19 learns the model M1 based on a composite image D1 generated from another combination of image pairs DP. The learning unit 19 learns the model M1 until it is determined that the learning of the model M1 is sufficient based on the predetermined evaluation value.

[0049] [Summary of Embodiment] According to the analysis device 10, an image pair DP of a good product image G1 having a good product as a subject and a defective product image G2 having a defective product as a subject is acquired by the acquisition unit 11. An image area R of a defective part of the defective product based on the image pair DP is extracted by the extraction unit 12. A plurality of image areas P1 to P4 of pseudo-defective parts with the feature amount of the image area R of the defective part changed are generated by the generation unit 13. Each of the plurality of image areas P1 to P4 of pseudo-defective parts and the good product image G1 are combined, and a plurality of composite images with different feature amounts are generated by the composite unit 14. The plurality of composite images are output to the determination device 20 by the image output unit 15. A label L1 corresponding to each of the plurality of composite images output from the determination device 20 is acquired by the result acquisition unit 16. An object indicating the label L1 corresponding to each of the plurality of composite images is displayed by the display control unit 17 in an array based on the feature amount. Thus, the analysis device 10 can visualize the accuracy of the determination device 20.

[0050] The tag unit 18 assigns a tag corresponding to the composite image D1 based on a user operation. In this case, the analysis device 10 can generate an accurate teacher image based on a user operation.

[0051] The learning unit 19 trains the judgment device 20 based on the composite image D1 to which tags have been assigned by the tagging unit 18. In this case, the analysis device 10 can cause the judgment device 20 to retrain with training images that reflect the user's intentions, based on user operation.

[0052] While embodiments of the present disclosure have been described above, the present disclosure is not limited to the embodiments described above. In the embodiments described above, the analysis device 10 and the determination device 20 are described in a configuration in which they are physically or logically separated, but the analysis device 10 and the determination device 20 may be integrated and physically or logically unified. In other words, the analysis device 10 may be configured to include the determination device 20. Model M2 is not limited to cases in which a neural network and parameters are included, but may also be a model learned by a support vector machine. Alternatively, model M2 may be a current rule-based classifier. Furthermore, model M2 is not limited to a neural network and a support vector machine, but is not limited to the above as long as it can make a judgment on an input, such as a decision tree.

[0053] 10...Analysis device, 11...Acquisition unit, 12...Extraction unit, 13...Generation unit, 14...Synthesis unit, 15...Image output unit, 16...Acquisition unit, 17...Display control unit, 18...Tagging unit, 19...Learning unit, 20...Determination device.

Claims

1. An analysis device that visualizes the accuracy of a trained judgment device that uses an image of an article as a subject to output either a good product label indicating that the article is a good product or a defective product label indicating that the article is a defective product, the analysis device comprising: an acquisition unit that acquires image pairs of a good product image of the good product as a subject and a defective product image of the defective product as a subject; an extraction unit that extracts image areas of defective parts of the defective product based on the image pairs; a generation unit that generates a plurality of image areas of pseudo-defective parts by changing feature amounts of the image areas of the defective parts; a synthesis unit that synthesizes each of the image areas of the plurality of pseudo-defective parts with the good product image to generate a plurality of synthesized images with different feature amounts of the pseudo-defective parts; an image output unit that outputs the plurality of synthesized images to the judgment device; a result acquisition unit that acquires labels corresponding to each of the plurality of synthesized images from the judgment device; and a display control unit that displays objects indicating the labels corresponding to each of the plurality of synthesized images in an arrangement based on the feature amounts.

2. The analysis device according to claim 1, wherein the display control unit displays the plurality of composite images and the object.

3. The analysis device according to claim 1, further comprising a tagging unit that assigns a corresponding tag to the composite image based on a user operation.

4. The analysis device according to claim 3, further comprising a learning unit that causes the determination device to learn based on the composite image and the tag corresponding to the composite image.

5. An analytical method for visualizing the accuracy of a trained judgment device that uses an image of an article as its subject to output either a good product label indicating that the article is a good product or a defective product label indicating that the article is a defective product, the analytical method comprising the steps of: acquiring an image pair of a good product image of the good product as its subject and a defective product image of the defective product as its subject; extracting an image area of ​​a defective part of the defective product based on the image pair; generating a plurality of image areas of pseudo-defective parts by changing a feature amount of the image area of ​​the defective part; combining each of the image areas of the plurality of pseudo-defective parts with the good product image to generate a plurality of composite images with different feature amounts of the pseudo-defective parts; outputting the plurality of composite images to the judgment device; acquiring a label corresponding to each of the plurality of composite images from the judgment device; and displaying an object indicating the label corresponding to each of the plurality of composite images in an arrangement based on the feature amount.