Data creation system, data creation method, and program

Abstract

The purpose of the present disclosure is to improve the accuracy of training data. This data creation system (5) comprises a first image acquisition unit (51), a second image acquisition unit (52), a division unit (71), a range generation unit (72), and a creation unit (73). The first image acquisition unit (51) acquires a first image (P1) pertaining to a first object (1) including a specific part (E1). A second image acquisition unit (52) acquires a second image (P2) pertaining to a second object (2). The division unit (71) divides at least one of the first image (P1) and the second image (P2) into a plurality of areas (3). The range generation unit (72) generates one or a plurality of range patterns (Q1) on the basis of the division result from the division unit (71). The creation unit (73) superposes the specific part (E1) on the second image (P2), creates one or a plurality of superposed images (P4), and outputs the superposed result as the training data on the basis of at least one range pattern (Q1) among one or a plurality of range patterns (Q1).

Description

[Technical Field]

[0001] The present disclosure generally relates to a data creation system, a data creation method, and a program. More particularly, the present disclosure relates to a data creation system, a data creation method, and a program related to data creation for machine learning. [Background technology]

[0002] Patent Document 1 discloses a data generation device that generates training data. This data generation device includes an acquisition unit that acquires an image of an object to be inspected, an input unit that accepts a designation of a partial image including a detection target portion, and a correction unit that corrects the partial image based on a feature amount of the detection target portion. The data generation device also includes a generation unit that generates a composite image by combining the corrected partial image with an image different from the image including the partial image, and generates new training data for training a classifier.

[0003] This data generation device also generates a composite image by combining partial images at locations on the object being inspected that are statistically likely to have defects, and identifies locations where the background patterns around defects are similar between the source and destination images.

[0004] The data generation device in Patent Document 1 statistically determines locations where defects (failures) are likely to occur, so if there is little defective data, the determination accuracy will be poor, and if the similarity between the source image and the destination image is low, there is a possibility that the synthesis location cannot be identified. As a result, the accuracy of the learning data may decrease. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2019-109563 Summary of the Invention

[0006] The present disclosure has been made in view of the above-mentioned circumstances, and aims to provide a data creation system, a data creation method, and a program that can improve the accuracy of learning data.

[0007] A data creation system according to one embodiment of the present disclosure creates training data for generating a trained model that identifies a specific region. The data creation system includes a first image acquisition unit, a second image acquisition unit, a division unit, a range generation unit, and a creation unit. The first image acquisition unit acquires a first image of a first object that includes the specific region. The second image acquisition unit acquires a second image of a second object. The division unit divides at least one of the first image and the second image into a plurality of regions. The range generation unit generates one or more range patterns based on the division result by the division unit. The creation unit creates one or more superimposed images by superimposing the specific region on the second image based on at least one of the one or more range patterns, and outputs the superimposed images as the training data. The plurality of regions includes a specific region where the specific part is located. The data creation system further includes an extraction unit that extracts one or more shape-similar regions from the first image or the second image, the shape of which is highly similar to that of the specific region. The range generation unit generates a range pattern that includes the specific region and the one or more shape-similar regions as one of the one or more range patterns. A data creation system according to one embodiment of the present disclosure creates training data for generating a trained model that identifies specific regions. The data creation system includes a first image acquisition unit, a second image acquisition unit, a division unit, a range generation unit, and a creation unit. The first image acquisition unit acquires a first image of a first object including the specific region. The second image acquisition unit acquires a second image of a second object. The division unit divides at least one of the first image and the second image into multiple regions. The range generation unit generates one or more range patterns based on the division result by the division unit. The creation unit creates one or more superimposed images by superimposing the specific region on the second image based on at least one of the one or more range patterns, and outputs the superimposed images as the training data. The multiple regions include a specific region in which the specific region is located. The data creation system further includes an extraction unit that extracts one or more pixel similar regions from the first image or the second image, the pixel similar regions including multiple pixels having pixel values ​​highly similar to pixel values ​​of multiple pixels in the specific region. The range generation unit generates a range pattern including the specific region and the one or more pixel similar regions as one of the one or more range patterns. A data creation system according to one embodiment of the present disclosure creates training data for generating a trained model that identifies a specific region. The data creation system includes a first image acquisition unit, a second image acquisition unit, a division unit, a range generation unit, and a creation unit. The first image acquisition unit acquires a first image of a first object including the specific region. The second image acquisition unit acquires a second image of a second object. The division unit divides at least one of the first image and the second image into a plurality of regions. The range generation unit generates one or more range patterns based on the division result by the division unit. The creation unit creates one or more superimposed images by superimposing the specific region on the second image based on at least one of the one or more range patterns, and outputs the superimposed images as the training data. The plurality of regions includes a specific region in which the specific region is located. The data creation system further includes an extraction unit that extracts from the first image or the second image one or more balanced similar regions that include multiple pixels that exhibit a balance of pixel values ​​highly similar to the balance of pixel values ​​across multiple pixels of the specific region. The range generation unit generates a range pattern that includes the specific region and the one or more balanced similar regions as one of the one or more range patterns.

[0008] A data creation system according to one aspect of the present disclosure creates training data for generating a trained model that identifies specific body parts. The data creation system includes a body part acquisition unit, an image acquisition unit, a division unit, a range generation unit, and a creation unit. The body part acquisition unit acquires information about the specific body part. The image acquisition unit: does not contain the specific site An object image relating to an object is acquired. The division unit divides the object image into a plurality of regions. The range generation unit generates one or more range patterns in the object image based on the division result by the division unit. The creation unit creates one or more superimposed images by superimposing the specific portion on the object image based on at least one of the one or more range patterns, and outputs the superimposed images as the learning data.

[0009] A data creation method according to one embodiment of the present disclosure creates learning data for generating a trained model that identifies a specific region. The data creation method includes a first image acquisition process, a second image acquisition process, a segmentation process, a range generation process, and a creation process. The first image acquisition process acquires a first image of a first object that includes the specific region. The second image acquisition process acquires a second image of a second object. The segmentation process divides at least one of the first image and the second image into a plurality of regions. The range generation process generates one or more range patterns based on the segmentation result of the segmentation process. The creation process creates one or more superimposed images by superimposing the specific region on the second image based on at least one of the one or more range patterns, and outputs the superimposed images as the learning data. The plurality of regions includes a specific region where the specific part is located. The data creation method further includes an extraction process of extracting one or more shape-similar regions, the shape of which is highly similar to that of the specific region, from the first image or the second image. The range generation process generates a range pattern including the specific region and the one or more shape-similar regions as one of the one or more range patterns. A data creation method according to one embodiment of the present disclosure creates training data for generating a trained model that classifies specific regions. The data creation method includes a first image acquisition process, a second image acquisition process, a segmentation process, a range generation process, and a creation process. The first image acquisition process acquires a first image of a first object including the specific region. The second image acquisition process acquires a second image of a second object. The segmentation process divides at least one of the first image and the second image into multiple regions. The range generation process generates one or more range patterns based on the segmentation results of the segmentation process. The creation process creates one or more superimposed images by superimposing the specific region on the second image based on at least one of the one or more range patterns, and outputs the superimposed images as the training data. The multiple regions include a specific region in which the specific region is located. The data creation method further includes an extraction process of extracting one or more pixel similar regions from the first image or the second image, the pixel similar regions including multiple pixels having pixel values ​​highly similar to pixel values ​​of multiple pixels in the specific region. In the range generation process, a range pattern including the specific region and the one or more pixel similar regions is generated as one of the one or more range patterns. A data creation method according to one embodiment of the present disclosure creates training data for generating a trained model that identifies a specific region. The data creation method includes a first image acquisition process, a second image acquisition process, a segmentation process, a range generation process, and a creation process. The first image acquisition process acquires a first image of a first object that includes the specific region. The second image acquisition process acquires a second image of a second object. The segmentation process divides at least one of the first image and the second image into multiple regions. The range generation process generates one or more range patterns based on the segmentation results of the segmentation process. The creation process creates one or more superimposed images by superimposing the specific region on the second image based on at least one of the one or more range patterns, and outputs the superimposed images as the training data. The multiple regions include a specific region in which the specific region is located. The data creation method further includes an extraction process for extracting from the first image or the second image one or more balanced similar regions including a plurality of pixels that exhibit a balance of pixel values ​​highly similar to the balance of pixel values ​​across the plurality of pixels in the specific region. In the range generation process, a range pattern including the specific region and the one or more balanced similar regions is generated as one of the one or more range patterns.

[0010] A data creation method according to one aspect of the present disclosure creates learning data for generating a trained model that identifies specific regions. The data creation method includes a region acquisition process, an image acquisition process, a segmentation process, a range generation process, and a creation process. In the region acquisition process, information about the specific region is acquired. In the image acquisition process, does not contain the specific site An object image relating to an object is obtained. In the segmentation process, the object image is segmented into a plurality of regions. In the range generation process, one or more range patterns are generated in the object image based on the segmentation results of the segmentation process. In the creation process, the specific portion is superimposed on the object image based on at least one of the one or more range patterns to create one or more superimposed images, which are output as the learning data.

[0011] A program according to one aspect of the present disclosure is a program for causing one or more processors to execute any one of the data creation methods described above. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram showing the overall configuration of a data creation system according to an embodiment. [Figure 2] 2A to 2E are conceptual diagrams for explaining range setting information in the data creation system. [Figure 3] FIG. 3 is a conceptual diagram for explaining an image generated in the flow of a series of operations in the data creation system. [Figure 4] FIG. 4 is a flowchart for explaining an example of the operation of the data creation system. [Figure 5] FIG. 5 is a conceptual diagram for explaining an image generated in the flow of a series of operations in the first modification of the data creation system. [Figure 6] FIG. 6 is a conceptual diagram for explaining an image generated in the flow of a series of operations in the second modification of the data creation system. [Figure 7] FIG. 7 is a conceptual diagram for explaining an image generated in the flow of a series of operations in the third modification of the data creation system. [Figure 8] FIG. 8 is a conceptual diagram for explaining an image generated in the flow of a series of operations in the fourth modification of the data creation system. DETAILED DESCRIPTION OF THE INVENTION

[0013] (Embodiment) The data creation system 5 according to the embodiment will be described below with reference to the drawings. However, the embodiment described below is merely one of various embodiments of the present disclosure. The embodiment described below can be modified in various ways depending on the design, etc., as long as the object of the present disclosure can be achieved. Furthermore, each figure described in the embodiment described below is a schematic diagram, and the ratios of the sizes and thicknesses of the components in the figures do not necessarily reflect the actual dimensional ratios.

[0014] (overview) A data creation system 5 (see FIG. 1) according to this embodiment is configured to create learning data for generating a trained model 82 (see FIG. 1) that performs classification related to a specific portion E1 (see FIG. 3). As an example, in this embodiment, the data creation system 5 creates a superimposed image P4 from a first image P1 and a second image P2, as shown in FIG. 1. The superimposed image P4 is learning data used to generate a model by machine learning.

[0015] In the present disclosure, a "model" is a program that, when input information about an object to be identified is input, identifies the state of the object to be identified and outputs the identification result. A "trained model" refers to a model for which machine learning using training data has been completed. Furthermore, "training data (training dataset)" is a dataset that combines input information (images) input to the model with labels assigned to the input information, and is so-called teacher data. In other words, in this embodiment, the trained model 82 is a model for which machine learning using supervised learning has been completed.

[0016] The trained model 82 here may include, for example, a model using a neural network or a model generated by deep learning using a multilayer neural network. The neural network may include, for example, a convolutional neural network (CNN) or a Bayesian neural network (BNN). The trained model 82 is realized by implementing a trained neural network in an integrated circuit such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). The trained model 82 is not limited to a model generated by deep learning. The trained model 82 may also be a model generated by a support vector machine, a decision tree, or the like.

[0017] In this embodiment, as an example, the object to be identified is a welded work. Fig. 3 shows an example of a superimposed image P4 (learning data). The superimposed image P4 shows an object 4. Like the object to be identified, the object 4 is also a welded work. As shown in Fig. 3, the object 4 includes a first metal plate 41, a second metal plate 42, and a bead 43.

[0018] The bead 43 is formed at the boundary B1 (see FIG. 3: weld location) between the first metal plate 41 and the second metal plate 42 when two or more base materials (here, the first metal plate 41 and the second metal plate 42) are welded using a metal welding material. The size and shape of the bead 43 depend mainly on the welding material. Furthermore, the object 4 includes a defective portion as a specific portion E1. Hereinafter, the location where the specific portion E1 (defective portion) exists may be referred to as the location where the defect occurs.

[0019] When an image of an object to be identified (inspection image P5: see FIG. 1) is input, the trained model 82 identifies the state of the object and outputs the identification result. As an identification of the specific part E1, the trained model 82 performs, for example, identification of at least one of the presence or absence of the specific part E1 (defective part) and the type of the specific part E1.

[0020] In this embodiment, as an example, the trained model 82 outputs, as an identification result, whether the object is a good or bad product, and if it is a bad product, outputs the type of the defective part. In other words, the trained model 82 is used for determining whether an object is good or bad, in other words, for welding appearance inspection to check whether welding has been performed correctly.

[0021] Whether an object is a non-defective product is determined based on the presence or absence of various specific portions E1 (defective portions) as shown in FIGS. 2A to 2E. FIGS. 2A to 2E illustrate a first object 1 (described later), which includes a first metal plate 11, a second metal plate 12, a bead 13, and the specific portion E1 (defective portion). As examples of the specific portion E1 (defective portion), FIG. 2A schematically illustrates a pit C1 in the bead 13, FIG. 2B illustrates a spatter C2 in the bead 13, FIG. 2C illustrates a protrusion C3 in the bead 13, FIG. 2D illustrates a burn-through (hole) C4 in the bead 13, and FIG. 2E illustrates an undercut C5 in the first object 1. For example, if any one of the defective portions listed above occurs, the object is determined to be defective. Whether an object is non-defective or not can also be determined based on whether the bead length, bead height, bead rise angle, bead throat thickness, bead excess, and misalignment of the bead welding point (including misalignment of the bead start point) are within the allowable ranges. For example, if any one of the conditions listed above is not within the allowable range, the object is determined to be defective.

[0022] In order to perform machine learning of a model, it is necessary to prepare a large number of image data, including defective products, as training data. However, if the frequency of defective products on an object production line is low, the training data required to generate a trained model 82 with a high classification rate is likely to be insufficient. Therefore, it is conceivable to perform data augmentation on training data (first image P1 and second image P2) obtained by actually capturing images of the object, thereby increasing the amount of training data and performing machine learning of the model. Data augmentation refers to a process of increasing the amount of training data by performing processes such as synthesis, translation, enlargement / reduction, rotation, inversion, or noise addition on the training data. In this embodiment, the data augmentation process is performed to create one or more (in the inventors' estimation, a large number) superimposed images P4, and each of the one or more superimposed images P4 is used as training data. In this embodiment, the superimposed image P4 is generated by performing a process of superimposing (combining) multiple original images on multiple original images (multiple training data) before they are processed by the data creation system 5.

[0023] It is not necessary that the multiple original images (first image P1 and second image P2) used to create the superimposed image P4 be used as training data for generating the trained model 82. In other words, the training data for generating the trained model 82 may include only the superimposed image P4, or may further include, in addition to the superimposed image P4, images not generated by the data creation system 5 (such as the first image P1 and the second image P2). In other words, the training data for generating the trained model 82 may or may not include the original image before it is processed by the data creation system 5. Furthermore, the training data for generating the trained model 82 may include images generated by a system other than the data creation system 5.

[0024] As shown in FIG. 1, the data creation system 5 of this embodiment includes a first image acquisition unit 51, a second image acquisition unit 52, a division unit 71, a range generation unit 72, and a creation unit 73.

[0025] The first image acquisition unit 51 acquires a first image P1 (e.g., a defective product image) of a first object 1 including a specific portion E1 (e.g., a defective portion). In this embodiment, it is assumed that the first image P1 is a captured image obtained by actually capturing an image of the first object 1 with an imaging device (imaging system). However, the first image P1 is not limited to a captured image, and may be an image obtained by performing image conversion processing (such as synthesis, translation, enlargement / reduction, rotation, inversion, or adding noise) on the captured image, or a CG image.

[0026] The second image acquisition unit 52 acquires a second image P2 (for example, a non-defective image) of the second object 2. In this embodiment, it is assumed that the second image P2 is a captured image obtained by actually capturing an image of the second object 2 with an imaging device (imaging system). However, the second image P2 is not limited to a captured image, and may be an image obtained by performing image conversion processing (such as synthesis, translation, enlargement / reduction, rotation, inversion, or adding noise) on the captured image, or a CG image.

[0027] The dividing unit 71 divides at least one of the first image P1 and the second image P2 into a plurality of regions. In the present embodiment, as an example, the dividing unit 71 divides the first image P1 into a plurality of regions 3 (see FIG. 3). In the present embodiment, the dividing unit 71 extracts a pixel region of the specific portion E1, and in the pixel region excluding the pixel region of the specific portion E1, identifies pixel regions of the first metal plate 11, the second metal plate 12, and the bead 13, and divides the image into three regions 3.

[0028] The range generating unit 72 generates one or more range patterns Q1 (i.e., one or more range candidate data) based on the division result by the dividing unit 71. For example, the range generating unit 72 predicts (generates) one or more range patterns Q1 from the three regions 3 of the first metal plate 11, the second metal plate 12, and the bead 13, and the occurrence location of the specific portion E1 (the location where the defect occurs).

[0029] The creation unit 73 creates one or more superimposed images P4 by superimposing the specific portion E1 on the second image P2 based on at least one range pattern Q1 from the one or more range patterns Q1, and outputs the superimposed images P4 as learning data. As an example, in this embodiment, it is assumed that the data creation system 5 displays one or more range patterns Q1 on the display device 58 (see FIG. 1) and allows the user U1 (see FIG. 3) to select at least one range pattern Q1 from the one or more range patterns Q1. However, the creation unit 73 may automatically select at least one range pattern Q1 from the one or more range patterns Q1.

[0030] According to this embodiment, an image (superimposed image P4) suitable for machine learning can be created. In particular, the specific portion E1 is superimposed on the second image P2 based on the range pattern Q1. Therefore, unlike the data generation device of Patent Document 1, for example, the location where the specific portion E1 is likely to exist (location where defects are likely to occur) is not statistically determined, so the accuracy of the learning data is less likely to deteriorate due to the small number of images (defective product images) containing the specific portion E1. Furthermore, unlike the data generation device of Patent Document 1, the location to be combined is not identified based on the similarity between the source image and the destination image, so the accuracy of the learning data is less likely to deteriorate. As a result, the data creation system 5 according to this embodiment has the advantage of being able to improve the accuracy of the learning data.

[0031] Furthermore, functions similar to those of the data creation system 5 can be embodied in a data creation method. The data creation method of this embodiment creates learning data for generating a trained model 82 that performs classification related to the specific portion E1. The data creation method includes a first image acquisition process, a second image acquisition process, a division process, a range generation process, and a creation process. The first image acquisition process acquires a first image P1 related to a first object 1 including the specific portion E1. The second image acquisition process acquires a second image P2 related to a second object 2. The division process divides at least one of the first image P1 and the second image P2 into multiple regions. The range generation process generates one or more range patterns Q1 based on the division results of the division process. The creation process superimposes the specific portion E1 on the second image P2 based on at least one range pattern Q1 of the one or more range patterns Q1 to create one or more superimposed images P4, which are output as learning data.

[0032] Furthermore, the data creation method is used on a computer system (data creation system 5). In other words, the data creation method can be embodied as a program. The program according to this embodiment is a program for causing one or more processors (of the computer system) to execute the data creation method according to this embodiment. The program may be recorded on a non-transitory recording medium readable by the computer system.

[0033] (detail) The data creation system 5 according to this embodiment will be described in more detail below.

[0034] (1) Overall structure 1 includes a computer system having one or more processors and a memory. At least some of the functions of the data creation system 5 are realized by the processor of the computer system executing a program recorded in the memory of the computer system. The program may be recorded in the memory, or may be provided via a telecommunications line such as the Internet, or may be provided by being recorded on a non-transitory recording medium such as a memory card.

[0035] The data creation system 5 may be installed within a factory that will be the welding site, or at least a portion of the configuration of the data creation system 5 may be installed outside the factory (for example, in a business establishment located in a different location from the factory).

[0036] As described above, the data creation system 5 has a function of performing data augmentation processing on original images (learning data) to increase the number of pieces of learning data. Hereinafter, a person who uses the data creation system 5 may be simply referred to as a "user U1" (see FIG. 3). User U1 is, for example, an operator who monitors manufacturing processes such as welding processes in a factory, or a manager.

[0037] In this embodiment, as described above, the trained model 82 is generated by machine learning. The trained model 82 may be implemented as any type of artificial intelligence or system. Here, the machine learning algorithm is, for example, a neural network. However, the machine learning algorithm is not limited to a neural network and may be, for example, eXtreme Gradient Boosting (XGB) regression, Random Forest, decision tree, logistic regression, support vector machine (SVM), Naive Bayes classifier, k-nearest neighbors, or the like. Furthermore, the machine learning algorithm may be, for example, a Gaussian Mixture Model (GMM), k-means clustering, or the like.

[0038] Furthermore, the trained model 82 is not limited to machine learning that classifies inspection images into two classes, good and bad, but may be generated by machine learning that classifies inspection images into multiple classes, such as good, pits C1, spatters C2, protrusions C3, burn-through C4, and undercuts C5. The trained model 82 may also be generated by machine learning that performs object detection or segmentation that detects the positions and types of defective parts in inspection images.

[0039] In addition, in the embodiment, the learning method is, for example, supervised learning, but the learning method is not limited to supervised learning and may be unsupervised learning or reinforcement learning.

[0040] 1, the data creation system 5 includes a storage unit 63 for storing (memorizing) the learning data. The storage unit 63 includes a rewritable nonvolatile memory such as an EEPROM (Electrically Erasable Programmable Read-Only Memory). The storage unit 63 may be a memory built into the data creation system 5. However, the storage unit 63 may also be provided outside the data creation system 5.

[0041] 1, the data creation system 5 includes a first image acquisition unit 51, a second image acquisition unit 52, an image processing unit 53, an input unit 54, a user interface 55, a setting information generation unit 56, a display output unit 57, and a display device 58. The data creation system 5 further includes an image output unit 59, a learning unit 60, a determination unit 61, an inspection image acquisition unit 62, and a storage unit 63.

[0042] At least the user interface 55, the display device 58, and the storage unit 63 are tangible components. On the other hand, the first image acquisition unit 51, the second image acquisition unit 52, the image processing unit 53, the input unit 54, the setting information generation unit 56, the display output unit 57, the image output unit 59, the learning unit 60, the determination unit 61, and the inspection image acquisition unit 62 merely indicate functions realized by one or more processors of the data creation system 5, and do not necessarily indicate tangible components.

[0043] (2) First image acquisition unit The first image acquisition unit 51 acquires a first image P1 of a first object 1 including a specific portion E1. In the present embodiment, as an example, the trained model 82 is used for a welding appearance inspection to check whether welding has been performed correctly, and therefore the specific portion E1 is a defective portion. However, the specific portion E1 is not limited to a defective portion. For example, if the trained model 82 is used to inspect whether an object is painted or plated, the specific portion E1 may be a painted portion or a plated portion, etc.

[0044] 3, the first image P1 is an image captured of the first object 1. The first image acquisition unit 51 may acquire the first image P1 from a device external to the data creation system 5, or may acquire the first image P1 from the storage unit 63 of the data creation system 5. In the example shown in FIG. 1, the first image acquisition unit 51 acquires the first image P1 from a device external to the data creation system 5. For example, the first image acquisition unit 51 acquires the first image P1 from a computer server.

[0045] Furthermore, the imaging device that captures the image of the first object 1 and generates the first image P1 may be provided in the data creation system 5 or may be provided outside the data creation system 5.

[0046] The first object 1 is an object that includes a defective portion. That is, the first object 1 is a defective product. A defective product is an item in which a defect has occurred. What state of an item is considered to be a state in which a defect has occurred can be appropriately determined by the user U1 of the data creation system 5.

[0047] The first image P1 is an image (defective product image) of at least the location where the defect occurred (specific location E1) of the first object 1. The first object 1 includes the first metal plate 11, the second metal plate 12, and the bead 13. It is preferable that the area captured in the first image P1 includes not only the location where the defect occurred, but also locations on the first object 1 other than the location where the defect occurred, for example, the surrounding area of ​​the location where the defect occurred. In the example shown in FIG. 3, the area captured in the first image P1 includes the entire bead 13 and the first metal plate 11 and the second metal plate 12 around the bead 13 so as to include the specific location E1.

[0048] In this embodiment, the first object 1 is an article formed by welding two or more base materials (here, a first metal plate 11 and a second metal plate 12). In other words, the first object 1 is a welded product. Types of defects that occur in welded products include, for example, pits, spatters, protrusions, burn-through (holes), and undercuts.

[0049] Pits are depressions that occur when bubbles generated in the weld metal rise to the surface, and occur on the surface of the bead 13. Spatter is metal particles or slag that fly off during welding, and is a spherical, conical, or other protrusion that occurs on the surface of the bead 13 and its surrounding surface. Protrusions are cylindrical protrusions that occur on the surface of the bead 13. Burn-through (holes) is when part of the bead 13 melts through and is lost. Undercut is a depression that occurs around the bead 13.

[0050] The specific portion E1 includes at least a portion of a defect related to at least one of pits, spatters, protrusions, burn-through (holes), and undercuts that occur in the weldment. In other words, the specific portion E1 may include not only one type of defect, but also two or more types of defect at the same time. Furthermore, the specific portion E1 is not limited to including the entirety of a certain defect, but may include a portion of it (for example, only a portion of an undercut). As an example, in this embodiment, it is assumed that the specific portion E1 is one type of defect and includes the entirety of that defect.

[0051] Furthermore, the specific portion E1 is not limited to a defective portion, and may be a portion where no defect occurs. Alternatively, the specific portion E1 may include a defective portion and a portion adjacent to the defective portion. In particular, when the defect occurring in the first object 1 is a defect occurring in a narrow range (such as a pit, spatter, or protrusion), it is preferable that the specific portion E1 include not only one defective portion, but also a portion adjacent to the defective portion or another defective portion. This ensures the length of the specific portion E1, and makes image processing related to the specific portion E1 more meaningful.

[0052] 2A to 2E show pits C1, spatters C2, protrusions C3, burn-through C4, and undercuts C5, which may correspond to specific region E1. In FIG. 3, specific region E1 is schematically shown as a black circle. In the example of FIG. 3, specific region E1 is located at the lower end in the longitudinal direction of the surface of elongated bead 13. Note that, for example, if the defective region is an undercut, the gradation of specific region E1 corresponds to the depth of the undercut.

[0053] (3) Second image acquisition unit The second image acquisition unit 52 acquires a second image P2 relating to the second object 2. As shown in FIG. 3, the second image P2 is an image captured of the second object 2. The second image acquisition unit 52 may acquire the second image P2 from a device external to the data creation system 5, or may acquire the second image P2 from the storage unit 63 of the data creation system 5. In the example shown in FIG. 1, the second image acquisition unit 52 acquires the second image P2 from a device external to the data creation system 5. For example, the second image acquisition unit 52 acquires the second image P2 from a computer server.

[0054] Furthermore, the imaging device that captures the image of the second object 2 and generates the second image P2 may be provided in the data creation system 5 or may be provided outside the data creation system 5.

[0055] The second object 2 is an object that does not include a defective portion. In other words, the second object 2 is a non-defective product. A non-defective product is an item that does not have any defects. The second object 2 includes a first metal plate 21, a second metal plate 22, and a bead 23. In the example shown in FIG. 3, the area captured in the second image P2 includes the entire bead 23 and the first metal plate 21 and the second metal plate 22 around the bead 23.

[0056] The second image P2 is the image that serves as the basis for the superimposed image P4. That is, the first metal plate 21, the second metal plate 22, and the bead 23 of the second object 2 correspond to the originals of the first metal plate 41, the second metal plate 42, and the bead 43 of the object 4 in the superimposed image P4. However, the superimposed image P4 includes a specific portion E1 (defective portion) that does not exist in the second image P2.

[0057] (4) First image and second image The first image P1 is, for example, a distance image including coordinate information in the depth direction (the direction from the imaging device toward the first object 1). The second image P2 is, for example, a distance image including coordinate information in the depth direction (the direction from the imaging device toward the second object 2). The coordinate information in the depth direction is represented, for example, by gradation. Specifically, the greater the density of a target point in the distance image, the further back the target point is. However, conversely, the lower the density of a target point in the distance image, the further back the target point is may also be represented.

[0058] The imaging device for generating the distance image is a distance image sensor such as a line sensor camera. The imaging device sequentially captures images of multiple objects to generate multiple images. From the multiple images generated by the imaging device, a first image P1 and a second image P2 are selected, for example, in response to an instruction from user U1. The data creation system 5 preferably includes an operation unit that accepts instructions regarding the selection. For example, a user interface 55 may be used as the operation unit.

[0059] The first image P1 and the second image P2 are not limited to distance images, but may be, for example, RGB color images.

[0060] (5) Image processing unit The image processing unit 53 is realized by, for example, a DSP (Digital Signal Processor) or an FPGA (Field-Programmable Gate Array). As shown in FIG. 1 , the image processing unit 53 includes a division unit 71, an area generation unit 72, and a creation unit 73. The image processing unit 53 executes the division process, the area generation process, and the creation process based on setting information 81. As will be described later, the setting information 81 may be input by a user U1 or may be automatically generated by the setting information generation unit 56. In this embodiment, the image generated by the creation unit 73 is called a superimposed image P4.

[0061] The dividing unit 71 performs a dividing process and divides the first image P1 into a plurality of regions 3 (see FIG. 3 ), for example, based on the pixel values ​​of a plurality of pixels in the first image P1. In this embodiment, the first image P1 includes pixel regions of the first metal plate 11, the second metal plate 12, and the bead 13. The dividing unit 71 identifies the pixel regions of the first metal plate 11, the second metal plate 12, and the bead 13, and divides the first image P1 into three regions 3 along the boundaries of these pixel regions.

[0062] 3, three regions 3 are shown: a first divided region 31, a second divided region 32, and a third divided region 33. The first divided region 31 indicates a pixel region of the first metal plate 11. The second divided region 32 indicates a pixel region of the second metal plate 12. The third divided region 33 indicates a pixel region of the bead 13.

[0063] Specifically, for example, the dividing unit 71 extracts the contours and characteristics of the specific portion E1, the first metal plate 11, the second metal plate 12, and the bead 13 from discontinuities in pixel values ​​in the first image P1 by edge detection or the like, and automatically identifies these pixel regions. Then, the dividing unit 71 identifies which region 3 among the multiple regions 3 the specific portion E1 is located in. In the example of FIG. 3, the specific portion E1 is included in the third divided region 33 (the pixel region of the bead 13). In other words, the multiple regions 3 include the specific region 3A (see FIG. 3) in which the specific portion E1 is located. In the example of FIG. 3, the third divided region 33 corresponds to the specific region 3A.

[0064] Alternatively, the dividing unit 71 may use designation information of pixel regions such as the specific portion E1 detected by the setting information generating unit 56, which will be described later. At least one pixel region (e.g., the specific portion E1) among the pixel regions of the specific portion E1, the first metal plate 11, the second metal plate 12, and the bead 13 may be designated by the user U1 via the user interface 55. In this case, the designation information of the pixel region may be included in the setting information 81.

[0065] Alternatively, the dividing unit 71 may divide the first image P1 into a plurality of regions 3 without using information about what objects appear in the first image P1. Specifically, the dividing unit 71 may divide the first image P1 into a plurality of regions 3 using an existing region division method such as a P-tile method, a mode method, a k-means method, or a region growing method, or division by a clustering method.

[0066] Furthermore, the type of the specific portion E1 (defective portion) (pit C1, sputter C2, protrusion C3, burn-through C4, or undercut C5) may be specified by the user U1 via the user interface 55. In this case, the type information of the specific portion E1 may be included in the setting information 81. Alternatively, the dividing unit 71 may automatically identify the type of the specific portion E1.

[0067] In short, the dividing unit 71 may divide the first image P1 into a plurality of regions 3 (see FIG. 3) based on the pixel values ​​in the first image P1 with reference to the setting information 81. However, when the data creation system 5 creates the superimposed image P4, information on the type of the specific portion E1 appearing in the first image P1 is not essential information.

[0068] The range generation unit 72 executes a range generation process and generates one or more (four in FIG. 3) range patterns Q1 (i.e., one or more range candidate data) based on the division results of the division unit 71. It is preferable that each of the multiple range patterns Q1 includes the specific portion E1. The number of range patterns Q1 is not particularly limited and may be one. Information on the number of range patterns Q1 to be generated may be included in the setting information 81, for example.

[0069] For example, the range generating unit 72 predicts (generates) one or more range patterns Q1 that are candidates for the defect occurrence range in the first image P1, based on the three regions 3 of the first metal plate 11, the second metal plate 12, and the bead 13 and the occurrence location of the specific portion E1 (the occurrence location of the defect). The above-mentioned "segmentation result" is assumed to relate to, for example, the relative positional relationship between the three divided regions 3 and the occurrence location of the specific portion E1 (the occurrence location of the defect) in the first image P1.

[0070] In this embodiment, because the first object 1 shown in the first image P1 is a welded workpiece, the location where the specific portion E1 occurs is narrowed down to a certain extent to the surface of the bead 13 or near the boundary between the bead 13 and the first metal plate 11 or the second metal plate 12. In particular, as shown in Figures 2A to 2E, the range where the specific portion E1 is likely to actually occur is predicted depending on the type of the specific portion E1.

[0071] Specifically, as shown in FIG. 2A, pits C1 may occur anywhere within a generation range R1 (shown by dotted hatching) that includes the entire surface of the bead 13, the vicinity of the boundary between the bead 13 and the first metal plate 11, and the vicinity of the boundary between the bead 13 and the second metal plate 12. As shown in FIG. 2B, spatters C2 may occur anywhere within a generation range R2 (shown by dotted hatching) that is slightly larger than the generation range R1 of the pit C1. As shown in FIG. 2C, protrusions C3 may occur anywhere within generation ranges R3 and R4 (shown by dotted hatching) that correspond to both ends of the elongated bead 13 in the longitudinal direction. As shown in FIG. 2D, burn-through C4 may occur anywhere within a generation range R5 (shown by dotted hatching) that corresponds to substantially the entire surface of the bead 13. As shown in Figure 2E, undercut C5 can occur anywhere in the occurrence range R6 (shown by the bold line) along the edge of the elongated bead 13, which corresponds to the boundary between the bead 13 and the first metal plate 11 and the boundary between the bead 13 and the second metal plate 12.

[0072] The occurrence ranges R1 to R6 are ranges within which a defective portion (specific portion E1) may actually occur.

[0073] In short, there may be a relatively limited positional relationship between the defect occurrence point and the bead 13, the first metal plate 11, and the second metal plate 12.

[0074] The generation ranges R1 to R6 shown in Figures 2A to 2E are merely examples and are not limited to these. Range setting information regarding the generation ranges R1 to R6 may be set in advance, or preferably can be registered and changed by the user U1 via the user interface 55. The range setting information may be included in the setting information 81. It is preferable that the setting information 81 includes correspondence information in which the type of specific portion E1 and the generation ranges R1 to R6 are associated one-to-one, one-to-many, or many-to-many.

[0075] The range generation unit 72 refers to the range setting information in the setting information 81 and predicts (generates) one or more range patterns Q1 from the relative positional relationship between the three regions 3 in the first image P1 and the occurrence location of the specific portion E1. In the example of FIG. 3, the occurrence location of the specific portion E1 is located at the bottom end of the surface of the bead 13 (the bottom end of the third divided region 33). As a result, the range generation unit 72 compares the relative positional relationship between the three divided regions 3 and the occurrence location of the specific portion E1 with the occurrence ranges R1 to R6 in the range setting information and determines one or more occurrence ranges that are likely to match. In the example of FIG. 3, the range generation unit 72 uses occurrence ranges R1, R3, R4, and R5 to generate four range patterns Q1 (Q11 to Q14).

[0076] Each range pattern Q1 is generated based on, for example, the pixel areas of the first divided region 31, the second divided region 32, the third divided region 33, and the specific portion E1 extracted from the first image P1.

[0077] The range pattern Q11 is generated using a generation range R1 that includes the entire surface of the bead 13 and the vicinity of the boundary between the bead 13 and the metal plates (11, 12). In other words, the range generation unit 72 generates the range pattern Q11, which covers the entire specific region 3A (third divided region 33), as one of one or more range patterns Q1.

[0078] The range pattern Q12 is generated by employing a generation range R4 corresponding to the vicinity of one end (second end 302 in FIG. 3 ) of both longitudinal ends (first end 301, second end 302) of the elongated bead 13. In other words, when the specific region 3A has an elongated shape (a shape elongated in the vertical direction in FIG. 3 ), the range generation unit 72 generates a range pattern Q12 that covers only one longitudinal end (second end 302) of the specific region 3A as one of one or more range patterns Q1. Although not shown in the drawings, the range generation unit 72 may also generate a range pattern Q1 that covers only the other longitudinal end (first end 301) of the specific region 3A as one of one or more range patterns Q1.

[0079] The range pattern Q13 is generated by employing generation ranges R3 and R4 corresponding to both longitudinal ends (first end 301, second end 302) of the vertically elongated bead 13. In other words, when the specific region 3A has an elongated shape (a shape elongated in the vertical direction in FIG. 3), the range generation unit 72 generates the range pattern Q13, which covers only both longitudinal ends (301, 302) of the specific region 3A, as one of one or more range patterns Q1.

[0080] The range pattern Q14 is generated by employing a generation range R5 that corresponds to substantially the entire surface of the bead 13. In other words, the range generation unit 72 generates the range pattern Q14, which covers the entire specific region 3A, as one of one or more range patterns Q1.

[0081] Although not shown in FIG. 3, the range generating unit 72 can also generate a range pattern Q1 that employs a generation range R2 (see FIG. 2) that is one size larger than the generation range R1.

[0082] Similarly, although not shown in Fig. 3, the range generation unit 72 can also generate a range pattern Q1 that employs a generation range R6 (see Fig. 2) that follows the edge of the vertically elongated bead 13. In other words, the range generation unit 72 can generate a range pattern Q1 that covers only the peripheral portion 303 (see Fig. 3) of the specific region 3A as one of the one or more range patterns Q1.

[0083] If the data creation system 5 has identified the type of specific part E1 in the first image P1, the range generation unit 72 may generate range pattern Q1 based on the type of specific part E1 in addition to the relative positional relationship. For example, the range generation unit 72 may determine an occurrence range corresponding to the type of specific part E1 from among occurrence ranges R1 to R6 based on the above-mentioned correspondence information, and generate range pattern Q1. In this case, the number of range patterns Q1 generated by the range generation unit 72 may be further narrowed down.

[0084] In the present embodiment, as an example, the data creation system 5 has a function of displaying one or more range patterns Q1 generated by the range generation unit 72 on the display device 58 (see FIG. 1). That is, the display output unit 57 (see FIGS. 1 and 3) displays one or more range patterns Q1 on the display device 58. The display output unit 57 displays one or more range patterns Q1 and prompts the user U1 to select at least one (e.g., one) range pattern Q1 from the displayed range patterns Q1. As shown in FIG. 3, the display output unit 57 includes a specific portion E1 in the same location as the first image P1 in each displayed range pattern Q1 to facilitate intuitive understanding by the user U1. In the example of FIG. 3, the specific portion E1 is illustrated as a schematic black circle. However, it is preferable that the specific portion E1 actually displayed on the display device 58 be displayed as an image of a pit, spatter, protrusion, burn-through, undercut, or the like that appears in the first image P1.

[0085] Furthermore, the display output unit 57 displays each range pattern Q1 so that the adopted generation range among the generation ranges R1 to R6 can be easily understood by the user U1 intuitively (see the ranges with light dot hatching). For example, the range pattern Q11 is displayed with the generation range R1 superimposed on the images of the first divided area 31, the second divided area 32, and the third divided area 33 so that the adopted generation range R1 can be easily understood.

[0086] Furthermore, the display output unit 57 displays not only the specific part E1 at the original position, but also one or more specific parts E1 (see dark dot-hatched circles) arranged at positions other than the original position within the generated range, so as to facilitate intuitive understanding by the user U1. Hereinafter, for ease of explanation and to facilitate distinction, the specific part E1 at the original position will be referred to as specific part E11, and one or more specific parts E1 arranged at positions other than the original position will also be referred to as specific part E12. For example, if the specific part E11 is a "protrusion," the same one or more protrusions (specific part E12) will be displayed within the adopted generated range.

[0087] The image of the specific portion E12 may be an exact copy of the image of the specific portion E11 in the first image P1, or may be an image obtained by subjecting the image of the specific portion E11 in the first image P1 to a predetermined image conversion process. The range generation unit 72 has the function of executing the predetermined image conversion process. This image conversion process may include transformation processes such as enlarging, reducing, rotating, inverting (mirroring), or adding noise to the specific portion E11. This image conversion process may also include processes such as adjusting the pixel values ​​of the specific portion E12 to eliminate incongruity with the surrounding pixel area where the specific portion E12 is located, and interpolation processes such as interpolating the pixels at the boundary between the specific portion E12 and the surrounding pixel area to smoothly connect them. This allows for the creation of a more natural-looking image of the range pattern Q1. The interpolation process may be implemented, for example, by linear interpolation.

[0088] In this embodiment, the first image P1 is a distance image, a three-dimensional image in which depth is displayed as gradation. That is, the range pattern Q1 includes information on the specific portion E1 and the depth coordinate of the first image P1 as gradation. The range generator 72 adjusts the depth coordinate by changing the gradation of the specific portion E1.

[0089] The number of specific parts E12 to be displayed is not particularly limited, but is preferably set in advance according to the size of the occurrence range (R1 to R6). Information on the number of specific parts E12 to be displayed can be included in the setting information 81, for example.

[0090] In the example of FIG. 3 , the display output unit 57 displays a total of 10 specific portions E1 (one specific portion E11 and nine specific portions E12) for the range pattern Q11 on the display device 58 in a manner that fits within the occurrence range R1. The 10 specific portions E1 are arranged at a predetermined density (e.g., at equal intervals). In other words, the display output unit 57 displays a range pattern Q1 in which multiple specific portions E1 are superimposed at a predetermined density as one or more range patterns Q1 on the display device 58. In this way, displaying multiple specific portions E1 at a predetermined density makes it easier for the user U1 to intuitively understand locations where the specific portion E1 may actually occur. This makes it easier for the user U1 to determine which range pattern Q1 is more appropriate among one or more range patterns Q1.

[0091] Also, in the example of Figure 3, the display output unit 57 displays range pattern Q12 on the display device 58 in a manner such that a total of two specific parts E1 (one specific part E11 and one specific part E12) fit within the occurrence range R4.

[0092] 3, the display output unit 57 displays range pattern Q13 on the display device 58 in such a manner that a total of three specific portions E1 (one specific portion E11 and two specific portions E12) fit within the generation ranges R3 and R4. That is, one specific portion E12 is arranged in each of the generation ranges R3 and R4.

[0093] 3, the display output unit 57 displays range pattern Q14 on the display device 58 in such a manner that a total of five specific portions E1 (one specific portion E11 and four specific portions E12) fit within the occurrence range R5. Unlike range pattern Q11, range pattern Q14 has multiple specific portions E12 arranged, for example, randomly. Random does not necessarily mean that all events occur with equal probability. For example, if the setting information 81 includes information (e.g., probability information) that a certain region 3 is particularly prone to defects, the range generation unit 72 may randomly determine the position of the specific portion E1 so that a position with a higher probability of occurrence is more likely to be selected as the position of the specific portion E1.

[0094] The input unit 54 of this embodiment accepts an instruction input for selecting at least one of the one or more displayed range patterns Q1. The user U1 selects at least one range pattern Q1, for example, using the user interface 55. The user U1 selects a range pattern Q1 that the user U1 deems appropriate based on his or her own knowledge and experience. The user U1 may also determine that none of the displayed range patterns Q1 are appropriate. The user U1 can also use the user interface 55 to instruct a correction to the range pattern Q1. That is, the input unit 54 also accepts an instruction input regarding a correction to the range pattern Q1. Possible corrections to the range pattern Q1 include, for example, changing the occurrence range to another occurrence range that is not displayed, enlarging or reducing the occurrence range, and correcting the position of the specific portion E1.

[0095] If the number of range patterns Q1 generated by the range generation unit 72 is one, the display output unit 57 displays that one range pattern Q1 and requests confirmation and an answer as to whether it is okay to create a superimposed image P4 using that one range pattern Q1. Furthermore, the number of range patterns Q1 that the user U1 can select is not particularly limited, and may be, for example, two.

[0096] The creation unit 73 executes a creation process to create, for example, multiple superimposed images P4 and output them as learning data. The creation unit 73 creates multiple superimposed images P4 by superimposing the specific portion E1 on the second image P2 based on one range pattern Q1 selected, for example, by the user U1, from one or more range patterns Q1. Note that the number of superimposed images P4 generated from one range pattern Q1 is not particularly limited. Information on the number of superimposed images P4 to be generated may be included in, for example, the setting information 81. The creation unit 73 creates one or more superimposed images P4 based on an instruction input received by the input unit 54. However, the creation unit 73 may automatically select at least one range pattern Q1 from one or more range patterns Q1 to create one or more superimposed images P4.

[0097] In the example of FIG. 3, the user U1 selects range pattern Q12 from four range patterns Q1. The creation unit 73 creates four superimposed images P4 (P41 to P44) based on the instruction input of "select range pattern Q12" received by the input unit 54. Note that in the example of FIG. 3, the user U1 not only selects range pattern Q12 using the user interface 55 but also instructs a modification to add an occurrence range R3 to range pattern Q12. As a result, the creation unit 73 randomly superimposes one or more specific portions E1 somewhere within the areas corresponding to the occurrence ranges R3 and R4 in the second image P2 to create superimposed images P41 to P44. The positions of the specific portions E1 in the four superimposed images P41 to P44 are different from one another. In Figure 3, for ease of understanding, the occurrence ranges R3 and R4 are shown in the four superimposed images P41 to P44, but the occurrence ranges R3 and R4 are not superimposed on the actual superimposed image P4 output from the creation unit 73.

[0098] When the creation unit 73 superimposes the specific portion E1 within the region corresponding to the generation ranges R3 and R4 in the second image P2, it performs, for example, image transformation processing on the specific portion E1. This image transformation processing may include transformation processing such as enlarging, reducing, rotating, inverting (mirroring), or adding noise to the specific portion E1 so that it fits the second image P2 on which it is to be superimposed. This image transformation processing may also include processing such as adjusting the pixel values ​​of the specific portion E1 so that it blends seamlessly with the surrounding pixel areas where the specific portion E1 is placed, and interpolation processing such as interpolating the pixels at the boundaries between the specific portion E1 and the surrounding pixel areas to smoothly connect them. This allows for the creation of a more natural-looking superimposed image P4. The interpolation processing is realized, for example, by linear interpolation.

[0099] In this embodiment, the superimposed image P4 is a distance image, a three-dimensional image in which depth is displayed as gradation. That is, the superimposed image P4 includes information on the specific portion E1 and the depth coordinates of the second image P2 as gradation. The creation unit 73 adjusts the depth coordinates by changing the gradation of the specific portion E1 to be superimposed.

[0100] The position of the specific portion E1 in the superimposed image P4 is preferably different from the position of the specific portion E1 in the original first image P1, but may be approximately the same. For example, the bead 13 in the first image P1 and the bead 23 in the second image P2 may be different in size or shape. Therefore, even if the position of the specific portion E1 in the superimposed image P4 and the first image P1 is approximately the same, the superimposed image P4 may not be an image that completely matches the first image P1, and thus may be valuable as learning data.

[0101] The superimposed image P4 thus created is an image (defective product image) in which the specific portion E1 is displayed.

[0102] (6) Setting information Next, a description will be given of the setting information 81 that defines the processing performed by the image processing unit 53. The setting information 81 is information related to the processing for creating the superimposed image P4.

[0103] The input unit 54 acquires the setting information 81. The input unit 54 acquires the setting information 81 from a user interface 55 that accepts an input operation of the setting information 81 by the user U1. The user interface 55 includes, for example, at least one of a mouse, a keyboard, a touchpad, and the like.

[0104] Examples of information included in the setting information 81 are listed below, but the setting information 81 is not limited to these and may include other information.

[0105] The setting information 81 may include designation information for designating at least one pixel region (e.g., the specific region E1) among the pixel regions of the specific region E1, the first metal plate 11, the second metal plate 12, and the bead 13 in the first image P1. The setting information 81 may also include type information for designating the type of the specific region E1 in the first image P1 (pit C1, sputter C2, protrusion C3, burn-through C4, or undercut C5). The setting information 81 may also include information for designating the number of range patterns Q1 to be generated. The setting information 81 may also include range setting information regarding occurrence ranges R1 to R6. The setting information 81 may also include correspondence information in which the types of the specific region E1 that may occur correspond to the occurrence ranges R1 to R6 in a one-to-one, one-to-many, or many-to-many relationship. The setting information 81 may also include information for designating the number of specific regions E12 to be displayed within the range pattern Q1 on the display device 58. The setting information 81 may also include information regarding selection and modification of the range pattern Q1 received from the user U1. The setting information 81 may also include information specifying the number of superimposed images P4 to be generated from one first image P1.

[0106] The setting information generation unit 56 generates at least a part of the setting information 81. The setting information generation unit 56 determines a pixel area of ​​the specific portion E1 in the first image P1, for example, by using a predetermined trained model for detecting the specific portion E1 (defective portion) from the first image P1.

[0107] Furthermore, the setting information generating unit 56 may assign a label to the superimposed image P4. For example, the setting information generating unit 56 may determine the label of the superimposed image P4 according to the label assigned to the first image P1. Specifically, if the label assigned to the first image P1 is "defective product," the label of the superimposed image P4 may be set to "defective product." In this case, the label may include the type of defect, and the type of defect may be the same as the type of defect in the first image P1.

[0108] (7) Display output unit and display device The display output unit 57 outputs the information to the display device 58. The display device 58 displays an image according to the information received from the display output unit 57.

[0109] The display device 58 includes a display. The display is, for example, a liquid crystal display or an organic electroluminescence (EL) display. The display device 58 displays, for example, a first image P1, a second image P2, a range pattern Q1, and a superimposed image P4. The display device 58 is used as an output interface that displays setting information 81 when the user U1 inputs setting information 81 to the user interface 55. For example, when the first image P1 is displayed on the display device 58, the user U1 can determine the range of the specific portion E1 in the first image P1 by operating the user interface 55 to move a cursor displayed on the display device 58. Furthermore, for example, when multiple range patterns Q1 are displayed on the display device 58, the user U1 can select an appropriate range pattern Q1 or modify the position or occurrence range (R1 to R6) of the specific portion E12 by operating the user interface 55 to move a cursor displayed on the display device 58. Furthermore, the display device 58 displays a determination result 83 output from a determination unit 61 (described later).

[0110] (8) Machine learning and pass / fail judgment The image output unit 59 outputs the superimposed image P4 generated by the image processing unit 53 to the learning unit 60. The learning unit 60 performs machine learning using the superimposed image P4 as learning data (learning data set). In this way, the learning unit 60 generates a trained model 82. As described above, the learning unit 60 may use not only the superimposed image P4 but also the first image P1 and the second image P2 as learning data.

[0111] The training dataset is generated by assigning labels to multiple image data indicating "good" or "defective," and in the case of defective products, the type and location of the defect. The labeling work is performed, for example, by a user U1 via a user interface 55, or by the setting information generating unit 56. The learning unit 60 generates a trained model 82 by performing machine learning on the state of the object (good state, defective state, type of defect, location of defect, etc.) using the training dataset.

[0112] The learning unit 60 may perform re-learning using a learning dataset including newly acquired learning data to improve the performance of the trained model 82. For example, if a new type of defect is found in an object, the learning unit 60 may be made to perform re-learning on the new defect.

[0113] In a factory production line, for example, an imaging device captures an image of an object and generates an inspection image P5. More specifically, the imaging device captures an image of an object on which a bead has been formed after the actual welding process, and generates the inspection image P5. The inspection image acquisition unit 62 acquires the inspection image P5 from the imaging device. The determination unit 61 uses the trained model 82 generated by the learning unit 60 to determine whether the inspection image P5 (object) acquired by the inspection image acquisition unit 62 is good or bad. If the object is defective, the determination unit 61 determines the type of defect and the location of the defect. The determination unit 61 outputs a determination result 83. The determination result 83 is output to, for example, the display device 58, which displays the determination result 83. A user U1 can check the determination result 83 via the display device 58. The production equipment may be controlled so that an object determined to be "defective" by the determination unit 61 is discarded before being transported to the next process. Furthermore, the determination result 83 is output to, for example, a data server and stored in the data server.

[0114] As described above, the data creation system 5 includes an inspection image acquisition unit 62 that acquires the inspection image P5, and a determination unit 61 that determines whether the inspection image P5 is good or bad using the trained model 82. The trained model 82 is generated based on the superimposed image P4 generated by the image processing unit 53.

[0115] (9) Operation flow Next, the flow of processing by which the data creation system 5 generates the superimposed image P4 will be described with reference to Fig. 4. Note that the flow shown in Fig. 4 is merely an example, and the order of processing may be changed as appropriate, and processing may be added or omitted as appropriate.

[0116] First, the first image acquisition unit 51 executes a first image acquisition process to acquire a first image P1 of a first object 1 including a specific portion E1 (step ST1). Next, the image processing unit 53 executes a division process to divide the first image P1 into a plurality of regions 3 (step ST2). As a result, a first divided region 31, a second divided region 32, and a third divided region 33 are obtained.

[0117] Next, the image processing unit 53 executes a range generation process to generate a plurality of range patterns Q1 (step ST3). Then, the display output unit 57 causes the display device 58 to display the generated plurality of range patterns Q1 (step ST4). The user U1 visually checks the plurality of range patterns Q1 displayed on the display device 58 and selects one (or more) range patterns Q1 that are deemed appropriate via the user interface 55. The input unit 54 accepts an instruction input by the user U1 to select one (or more) range patterns Q1. The image processing unit 53 determines the range pattern Q1 based on the instruction input accepted by the input unit 54 (step ST5).

[0118] Next, the second image acquisition unit 52 executes the second image acquisition process to acquire a second image P2 relating to the second object 2 (step ST6).

[0119] Then, the image processing unit 53 executes a creation process to superimpose the specific portion E1 on the second image P2 based on the determined range pattern Q1 to create one or more superimposed images P4 (step ST7). If the number of superimposed images P4 to be created is set, the image processing unit 53 creates superimposed images P4 until the number of superimposed images P4 to be created is reached. Thereafter, the image processing unit 53 outputs the one or more superimposed images P4 created by the creation process as learning data (step ST8) and ends the process.

[0120] The data creation system 5 may use the superimposed image P4 created by the image processing unit 53 as the first image P1 the next time another superimposed image P4 is created.

[0121] <Advantages> The data creation system 5 according to this embodiment can create an image (superimposed image P4) suitable for machine learning. In particular, the specific portion E1 is superimposed on the second image P2 based on the range pattern Q1. Therefore, unlike the data generation device of Patent Document 1, for example, the system does not statistically determine the location where the specific portion E1 is likely to exist (location where defects are likely to occur). Therefore, the accuracy of the learning data is less likely to deteriorate due to the small number of images containing the specific portion E1 (defective product images). Furthermore, unlike the data generation device of Patent Document 1, the system does not identify the compositing location based on the similarity between the source image and the destination image. Consequently, the data creation system 5 according to this embodiment has the advantage of being able to improve the accuracy of the learning data.

[0122] Furthermore, the specific part E1 is a defective part, and the first object 1 is an object that includes the defective part. The second object 2 is an object that does not include the defective part. This improves the accuracy of the training data used to generate the trained model 82 that identifies defective parts.

[0123] In particular, the data creation system 5 according to this embodiment displays the range pattern Q1 on the display device 58 and creates one or more superimposed images P4 based on the range pattern Q1 selected by the user U1. Therefore, compared to when the data creation system 5 automatically executes all processes without visual confirmation by the user U1, it is easier to generate a superimposed image P4 arranged in a location where the specific portion E1 may actually occur.

[0124] (Variation 1) A data creation system 5 according to Modification 1 will be described below with reference to Fig. 5. In the following, components that are substantially the same as those in the above embodiment will be given the same reference numerals and descriptions thereof may be omitted.

[0125] In the above embodiment, an example has been described in which the object is a weldment. The data creation system 5 can also be applied to cases in which the object is an unknown object other than a weldment. In Modification 1, the object is assumed to be an object other than a weldment, for example, a plate-shaped part with multiple protrusions (e.g., screw heads) arranged in a matrix on its surface. In Modification 1, the trained model 82 identifies defective parts that may occur in such a plate-shaped part.

[0126] Specifically, the first image P1 is, for example, a distance image. The first object 1 shown in the first image P1 includes a plate-shaped base material 14 (e.g., a metal plate) and nine protrusions 15 (e.g., metal protrusions such as screw heads) arranged in a matrix on the surface of the base material 14. As shown in FIG. 5, the surface of the base material 14 has a gently curved surface like the surface of a cylinder. The first object 1 is an object that includes a defective portion. In other words, the first object 1 is a defective product. The first image P1 is an image (defective product image) of at least a location (specific portion E1) of the first object 1 where a defect has occurred. The specific portion E1 is, for example, a scratch, a dent, or a dent that may occur on the surface of the protrusion 15. In the example of FIG. 5, the specific portion E1 is present on the surface of the lower-left protrusion 15 among the nine protrusions 15.

[0127] Furthermore, the second image P2 is, for example, a distance image. The second object 2 shown in the second image P2 is an object that does not include any defective portions. In other words, the second object 2 is a non-defective product. The second object 2 includes a plate-shaped base material 24 (e.g., a metal plate) and nine protrusions 25 (e.g., metal protrusions such as screw heads) arranged in a matrix on the surface of the base material 24. However, unlike the base material 14 of the first object 1, the surface of the base material 24 has a flat surface. In other words, the shape of the base material of the first object 1 differs from that of the second object 2.

[0128] Moreover, the superimposed image P4 is, for example, a distance image. The object 4 shown in the superimposed image P4 includes a plate-shaped base material 44 and nine protrusions 45 arranged in a matrix on the surface of the base material 44. The second image P2 is the image that forms the basis of the superimposed image P4. That is, the base material 24 and the nine protrusions 25 of the second object 2 correspond to the originals of the base material 44 and the nine protrusions 45 of the object 4 in the superimposed image P4. However, the superimposed image P4 includes a specific portion E1 (defective portion) that does not exist in the second image P2.

[0129] In Modification 1, the dividing unit 71 performs a dividing process to divide the first image P1 into a plurality of regions 3 (see FIG. 5). In Modification 1, the dividing unit 71 identifies pixel regions of the base material 14 and nine protrusions 15, and divides the first image P1 into ten regions 3 along the boundaries of these pixel regions.

[0130] In the example of FIG. 5, ten regions 3 are shown, including a first divided region 34 and nine second divided regions 35. The first divided region 34 indicates a pixel region of the base material 14. Each second divided region 35 indicates a pixel region of a corresponding one protrusion 15. The dividing unit 71 identifies which region 3 among the multiple regions 3 the specific portion E1 is located in. In the example of FIG. 5, the specific portion E1 is included in the second divided region 35 at the bottom left (the pixel region of the protrusion 15). In other words, the multiple regions 3 include a specific region 3A (see FIG. 5) in which the specific portion E1 is located. In the example of FIG. 5, the second divided region 35 at the bottom left corresponds to the specific region 3A.

[0131] Unlike the above embodiment, the data creation system 5 according to the first modification further includes an extraction unit 74 (see FIG. 5) that executes first to third extraction processes described below. The function of the extraction unit 74 is provided in the image processing unit 53 (see FIG. 1).

[0132] The extraction unit 74 executes a first extraction process to extract one or more shape-similar regions 3B, each having a shape highly similar to that of the specific region 3A, from multiple regions 3 (i.e., the first image P1). In the example of FIG. 5, the specific region 3A (the second divided region 35 at the lower left) has a circular geometric shape, and the remaining eight second divided regions 35 also have circular geometric shapes. As a result, the extraction unit 74 extracts the remaining eight second divided regions 35, each having a geometric shape highly similar to that of the specific region 3A, as shape-similar regions 3B. The shape similarity may be calculated using an appropriate method. The extraction unit 74 extracts geometric features (edges, etc.) from the specific region 3A to generate a template image. Furthermore, the extraction unit 74 performs pattern matching with the template image for each of the other regions 3 and calculates a correlation coefficient (similarity) with the template image. The extraction unit 74 designates a region 3 having a similarity higher than a predetermined value as a shape-similar region 3B. The index of similarity is not limited to the correlation coefficient (similarity), but can also be the "distance (dissimilarity)" where the smaller the distance, the higher the similarity. In this case, the area 3 where the distance is lower than the reference value is determined to be a shape-similar area 3B.

[0133] If no highly similar regions are extracted, the predetermined value (threshold value) may be adjusted, for example, to extract at least one highly similar region. The predetermined value (threshold value) is preferably changeable via the user interface 55, for example.

[0134] The extraction unit 74 also executes a second extraction process to extract one or more pixel-similar regions 3C from the multiple regions 3 (i.e., the first image P1), each of which includes a plurality of pixels having pixel values ​​highly similar to the pixel values ​​of the plurality of pixels in the specific region 3A. In the example of FIG. 5, the pixel values ​​of the plurality of pixels in the specific region 3A (the second divided region 35 at the lower left) are highly similar to the pixel values ​​of the plurality of pixels in each of the remaining eight second divided regions 35. As a result, the extraction unit 74 extracts the remaining eight second divided regions 35 as pixel-similar regions 3C. The calculation of the similarity of the pixel values ​​of the plurality of pixels may also be performed using an appropriate method. The extraction unit 74 performs pattern matching to calculate the correlation coefficient (or distance) between each pixel in the specific region 3A and other regions 3, and extracts the pixel-similar regions 3C.

[0135] The extraction unit 74 also executes a third extraction process to extract one or more balanced-similar regions 3D from the multiple regions 3 (i.e., the first image P1) that include multiple pixels exhibiting a balance of pixel values ​​highly similar to the balance of pixel values ​​across the multiple pixels of the specific region 3A. In the example of FIG. 5 , the balance of pixel values ​​across the multiple pixels in the specific region 3A (the second divided region 35 at the lower left), specifically the balance of the number of pixels from white to black, is highly similar to the balance of pixel values ​​across the multiple pixels of each of the remaining eight second divided regions 35. As a result, the extraction unit 74 extracts the remaining eight second divided regions 35 as balanced-similar regions 3D. The calculation of the similarity of the balance of pixel values ​​(balance of the number of pixels from white to black) may also be performed using an appropriate method. The extraction unit 74 performs pattern matching to calculate the correlation coefficient (or distance) between the balance of the number of pixels in the specific region 3A and the balance of the number of pixels in the other regions 3, and extracts the balanced-similar regions 3D.

[0136] In the first modification, the extraction unit 74 executes all of the first to third extraction processes, but may execute only one or two of the first to third extraction processes.

[0137] In the example of Fig. 5, the extraction results of the first to third extraction processes are completely consistent with each other. That is, the extraction unit 74 extracts the same regions 3 (the remaining eight second divided regions 35) for all of the shape-similar regions 3B, pixel-similar regions 3C, and balance-similar regions 3D. However, the extraction results of the first to third extraction processes do not necessarily completely match. If there are differences, the extraction unit 74 may input only the matching extraction results to the subsequent range generation unit 72, or may input all extraction results to the subsequent range generation unit 72 regardless of whether there are differences.

[0138] The extraction result of the extraction unit 74 may be displayed on the display device 58 via the display output unit 57 and presented to the user U1. The user U1 may visually check the extraction result displayed on the display device 58 and input some kind of instruction (for example, an answer that there is an error in the shape similar region 3B) to the input unit 54. The answer from the user received by the input unit 54 may be fed back to the extraction process of the extraction unit 74.

[0139] As a result of the first to third extraction processes, there may be no region 3 highly similar to the specific region 3A, and no shape-similar region 3B, pixel-similar region 3C, or balance-similar region 3D may be extracted. In this case, it is preferable that the display device 58 notify the user U1 via the display output unit 57 that no highly similar region 3 was found. In this case, the user U1 may stop the process of creating the superimposed image P4 and input to the input unit 54 to change at least one of the first image P1 or the second image P2 to a different image.

[0140] The range generation unit 72 executes a range generation process to generate one or more range patterns Q1 (four range patterns Q15 to Q18 in the example of FIG. 5 ) in the first image P1 based on the segmentation result by the segmentation unit 71 and the extraction result by the extraction unit 74. Specifically, the range generation unit 72 generates a range pattern Q1 including a specific region 3A and one or more shape-similar regions 3B as one of the one or more range patterns Q1. The range generation unit 72 also generates a range pattern Q1 including a specific region 3A and one or more pixel-similar regions 3C as one of the one or more range patterns Q1. The range generation unit 72 also generates a range pattern Q1 including a specific region 3A and one or more balance-similar regions 3D as one of the one or more range patterns Q1. Each of the multiple range patterns Q1 preferably includes a specific portion E1.

[0141] For example, the range generation unit 72 predicts (generates) one or more range patterns Q1 from a total of 10 regions 3 of the base material 14 and nine protrusions 15, the location where the specific part E1 occurs (the location where the defect occurs), the shape similarity region 3B, the pixel similarity region 3C, and the balance similarity region 3D.

[0142] In the above embodiment, there is a somewhat limited relative positional relationship between the location where the defect occurs and the bead 13, the first metal plate 11, and the second metal plate 12, and range setting information regarding the occurrence ranges R1 to R6 is included in advance in the setting information 81. In the first modification, there may also be a somewhat limited relative positional relationship between the location where the defect, such as a scratch, dent, or dent, occurs and the base material 14 and the nine protrusions 15 ... range setting information regarding the occurrence range of the specific portion E1, such as a scratch, dent, or dent, included in the setting information 81, or the range setting information may not be included.

[0143] The range generation unit 72 predicts (generates) one or more range patterns Q1 based on the relative positional relationship between the ten regions 3 in the first image P1 and the location where the specific portion E1 occurs (by referencing range setting information, if any, in the setting information 81). In the first modification, as described above, the extraction results by the extraction unit 74 are available, so the range generation unit 72 can predict one or more range patterns Q1 even without range setting information. In the example of FIG. 5, the location where the specific portion E1 occurs is located in the lower left protrusion 15 (lower left second divided region 35), and the remaining eight second divided regions 35 correspond to the shape-similar region 3B, pixel-similar region 3C, and balance-similar region 3D. Based on this information, the range generation unit 72 generates four range patterns Q1 (Q15 to Q18).

[0144] Each range pattern Q1 is generated based on, for example, the first divided region 34, nine second divided regions 35, and the pixel region of the specific portion E1 extracted from the first image P1.

[0145] Range pattern Q15 is a range pattern Q1 in which only the first divided region 34 (pixel region of the base material 14) is set as the range R7 (see dot hatching) in which the specific portion E1 can occur. If the defect is, for example, a scratch, it can also occur on the surface of the base material 14. Furthermore, if there are only a small number of such defective product images, they can be valuable images for learning data.

[0146] Range pattern Q16 is a range pattern Q1 in which only the specific region 3A and the entire remaining eight second divided regions 35 (shape similar region 3B, pixel similar region 3C, and balance similar region 3D) are set as the range R8 (see dot hatching) in which the specific part E1 can occur.

[0147] The range pattern Q17 is a range pattern Q1 in which only the specific region 3A and the peripheral portions of the remaining eight second divided regions 35 are set as the range R9 in which the specific portion E1 can occur.

[0148] Range pattern Q18 is a range pattern Q1 in which the range R10 (see dot hatching) in which the specific portion E1 can occur is limited to the specific region 3A and approximately the right half of each of the remaining eight second divided regions 35. Conversely, although not shown, a range pattern Q1 may be generated in which the range in which the specific portion E1 can occur is limited to approximately the left half, approximately the upper half, or approximately the lower half of each of the specific region 3A and the remaining eight second divided regions 35.

[0149] If the data creation system 5 identifies the type of specific part E1 appearing in the first image P1, the range generation unit 72 may generate the range pattern Q1 based on the type of specific part E1 in addition to the relative positional relationship.

[0150] The display output unit 57 displays a plurality of range patterns Q1 on the display device 58. As shown in FIG. 5, the display output unit 57 displays each range pattern Q1 to include a specific part E1 in the same location as the first image P1, so as to facilitate intuitive understanding by the user U1. Furthermore, the display output unit 57 displays ranges R7 to R10 in which the specific part E1 may occur, so as to facilitate intuitive understanding by the user U1. Although not shown in FIG. 5, the display output unit 57 may arrange, within the ranges R7 to R10, not only the specific part E1 at the original position, but also one or more specific parts E1 arranged at positions different from the original position.

[0151] The creation unit 73 executes a creation process to create, for example, multiple superimposed images P4 and output them as learning data. The example in FIG. 5 shows an example in which the user U1 selects range pattern Q16 from four range patterns Q1. The creation unit 73 creates four superimposed images P4 (P45 to P48) based on the instruction input of "select range pattern Q16" received by the input unit 54. The positions of the specific portions E1 in the four superimposed images P45 to P48 are different from each other. In particular, two specific portions E1 are superimposed in the superimposed image P48. The creation unit 73 superimposes the specific portion E1 in an area corresponding to range R8 in the second image P2. When superimposing the specific portion E1, the creation unit 73 performs image conversion processing on the specific portion E1. In the example of Figure 5, based on the type of the superimposed specific portion E1, in all four superimposed images P4, the specific portion E1 is positioned adjacent to the base material 44 within the pixel region of the corresponding protrusion 45.

[0152] The superimposed image P4 thus created is an image (defective product image) in which the specific portion E1 is displayed. In the first modification, the superimposed image P4 arranged in a location where the specific portion E1 may actually occur is also more likely to be generated, further improving the accuracy of the learning data.

[0153] (Variation 2) A data creation system 5 according to Modification 2 will be described below with reference to Fig. 6. In the following, Modification 2 is a further modification of the above-described Modification 1. Therefore, components that are substantially the same as those in Modification 1 may be assigned the same reference numerals and descriptions thereof may be omitted.

[0154] The first image P1 and the second image P2 may be acquired by capturing images of a first object 1 (defective product) and a second object 2 (non-defective product) while they are being transported (moved) by a transport device such as a conveyor. In this case, as shown in Fig. 6, there is a possibility that the multiple protrusions 15 (e.g., screw heads, etc.) in the first image P1 and the multiple protrusions 25 (e.g., screw heads, etc.) in the second image P2 may be captured with their positions shifted from each other.

[0155] The first image P1 in Fig. 6 includes 12 protrusions 15, but only about half of the area of ​​the three protrusions 15 on the left and the three protrusions 15 on the right are visible. On the other hand, the second image P2 in Fig. 6 includes nine protrusions 25, which is the same as the second image P2 in Modification Example 1.

[0156] In Modification 2, the dividing unit 71 performs a dividing process to divide the first image P1 into a plurality of regions 3 (see FIG. 6). In Modification 2, the dividing unit 71 identifies pixel regions of the base material 14 and the 12 protrusions 15, and divides the first image P1 into 13 regions 3 along the boundaries of these pixel regions.

[0157] In the example of FIG. 6, 13 regions 3 are shown, including a first divided region 34, six second divided regions 35, three third divided regions 36, and three fourth divided regions 37. The first divided region 34 indicates a pixel region of the base material 14. Each second divided region 35 indicates a pixel region of a corresponding one protrusion 15. Each third divided region 36 indicates a pixel region of the right half of the corresponding one protrusion 15. Each fourth divided region 37 indicates a pixel region of the left half of the corresponding one protrusion 15.

[0158] The dividing unit 71 identifies which of the multiple regions 3 the specific portion E1 is located in. In the example of FIG. 6, the specific portion E1 is included in the second divided region 35 at the bottom left (pixel region of the protrusion 15). In other words, the multiple regions 3 include the specific region 3A (see FIG. 6) where the specific portion E1 is located. In the example of FIG. 6, the second divided region 35 at the bottom left corresponds to the specific region 3A.

[0159] Furthermore, unlike the above-described embodiment and modified example 1, the dividing unit 71 performs the dividing process not only on the first image P1 but also on the second image P2. In other words, the dividing unit 71 of modified example 2 divides both the first image P1 and the second image P2 into a plurality of regions (3, 3X). The dividing unit 71 divides the base material 24 and 9 The pixel regions of the protrusions 25 are identified, and the second image P2 is divided into ten regions 3X along the boundaries of these pixel regions.

[0160] In the example of FIG. 6, a first divided region 34X and nine second divided regions 35X are shown as the ten regions 3X in the second image P2.

[0161] In Modification 2, the extraction unit 74 executes first to third extraction processes. Here, unlike Modification 1, the extraction unit 74 extracts an area that has a high similarity to the shape of the specific area 3A from the second image P2, not from the first image P1.

[0162] The extraction unit 74 executes a first extraction process to extract one or more shape-similar regions 3B, which have a shape highly similar to that of the identified region 3A, from ten regions 3X (i.e., the second image P2). In the example of FIG. 6, the identified region 3A (the second divided region 35 at the lower left) has a circular geometric shape, and the nine second divided regions 35X of the second image P2 also have circular geometric shapes. As a result, the extraction unit 74 extracts the nine second divided regions 35X as shape-similar regions 3B.

[0163] In addition, the extraction unit 74 performs a second extraction process to extract one or more pixel similar regions 3C from the ten regions 3X (i.e., the second image P2) that contain multiple pixels with pixel values ​​that are highly similar to the pixel values ​​of multiple pixels in the specific region 3A.

[0164] In addition, the extraction unit 74 performs a third extraction process to extract one or more balance similarity regions 3D from the ten regions 3X (i.e., the second image P2) that contain multiple pixels that exhibit a balance of pixel values ​​that is highly similar to the balance of pixel values ​​across multiple pixels in the specific region 3A.

[0165] In the second modification, the extraction unit 74 also executes all of the first to third extraction processes, but may execute only one or two of the first to third extraction processes.

[0166] 6, the extraction results of the first to third extraction processes are completely consistent with each other. That is, the extraction unit 74 extracts the same regions 3X (nine second divided regions 35X) for all of the shape-similar regions 3B, pixel-similar regions 3C, and balance-similar regions 3D.

[0167] As in the first modification, the extraction result of the extraction unit 74 may be displayed on the display device 58 via the display output unit 57 and presented to the user U1.

[0168] As a result of the first to third extraction processes, there is a possibility that no region 3X highly similar to the specific region 3A is present in the second image P2, and no shape-similar region 3B, pixel-similar region 3C, or balance-similar region 3D is extracted. In this case, it is preferable that the display device 58 notify the user U1 via the display output unit 57 that no highly similar region 3X is present. In this case, the user U1 can stop the process of creating the superimposed image P4 and input to the input unit 54 to change at least one of the first image P1 or the second image P2 to a different image.

[0169] The range generation unit 72 executes a range generation process and generates one or more range patterns Q1 (four range patterns Q15 to Q18 in the example of FIG. 6) in the second image P2 based on the division result by the division unit 71 and the extraction result by the extraction unit 74. That is, in the above embodiment and modified example 1, the range generation unit 72 generates the range pattern Q1 based on the first image P1, but in modified example 2, the range pattern Q1 is generated based on the second image P2, not the first image P1.

[0170] Each range pattern Q1 is generated based on, for example, the first divided region 34X, nine second divided regions 35X, and the pixel region of the specific portion E1 extracted from the second image P2. The display output unit 57 displays the multiple range patterns Q1 on the display device 58.

[0171] The creating unit 73 executes the creating process, and based on the determined range pattern Q1, creates a plurality of superimposed images P4 by superimposing the specific portion E1 on the second image P2, and outputs the superimposed images P4 as learning data.

[0172] The superimposed image P4 thus created is an image (defective product image) in which the specific portion E1 is displayed. In the second modification, it is also easier to generate the superimposed image P4 in a location where the specific portion E1 may actually occur, thereby further improving the accuracy of the learning data.

[0173] As described above, the dividing unit 71 of Modification 2 divides both the first image P1 and the second image P2 into a plurality of regions (3, 3X). However, it is also possible to divide only the second image P2 into a plurality of regions 3X. In this case, the data creation system 5 may acquire information specifying the plurality of regions 3 from an external source, instead of having the dividing unit 71 divide the first image P1. For example, the data creation system 5 may receive information specifying the plurality of regions 3 from the user U1 via the user interface 55. The range generation unit 72 may generate one or more range patterns Q1 based on the information specified by the user U1, the division result by the dividing unit 71, and the extraction result by the extraction unit 74.

[0174] (Variation 3) A data creation system 5 according to Modification 3 will be described below with reference to Fig. 7. In the following, components that are substantially the same as those in the above embodiment will be given the same reference numerals and descriptions thereof may be omitted.

[0175] In both the above-described embodiment and modification example 1, the first image P1 in which the first object 1 including the specific portion E1 (defective portion) is captured is acquired, and the first image P1 is subjected to the segmentation process.

[0176] In Modification 3, the first image acquisition unit 51 (see FIG. 1) functions as a "region acquisition unit" that acquires only the image of the specific region E1 (region image P1A) as information about the specific region E1. Note that, in Modification 3 as well, the specific region E1 is assumed to be a defective region.

[0177] In addition, in Modification 3, the second image acquisition unit 52 (see FIG. 1) functions as an "image acquisition unit" that acquires an object image P2A related to the object 2A. In Modification 3, the data creation system 5 subjects the object image P2A to division processing. Modification 3 will be specifically described below.

[0178] The part image P1A is, for example, a distance image. The part image P1A is a local image (partial image) corresponding to a pixel area of ​​almost only the specific part E1. The specific part E1 in the part image P1A is a defective part that may occur in the weldment. The part image P1A is, for example, substantially the same as the image corresponding to the pixel area of ​​the specific part E1 in the above embodiment. Therefore, a detailed description of the specific part E1 will be omitted.

[0179] The object image P2A is, for example, a distance image. The object 2A shown in the object image P2A is an object that does not include the specific portion E1 (defective portion). In other words, the object 2A is a non-defective product. The object 2A is, for example, a welded product that is substantially the same as the second object 2 in the above embodiment. Therefore, a detailed description of the object 2A will be omitted.

[0180] The superimposed image P4 is, for example, a distance image. The object image P2A is an image that serves as the basis for the superimposed image P4. That is, the first metal plate 21, the second metal plate 22, and the bead 23 of the object 2A correspond to the originals of the first metal plate 41, the second metal plate 42, and the bead 43 of the object 4 in the superimposed image P4. However, the superimposed image P4 includes a specific portion E1 (defective portion) that does not exist in the object image P2A.

[0181] The data creation system 5 according to the third modification includes a region acquisition unit (first image acquisition unit 51), an image acquisition unit (second image acquisition unit 52), a division unit 71, a range generation unit 72, and a creation unit 73. The other components of the data creation system 5 are substantially the same as those of the above embodiment. The region acquisition unit executes a region acquisition process to acquire information (region image P1A) related to the specific region E1. The image acquisition unit executes an image acquisition process to acquire an object image P2A related to the object 2A.

[0182] The dividing unit 71 performs a dividing process to divide the object image P2A into a plurality of regions 3 (see FIG. 7). In the third modification, the dividing unit 71 identifies the pixel regions of the first metal plate 21, the second metal plate 22, and the bead 23, and divides the object image P2A into three regions 3 along the boundaries of these pixel regions.

[0183] In the example of FIG. 7, three regions 3 are shown: a first divided region 31, a second divided region 32, and a third divided region 33. The first divided region 31 indicates a pixel region of the first metal plate 21. The second divided region 32 indicates a pixel region of the second metal plate 22. The third divided region 33 indicates a pixel region of the bead 23. Note that the object image P2A is a non-defective product image and does not include the specific portion E1, so the dividing unit 71 does not need to identify in which region 3 among the multiple regions 3 the specific portion E1 exists.

[0184] The range generating unit 72 executes a range generating process and generates one or more range patterns Q1 (four range patterns Q11A, Q12A, Q13A, and Q14A in the example of FIG. 7) in the object image P2A based on the division result by the dividing unit 71. Although not shown in FIG. 7, each of the multiple range patterns Q1 may include a specific portion E1.

[0185] In the above embodiment, range setting information regarding the occurrence ranges R1 to R6 was previously included in the setting information 81. In the third modification, range setting information regarding the occurrence range of the specific part E1 may also be included in the setting information 81, or the range setting information may not be present.

[0186] The range generation unit 72 may generate range pattern Q1 by referring to range setting information if it is present in the setting information 81, but as an example, it generates four range patterns Q1 (Q11A, Q12A, Q13A, Q14A) based only on the three divided areas 3.

[0187] The range pattern Q11A is a range pattern Q1 in which only the third divided region 33 (pixel region of the bead 23) is set as the range R11 (see dot hatching) in which the specific portion E1 can occur.

[0188] The range pattern Q12A is a range pattern Q1 in which only the first divided region 31 (pixel region of the first metal plate 21) is set as a range R12 (see dot hatching) in which the specific portion E1 can occur.

[0189] The range pattern Q13A is a range pattern Q1 in which only the second divided region 32 (pixel region of the second metal plate 22) is set as a range R13 (see dot hatching) in which the specific portion E1 can occur.

[0190] The range pattern Q14A is a range pattern Q1 in which only the first divided region 31 and the second divided region 32 are set as ranges R12 and R13 (see dot hatching) where the specific portion E1 can occur.

[0191] If the data creation system 5 identifies the type of specific part E1 shown in the part image P1A, the range generation unit 72 may generate a range pattern Q1 based on the type of specific part E1 and the three divided areas 3.

[0192] The display output unit 57 displays a plurality of range patterns Q1 on the display device 58. As shown in Fig. 7, the display output unit 57 displays ranges R11 to R13 in which the specific part E1 may occur so that the user U1 can easily understand intuitively. Although not shown in Fig. 7, the display output unit 57 may display one or more specific parts E1 in each of the range patterns Q1 to be displayed so that the user U1 can easily understand intuitively.

[0193] The creation unit 73 executes a creation process to superimpose a specific portion E1 on the object image P2A based on at least one of the one or more range patterns Q1, thereby creating one or more superimposed images P4 and outputting the superimposed images P4 as learning data. The example in FIG. 7 shows an example in which the user U1 selects range pattern Q12A from among the four range patterns Q1. The creation unit 73 creates four superimposed images P4 (P41A, P42A, P43A, and P44A) based on the instruction input of "select range pattern Q12A" received by the input unit 54. The positions of the specific portion E1 in the four superimposed images P41A, P42A, P43A, and P44A are different from one another. The creation unit 73 superimposes the specific portion E1 in a region corresponding to range R12 in the object image P2A. When superimposing the specific portion E1, the creation unit 73 performs image conversion processing on the specific portion E1. In the example of Figure 7, based on the type of the superimposed specific portion E1, in all four superimposed images P4, the specific portion E1 is positioned adjacent to the bead 43 within the pixel region of the first metal plate 41.

[0194] The superimposed image P4 thus created is an image (defective product image) in which the specific portion E1 is displayed. In the third modification, it is also easier to generate the superimposed image P4 in a location where the specific portion E1 may actually occur, thereby further improving the accuracy of the learning data.

[0195] Note that functions similar to those of the data creation system 5 according to Modification 3 can be realized by a data creation method. The data creation method according to Modification 3 includes a region acquisition process, an image acquisition process, a division process, a range generation process, and a creation process. The region acquisition process acquires information (region image P1A) related to the specific region E1. The image acquisition process acquires an object image P2A related to the object 2A. The division process divides the object image P2A into multiple regions 3. The range generation process generates one or more range patterns Q1 in the object image P2A based on the division results of the division process. The creation process creates one or more superimposed images P4 by superimposing the specific region E1 on the object image P2A based on at least one of the one or more range patterns Q1, and outputs the superimposed images P4 as learning data. Furthermore, a program according to Modification 3 causes one or more processors (of a computer system) to execute the data creation method according to Modification 3. The program may be recorded on a non-transitory recording medium readable by the computer system.

[0196] (Variation 4) A data creation system 5 according to Modification 4 will be described below with reference to Fig. 8. In the following, components that are substantially the same as those in the above embodiment will be given the same reference numerals and descriptions thereof may be omitted.

[0197] Modification 4 is another example of Modification 3. In Modification 4 as well, the first image acquisition unit 51 (see FIG. 1) functions as a "part acquisition unit" that acquires only the image of the specific part E1 (part image P1A) as information about the specific part E1. Note that in Modification 4 as well, the specific part E1 is assumed to be a defective part.

[0198] Also in the fourth modification, the second image acquisition unit 52 (see FIG. 1) functions as an "image acquisition unit" that acquires an object image P2A related to the object 2A. Also in the fourth modification, the data creation system 5 subjects the object image P2A to division processing.

[0199] However, in Modification 4, unlike Modification 3, the object is assumed to be an article other than a weldment, for example, a plate-like part with a plurality of protrusions arranged in a matrix on its surface. As an example, it is assumed that object 2A in Modification 4 is substantially the same as second object 2 in Modification 1. Modification 4 will be described in detail below.

[0200] The part image P1A is, for example, a distance image. The part image P1A is a local image (partial image) corresponding to a pixel area of ​​almost only the specific part E1. The specific part E1 of the part image P1A is, for example, a defective part such as a scratch, a dent, or a dent. The part image P1A is, for example, substantially the same as an image corresponding to the pixel area of ​​the specific part E1 in the first modification. Therefore, a detailed description of the specific part E1 will be omitted.

[0201] The object image P2A is, for example, a distance image. The object 2A shown in the object image P2A is an object that does not include the specific portion E1 (defective portion). In other words, the object 2A is a non-defective product. The object 2A is, for example, substantially the same as the second object 2 in the first modification. Therefore, a detailed description of the object 2A will be omitted.

[0202] The superimposed image P4 is, for example, a distance image. The object image P2A is the image that forms the basis of the superimposed image P4. That is, the base material 24 and nine protrusions 25 of the object 2A correspond to the originals of the base material 44 and nine protrusions 45 of the object 4 in the superimposed image P4. However, the superimposed image P4 includes a specific portion E1 (defective portion) that does not exist in the object image P2A.

[0203] The data creation system 5 according to the fourth modification includes a region acquisition unit (first image acquisition unit 51), an image acquisition unit (second image acquisition unit 52), a division unit 71, a range generation unit 72, and a creation unit 73. The other components of the data creation system 5 are substantially the same as those of the above-described embodiment. The region acquisition unit executes a region acquisition process to acquire information (region image P1A) related to the specific region E1. The image acquisition unit executes an image acquisition process to acquire an object image P2A related to the object 2A.

[0204] In the fourth modification, the dividing unit 71 performs a dividing process to divide the object image P2A into a plurality of regions 3 (see FIG. 8). In the fourth modification, the dividing unit 71 identifies the pixel regions of the base material 24 and the nine protrusions 25, and divides the object image P2A into ten regions 3 along the boundaries of these pixel regions.

[0205] In the example of Fig. 8, ten regions 3 are shown, namely, a first divided region 34 and nine second divided regions 35. The first divided region 34 indicates the pixel region of the base material 24. Each second divided region 35 indicates the pixel region of a corresponding one protrusion 25. Note that the object image P2A is a non-defective product image and does not include the specific portion E1, so the dividing unit 71 does not need to identify in which region 3 among the multiple regions 3 the specific portion E1 exists.

[0206] The range generating unit 72 executes a range generating process to generate one or more range patterns Q1 (four range patterns Q15A, Q16A, Q17A, and Q18A in the example of FIG. 8) in the object image P2A based on the segmentation result by the segmentation unit 71. Although not shown in FIG. 8, each of the multiple range patterns Q1 may include a specific portion E1.

[0207] In the fourth modification, the setting information 81 may also include range setting information regarding the occurrence range of the specific portion E1, or the range setting information may not be present.

[0208] The range generation unit 72 may generate range pattern Q1 by referring to range setting information if it is present in the setting information 81, but as an example, it generates four range patterns Q1 (Q15A, Q16A, Q17A, Q18A) based only on the 10 divided areas 3.

[0209] The range pattern Q15A is a range pattern Q1 in which only the first divided region 34 (pixel region of the base material 24) is set as a range R14 (see dot hatching) in which the specific portion E1 can occur.

[0210] Range pattern Q16A is a range pattern Q1 in which only the entirety of the nine second divided regions 35 (pixel regions of the nine protrusions 25) is set as a range R15 (see dot hatching) in which the specific portion E1 can occur.

[0211] The range pattern Q17A is a range pattern Q1 in which only the peripheral portions of the nine second divided regions 35 (pixel regions of the nine protrusions 25) are set as the range R16 in which the specific portion E1 can occur.

[0212] The range pattern Q18A is a range pattern Q1 in which only the substantially right half of each of the nine second divided regions 35 is set as a range R17 (see dot hatching) in which the specific portion E1 can occur.

[0213] If the data creation system 5 identifies the type of specific part E1 shown in the part image P1A, the range generation unit 72 may generate a range pattern Q1 based on the type of specific part E1 and the 10 divided areas 3.

[0214] The display output unit 57 displays a plurality of range patterns Q1 on the display device 58. As shown in Fig. 8, the display output unit 57 displays ranges R14 to R17 in which the specific part E1 may occur so that the user U1 can easily understand intuitively. Although not shown in Fig. 8, the display output unit 57 may display one or more specific parts E1 in each range pattern Q1 to be displayed so that the user U1 can easily understand intuitively.

[0215] The creation unit 73 executes a creation process to superimpose a specific portion E1 on the object image P2A based on at least one of the one or more range patterns Q1, thereby creating one or more superimposed images P4 and outputting the superimposed images P4 as learning data. The example in FIG. 8 shows an example in which the user U1 selects range pattern Q16A from the four range patterns Q1. The creation unit 73 creates four superimposed images P4 (P45A, P46A, P47A, and P48A) based on the instruction input of "select range pattern Q16A" received by the input unit 54. The positions of the specific portion E1 in the four superimposed images P45A, P46A, P47A, and P48A are different from one another. In particular, two specific portions E1 are superimposed in the superimposed image P48A. The creation unit 73 superimposes the specific portion E1 in the area corresponding to range R15 in the object image P2A. When the creation unit 73 superimposes the specific portion E1, the creation unit 73 performs image conversion processing on the specific portion E1. In the example of Fig. 8, based on the type of the specific portion E1 to be superimposed, in each of the four superimposed images P4, the specific portion E1 is positioned adjacent to the base material 44 within the pixel region of the corresponding protrusion 45.

[0216] The superimposed image P4 thus created is an image (defective product image) in which the specific portion E1 is displayed. In the fourth modification, it is also easier to generate the superimposed image P4 in a location where the specific portion E1 may actually occur, thereby further improving the accuracy of the learning data.

[0217] (Other Modifications of the Embodiments) Other variations of the above embodiment are listed below. The variations listed below may be realized in appropriate combination with each other. Furthermore, the variations listed below may be realized in appropriate combination with the above embodiment and variations 1 to 4.

[0218] The trained model 82 is not limited to a model used for welding visual inspection, but may be a model used for various types of inspection, etc. Furthermore, the trained model 82 is not limited to a model used for inspection, but may be a model used for various types of image recognition.

[0219] The entity that executes at least one of the division process, range generation process, and creation process is not limited to the image processing unit 53, and may be a configuration external to the image processing unit 53.

[0220] At least one of the user interface 55, the display device 58, the learning unit 60, and the determination unit 61 may be configured external to the data creation system 5. In addition, the display device 58 may be a portable terminal such as a smartphone or a tablet terminal.

[0221] The images handled by the data creation system 5 (first image P1, second image P2, superimposed image P4, and inspection image P5, etc.) are not limited to three-dimensional images, but may be two-dimensional images or images with four or more dimensions.

[0222] The superimposed image P4 is not limited to a defective product image, but may also be a non-defective product image.

[0223] The first object 1 in the first image P1 may be an article of the same shape as the second object 2 in the second image P2, or may be an article of a different shape.

[0224] The first image P1 may be the same image as the second image P2. That is, the specific portion E1 in the first image P1 may be superimposed on a specific portion E1 in the second image P2 at a different position from the specific portion E1, thereby creating a superimposed image P4 including two specific portions E1. In other words, the first image P1 and the second image P2 may be images of defective products.

[0225] Furthermore, the specific portion E1 is not limited to being a defective portion, and conversely, the first image P1 and the second image P2 may be images of a non-defective product. Furthermore, one of the first image P1 and the second image P2 may be an image of a non-defective product and the other may be an image of a defective product.

[0226] The first image P1 may be an image obtained by capturing a part or all of the first object 1. The second image P2 may be an image obtained by capturing a part or all of the second object 2.

[0227] The first image P1, the second image P2, the superimposed image P4, and the inspection image P5 may be luminance image data that represents the luminance of an object using gradation. In the above embodiment, "gradation" refers to the depth of a single color (e.g., black). In contrast, "gradation" may refer to the depth of each of multiple colors (e.g., the three RGB colors).

[0228] In the above-described third and fourth variations, the data creation system 5 includes a region acquisition unit that acquires a region image P1A, which is a localized image (partial image) corresponding to a pixel region consisting essentially of the specific region E1, as "information about the specific region E1." However, the data creation system 5 is only required to acquire information that can identify the specific region E1, and does not necessarily need to acquire an "image" about the specific region E1. For example, the region acquisition unit may acquire, as information about the specific region E1, numerical data such as the distance of the specific region E1, or time-series data such as one-dimensional waveform data about the specific region E1. The data creation system 5 may create an image of the specific region E1 based on such data and create a superimposed image P4 that includes the specific region E1.

[0229] The same functions as those of the data creation system 5 according to the above embodiment may be realized by a data creation method, a computer program, or a non-transitory recording medium on which a computer program is recorded.

[0230] The data creation system 5 in the present disclosure includes a computer system. The computer system is primarily composed of a processor and memory as hardware. The processor executes a program stored in the memory of the computer system to realize the functions of the data creation system 5 in the present disclosure. The program may be pre-stored in the memory of the computer system, provided via a telecommunications line, or provided in a non-transitory recording medium readable by the computer system, such as a memory card, optical disk, or hard disk drive. The processor of the computer system is composed of one or more electronic circuits including a semiconductor integrated circuit (IC) or a large-scale integrated circuit (LSI). The integrated circuits, such as ICs and LSIs, are referred to by different names depending on the degree of integration, and include integrated circuits called system LSIs, very large-scale integrations (VLSIs), or ultra-large-scale integrations (ULSIs). Furthermore, field-programmable gate arrays (FPGAs), which are programmed after the LSI is manufactured, or logic devices that allow the reconfiguration of internal connections or internal circuit partitions of the LSI, can also be used as processors. The electronic circuits may be integrated into one chip or distributed across multiple chips. The chips may be integrated into one device or distributed across multiple devices. The computer system referred to here includes a microcontroller having one or more processors and one or more memories. Therefore, the microcontroller is also composed of one or more electronic circuits including a semiconductor integrated circuit or a large-scale integrated circuit.

[0231] Furthermore, it is not essential that the multiple functions of the data creation system 5 are concentrated in one housing. For example, the components of the data creation system 5 may be distributed across multiple housings.

[0232] Conversely, multiple functions of the data creation system 5 may be integrated into one housing. Furthermore, at least some of the functions of the data creation system 5, for example, some of the functions of the data creation system 5, may be realized by the cloud (cloud computing) or the like.

[0233] (summary) The above-described embodiments and the like disclose the following aspects.

[0234] A data creation system (5) according to a first aspect creates learning data for generating a trained model (82) that performs classification regarding a specific portion (E1). The data creation system (5) includes a first image acquisition unit (51), a second image acquisition unit (52), a division unit (71), a range generation unit (72), and a creation unit (73). The first image acquisition unit (51) acquires a first image (P1) regarding a first object (1) including the specific portion (E1). The second image acquisition unit (52) acquires a second image (P2) regarding a second object (2). The division unit (71) divides at least one of the first image (P1) and the second image (P2) into a plurality of regions (3, 3X). The range generation unit (72) generates one or a plurality of range patterns (Q1) based on the division result by the division unit (71). The creation unit (73) creates one or more superimposed images (P4) by superimposing a specific portion (E1) on a second image (P2) based on at least one range pattern (Q1) out of one or more range patterns (Q1), and outputs the superimposed images (P4) as learning data.

[0235] According to this aspect, the specific portion (E1) is superimposed on the second image (P2) based on the range pattern (Q1), thereby improving the accuracy of the learning data.

[0236] Regarding the data creation system (5) according to the second aspect, in the first aspect, the specific part (E1) is a defective part, the first object (1) is an object including the defective part, and the second object (2) is an object not including the defective part.

[0237] According to this aspect, the accuracy of the learning data for generating the learned model (82) that identifies defective portions is improved.

[0238] Regarding the data creation system (5) according to the third aspect, in the first or second aspect, the multiple regions (3) include a specific region (3A) in which a specific portion (E1) is located. The data creation system (5) further includes an extraction unit (74). The extraction unit (74) extracts one or more shape-similar regions (3B) having a shape highly similar to that of the specific region (3A) from the first image (P1) or the second image (P2). The range generation unit (72) generates a range pattern (Q1) including the specific region (3A) and the one or more shape-similar regions (3B) as one of the one or more range patterns (Q1).

[0239] According to this aspect, it becomes easier to generate a superimposed image (P4) arranged at a location where the specific portion (E1) may actually occur, and the accuracy of the learning data is further improved.

[0240] With respect to a data creation system (5) according to a fourth aspect, in any one of the first to third aspects, the multiple areas (3) include a specific area (3A) in which a specific portion (E1) is located. The data creation system (5) further includes an extraction unit (74). The extraction unit (74) extracts one or more pixel similar areas (3C) from the first image (P1) or the second image (P2), the pixel similar areas (3C) including a plurality of pixels having pixel values ​​highly similar to the pixel values ​​of the plurality of pixels in the specific area (3A). The range generation unit (72) generates a range pattern (Q1) including the specific area (3A) and the one or more pixel similar areas (3C) as one of the one or more range patterns (Q1).

[0241] According to this aspect, it becomes easier to generate a superimposed image (P4) arranged at a location where the specific portion (E1) may actually occur, and the accuracy of the learning data is further improved.

[0242] With respect to a data creation system (5) according to a fifth aspect, in any one of the first to fourth aspects, the multiple regions (3) include a specific region (3A) in which a specific portion (E1) is located. The data creation system (5) further includes an extraction unit (74). The extraction unit (74) extracts one or more balanced similar regions (3D) from the first image (P1) or the second image (P2), the balanced similar regions (3D) including multiple pixels exhibiting a balance of pixel values ​​highly similar to the balance of pixel values ​​across multiple pixels in the specific region (3A). The range generation unit (72) generates a range pattern (Q1) including the specific region (3A) and the one or more balanced similar regions (3D) as one of the one or more range patterns (Q1).

[0243] According to this aspect, it becomes easier to generate a superimposed image (P4) arranged at a location where the specific portion (E1) may actually occur, and the accuracy of the learning data is further improved.

[0244] In a data creation system (5) according to a sixth aspect, in any one of the first to fifth aspects, the plurality of areas (3) includes a specific area (3A) in which a specific portion (E1) is located. The range generating unit (72) generates at least one of a range pattern (Q1) whose range is only a peripheral portion (303) of the specific area (3A) and a range pattern (Q1) whose range is the entire specific area (3A), as at least one of the one or more range patterns (Q1).

[0245] According to this aspect, it becomes easier to generate a superimposed image (P4) arranged at a location where the specific portion (E1) may actually occur, and the accuracy of the learning data is further improved.

[0246] With respect to the data creation system (5) according to the seventh aspect, in any one of the first to sixth aspects, the multiple areas (3) include a specific area (3A) where a specific portion (E1) is located. When the specific area (3A) has an elongated shape, the area generation unit (72) generates at least one of the following range patterns (Q1): an area pattern (Q1) covering only one end (first end 301 or second end 302) in the longitudinal direction of the specific area (3A) and an area pattern (Q1) covering only both end portions (first end 301 and second end 302) in the longitudinal direction of the specific area (3A).

[0247] According to this aspect, it becomes easier to generate a superimposed image (P4) arranged at a location where the specific portion (E1) may actually occur, and the accuracy of the learning data is further improved.

[0248] A data creation system (5) according to an eighth aspect creates learning data for generating a trained model (82) that performs classification regarding a specific portion (E1). The data creation system (5) includes a portion acquisition unit (first image acquisition unit 51), an image acquisition unit (second image acquisition unit 52), a division unit (71), a range generation unit (72), and a creation unit (73). The portion acquisition unit acquires information regarding the specific portion (E1). The image acquisition unit acquires an object image (P2A) regarding an object (2A). The division unit (71) divides the object image (P2A) into multiple regions (3). The range generation unit (72) generates one or more range patterns (Q1) in the object image (P2A) based on the division result by the division unit (71). The creation unit (73) creates one or more superimposed images (P4) by superimposing a specific portion (E1) on an object image (P2A) based on at least one range pattern (Q1) out of one or more range patterns (Q1), and outputs the superimposed images (P4) as learning data.

[0249] According to this aspect, the specific portion (E1) is superimposed on the object image (P2A) based on the range pattern (Q1), thereby improving the accuracy of the learning data.

[0250] In the data creation system (5) according to the ninth aspect, in the eighth aspect, the specific portion (E1) is a defective portion, and the object (2A) is an object that does not include a defective portion.

[0251] According to this aspect, the accuracy of the learning data for generating the learned model (82) that identifies defective portions is improved.

[0252] A data creation system (5) according to a tenth aspect is any one of the first to ninth aspects, and further includes a display output unit (57) and an input unit (54). The display output unit (57) displays one or more range patterns (Q1) on a display device (58). The input unit (54) accepts an instruction input for selecting at least one of the one or more displayed range patterns (Q1). The creation unit (73) creates one or more superimposed images (P4) based on the instruction input accepted by the input unit (54).

[0253] According to this aspect, at least one appropriate range pattern (Q1) can be selected from one or more range patterns (Q1) through visual confirmation by the user (U1). Therefore, compared to when the data creation system (5) automatically executes all processes without the intervention of visual confirmation by the user (U1), it becomes easier to generate a superimposed image (P4) arranged at a location where the specific portion (E1) may actually occur.

[0254] Regarding the data creation system (5) according to the eleventh aspect, in the tenth aspect, the display output unit (57) displays a range pattern (Q1) in which multiple specific parts (E1, E2) are superimposed at a predetermined density on the display device (58) as one of one or more range patterns (Q1).

[0255] According to this aspect, it becomes easier for the user (U1) to intuitively understand the locations where the specific parts (E1, E2) may actually occur, and to determine which range pattern (Q1) is more appropriate among one or more range patterns (Q1).

[0256] A data creation method according to a twelfth aspect creates learning data for generating a trained model (82) that performs classification regarding a specific portion (E1). The data creation method includes a first image acquisition process, a second image acquisition process, a division process, a range generation process, and a creation process. In the first image acquisition process, a first image (P1) regarding a first object (1) including the specific portion (E1) is acquired. In the second image acquisition process, a second image (P2) regarding a second object (2) is acquired. In the division process, at least one of the first image (P1) and the second image (P2) is divided into a plurality of regions (3, 3X). In the range generation process, one or more range patterns (Q1) are generated based on the division result of the division process. In the creation process, the specific portion (E1) is superimposed on the second image (P2) based on at least one range pattern (Q1) of the one or more range patterns (Q1) to create one or more superimposed images (P4), and output the superimposed images as learning data.

[0257] According to this aspect, the specific portion (E1) is superimposed on the second image (P2) based on the range pattern (Q1), so that it is possible to provide a data creation method that can improve the accuracy of learning data.

[0258] A data creation method according to a thirteenth aspect creates learning data for generating a trained model (82) that performs classification regarding a specific portion (E1). The data creation method includes a portion acquisition process, an image acquisition process, a division process, a range generation process, and a creation process. The portion acquisition process acquires information regarding the specific portion (E1). The image acquisition process acquires an object image (P2A) regarding an object (2A). The division process divides the object image (P2A) into multiple regions (3). The range generation process generates one or more range patterns (Q1) in the object image (P2A) based on the division result of the division process. The creation process creates one or more superimposed images (P4) by superimposing the specific portion (E1) on the object image (P2A) based on at least one range pattern (Q1) of the one or more range patterns (Q1), and outputs the superimposed images as learning data.

[0259] According to this aspect, the specific portion (E1) is superimposed on the object image (P2A) based on the range pattern (Q1), so that it is possible to provide a data creation method that can improve the accuracy of learning data.

[0260] A program according to a fourteenth aspect is a program for causing one or more processors to execute the data creation method according to the twelfth or thirteenth aspect.

[0261] According to this aspect, it is possible to provide a function that can improve the accuracy of the learning data.

[0262] The configurations according to the second to seventh and tenth to eleventh aspects are not essential for the data creation system (5) according to the first aspect and may be omitted as appropriate. The configurations according to the ninth to eleventh aspects are not essential for the data creation system (5) according to the eighth aspect and may be omitted as appropriate. [Explanation of symbols]

[0263] 1. First Object 2 Second Object 2A Object 3,3X area 3A Specific area 301 First end (end) 302 Second end (end) 303 Periphery 3B Shape similar region 3C Pixel Similarity Region 3D Balance Similarity Region 5 Data Creation System 51 First image acquisition unit (region acquisition unit) 52 second image acquisition unit (image acquisition unit) 54 Input section 57 Display output section 58 Display device 71 Split part 72 Range Generation Unit 73 Creation Department 74 Extraction part 82 trained models E1 Specific part P1 First image P2 Second image P2A Object Image P4 Superimposed image Q1 Range Pattern

Claims

1. A data creation system that creates learning data for generating a trained model that performs identification of a specific part, a first image acquisition unit that acquires a first image of a first object including the specific portion; a second image acquisition unit that acquires a second image relating to a second object; a dividing unit that divides at least one of the first image and the second image into a plurality of regions; a range generation unit that generates one or more range patterns based on the division result by the division unit; a creating unit that creates one or more superimposed images by superimposing the specific portion on the second image based on at least one of the one or more range patterns, and outputs the superimposed images as the learning data; Equipped with the plurality of regions includes a specific region in which the specific site is located, an extracting unit that extracts one or more shape-similar regions from the first image or the second image, the shape of the shape-similar regions being highly similar to the shape of the specific region; the range generation unit generates a range pattern including the specific region and the one or more shape similar regions as one of the one or more range patterns; Data creation system.

2. A data creation system for creating learning data for generating a trained model that identifies specific parts, comprising: a first image acquisition unit that acquires a first image of a first object including the specific portion; a second image acquisition unit that acquires a second image relating to a second object; a dividing unit that divides at least one of the first image and the second image into a plurality of regions; a range generation unit that generates one or more range patterns based on the division result by the division unit; a creating unit that creates one or more superimposed images by superimposing the specific portion on the second image based on at least one of the one or more range patterns, and outputs the superimposed images as the learning data; Equipped with the plurality of regions includes a specific region in which the specific site is located, an extracting unit that extracts one or more pixel similar regions from the first image or the second image, the pixel similar regions including a plurality of pixels having pixel values ​​that are highly similar to pixel values ​​of the plurality of pixels in the specific region; the range generation unit generates a range pattern including the specific region and the one or more pixel similar regions as one of the one or more range patterns; Data creation system.

3. A data creation system for creating learning data for generating a trained model that performs identification of a specific part, a first image acquisition unit that acquires a first image of a first object including the specific portion; a second image acquisition unit that acquires a second image relating to a second object; a dividing unit that divides at least one of the first image and the second image into a plurality of regions; a range generation unit that generates one or more range patterns based on the division result by the division unit; a creating unit that creates one or more superimposed images by superimposing the specific portion on the second image based on at least one of the one or more range patterns, and outputs the superimposed images as the learning data; Equipped with the plurality of regions includes a specific region in which the specific site is located, an extracting unit that extracts one or more similarly balanced regions from the first image or the second image, the similarly balanced regions including a plurality of pixels that exhibit a balance of pixel values ​​that is highly similar to the balance of pixel values ​​across the plurality of pixels in the specific region; the range generation unit generates a range pattern including the specific region and the one or more balanced similar regions as one of the one or more range patterns; Data creation system.

4. The specific portion is a defective portion, the first object is an object including the defective portion, the second object is an object that does not include the defective portion. The data creation system according to any one of claims 1 to 3.

5. The range generation unit further generates, as at least one of the one or more range patterns, a range pattern that covers only the peripheral portion of the specific region and a range pattern that covers the entire specific region. The data creation system according to any one of claims 1 to 3.

6. When the specific region has an elongated shape, the range generation unit further generates at least one of a range pattern that covers only one end of the specific region in the longitudinal direction, and a range pattern that covers only both ends of the specific region in the longitudinal direction, as at least one of the one or more range patterns. The data creation system according to any one of claims 1 to 3.

7. A data creation system for creating learning data for generating a trained model that performs identification of a specific part, comprising: a part acquisition unit that acquires information about the specific part; an image acquisition unit that acquires an object image related to an object that does not include the specific portion; a dividing unit that divides the object image into a plurality of regions; an area generation unit that generates one or more area patterns in the object image based on the division result by the division unit; a creating unit that creates one or more superimposed images by superimposing the specific portion on the object image based on at least one of the one or more range patterns, and outputs the superimposed images as the learning data; Equipped with Data creation system.

8. The specific portion is a defective portion, the object is an object that does not include the defective portion, The data creation system according to claim 7.

9. A display output unit that displays the one or more range patterns on a display device; an input unit that accepts an instruction input for selecting at least one of the one or more displayed range patterns; Further provided with the creation unit creates the one or more superimposed images based on the instruction input received by the input unit. The data creation system according to any one of claims 1 to 3 and 7.

10. The display output unit displays a range pattern in which the specific portion is superimposed at a predetermined density as one of the one or more range patterns on the display device. The data creation system according to claim 9.

11. A data creation method for creating training data for generating a trained model that performs identification of a specific part, comprising: a first image acquisition process for acquiring a first image of a first object including the specific portion; a second image acquisition process for acquiring a second image relating to a second object; a division process of dividing at least one of the first image and the second image into a plurality of regions; a range generation process for generating one or more range patterns based on the division result of the division process; a creation process of creating one or more superimposed images by superimposing the specific portion on the second image based on at least one of the one or more range patterns, and outputting the superimposed images as the learning data; Including, the plurality of regions includes a specific region in which the specific site is located, an extraction process of extracting one or more shape-similar regions having a shape highly similar to a shape of the specific region from the first image or the second image; In the range generation process, a range pattern including the specific region and the one or more shape similar regions is generated as one of the one or more range patterns. Data creation method.

12. A data creation method for creating training data for generating a trained model that performs identification of a specific part, comprising: a first image acquisition process for acquiring a first image of a first object including the specific portion; a second image acquisition process for acquiring a second image relating to a second object; a division process of dividing at least one of the first image and the second image into a plurality of regions; a range generation process for generating one or more range patterns based on the division result of the division process; a creation process of creating one or more superimposed images by superimposing the specific portion on the second image based on at least one of the one or more range patterns, and outputting the superimposed images as the learning data; Including, the plurality of regions includes a specific region in which the specific site is located, an extraction process for extracting one or more pixel similar regions from the first image or the second image, the pixel similar regions including a plurality of pixels having pixel values ​​highly similar to pixel values ​​of a plurality of pixels in the specific region; In the range generation process, a range pattern including the specific region and the one or more pixel similar regions is generated as one of the one or more range patterns. Data creation method.

13. A data creation method for creating training data for generating a trained model that performs identification of a specific part, comprising: a first image acquisition process for acquiring a first image of a first object including the specific portion; a second image acquisition process for acquiring a second image relating to a second object; a division process of dividing at least one of the first image and the second image into a plurality of regions; a range generation process for generating one or more range patterns based on the division result of the division process; a creation process of creating one or more superimposed images by superimposing the specific portion on the second image based on at least one of the one or more range patterns, and outputting the superimposed images as the learning data; Including, the plurality of regions includes a specific region in which the specific site is located, an extraction process for extracting one or more similarly balanced regions from the first image or the second image, the similarly balanced regions including a plurality of pixels that exhibit a balance of pixel values ​​highly similar to the balance of pixel values ​​across the plurality of pixels in the specific region; In the range generation process, a range pattern including the specific region and the one or more balanced similar regions is generated as one of the one or more range patterns. Data creation method.

14. A data creation method for creating training data for generating a trained model that performs identification of a specific part, comprising: a part acquisition process for acquiring information about the specific part; an image acquisition process for acquiring an object image relating to an object that does not include the specific portion; a division process for dividing the object image into a plurality of regions; a range generation process for generating one or more range patterns in the object image based on the division result of the division process; a creation process of creating one or more superimposed images by superimposing the specific portion on the object image based on at least one of the one or more range patterns, and outputting the superimposed images as the learning data; Including, Data creation method.

15. A program for causing one or more processors to execute the data creation method described in any one of claims 11 to 14.

Images