Defective product data generation device, defective product data generation method, program, and defect generation step identification device

Abstract

Provided is a defective product data generation device comprising: a non-defective product data acquisition unit which acquires non-defective product data pertaining to a predetermined object; an elapsed information acquisition unit which acquires elapsed information including at least one of manufacturing step information pertaining to a step of manufacturing the object or elapsed step information pertaining to an elapsed step after the manufacturing of the object; and a defective product data generation unit which uses a predetermined generation model to generate defective product data pertaining to the object on the basis of the non-defective product data and the elapsed information.

Description

Defective product data generation device, defective product data generation method, program, and defect occurrence step identification device

[0001] The present invention relates to a defective product data generating device, a defective product data generating method, a program, and a defect occurrence step identifying device.

[0002] It is known to generate image data by applying extended data obtained by extending a characteristic portion of an image to another image (see, for example, Patent Document 1). General disclosure

[0003] In a first aspect of the present invention, there is provided a defective product data generation device comprising: a good product data acquisition unit that acquires good product data regarding a predetermined object; a progress information acquisition unit that acquires progress information including at least one of manufacturing step information regarding a manufacturing step of the object or progress step information regarding a progress step after the object is manufactured; and a defective product data generation unit that generates defective product data regarding the object using a predetermined generation model based on the good product data and the progress information.

[0004] In the defective product data generating device, the non-defective product data may include at least one of an image, three-dimensional data, drawing data, and video.

[0005] In any of the above-described defective product data generation devices, the manufacturing step information may include at least one of words, sentences, images, videos, and structured data indicating a manufacturing step of the target product.

[0006] In any of the above-described defective product data generation devices, the progress step information may include at least one of a word, a sentence, an image, a video, and structured data indicating a progress step after the manufacturing of the object.

[0007] In any of the above-described defective product data generation devices, the defective product data generation unit may specify at least one of the type of the target product, the generation model, position information of the defective parts to be generated, morphological information of the defective parts to be generated, failure mode of the defective parts to be generated, or the number of defective parts to be generated, and generate the defective product data using the generation model.

[0008] Any of the above-described defective product data generation devices may include a defect occurrence data acquisition unit that acquires defect occurrence data relating to defective locations that occur in the manufacturing step and / or the elapsed step, and a paired data storage unit that stores the manufacturing step and / or the elapsed step and the defective locations as paired data.

[0009] Any of the above-described defective product data generation devices may include a generative model learning unit that learns the generative model based on the paired data stored in the paired data storage unit.

[0010] In any of the above-described defective product data generation devices, the defective product data generation unit may specify the manufacturing step and / or the progress step, and a failure mode of the defective part that occurs in the corresponding step stored as the pair data, and generate the defective product data using the generation model.

[0011] Any of the above-described defective product data generation devices may include a storage unit that stores the progress information and the learning data used to learn the generative model in association with each other.

[0012] Any of the above-described defective product data generation devices may include an original data acquisition unit that acquires original data of the defective product of the target, and an evaluation unit that evaluates the defective product data based on the original data.

[0013] Any of the above-described defective product data generation devices may include a defect occurrence step designation unit that designates a defect occurrence step in which a defective location will occur, and the defective product data generation unit may generate the defective product data by causing a defect location that may occur in the designated defect occurrence step in the target object.

[0014] Any of the above-described defective product data generating devices may include a defect occurrence time prediction unit that predicts a defect occurrence time when the defective portion will occur in the defect occurrence step.

[0015] In any of the above-described defective product data generation devices, the elapsed step may be aging of the object, and the defective product data generation unit may output, as the defective product data related to the object, a defective product image of the object that has aged over time and data on the number of years that have passed since the aging.

[0016] In any of the above-described defective product data generation devices, the progress information acquisition unit may acquire at least one of the manufacturing step information related to a plurality of manufacturing steps or the progress step information related to a plurality of progress steps, and the defective product data generation unit may generate the defective product data related to the target object in accordance with at least two combinations of the plurality of manufacturing steps or at least two combinations of the plurality of progress steps.

[0017] Any of the above-mentioned defective product data generation devices may include a supplementary information acquisition unit that acquires supplementary information for the defective product data according to the manufacturing step or the progress step of the object, and an output unit that outputs the defective product data generated based on the progress information and the supplementary information corresponding to the defective product data.

[0018] Any of the above-mentioned defective product data generation devices may include a warning unit that issues a warning when there is a possibility that a defect will occur in the object due to the manufacturing step or the progress step of the object acquired by the progress information acquisition unit.

[0019] Any of the above-described defective product data generation devices may further include a process condition acquisition unit that acquires process conditions for the manufacturing steps for manufacturing the target product, and the warning unit may issue a warning in accordance with the process conditions.

[0020] Any of the above-mentioned defective product data generation devices may include a feedback unit that feeds back information to reduce the probability of defects occurring in the object, depending on the manufacturing step or the progress step of the object acquired by the progress information acquisition unit.

[0021] In a second aspect of the present invention, there is provided a defective product data generation method comprising the steps of: a computer acquiring good product data relating to a predetermined object; a computer acquiring progress information including at least one of manufacturing step information relating to a manufacturing step of the object or progress step information relating to a progress step after the object is manufactured; and a computer generating defective product data relating to the object using a predetermined generative model based on the good product data and the progress information.

[0022] In a third aspect of the present invention, there is provided a program that, when executed by a computer, causes the computer to acquire good product data regarding a predetermined object, acquire progress information including at least one of manufacturing step information regarding a manufacturing step of the object or progress step information regarding a progress step after the manufacturing of the object, and generate defective product data regarding the object using a predetermined generative model based on the good product data and the progress information.

[0023] In a fourth aspect of the present invention, there is provided a defect-occurrence step identification device comprising: a defective product data acquisition unit that acquires defective product original data related to a predetermined object; a progress information acquisition unit that acquires progress information including at least one of manufacturing step information related to a manufacturing step of the object or progress step information related to a progress step after the object is manufactured; and a defect-occurrence step identification unit that uses a predetermined machine learning model based on the defective product original data and the progress information to identify a defect-occurrence step at which a defect occurred in the object.

[0024] The above summary of the invention does not list all of the features of the present invention, and subcombinations of these features may also be inventions.

[0025] 1 shows an overview of the configuration of the defective data generation device 100. 1 shows an example of manufacturing steps and progress steps. 1 shows an example of defective data generation based on manufacturing step information. 1 shows an example of defective data generation based on manufacturing step information. 1 shows an example of defective data generation based on manufacturing step information. 1 shows an example of defective data generation based on manufacturing step information. 1 shows an example of defective data generation based on progress step information. 1 shows an example of defective data generation based on progress step information. 1 shows a modified example of the defective data generation device 100. 1 shows a modified example of the defective data generation device 100. 1 shows an overview of the configuration of the defect occurrence step identification device 200. 1 shows an example of a computer 2200 in which multiple aspects of the present invention may be embodied in whole or in part.

[0026] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the scope of the invention as claimed. Furthermore, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0027] FIG. 1 shows an overview of the configuration of a defective product data generation device 100. The defective product data generation device 100 of this example includes a non-defective product data acquisition unit 110, a historical information acquisition unit 115, and a defective product data generation unit 120. The defective product data generation device 100 may also include a model storage unit 125. The illustrated blocks are functionally separated functional blocks and may not necessarily correspond to the actual device configuration. That is, blocks shown as a single block in this diagram may not necessarily be configured by a single device. Furthermore, blocks shown as separate blocks in this diagram may not necessarily be configured by separate devices. The same applies to blocks in other diagrams.

[0028] The defective product data generation device 100 generates defective product data for a predetermined object based on non-defective product data and historical information for the object. The object may be an industrial product, food, or household item, but is not limited to these.

[0029] The non-defective product data acquisition unit 110 acquires non-defective product data related to a predetermined target object. The non-defective product data may include at least one of an image, three-dimensional data, drawing data, or video. However, the type of non-defective product data is not limited to these. The non-defective product data acquisition unit 110 may acquire multiple non-defective product data. The non-defective product data acquisition unit 110 may acquire multiple non-defective product data of the same type, or may acquire multiple types of non-defective product data. The non-defective product data acquisition unit 110 may supply the acquired non-defective product data to the defective product data generation unit 120.

[0030] The progress information acquisition unit 115 acquires progress information including at least one of manufacturing step information related to a manufacturing step of the object or progress step information related to a progress step after the object is manufactured. The manufacturing steps and progress steps will be described later. The progress information acquisition unit 115 may acquire multiple pieces of progress information. For example, the progress information acquisition unit 115 acquires at least one of manufacturing step information related to multiple manufacturing steps or progress step information related to multiple progress steps. The progress information acquisition unit 115 may supply the acquired progress information to the defective product data generation unit 120.

[0031] The manufacturing step information may include at least one of words, sentences, images, videos, or structured data that indicate a manufacturing step of the object. The manufacturing step information may include information about defects related to a product to which the object is applied, the material of the object, or a manufacturing step of the object. For example, the manufacturing step information may include a dataset of images of good and bad products that occurred during an actual manufacturing step, sentences indicating the cause of the defect, and the rate of good and / or bad products in products similar to the object.

[0032] When the manufacturing step information includes a word or a sentence, the word or sentence may indicate a manufacturing step of the object, such as "pressing," "cooling," "molding," "casting," or "etching." For example, the manufacturing step information may include product information such as "for automobile parts" or "for metal products," information on the material of the object, such as "made of metal" or "wood," and information on defects related to the manufacturing step of the object, such as "defect during pressing" or "defect during processing."

[0033] When the manufacturing step information includes an image, the image may or may not include the object. For example, the manufacturing step information may include an image of the object itself, or may include an image of a manufacturing device for manufacturing the object. The manufacturing step information may include at least one of an image during the manufacturing step and an image after the manufacturing step. For example, if the manufacturing step is casting, the manufacturing step information may include an image of the object and / or the casting device during casting, or may include an image of the object and / or the casting device after casting.

[0034] When the manufacturing step information includes a video, the manufacturing step information may include a video during the manufacturing step. For example, the manufacturing step information may include a video of a manufacturing device during the manufacturing step, or may include a video of an object during the manufacturing step.

[0035] The structured data may be data structured in rows and columns. When the manufacturing step information includes structured data, the manufacturing step information may include structured data indicating manufacturing conditions for each of a plurality of manufacturing steps. For example, the manufacturing step information includes structured data in a matrix format in which one of the rows or columns corresponds to a manufacturing step such as "pressurization" and the other corresponds to a manufacturing condition such as "pressure."

[0036] The process step information may include at least one of words, sentences, images, videos, or structured data indicating a process step after manufacturing of the object. The process step information may include information on defects related to the process step of the object. For example, the process step information may include a dataset of images of good and defective products that occurred during the actual process step, may include sentences indicating the cause of the defect, and may include the rate of good and / or defective products in products similar to the object.

[0037] When the process step information includes a word or a sentence, the word or sentence may indicate a process step of the object, such as “transportation,” “installation,” or “use.” For example, the process step information may include defect information related to a process step of the object, such as “corrosion in water,” “weathering in the atmosphere,” “aging,” or “corrosion.”

[0038] When the progress step information includes an image, the image may or may not include the object. For example, the progress step information may include an image of the object itself, an image of a product to which the object is applied, or an image of the environment in which the object is used. The progress step information may include at least one of an image during the progress step or an image after the progress step. For example, if the progress step is "use underwater," the progress step information may include an image of the object being used underwater, or an image of the object after being used underwater.

[0039] When the progress step information includes a video, the progress step information may include a video during the progress step. For example, the progress step information may include a video of an environment in which an object is placed during the progress step, or may include a video of the object during the progress step.

[0040] When the progress step information includes structured data, the progress step information may include structured data indicating progress conditions for each of a plurality of progress step information. For example, the progress step information includes structured data in a matrix format in which one of the rows or columns corresponds to a progress step such as "transportation" and the other corresponds to a progress condition such as "time."

[0041] The defective product data generation unit 120 generates defective product data for the target object using a predetermined generation model 126 based on the non-defective product data and historical information. The defective product data may include at least one of images, three-dimensional data, drawing data, and video. However, the types of defective product data are not limited to these. The defective product data generation unit 120 may generate multiple pieces of defective product data. The defective product data generation unit 120 may generate multiple pieces of the same type of defective product data, or multiple types of defective product data. For example, the defective product data generation unit 120 may generate defective product images, sentences indicating the causes of defects, or data indicating the probability of defects occurring.

[0042] The defective product data generation unit 120 may generate defective product data using the generative model 126 by specifying at least one of the type of the target product, the generative model 126, position information of the defective parts to be generated, morphological information of the defective parts to be generated, the defect mode of the defective parts to be generated, or the number of defective parts to be generated.

[0043] The type of the target product may be, for example, a metal product, a pharmaceutical product, a glass product, or an automobile part. The type of the target product may be a further subdivision of the above products. For example, the type of the target product may be a bearing, a washer, a screw, a pill, a syringe container, a bottle, an engine part, or an automobile body. However, the type of the target product is not limited to these.

[0044] The generative model 126 may be a model that has been trained in advance to generate defective product data, such as a generative adversarial network (GAN) or a diffusion model. However, the type of the generative model 126 is not limited to these. The generative model 126 may be additionally trained.

[0045] The generated position information of the defective portion may be information on the position where the defective portion is to be provided in the non-defective data. The position information of the defective portion may be specified in a paint area, may be specified by manually specifying points, or may be specified by a grid with an arbitrary interval. The point specification may specify the position of the defective portion on the non-defective data, or may specify the position of the defective portion by coordinates. The position information of the defective portion may be specified in two dimensions or in three dimensions.

[0046] The generated morphological information of the defective portion may include at least one of the area, major axis, angle, degree of distortion, shape, scratch depth, color, brightness, contrast, and texture information of the defective portion. The morphology of the defective portion may be specified by a magnification of the defective portion when applying the defective portion to the non-defective data, the number of area pixels, the number of major axis pixels, etc.

[0047] The failure mode of the generated defect portion may refer to at least one of a defect, a scratch, a dent, a distortion, a noise, a crack, or an attachment at the defect portion, but the types of the failure mode of the defect portion are not limited to these.

[0048] The number of defect locations to be generated may be the number of defect locations to be applied to the non-defective data. The number of defect locations may be any integer equal to or greater than 1. When the number of defect locations is two or more, other information such as position information of the defect locations, shape information of the defect locations, and / or failure modes of the defect locations may be specified for each defect location.

[0049] The defective product data generation unit 120 may generate defective product data using a generative model 126 stored in the model storage unit 125. The model storage unit 125 may store a plurality of generative models 126. The defective product data generation unit 120 may select a generative model 126 corresponding to the progress information from the plurality of generative models 126 stored in the model storage unit 125, and generate defective product data using the selected generative model 126. The defective product data generation unit 120 may input good product data and progress information to the generative model 126. The generative model 126 may output defective product data in response to the input of good product data and progress information from the defective product data generation unit 120.

[0050] The defective product data generating unit 120 may generate defective product data for the target object according to at least two combinations of a plurality of manufacturing steps or at least two combinations of a plurality of progress steps. For example, when the progress information acquiring unit 115 acquires at least one of manufacturing step information for a plurality of manufacturing steps or progress step information for a plurality of progress steps, the defective product data generating unit 120 may generate defective product data according to the combination.

[0051] The defective product data generating device 100 of this example generates defective product data based on progress information including manufacturing step information and / or progress step information. This allows the defective product data generating device 100 to generate natural defective product data corresponding to the progress information. The defective product data generating device 100 of this example can provide examples of defects of objects that serve as the basis for AI inspections that determine defects in objects. The defective product data generating device 100 can provide examples of possible defects to the discriminator even when defects in objects are manually determined in an object manufacturing factory, etc. Furthermore, by comparing the defective product data generated by the defective product data generating device 100 with defects that actually occurred, the manufacturing step and / or progress step in which the defect occurred can be identified. Furthermore, the defective product data generated by the defective product data generating device 100 can be visualized and provided to the user of the object as examples of defects that may occur depending on the progress step of the object.

[0052] 2 shows an example of manufacturing and process steps. The object in this example is a screw, but the object is not limited to this.

[0053] The target screw is manufactured through, for example, step S100 of wiredrawing a material, step S110 of heading the head of the screw, step S120 of rolling the thread, step S130 of heat treating the screw, and step S140 of surface treating the screw by plating or the like. These manufacturing processes may be manufacturing steps. As an example, if the progress information acquired by the progress information acquisition unit 115 includes manufacturing step information related to the rolling step S120, the defective product data generation unit 120 may generate defective product data related to defects that may occur in the rolling step S120.

[0054] After the manufacturing of the target screw is completed, for example, it is transported (step S200), installed (step S210), and used (step S220). These processes after the manufacturing end may be the progress steps. As an example, if the progress information acquired by the progress information acquisition unit 115 includes progress step information related to the transportation step S200, the defective product data generation unit 120 may generate defective product data related to defects that may occur in the transportation step S200.

[0055] In the above example, the object is a screw, but the same applies when the object is another product. The manufacturing step may indicate at least one step in the process of manufacturing the object, and the process step may indicate at least one step in the processing after the object is manufactured.

[0056] 3A shows an example of defective product data generation based on manufacturing step information. The manufacturing step information in this example includes words that indicate the manufacturing step of the target object. As an example, the manufacturing step information includes the character string "defect during processing."

[0057] The non-defective data acquisition unit 110 of this example acquires a non-defective image of a screw. The progress information acquisition unit 115 of this example acquires a character string "damage during processing" as manufacturing step information related to the manufacturing step of the screw. The defective data generation unit 120 of this example generates a defective image of the screw by applying a defective defect to the non-defective image of the screw based on the non-defective image of the screw and the character string "damage during processing."

[0058] 3B shows an example of generating defective product data based on manufacturing step information. The manufacturing step information in this example includes images showing manufacturing steps of the target object. As an example, the manufacturing step information includes images showing defects during processing.

[0059] The non-defective data acquisition unit 110 of this example acquires non-defective images of the screw. The progress information acquisition unit 115 of this example acquires images showing defects caused during processing as manufacturing step information related to the screw manufacturing steps. The defective data generation unit 120 of this example generates a defective image of the screw by applying the defective defects to the non-defective image of the screw, based on the non-defective image of the screw and the image showing the defects caused during processing.

[0060] When the manufacturing step information includes an image indicating a defect, the position of the defect shown in the image and the position of the defect in the defective product data generated by the defective product data generation unit 120 may be the same or different. That is, the defective product data generation unit 120 may specify any position as the position information of the defect location to be generated. The same applies to the shape information of the defect location or the number of defect locations. The defective product data generation unit 120 may generate defective product data that includes defects of a different shape from the shape of the defect shown in the image included in the manufacturing step information and / or a different number of defects from the number of defects shown in the image included in the manufacturing step information.

[0061] 3C shows an example of generating defective product data based on manufacturing step information. The manufacturing step information in this example includes structured data indicating the manufacturing steps of the target object. As an example, the manufacturing step information includes structured data in a matrix format, where columns correspond to manufacturing steps and rows correspond to manufacturing conditions.

[0062] The non-defective product data acquisition unit 110 of this example acquires non-defective product images of screws. The progress information acquisition unit 115 of this example acquires structured data indicating the correspondence between manufacturing steps and manufacturing conditions as manufacturing step information related to the manufacturing steps of the screws. The progress information acquisition unit 115 may acquire manufacturing step information related to multiple manufacturing steps. The defective product data generation unit 120 of this example generates defective product images of the screws by applying defective defects to the non-defective product images of the screws based on the non-defective product images of the screws and the structured data. The defective product data generation unit 120 may generate defective product data related to the target object according to at least two combinations of the multiple manufacturing steps.

[0063] 4A shows an example of generating defective product data based on elapsed step information. The elapsed step information in this example includes words indicating elapsed steps after the manufacturing of the target object. As an example, the elapsed step information includes the character string "aging."

[0064] The non-defective product data acquisition unit 110 of this example acquires a non-defective product image of a screw. The progress information acquisition unit 115 of this example acquires a character string "aging" as progress step information relating to progress steps after the manufacture of the screw. The defective product data generation unit 120 of this example generates a defective product image of the screw by applying rust, which is a defect, to the non-defective product image of the screw, based on the non-defective product image of the screw and the character string "aging." However, defects caused by aging are not limited to rust.

[0065] When the elapsed step is the aging of an object, the defective product data generation unit 120 may output, as defective product data related to the object, a defective product image of the aged object and data on the number of years since the aging. In this example, the defective product data generation unit 120 outputs, as defective product data, a defective product image of a screw that has rusted due to aging and data on the number of years since the aging. The defective product data generation unit 120 may output, for each of a plurality of pieces of elapsed age data, defective product images of the object that have aged over time. For example, the defective product data generation unit 120 outputs, as defective product data, elapsed age data indicating that the number of years since the aging is five years and a defective product image of the object that has aged over five years, and elapsed age data indicating that the number of years since the aging is ten years and a defective product image of the object that has aged over ten years.

[0066] 4B shows an example of generating defective product data based on progress step information. The progress step information in this example includes images showing progress steps after the manufacturing of the object. As an example, the progress step information includes images showing rust.

[0067] The non-defective data acquisition unit 110 of this example acquires non-defective images of the screw. The progress information acquisition unit 115 of this example acquires images showing rust as progress step information relating to progress steps after the manufacture of the screw. The defective data generation unit 120 of this example generates a defective image of the screw by applying defective rust to the non-defective image of the screw, based on the non-defective image of the screw and the image showing rust.

[0068] When the elapsed step information includes an image indicating a defect, the position of the defect shown in the image and the position of the defect in the defective product data generated by the defective product data generation unit 120 may be the same or different. That is, the defective product data generation unit 120 may specify any position as the position information of the defect location to be generated. The same applies to the shape information of the defect location or the number of defect locations. The defective product data generation unit 120 may generate defective product data that includes defects of a different shape from the shape of the defect shown in the image included in the elapsed step information and / or a different number of defects from the number of defects shown in the image included in the elapsed step information.

[0069] 4C shows an example of generating defective product data based on progress step information. The progress step information in this example includes structured data indicating progress steps after manufacturing of the target object. As an example, the progress step information includes structured data in a matrix format in which columns correspond to progress steps and rows correspond to progress conditions.

[0070] The non-defective product data acquisition unit 110 of this example acquires a non-defective product image of a screw. The progress information acquisition unit 115 of this example acquires structured data indicating the correspondence between progress steps and progress conditions as progress step information regarding progress steps after the manufacture of the screw. The progress information acquisition unit 115 may acquire progress step information regarding a plurality of progress steps. The defective product data generation unit 120 of this example generates a defective product image of the screw by applying a defect, rust, to the non-defective product image of the screw based on the non-defective product image of the screw and the structured data. The defective product data generation unit 120 may generate defective product data regarding the object according to at least two combinations of the plurality of progress steps.

[0071] 5 shows a modified example of the defective product data generation device 100. The defective product data generation device 100 of this example differs from the embodiment of FIG. 1 in that it includes a defect occurrence data acquisition unit 130, a paired data storage unit 135, a generative model learning unit 140, a defect occurrence step designation unit 145, and a defect occurrence time prediction unit 150. In this example, differences from the embodiment of FIG. 1 will be particularly described, and the rest may be the same as the embodiment of FIG. 1.

[0072] The defect occurrence data acquiring section 130 may acquire defect occurrence data relating to defect locations occurring in the manufacturing steps and / or elapsed steps. The defect occurrence data acquiring section 130 may acquire defect occurrence data relating to defect locations occurring in each of a plurality of manufacturing steps, or may acquire defect occurrence data relating to defect locations occurring in each of a plurality of elapsed steps. The defect occurrence data acquiring section 130 may supply the acquired defect occurrence data to the pair data storage section 135.

[0073] The paired data storage section 135 may store a manufacturing step and / or a progress step and a defective portion as paired data. The paired data storage section 135 may store a plurality of manufacturing steps and a defective portion occurring in each of the plurality of manufacturing steps as paired data, or may store a plurality of progress steps and a defective portion occurring in each of the plurality of progress steps as paired data.

[0074] The paired data storage unit 135 may update the stored paired data. For example, when new data is obtained for one of the manufacturing steps or defective locations that constitute the paired data, the paired data storage unit 135 updates the existing paired data as new paired data by combining the obtained new data with the other. Furthermore, when new data is obtained for one of the progress steps or defective locations that constitute the paired data, the paired data storage unit 135 may update the existing paired data as new paired data by combining the obtained new data with the other.

[0075] The generative model learning unit 140 may learn the generative model 126 based on the paired data stored in the paired data storage unit 135. For example, when a new defect occurs, the paired data storage unit 135 selects a defect similar to the new defect, generates paired data of the similar defect, and stores it. Then, the generative model learning unit 140 may select the generative model 126 learned based on the similar defect, and additionally learn the generative model 126 based on the paired data stored in the paired data storage unit 135. This allows the generative model 126 to be updated even when an unlearned defect occurs. Furthermore, the time required for pre-learning the generative model 126 can be reduced.

[0076] The defective product data generation unit 120 may specify a manufacturing step and / or a progress step and a failure mode of a defective part occurring in the corresponding step stored as paired data, and generate defective product data using the generative model 126. This allows the defective product data generation device 100 to generate natural defective product data according to the progress information.

[0077] The defect occurrence step designation unit 145 may designate a defect occurrence step in which a defect occurs. The defective product data generation unit 120 may generate defective product data in which a defect that may occur in the designated defect occurrence step is created in the object. For example, if the object is a screw and the defect occurrence step designation unit 145 designates a rolling step in the manufacturing step as the defect occurrence step, the defective product data generation unit 120 generates defective product data in which a defect that may occur in the rolling step is created in the screw. As another example, if the object is a screw and the defect occurrence step designation unit 145 designates a use step in the elapsed step as the defect occurrence step, the defective product data generation unit 120 may generate defective product data in which a defect that may occur in the use step is created in the screw.

[0078] The defect occurrence time prediction unit 150 may predict a defect occurrence time at which a defect occurs in a defect occurrence step. For example, when the target object is a screw and the defect occurrence step designation unit 145 designates a rolling step of the manufacturing steps as a defect occurrence step, the defect occurrence time prediction unit 150 may predict the time from the start of rolling until the defect occurs as the defect occurrence time at which the defect occurs. Alternatively, the defect occurrence time prediction unit 150 may further classify the rolling step into smaller processes and predict a process among the subdivided processes at which a defect occurs as the defect occurrence time at which the defect occurs.

[0079] The defective product data generation device 100 of this example includes a defect occurrence time prediction unit 150. As a result, the defective product data generation device 100 can predict the time when a defect will occur in accordance with the historical information, in addition to or instead of generating natural defective product data in accordance with the historical information.

[0080] 6 shows a modified example of the defective product data generation device 100. The defective product data generation device 100 of this example differs from the embodiment of FIG. 1 in that it includes a storage unit 155, a process condition acquisition unit 160, a warning unit 165, and a feedback unit 170. In this example, differences from the embodiment of FIG. 1 will be particularly described, and the rest may be the same as the embodiment of FIG. 1.

[0081] The storage unit 155 may store the progress information in association with the learning data used to train the generative model 126. The defective product data generation unit 120 may generate defective product data by specifying the generative model 126. The defective product data generation unit 120 may generate defective product data by specifying the generative model 126 that has been trained using the learning data stored in association with the progress information acquired by the progress information acquisition unit 115.

[0082] The process condition acquisition unit 160 may acquire process conditions for manufacturing steps for manufacturing the target object, and may provide the acquired process conditions to the warning unit 165.

[0083] The warning unit 165 may issue a warning when a manufacturing step or progress step of the object acquired by the progress information acquisition unit 115 indicates a possibility of a defect occurring in the object. For example, the warning unit 165 issues a warning when the progress information includes a manufacturing step or progress step that may result in a defect in the object. The warning unit 165 may issue a warning depending on the process conditions. For example, the warning unit 165 issues a warning when the process conditions acquired by the process condition acquisition unit 160 include a process condition that may result in a defect in the object.

[0084] The defective product data generation device 100 of this example includes a warning unit 165 that issues a warning when there is a possibility that a defect will occur in the object. As a result, when a warning is issued from the warning unit 165, the manufacturer and / or user of the object can review the manufacturing steps and / or progress steps to reduce the possibility of a defect occurring in the object. Furthermore, the defective product data generation device 100 of this example includes a process condition acquisition unit 160 in addition to the warning unit 165, and the warning unit 165 issues a warning according to the process conditions. As a result, when a warning is issued from the warning unit 165, the manufacturer of the object can review the process conditions of the manufacturing steps for manufacturing the object to reduce the possibility of a defect occurring in the object.

[0085] The feedback unit 170 may feed back information for reducing the probability of defects occurring in the object, depending on the manufacturing steps or progress steps of the object acquired by the progress information acquisition unit 115. For example, the feedback unit 170 may feed back information suggesting changing a manufacturing step or progress step with a high probability of defects to another manufacturing step or progress step with a low probability of defects. As another example, the feedback unit 170 may feed back information suggesting changing the process conditions of the manufacturing step to reduce the probability of defects. Since the defective product data generation device 100 of this example includes the feedback unit 170, the manufacturer and / or user of the object can review the manufacturing steps and / or progress steps to reduce the possibility of defects occurring in the object.

[0086] 7 shows a modified example of the defective product data generation device 100. The defective product data generation device 100 of this example differs from the embodiment of FIG. 1 in that it includes an original data acquisition unit 175, an evaluation unit 180, a supplemental information acquisition unit 185, and an output unit 190. In this example, differences from the embodiment of FIG. 1 will be particularly described, and the rest may be the same as the embodiment of FIG. 1.

[0087] The original data acquisition unit 175 may acquire original data of a defective object. The original data may include at least one of an image, three-dimensional data, drawing data, or video. However, the type of original data is not limited to these. The original data acquisition unit 175 may acquire multiple pieces of original data of a defective object. The original data acquisition unit 175 may acquire multiple pieces of original data of the same type, or may acquire multiple types of original data. The original data acquisition unit 175 may supply the acquired original data to the evaluation unit 180.

[0088] The original data acquiring unit 175 may acquire original data corresponding to the progress information used to generate the defective product data. For example, if the manufacturing step information and / or progress step information includes words, sentences, images, videos, or structured data indicating the manufacturing steps and / or progress steps, the original data acquiring unit 175 may acquire original data of the defective product of the object associated with the words, sentences, images, videos, or structured data. As another example, if the progress information includes an image or video of the defective product, the original data acquiring unit 175 may acquire the image or video of the defective product as original data.

[0089] The evaluation unit 180 may evaluate the defective product data based on the original data. The evaluation unit 180 may evaluate whether the generated defective product data is natural based on the original data. If the original data includes an image or video of a defective product, the evaluation unit 180 may determine the similarity between the original data and the defective product data. For example, defective product data whose determined similarity is higher than a predetermined reference value may be stored, and defective product data whose determined similarity is lower than a predetermined reference value may be discarded. This can improve the quality of defective product data generation by the defective product data generation device 100.

[0090] The supplemental information acquisition unit 185 may acquire supplemental information for defective product data corresponding to the manufacturing step or progress step of the object. The supplemental information for defective product data may include at least one of words, sentences, images, videos, or structured data indicating the manufacturing step of the object, and may include at least one of words, sentences, images, videos, or structured data indicating the progress step after the manufacturing of the object. The supplemental information for defective product data may include information on defects related to the product to which the object is applied, the material of the object, or the manufacturing step and / or progress step of the object, may include sentences indicating the cause of the defect, may include the probability of the defect occurring, and may include the non-defective product rate and / or defective product rate of products similar to the object. The supplemental information acquisition unit 185 may supply the acquired supplemental information to the output unit 190.

[0091] The output unit 190 may output the defective product data generated based on the historical information and the supplementary information corresponding to the defective product data. For example, if the supplementary information includes a sentence indicating the cause of the defect, the output unit 190 outputs the defective product data and the cause of the defect corresponding to the defective product data. As another example, if the supplementary information includes the probability of the defect occurring, the output unit 190 may output the defective product data and the probability of the defect occurring corresponding to the defective product data. In this way, by the output unit 190 outputting the defective product data generated based on the historical information and the supplementary information corresponding to the defective product data, users of the defective product data can easily understand the supplementary information for the defective product data.

[0092] Each functional block illustrated in each of Figures 1 and 5 to 7 may be provided together with functional blocks illustrated in other figures. That is, the defect data generation device 100 according to the present invention may include all or any combination of the functional blocks illustrated in Figures 1 and 5 to 7.

[0093] 8 shows an outline of the configuration of the defect-occurrence step identifying device 200. The defect-occurrence step identifying device 200 of this example includes a defective product data acquiring unit 210, a progress information acquiring unit 215, and a defect-occurrence step identifying unit 220. The defect-occurrence step identifying device 200 may include a model storage unit 225.

[0094] The defect occurrence step identifying device 200 identifies a defect occurrence step in which a defect occurs in a predetermined object based on the defective product original data and historical information related to the object. The object may be an industrial product, a food product, or a daily necessities, but is not limited to these.

[0095] The defective product data acquisition unit 210 acquires original defective product data related to a predetermined object. The original defective product data may include at least one of an image, three-dimensional data, drawing data, or video. However, the type of original defective product data is not limited to these. The defective product data acquisition unit 210 may acquire multiple original defective product data. The defective product data acquisition unit 210 may acquire multiple original defective product data of the same type, or may acquire multiple types of original defective product data. The defective product data acquisition unit 210 may supply the acquired original defective product data to the defect occurrence step identification unit 220.

[0096] The progress information acquiring unit 215 acquires progress information including at least one of manufacturing step information related to a manufacturing step of the object or progress step information related to a progress step after the manufacturing of the object. The explanation of the progress information has been omitted as it has been explained with reference to Figures 1 to 7. That is, the progress information acquiring unit 215 included in the defect occurrence step identifying device 200 may have a configuration similar to that of the progress information acquiring unit 115 included in the defective product data generation device 100.

[0097] The defect-occurrence step identifying unit 220 identifies a defect-occurrence step at which a defect in the object occurred, based on the defective product original data and the progress information, using a predetermined machine learning model 226. The defect-occurrence step identifying unit 220 may identify at least one manufacturing step as the defect-occurrence step at which a defect in the object occurred, and may identify at least one progress step as the defect-occurrence step at which a defect in the object occurred.

[0098] The machine learning model 226 may be a model that has been trained in advance to identify a defect occurrence step, such as a Generative Adversarial Network (GAN) or a diffusion model. However, the type of the machine learning model 226 is not limited to these. The machine learning model 226 may be additionally trained.

[0099] The defect occurrence step identifying unit 220 may identify the defect occurrence step using a machine learning model 226 stored in the model storage unit 225. The model storage unit 225 may store a plurality of machine learning models 226. The defect occurrence step identifying unit 220 may select a machine learning model 226 corresponding to the progress information from the plurality of machine learning models 226 stored in the model storage unit 225, and identify the defect occurrence step using the selected machine learning model 226. The defect occurrence step identifying unit 220 may input the defective product original data and the progress information to the machine learning model 226. The machine learning model 226 may output the identified defect occurrence step in response to the input of the defective product original data and the progress information from the defect occurrence step identifying unit 220.

[0100] The defect-causing step identifying device 200 of this example identifies a defect-causing step based on progress information including manufacturing step information and / or progress step information. This allows the defect-causing step identifying device 200 to accurately identify the defect-causing step in the defective product original data where a defect occurred in the object. Furthermore, the manufacturer and / or user of the object can reduce the possibility of a defect occurring in the object by reviewing the identified defect-causing step.

[0101] As described above, the progress information acquisition unit 215 included in the defect occurrence step identification device 200 may have the same configuration as the progress information acquisition unit 115 included in the defective data generation device 100. Therefore, it can be understood that the defect occurrence step identification device 200 and the defective data generation device 100 can be configured in a composite manner as the same device.

[0102] 9 illustrates an example of a computer 2200 in which aspects of the present invention may be embodied, in whole or in part. Programs installed on the computer 2200 may cause the computer 2200 to function as or perform operations associated with an apparatus or one or more sections of the apparatus according to embodiments of the present invention, and / or to perform a process or steps of a process according to embodiments of the present invention. Such programs may be executed by the CPU 2212 to cause the computer 2200 to perform specific operations associated with some or all of the blocks of the flowcharts and block diagrams described herein.

[0103] A computer 2200 according to this embodiment includes a CPU 2212, a RAM 2214, a graphics controller 2216, and a display device 2218, which are interconnected by a host controller 2210. The computer 2200 also includes input / output units such as a communication interface 2222, a hard disk drive 2224, a DVD-ROM drive 2226, and an IC card drive, which are connected to the host controller 2210 via an input / output controller 2220. The computer also includes legacy input / output units such as a ROM 2230 and a keyboard 2242, which are connected to the input / output controller 2220 via an input / output chip 2240.

[0104] The CPU 2212 operates according to programs stored in the ROM 2230 and RAM 2214, thereby controlling each unit. The graphics controller 2216 acquires image data generated by the CPU 2212 into a frame buffer or the like provided in the RAM 2214 or into the graphics controller 2216 itself, and causes the image data to be displayed on the display device 2218.

[0105] The communication interface 2222 communicates with other electronic devices via a network. The hard disk drive 2224 stores programs and data used by the CPU 2212 in the computer 2200. The DVD-ROM drive 2226 reads programs or data from the DVD-ROM 2201 and provides the programs or data to the hard disk drive 2224 via the RAM 2214. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0106] ROM 2230 stores therein a boot program or the like that is executed by computer 2200 upon activation, and / or programs that depend on the hardware of computer 2200. I / O chip 2240 may also connect various I / O units to I / O controller 2220 via a parallel port, a serial port, a keyboard port, a mouse port, etc.

[0107] The programs are provided by a computer-readable medium such as a DVD-ROM 2201 or an IC card. The programs are read from the computer-readable medium, installed in the hard disk drive 2224, RAM 2214, or ROM 2230, which are also examples of computer-readable media, and executed by the CPU 2212. Information processing described in these programs is read by the computer 2200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be configured by realizing information manipulation or processing in accordance with the use of the computer 2200.

[0108] For example, when communication is performed between computer 2200 and an external device, CPU 2212 may execute a communication program loaded into RAM 2214 and instruct communication interface 2222 to perform communication processing based on the processing described in the communication program. Under the control of CPU 2212, communication interface 2222 reads transmission data stored in a transmission buffer processing area provided in RAM 2214, hard disk drive 2224, DVD-ROM 2201, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes received data received from the network to a reception buffer processing area or the like provided on the recording medium.

[0109] Furthermore, the CPU 2212 may cause all or a necessary portion of a file or database stored on an external recording medium such as the hard disk drive 2224, the DVD-ROM drive 2226 (DVD-ROM 2201), an IC card, etc. to be read into the RAM 2214, and may perform various types of processing on the data on the RAM 2214. The CPU 2212 then writes back the processed data to the external recording medium.

[0110] Various types of information, such as various types of programs, data, tables, and databases, may be stored on the recording medium and may undergo information processing. The CPU 2212 may perform various types of processing on data read from the RAM 2214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout this disclosure and specified by the instruction sequences of the programs, and write the results back to the RAM 2214. The CPU 2212 may also search for information in a file, database, etc. on the recording medium. For example, if multiple entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored on the recording medium, the CPU 2212 may search for an entry that matches a condition specified by the attribute value of the first attribute from among the multiple entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0111] The above-described programs or software modules may be stored in a computer-readable medium on or near the computer 2200. A recording medium such as a hard disk or RAM provided in a server system connected to a dedicated communication network or the Internet can also be used as a computer-readable medium, thereby providing the programs to the computer 2200 via the network.

[0112] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It will be apparent to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the claims that such modifications and improvements can also be included within the technical scope of the present invention.

[0113] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before," "prior to," etc., and that the processes can be performed in any order unless the output of a previous process is used in a subsequent process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," etc. for convenience, this does not mean that the processes must be performed in this order.

[0114] 100... Defective product data generation device, 110... Good product data acquisition unit, 115... Progress information acquisition unit, 120... Defective product data generation unit, 125... Model storage unit, 126... Generative model, 130... Defect occurrence data acquisition unit, 135... Pair data storage unit, 140... Generative model learning unit, 145... Defect occurrence step designation unit, 150... Defect occurrence time prediction unit, 155... Memory unit, 160... Process condition acquisition unit, 165... Warning unit, 170... Feedback unit, 175... Original data acquisition unit, 180... Evaluation unit, 185... Supplementary information acquisition unit, 190... Output unit, 200... Defect occurrence step identification device, 2 10...Defective product data acquisition unit, 215...Progress information acquisition unit, 220...Defect occurrence step identification unit, 225...Model storage unit, 226...Machine learning model, 2200...Computer, 2201...DVD-ROM, 2210...Host controller, 2212...CPU, 2214...RAM, 2216...Graphics controller, 2218...Display device, 2220...Input / output controller, 2222...Communication interface, 2224...Hard disk drive, 2226...DVD-ROM drive, 2230...ROM, 2240...Input / output chip, 2242...Keyboard

Claims

1. A defective product data generation device comprising: a good product data acquisition unit that acquires good product data regarding a predetermined object; a progress information acquisition unit that acquires progress information including at least one of manufacturing step information regarding a manufacturing step of the object or progress step information regarding a progress step after the object is manufactured; and a defective product data generation unit that generates defective product data regarding the object using a predetermined generation model based on the good product data and the progress information.

2. The defective product data generating device according to claim 1, wherein the non-defective product data includes at least one of an image, three-dimensional data, drawing data, or video.

3. The defective product data generation device according to claim 1, wherein the manufacturing step information includes at least one of words, sentences, images, videos, and structured data indicating the manufacturing steps of the object.

4. The defective product data generation device according to claim 1, wherein the progress step information includes at least one of a word, a sentence, an image, a video, or structured data indicating a progress step after the manufacturing of the object.

5. The defective product data generation device according to claim 1, wherein the defective product data generation unit generates the defective product data using the generative model by specifying at least one of the type of the target product, the generative model, position information of the defective parts to be generated, morphological information of the defective parts to be generated, failure mode of the defective parts to be generated, or the number of defective parts to be generated.

6. The defective product data generation device according to claim 1, comprising: a defect occurrence data acquisition unit that acquires defect occurrence data relating to defective locations that occur in the manufacturing steps and / or the elapsed steps; and a paired data storage unit that stores the manufacturing steps and / or the elapsed steps and the defective locations as paired data.

7. The defective product data generation device according to claim 6, further comprising a generative model learning unit that learns the generative model based on the paired data stored in the paired data storage unit.

8. The defective product data generation device according to claim 6, wherein the defective product data generation unit generates the defective product data using the generative model by specifying the manufacturing step and / or the progress step and a failure mode of the defective part that occurs in the corresponding step stored as the paired data.

9. The defective product data generation device according to claim 1, further comprising a storage unit that stores the progress information in association with the learning data used to train the generative model.

10. A defective product data generation device according to any one of claims 1 to 9, comprising: an original data acquisition unit that acquires original data of the defective product of the target object; and an evaluation unit that evaluates the defective product data based on the original data.

11. A defective product data generation device as described in any one of claims 1 to 9, further comprising a defect occurrence step designation unit that designates a defect occurrence step in which a defective part will occur, and the defective product data generation unit generates the defective product data in which a defective part that may occur in the designated defect occurrence step has been generated in the target object.

12. The defective product data generating device according to claim 11, further comprising a defect occurrence time prediction unit that predicts the time when the defective portion will occur during the defect occurrence step.

13. A defective product data generation device as claimed in any one of claims 1 to 9, wherein the step of progression is the deterioration of the object over time, and the defective product data generation unit outputs, as the defective product data relating to the object, a defective product image of the object that has deteriorated over time and data on the number of years that have passed since the deterioration.

14. A defective product data generation device as described in any one of claims 1 to 9, wherein the progress information acquisition unit acquires at least one of the manufacturing step information regarding a plurality of manufacturing steps or the progress step information regarding a plurality of progress steps, and the defective product data generation unit generates the defective product data regarding the target object according to at least two combinations of the plurality of manufacturing steps or at least two combinations of the plurality of progress steps.

15. A defective product data generation device as described in any one of claims 1 to 9, comprising: a supplementary information acquisition unit that acquires supplementary information for the defective product data according to the manufacturing step or the progress step of the object; and an output unit that outputs the defective product data generated based on the progress information and the supplementary information corresponding to the defective product data.

16. A defective product data generation device as described in any one of claims 1 to 9, further comprising a warning unit that issues a warning when there is a possibility that a defect will occur in the object due to the manufacturing step or the progress step of the object acquired by the progress information acquisition unit.

17. The defective product data generation device according to claim 16, further comprising a process condition acquisition unit that acquires process conditions for the manufacturing steps for manufacturing the target object, and the warning unit issues a warning in accordance with the process conditions.

18. A defective product data generation device as claimed in any one of claims 1 to 9, further comprising a feedback unit that feeds back information to reduce the probability of defects occurring in the object in accordance with the manufacturing step or progress step of the object acquired by the progress information acquisition unit.

19. A defective product data generation method comprising the steps of: a computer acquiring non-defective product data regarding a predetermined object; a computer acquiring progress information including at least one of manufacturing step information regarding a manufacturing step of the object or progress step information regarding a progress step after the object is manufactured; and a computer generating defective product data regarding the object using a predetermined generative model based on the non-defective product data and the progress information.

20. A program that, when executed by a computer, causes the computer to: acquire good product data regarding a predetermined object; acquire progress information including at least one of manufacturing step information regarding a manufacturing step of the object or progress step information regarding a progress step after the object is manufactured; and generate defective product data regarding the object using a predetermined generative model based on the good product data and the progress information.

21. A defect-occurrence step identification device comprising: a defective product data acquisition unit that acquires defective product original data related to a predetermined object; a progress information acquisition unit that acquires progress information including at least one of manufacturing step information related to a manufacturing step of the object or progress step information related to a progress step after the object is manufactured; and a defect-occurrence step identification unit that identifies a defect-occurrence step at which a defect occurred in the object using a predetermined machine learning model based on the defective product original data and the progress information.

Images