Defect data generation method and device, electronic equipment and storage medium

Abstract

The invention provides a defect data generation method and device, electronic equipment and a storage medium, and the method comprises the steps: generating a semantic protection mask based on an obtained text describing a defect; the semantic protection mask is used for determining the attribute of the defect in the image; generating a defect image according to the semantic protection mask and the random noise image; fusing the semantic protection mask, the defect image and the defect-free image to obtain a target defect image; wherein the defect-free image is obtained through a defect-free image data set; and obtaining a defect data set based on the target defect image. According to the method, the defect image is accurate through the semantic protection mask, and the high-quality target defect image is obtained through the defect-free image, so that the defect database is obtained, and the quality of the defect data is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, electronic device and storage medium for generating defect data. Background Art

[0002] In industrial production, product defect detection is an important link, and the efficiency and accuracy of defect detection directly affect product quality and production efficiency. Therefore, it is very important to collect a large amount of defect data to perform corresponding inspections on products.

[0003] In the related art, a diffusion model or a generation model is generally used to generate defect images. However, when the model is used to generate defect images, there is a situation of attribute confusion, which causes the generated defect data to be inaccurate and of low quality. Summary of the invention

[0004] The present application provides a method, device, electronic device and storage medium for generating defect data, so as to at least solve the above technical problems existing in the prior art.

[0005] According to a first aspect of the present application, a method for generating defect data is provided, the method comprising:

[0006] Generate a semantic protection mask based on the acquired text describing the defect; the semantic protection mask is used to determine the attribute of the defect in the image;

[0007] generating a defect image according to the semantic protection mask and the random noise image;

[0008] Fusion of the semantic protection mask, the defect image and the non-defect image to obtain a target defect image; wherein the non-defect image is obtained through a non-defect image dataset;

[0009] Based on the target defect image, a defect data set is obtained.

[0010] In one possible implementation, the generating of the semantic protection mask based on the acquired text describing the defect includes:

[0011] The text of the defect description is input into the semantic protection mask generation model for semantic understanding, noun blocks are extracted, and corresponding defect categories and confidence maps corresponding to the defect categories are generated based on the noun blocks. The confidence map corresponding to the defect category is used as a semantic protection mask; wherein the confidence map is used to characterize the size of the defect and the position of the defect in the image.

[0012] In one possible implementation, the defect categories include multiple ones, and the generating of the corresponding defect categories and the confidence graphs corresponding to the defect categories based on the noun blocks includes:

[0013] Based on the noun block, more than one defect category and a confidence map corresponding to each defect category are generated; the defect categories include: bulge, crack, and chipping.

[0014] In one possible implementation manner, generating a defect image according to the semantic protection mask and the random noise image includes:

[0015] Get a random noise image;

[0016] The random noise image and the semantic protection mask are input into a semantic protection attention module in an image generation model to obtain a defect image.

[0017] In one possible implementation manner, inputting the random noise image and the semantic protection mask into an image generation model to obtain a defect image includes:

[0018] Determining shape parameters of a defect and a position of the defect in an image based on the semantic protection mask;

[0019] The shape parameters of the defect and the position of the defect in the image are set in the random noise image to obtain a defect image.

[0020] In one possible implementation manner, the fusing the semantic protection mask, the defect image and the non-defective image to obtain a target defect image includes:

[0021] Acquire the defect-free image through a defect-free image dataset;

[0022] Inputting the defect image, the defect-free image and the semantic protection mask into an image fusion model for fusion to obtain a spliced ​​image;

[0023] The spliced ​​image is spliced ​​and expanded in channel dimension to obtain the target defect image.

[0024] In one possible implementation manner, the step of inputting the defective image, the defect-free image, and the semantic protection mask into an image fusion model for fusion includes:

[0025] The defect image is multiplied by the semantic protection mask to obtain a mask image of a corresponding defect category;

[0026] The mask image is added with the defect-free image and the semantic protection mask is subtracted.

[0027] In one possible implementation manner, after obtaining the defect data set, the method further includes:

[0028] Update the defect dataset.

[0029] In one possible implementation manner, the updating of the defect data set includes:

[0030] Update the text describing the defect according to the required target defect data to update the target defect image; wherein the required target defect data is defect data that needs to be added to the defect data set;

[0031] Update the defect dataset according to the updated target defect image.

[0032] In one possible implementation, the semantic protection mask generation model generates a corresponding confidence map for each defect category.

[0033] According to a second aspect of the present application, a device for generating defect data is provided, the device comprising:

[0034] A first generating module is used to generate a semantic protection mask based on the acquired text describing the defect; the semantic protection mask is used to determine the attribute of the defect in the image;

[0035] A second generating module, used for generating a defect image according to the semantic protection mask and the random noise image;

[0036] A fusion module, used for fusing the semantic protection mask, the defect image and the non-defect image to obtain a target defect image; wherein the non-defect image is obtained through a non-defect image dataset;

[0037] The determination module is used to obtain a defect data set based on the target defect image.

[0038] In one possible implementation, the first generation module includes:

[0039] An extraction unit is used to input the text describing the defect into the semantic protection mask generation model for semantic understanding, extract noun blocks, and generate corresponding defect categories and confidence maps corresponding to the defect categories based on the noun blocks, and use the confidence maps corresponding to the defect categories as semantic protection masks; wherein the confidence maps are used to characterize the size of the defect and the position of the defect in the image.

[0040] In one possible implementation manner, the defect category includes multiple categories, and the generating unit includes:

[0041] A generating subunit is used to generate one or more defect categories and a confidence map corresponding to each defect category based on the noun block; the defect categories include: bulge, crack, and chipping.

[0042] In one possible implementation, the second generation module includes:

[0043] A first acquisition unit, used for acquiring a random noise image;

[0044] The second generating unit is used to input the random noise image and the semantic protection mask into a semantic protection attention module in the image generation model to obtain a defect image.

[0045] In one possible implementation, the second generating unit includes:

[0046] A determining unit, configured to determine a shape parameter of a defect and a position of the defect in an image based on the semantic protection mask;

[0047] The setting unit is used to set the shape parameters of the defect and the position of the defect in the image in the random noise image to obtain a defect image.

[0048] In one possible implementation, the fusion module includes:

[0049] A second acquisition unit, configured to acquire the defect-free image through a defect-free image data set;

[0050] A fusion unit, used for inputting the defective image, the defect-free image and the semantic protection mask into an image fusion model for fusion to obtain a spliced ​​image;

[0051] The stitching unit is used to perform channel dimension stitching expansion on the stitched image to obtain a target defect image.

[0052] In one embodiment, the fusion unit comprises:

[0053] A first calculation subunit, configured to multiply the defect image by the semantic protection mask to obtain a mask image of a corresponding defect category;

[0054] The second calculation subunit is configured to add the defect-free image to the mask image and then subtract the semantic protection mask.

[0055] In one embodiment, the device further comprises:

[0056] The update module is used to update the defect dataset.

[0057] In one possible implementation, the update module includes:

[0058] A first updating unit, configured to update the text describing the defect according to the required target defect data, so as to update the target defect image; wherein the required target defect data is defect data that needs to be added to the defect data set;

[0059] The second updating unit is used to update the defect data set according to the updated target defect image.

[0060] In one possible implementation, the semantic protection mask generation model generates a corresponding confidence map for each defect category.

[0061] According to a third aspect of the present application, an electronic device is provided, including:

[0062] at least one processor; and

[0063] a memory communicatively connected to the at least one processor; wherein,

[0064] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present application.

[0065] According to a fourth aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method described in the present application.

[0066] In this application, firstly, a text describing a defect is received, and then a semantic protection mask is generated based on the obtained text describing the defect; the semantic protection mask is used to determine the attributes of the defect in the image; a defect image is generated based on the semantic protection mask and the random noise image; the semantic protection mask, the defect image and the non-defect image are fused to obtain a target defect image; wherein the non-defect image is obtained through a non-defect image data set; based on the target defect image, a defect data set is obtained. This application obtains a defect database by accurately obtaining a defect image through a semantic protection mask, and then obtains a high-quality target defect image through a non-defect image, thereby improving the quality of the defect data.

[0067] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] By reading the detailed description below with reference to the accompanying drawings, the above and other purposes, features and advantages of the exemplary embodiments of the present application will become readily understood. In the accompanying drawings, several embodiments of the present application are shown in an exemplary and non-limiting manner, wherein:

[0069] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0070] Figure 1 The schematic diagram shows the implementation process of the method for generating defect data in the embodiment of the present application. Figure 1 ;

[0071] Figure 2The schematic diagram shows the implementation process of the method for generating defect data in the embodiment of the present application. Figure 2 ;

[0072] Figure 3 The schematic diagram shows the implementation process of the method for generating defect data in the embodiment of the present application. Figure 3 ;

[0073] Figure 4 A schematic diagram showing the composition structure of a device for generating defect data according to an embodiment of the present application is shown;

[0074] Figure 5 A schematic diagram of the structure of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0075] In order to make the purpose, features, and advantages of the present application more obvious and easy to understand, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present application.

[0076] It can be understood that in industrial sites, a trained deep learning model is usually used to detect workpieces on the production line, and each workpiece is given an OK (good product) or NG (defective product) result. The embodiment of the present application provides a method for generating defect data, by describing the required defects in text, thereby generating a semantic protection mask based on the text, and the semantic protection mask can determine the attributes of the defect in the image, including size and position, and then generate a defect image, and then fuse the defect image and the defect-free image to obtain a high-quality target defect image, and save multiple target defect images to obtain a defect data set. The present application solves the semantic confusion problem through a semantic protection mask, and controls the generation size of the defect through the constraints of the semantic mask, so that the user can generate the required NG image by adjusting the description text.

[0077] The present application embodiment provides a method for generating defect data, such as Figure 1 As shown, the method includes:

[0078] S101: Generate a semantic protection mask based on the acquired text describing the defect; the semantic protection mask is used to determine the attribute of the defect in the image;

[0079] In this step, the text describing the defect can be the description text input by the user according to the desired defect data. It can be understood that the defect description text can include one or more defects. The attributes of the defect in the image in this application include the size of the defect and the position of the defect in the image. This application generates a semantic protection mask of the defect through text. The semantic protection mask can determine the size of the defect and the position of the defect in the image. The semantic protection mask is used to guide attention to the correct concept, thereby improving the consistency between the image and the text.

[0080] S102: Generate a defect image according to the semantic protection mask and the random noise image;

[0081] This step can be implemented by an image generation model, wherein the image generation model can adopt a Unet model of semantic segmentation or other models, which is not limited in this application. Taking the Unet model as an example, in this application, the semantic protection mask and the random noise image are input into the Unet model to generate the corresponding defect image.

[0082] It should be noted that the defect image in this application is generated in the model through a semantic protection mask generated by text description. Therefore, there are large areas of blank space in the defect image, and only the defective position has a defective image. The defect image is an intermediate product of the entire process.

[0083] S103: fusing the semantic protection mask, the defect image and the non-defect image to obtain a target defect image; wherein the non-defect image is obtained through a non-defect image dataset;

[0084] In this step, a defect-free image data set, that is, an OK set, is pre-set. The OK set contains images collected from normal samples in the industrial production line, that is, defect-free images. When performing this step, first randomly select a defect-free image from the OK set, and fuse the semantic protection mask, the defect image, and the defect-free image to obtain a target defect image, that is, the required defect image. It can be understood that the fusion method in this application can be through an image fusion model, or it can be other fusion methods that can achieve image fusion, and this application does not limit it here.

[0085] S104: Obtain a defect data set based on the target defect image.

[0086] In the present application, as the number of obtained target defect images increases, the defect data in the defect database increases, that is, the fuller the defect database is, the more defect types can be supported for similar defect retrieval.

[0087] The defect data generation method provided by the present application generates a semantic protection mask by acquiring defect description text, obtains a defect image based on the semantic protection mask and a random noise image, and then fuses the defect image with the semantic protection mask and the defect-free image to obtain a target defect image. As the number of target defect images increases, a defect data set is obtained. The present application solves the semantic confusion problem through a semantic protection mask, and controls the size of defect generation through the constraints of the semantic mask, so that the user can generate the required NG image by adjusting the description text. In addition, after generating the defect image, the present application fuses the defect image with the defect-free image to obtain a high-quality defect image and improve the quality of the defect data.

[0088] In some embodiments, generating a semantic protection mask based on the acquired text describing the defect includes:

[0089] The text of the defect description is input into the semantic protection mask generation model for semantic understanding, noun blocks are extracted, and corresponding defect categories and confidence maps corresponding to the defect categories are generated based on the noun blocks. The confidence map corresponding to the defect category is used as a semantic protection mask; wherein the confidence map is used to characterize the size of the defect and the position of the defect in the image.

[0090] It is understandable that when the defect data of the defect of the required defect is determined, the user can input the descriptive text of the defect, and then the semantic protection mask generation model semantically understands the input descriptive text to obtain the noun block, and the semantic protection mask generation model processes the noun block to generate the defect category corresponding to the noun block and the confidence map corresponding to the defect category. The semantic protection mask generation model in the present application can be but not limited to a diffusion model. The semantic protection mask generation model in the present application is trained in advance according to the recognition needs, and is specifically trained using NG images and text of similar scenes, so that the semantic protection mask generation model obtained after training can output the confidence map through the input text and random noise images. It should be noted that each point of the image output by the semantic protection mask generation model is no longer a pixel value, but a plurality of confidence maps of 0-1. Among them, the confidence is the probability that the pixel point is a certain type, and the probability is definitely between 0 and 1. The confidence map is that the value of each point is a floating point number, and the floating point number range is between 0 and 1.

[0091] For example, Figure 2 As shown in the figure, the input descriptive text is "a very small bulge", and the semantic protection mask generation model performs semantic understanding on "a very small bulge", obtains multiple noun blocks "bulge" and "very small", obtains the defect category of the bulge, and the confidence map of the bulge, which is the semantic protection mask.

[0092] In some embodiments, the defect categories include multiple ones, and generating corresponding defect categories and confidence maps corresponding to the defect categories based on the noun blocks includes:

[0093] Based on the noun block, more than one defect category and a confidence map corresponding to each defect category are generated; the defect categories include: bulge, crack, and chipping.

[0094] Among them, each category generates a corresponding confidence map.

[0095] It is understandable that the defect categories in this application may include multiple ones. For example, if the input text is "generate a crack and chipping", it corresponds to two types of defects: "crack" and "chipping". The model will automatically generate two confidence maps based on the extracted key terms, corresponding to the above two categories of "crack" and "chipping". The defect categories may also include other defects, which are not limited in this application.

[0096] In some embodiments, generating a defect image according to the semantic protection mask and the random noise image includes:

[0097] Get a random noise image;

[0098] The random noise image and the semantic protection mask are input into a semantic protection attention module in an image generation model to obtain a defect image.

[0099] It can be understood that a semantic protection attention module is provided in the image generation model in the present application. The present application first obtains a random noise image, and then inputs the semantic protection mask obtained in the above steps and the random noise image into the semantic protection attention module to generate the corresponding defect image. The semantic protection mask guides attention to the correct concept through the semantic protection attention module, thereby improving the consistency between the image and the text.

[0100] In some embodiments, inputting the random noise image and the semantic protection mask into an image generation model to obtain a defect image includes:

[0101] Determining shape parameters of a defect and a position of the defect in an image based on the semantic protection mask;

[0102] The shape parameters of the defect and the position of the defect in the image are set in the random noise image to obtain a defect image.

[0103] Specifically, the shape parameters of the defect in the present application include the size and length of the defect. In the present application, the defect size and the position of the defect in the image are set on the random noise image to obtain a defect image. The defect image has a large blank area, and only the defect position has a defective image.

[0104] In some embodiments, the fusing the semantic protection mask, the defect image and the non-defective image to obtain a target defect image includes:

[0105] Acquire defect-free images through a defect-free image dataset;

[0106] Inputting the defect image, the defect-free image and the semantic protection mask into an image fusion model for fusion to obtain a spliced ​​image;

[0107] The spliced ​​image is spliced ​​and expanded in channel dimension to obtain a target defect image.

[0108] Specifically, in this application, a defect-free image is obtained from a pre-set defect-free image data set, and then the defect image, the defect-free image and the semantic protection mask are input into the image fusion model for fusion to obtain a spliced ​​image, and then the spliced ​​image is spliced ​​and expanded in the channel dimension to obtain the final target defect image. The image generation model in this application can use but is not limited to various image generation models. This application takes the Unet model as an example. The semantic protection mask, defect image, and OK image are spliced ​​into one image, and then spliced ​​and expanded in the channel dimension, and the output is the final NG image.

[0109] In some embodiments, the step of inputting the defective image, the defect-free image, and the semantic protection mask into an image fusion model for fusion includes:

[0110] The defect image is multiplied by the semantic protection mask to obtain a mask image of the corresponding defect category;

[0111] The mask map is added to the defect-free image and the semantic protection mask is subtracted.

[0112] Specifically, in this application, the defect image is multiplied by the semantic protection mask and the OK image is multiplied by 1, and then the semantic mask is subtracted to obtain the fusion result, that is, the target defect image. It can be understood that the target defect image in this application is the image of the small part of the defect, and the confidence can be directly added, which is similar to cutting and pasting, to obtain the target defect image.

[0113] In some embodiments, such as Figure 3 As shown, the defect data generation method provided by the present application, after obtaining the defect data set, further includes:

[0114] S301, updating the defect data set.

[0115] It is understandable that in the present application, the defect data set can also be updated according to the update changes of the defect data, or when new categories of defects appear, the defect data also needs to be updated, so that the defect database is full and can support more defect types for similar defect retrieval.

[0116] In some embodiments, updating the defect dataset includes:

[0117] Update the text describing the defect according to the required target defect data to update the target defect image; wherein the required target defect data is defect data that needs to be added to the defect data set;

[0118] Update the defect dataset according to the updated target defect image.

[0119] It is understandable that when a new defect category appears, the text describing the defect can be updated according to the required target defect data, thereby generating a target defect image of the corresponding defect category, and then using the generated target defect image of the defect category to update the defect dataset. The required target defect data is the defect data that needs to be added to the defect dataset. For example, if the user wants to add defect data related to bulges, then the bulge image is the required target defect data.

[0120] The defect data generation method provided in this application solves the common semantic confusion problem of the diffusion model through the semantic protection mask, and controls the generation size of the defect through the constraint of the semantic mask, so that the user can generate the required NG image by adjusting the description text. Instead of directly generating the NG image, the defect image is generated, and then merged with the OK image to obtain a high-quality NG image.

[0121] like Figure 4 As shown, an embodiment of the present application provides a device for generating defect data, the device comprising:

[0122] The first generating module 401 is used to generate a semantic protection mask based on the acquired text describing the defect; the semantic protection mask is used to determine the attribute of the defect in the image;

[0123] A second generating module 402, configured to generate a defect image according to the semantic protection mask and the random noise image;

[0124] A fusion module 403 is used to fuse the semantic protection mask, the defect image and the non-defect image to obtain a target defect image; wherein the non-defect image is obtained through a non-defect image dataset;

[0125] The determination module 404 is used to obtain a defect data set based on the target defect image.

[0126] In some embodiments, the first generating module 401 includes:

[0127] An extraction unit is used to input the text describing the defect into the semantic protection mask generation model for semantic understanding, extract noun blocks, and generate corresponding defect categories and confidence maps corresponding to the defect categories based on the noun blocks, and use the confidence maps corresponding to the defect categories as semantic protection masks; wherein the confidence maps are used to characterize the size of the defect and the position of the defect in the image.

[0128] In some embodiments, the defect category includes multiple categories, and the generating unit includes:

[0129] A generating subunit is used to generate one or more defect categories and a confidence map corresponding to each defect category based on the noun block; the defect categories include: bulge, crack, and chipping.

[0130] In some embodiments, the second generating module 402 includes:

[0131] A first acquisition unit, used for acquiring a random noise image;

[0132] The second generating unit is used to input the random noise image and the semantic protection mask into a semantic protection attention module in the image generation model to obtain a defect image.

[0133] In some embodiments, the second generating unit includes:

[0134] A determining unit, configured to determine a shape parameter of a defect and a position of the defect in an image based on the semantic protection mask;

[0135] The setting unit is used to set the shape parameters of the defect and the position of the defect in the image in the random noise image to obtain a defect image.

[0136] In some embodiments, the fusion module 403 includes:

[0137] A second acquisition unit, configured to acquire a defect-free image through a defect-free image data set;

[0138] A fusion unit, used for inputting the defective image, the defect-free image and the semantic protection mask into an image fusion model for fusion to obtain a spliced ​​image;

[0139] The stitching unit is used to perform channel dimension stitching expansion on the stitched image to obtain a target defect image.

[0140] In some embodiments, the fusion unit comprises:

[0141] A first calculation subunit, configured to multiply the defect image by the semantic protection mask to obtain a mask image of a corresponding defect category;

[0142] The second calculation subunit is used for adding the defect-free image to the mask image and then subtracting the semantic protection mask.

[0143] In some embodiments, it is characterized in that the device further comprises:

[0144] The update module is used to update the defect dataset.

[0145] In some embodiments, the update module includes:

[0146] A first updating unit, used to update the text describing the defect according to the required target defect data, so as to update the target defect image;

[0147] The second updating unit is used to update the defect data set according to the updated target defect image.

[0148] In some embodiments, the semantic protection mask generation model generates a corresponding confidence map for each defect category.

[0149] It should be noted that the defect data generating device of the embodiment of the present application solves the problem based on a principle similar to that of the defect data generating method described above. Therefore, the implementation process, implementation principle and beneficial effects of the defect data generating device can all be referred to the description of the implementation process, implementation principle and beneficial effects of the described method, and the repeated parts will not be repeated here.

[0150] An embodiment of the present application provides an electronic device, including:

[0151] at least one processor; and

[0152] a memory communicatively connected to the at least one processor; wherein,

[0153] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of the above embodiments.

[0154] An embodiment of the present application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the method described in any of the above embodiments.

[0155] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0156] Figure 5A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0157] like Figure 5 As shown, the electronic device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0158] Multiple components in the electronic device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the electronic device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0159] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the generation method of defect data. For example, in some embodiments, the generation method of defect data may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded into the RAM 503 and executed by the computing unit 501, one or more steps of the generation method of defect data described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the generation method of defect data in any other appropriate manner (e.g., by means of firmware).

[0160] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0161] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0162] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0163] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0164] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0165] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0166] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0167] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0168] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for generating defect data, characterized in that: The method comprises: Generate a semantic protection mask based on the acquired text describing the defect; the semantic protection mask is used to determine the attribute of the defect in the image; generating a defect image according to the semantic protection mask and the random noise image; Fusion of the semantic protection mask, the defect image and the non-defect image to obtain a target defect image; wherein the non-defect image is obtained through a non-defect image dataset; Based on the target defect image, a defect data set is obtained.

2. The method according to claim 1, characterized in that: The generating of a semantic protection mask based on the acquired text describing the defect includes: The text of the defect description is input into the semantic protection mask generation model for semantic understanding, noun blocks are extracted, and corresponding defect categories and confidence maps corresponding to the defect categories are generated based on the noun blocks. The confidence map corresponding to the defect category is used as a semantic protection mask; wherein the confidence map is used to characterize the size of the defect and the position of the defect in the image.

3. The method according to claim 2, characterized in that The defect categories include multiple ones, and the generating of corresponding defect categories and confidence maps corresponding to the defect categories based on the noun blocks includes: Based on the noun block, more than one defect category and a confidence map corresponding to each defect category are generated; the defect categories include: bulge, crack, and chipping.

4. The method according to claim 1, characterized in that The step of generating a defect image according to the semantic protection mask and the random noise image comprises: Get a random noise image; The random noise image and the semantic protection mask are input into a semantic protection attention module in an image generation model to obtain a defect image.

5. The method according to claim 4, characterized in that The step of inputting the random noise image and the semantic protection mask into an image generation model to obtain a defect image includes: Determining shape parameters of a defect and a position of the defect in an image based on the semantic protection mask; The shape parameters of the defect and the position of the defect in the image are set in the random noise image to obtain a defect image.

6. The method according to claim 1, characterized in that The fusing the semantic protection mask, the defect image and the non-defect image to obtain a target defect image includes: Acquire the defect-free image through a defect-free image dataset; Inputting the defect image, the defect-free image and the semantic protection mask into an image fusion model for fusion to obtain a spliced ​​image; The spliced ​​image is spliced ​​and expanded in channel dimension to obtain the target defect image.

7. The method according to claim 6, characterized in that The step of inputting the defect image, the defect-free image, and the semantic protection mask into an image fusion model for fusion includes: The defect image is multiplied by the semantic protection mask to obtain a mask image of a corresponding defect category; The mask image is added with the defect-free image and the semantic protection mask is subtracted.

8. A defect data generating device, characterized in that: The device comprises: A first generating module is used to generate a semantic protection mask based on the acquired text describing the defect; the semantic protection mask is used to determine the attribute of the defect in the image; A second generating module, used for generating a defect image according to the semantic protection mask and the random noise image; A fusion module, used for fusing the semantic protection mask, the defect image and the non-defect image to obtain a target defect image; wherein the non-defect image is obtained through a non-defect image dataset; The determination module is used to obtain a defect data set based on the target defect image.

9. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.

Images