Industrial defect image generation method and device, storage medium and electronic device

Abstract

The present disclosure provides an industrial defect image generation method, an industrial defect image generation device, a computer readable storage medium and an electronic device, and relates to the technical field of image processing. The industrial defect image generation method comprises: performing an encoding processing process on a to-be-de-noised industrial defect image to obtain an intermediate feature; the encoding processing process comprises at least one first down-sampling process and at least one second down-sampling process, the first down-sampling process comprises an operation of extracting context information, and the second down-sampling process is realized based on a Transformer structure; performing a decoding processing process on the intermediate feature to generate a de-noised industrial defect image; the decoding processing process comprises at least one first up-sampling process and at least one second up-sampling process, the first up-sampling process is realized based on a Transformer structure, and the second up-sampling process comprises an operation of extracting context information. The present disclosure can generate an industrial defect image, and further expand sample data for model training.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of image processing, and in particular, to an industrial defect image generation method, an industrial defect image generation apparatus, a computer readable storage medium and an electronic device. BACKGROUND

[0002] In modern industrial scenarios, machine learning can be used to analyze possible industrial defects in industrial images to guide production and research and development. The performance of a machine learning model depends largely on the diversity and richness of the training data.

[0003] However, it is difficult to accumulate a large number of industrial defect images in the current industrial scenario, resulting in insufficient sample data for training machine learning models.

[0004] It should be noted that the information disclosed in the above background section is only used to strengthen the understanding of the background of the present disclosure, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0005] The purpose of the present disclosure is to provide an industrial defect image generation method, an industrial defect image generation apparatus, a computer readable storage medium and an electronic device, thereby at least partially overcoming the problem of insufficient industrial defect image sample data.

[0006] According to a first aspect of the present disclosure, an industrial defect image generation method is provided, comprising: obtaining an industrial defect image to be denoised; performing an encoding process on the industrial defect image to be denoised to obtain intermediate features; wherein the encoding process comprises at least one first down-sampling process and at least one second down-sampling process, the first down-sampling process comprises an operation of extracting context information, and the second down-sampling process is implemented based on a Transformer structure; performing a decoding process on the intermediate features to generate a denoised industrial defect image corresponding to the industrial defect image to be denoised; wherein the decoding process comprises at least one first up-sampling process and at least one second up-sampling process, the first up-sampling process is implemented based on a Transformer structure, and the second up-sampling process comprises an operation of extracting context information.

[0007] Optionally, obtaining the industrial defect image to be denoised comprises: obtaining an original industrial defect image; performing forward diffusion on the original industrial defect image to generate a plurality of Gaussian noise images; wherein the industrial defect image to be denoised is any one of the plurality of Gaussian noise images.

[0008] Optionally, the encoding process is performed on the industrial defect image to be denoised to obtain intermediate features, including: performing the encoding process on the industrial defect image to be denoised to generate encoded features; and performing at least one global feature extraction process on the encoded features to obtain the intermediate features.

[0009] Optionally, the global feature extraction process is implemented based on a Transformer structure that is invariant to feature map resolution.

[0010] Optionally, the first down-sampling process and / or the second up-sampling process is implemented based on a convolution block, and the convolution block includes two residual convolution units, and each residual convolution unit includes a linear layer, a convolution layer, and two context information extraction layers.

[0011] Optionally, the Transformer structure includes a multi-layer perceptron unit, a multi-head self-attention unit, and a normalization operator.

[0012] Optionally, the industrial defect image generation method further includes: connecting the encoding process and a process with the same feature map resolution as the decoding process.

[0013] According to a second aspect of the present disclosure, an industrial defect image generation apparatus is provided, including: an image acquisition module configured to acquire an industrial defect image to be denoised; an image encoding module configured to perform an encoding process on the industrial defect image to be denoised to obtain intermediate features; wherein the encoding process includes at least one first down-sampling process and at least one second down-sampling process, the first down-sampling process includes an operation of extracting context information, and the second down-sampling process is implemented based on a Transformer structure; and an image generation module configured to perform a decoding process on the intermediate features to generate a denoised industrial defect image corresponding to the industrial defect image to be denoised; wherein the decoding process includes at least one first up-sampling process and at least one second up-sampling process, the first up-sampling process is implemented based on the Transformer structure, and the second up-sampling process includes an operation of extracting context information.

[0014] Optionally, the image acquisition module is configured to: acquire an original industrial defect image; and perform forward diffusion on the original industrial defect image to generate a plurality of Gaussian noise images; wherein the industrial defect image to be denoised is any one of the plurality of Gaussian noise images.

[0015] Optionally, the image encoding module is configured to: perform the encoding process on the industrial defect image to be denoised to generate encoded features; and perform at least one global feature extraction process on the encoded features to obtain the intermediate features.

[0016] Optionally, the global feature extraction process is implemented based on a Transformer structure that is invariant to feature map resolution.

[0017] Optionally, the first down-sampling process and / or the second up-sampling process is implemented based on a convolution block, the convolution block comprising two residual convolution units, each residual convolution unit comprising one linear layer, one convolution layer and two context information extraction layers.

[0018] Optionally, the Transformer structure comprises a multi-layer perception unit, a multi-head self-attention unit and a normalization operator.

[0019] Optionally, the encoding process and the decoding process are connected with a process having the same feature map resolution.

[0020] According to a third aspect of the present disclosure, a computer readable storage medium is provided, which stores a computer program, the computer program being executed by a processor to implement any of the above industrial defect image generation methods.

[0021] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; the processor being configured to implement any of the above industrial defect image generation methods via execution of the executable instructions.

[0022] In the technical solutions provided by some embodiments of the present disclosure, on one hand, the present disclosure solves the problem of insufficient defect images in actual industrial scenarios by generating industrial defect images by computer, which helps to expand the training data of the defect analysis model, and further improves the accuracy of industrial defect detection, recognition and analysis, and helps to improve the generalization ability of the model. On the other hand, the image processing process of the present disclosure adds the extraction operation of context information and the attention mechanism of Transformer, so that the generated images have high quality and good details.

[0023] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0024] The accompanying drawings incorporated in and forming a part of the specification, illustrate embodiments consistent with the present disclosure and serve to explain the principles of the present disclosure. It is apparent that the accompanying drawings in the following description are only some embodiments of the present disclosure, and other drawings can be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0025] Figure 1 A schematic diagram of the processing stages involved in the image generation scheme of the embodiments of the present disclosure is shown.

[0026] Figure 2A schematic diagram showing the forward diffusion of the embodiment of the present disclosure is shown.

[0027] Figure 3 A flow chart of the industrial defect image generation method according to the exemplary embodiment of the present disclosure is shown.

[0028] Figure 4 A schematic diagram showing the encoding process and the decoding process of the embodiment of the present disclosure is shown.

[0029] Figure 5 A schematic diagram showing the encoding and decoding process of the embodiment of the present disclosure in combination with the reverse noise reduction stage is shown.

[0030] Figure 6 A structural schematic diagram of the convolution block of the embodiment of the present disclosure is shown.

[0031] Figure 7 A structural schematic diagram of the Transformer module of the embodiment of the present disclosure is shown.

[0032] Figure 8 A schematic diagram showing the entire network structure containing encoding and decoding of the embodiment of the present disclosure is shown.

[0033] Figure 9 A comparison chart of the effect of the generated industrial defect image of the embodiment of the present disclosure is shown, in which A is the original industrial defect image, B is the industrial defect image generated by applying the network without the context information extraction operation and the Transformer structure, and C is the industrial defect image generated by applying the image generation scheme of the embodiment of the present disclosure.

[0034] Figure 10 A block diagram of the industrial defect image generation device according to the exemplary embodiment of the present disclosure is shown.

[0035] Figure 11 A block diagram of the electronic device according to the exemplary embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0036] Example implementations are now described with reference to the following drawings. The example implementations, can, however, be implemented in many different forms and should not be construed as limited to the examples set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example implementations to those skilled in the art. The described features, structures, or characteristics can be combined in one or more implementations. In the following description, numerous specific details are provided to give a thorough understanding of example implementations. One skilled in the relevant art will recognize, however, that the implementations can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures have not been described in detail to avoid obscuring the aspects of the disclosure. Unless otherwise noted, the terms "example" and "exemplary" are used herein to mean "serving as an example, instance, or illustration." Any implementation described herein as an "example" or "exemplary" is not necessarily to be construed as preferred or advantageous over other implementations.

[0037] In addition, the accompanying drawings are included to provide a further understanding of the disclosure, and are incorporated in and constitute a part of this specification. The drawings illustrate examples of the present disclosure and, as such, a change can be made in the size of some of the elements for the sake of illustration in the drawings. Like reference numerals refer to like elements throughout the drawings. It will be understood that when a certain embodiment is "explained" or "described," it is meant to be an example, and not a limitation of the disclosure. It is to be understood that the use of certain specific language, words, or phrases, such as "preferably," "preferably," "more preferably," "most preferably," "corresponding," "exemplary," "for example," and the like, are intended to convey that the particular identifiers are included in the present disclosure, but are not intended to limit the scope of the disclosure, unless otherwise specifically indicated. In the drawings:

[0038] The flowcharts shown in the drawings are merely illustrative and do not necessarily include all the steps. For example, some steps can be further divided, and some steps can be combined or partially combined, so the actual execution order can be changed according to the actual situation. In addition, all the terms "first", "second", and the like below are only for the purpose of distinction, and should not be considered as a limitation of the disclosure.

[0039] The embodiments of the present disclosure provide a CoT-TransDDPM-based industrial defect image generation scheme. In order to make the present disclosure easy to understand, some terms are explained below.

[0040] DDPM (Denoising Diffusion Probabilistic Model) is also known as diffusion model, which is a model that can be used to generate artificial intelligence content. The theoretical basis of DDPM is to train a parameterized Markov chain through variational inference. The diffusion model can be divided into two stages, forward diffusion stage and reverse denoising stage. The forward diffusion stage is used to add Gaussian noise to the initial data distribution, and the forward diffusion stage is a Markov process. The reverse denoising stage is used to restore the noise to the initial data, specifically, the noise added at each step is predicted, and the noise is removed to gradually restore the noise-free image.

[0041] The CoT module combines convolution and self-attention mechanism, and the CoT module can be used to extract context information.

[0042] The Transformer module can integrate self-attention mechanism, position encoding and residual connection technology, which helps to improve the performance and stability of the model.

[0043] The CoT-TransDDPM in the embodiments of the present disclosure is a diffusion model containing CoT module and Transformer module, that is, the CoT module and Transformer module are integrated into the DDPM model to enhance the attention mechanism. The present disclosure uses the model to realize the generation scheme of industrial defect images.

[0044] It should be noted that the industrial defect image referred to in the present disclosure refers to images of various objects appearing defects in industry, which can include but is not limited to PCB (Printed Circuit Board) defect images, impeller defect images, weld (or weld) defect images, etc.

[0045] Figure 1 The schematic diagram of the processing stage involved in the image generation scheme of the embodiments of the present disclosure is shown. Referring to Figure 1 The image generation scheme of the embodiments of the present disclosure can include forward diffusion stage 11 and reverse denoising stage 12.

[0046] Before implementing the scheme of the present disclosure, the preparation of the data set can be carried out. Specifically, the original industrial defect image with high clarity can be obtained. The present disclosure does not limit the number of images and the resolution of the images contained in the data set, for example, the data set can contain 693 images with a resolution of 3034x1586.

[0047] For the forward diffusion stage 11, the original industrial defect image can be recorded as X0, Gaussian noise is added to the image step by step until almost pure noise image X T . The process can be defined as a Markov process, that is, the noise image at the noise number n only depends on the noise image at n-1, and each noise image generated in the process can be used as a training set.

[0048] Figure 2 A schematic diagram of the forward diffusion of the embodiments of the present disclosure is shown. Referring to Figure 2 , taking a PCB image as an example, on the basis of the image X0, by continuously adding noise, the image X1, the image X2, …, the image X T-2 , the image X T-1 and the image X T can be obtained.

[0049] The noise adding process involved in X n-1 to X n is shown in formula 1:

[0050]

[0051] Where N is a normal distribution, β n is the variance at the noise number n, and I represents the unit matrix.

[0052] Substituting α n = 1-β n into formula 1, the noise image generated at any X n can be obtained, as shown in formula 2:

[0053]

[0054] Where ∈ n ~N(0,1) is the noise sampled from the normal distribution.

[0055] For the reverse denoising stage 12, the random Gaussian noise image X n is recursively denoised to the image X n-1 in reverse until the corresponding new X n-1 is generated. The conditional probability parameters involved can be shown in formula 3:

[0056]

[0057] The industrial defect image generation scheme of the embodiments of the present disclosure can be implemented by an electronic device. That is, each step of the industrial defect image generation method of the embodiments of the present disclosure can be performed by an electronic device, and the industrial defect image generation apparatus described below can be configured in the electronic device. The present disclosure does not limit the type of electronic device, which can be, for example, an industrial server, a smart terminal, a personal computer, etc.

[0058] Figure 3 A flowchart of an industrial defect image generation method of an exemplary embodiment of the present disclosure is schematically shown. Referring to Figure 3 , the industrial defect image generation method can include the following steps:

[0059] S32. Obtain an industrial defect image to be denoised.

[0060] It should be understood that the present disclosure can be directed to different industrial objects, which can include but are not limited to PCB circuit boards, impellers, weldments, pipe fittings, etc. Different industrial objects correspond to different defect types, which are not limited by the present disclosure.

[0061] According to some embodiments of the present disclosure, an electronic device can obtain an original industrial defect image based on an industrial site shot, and perform forward diffusion on the original industrial defect image to generate a plurality of Gaussian noise images. The industrial defect image to be denoised according to the present disclosure can be any one of the Gaussian noise images.

[0062] According to other embodiments of the present disclosure, after an original industrial defect image of an industrial site is shot, a developer can manually add noise to the original industrial defect image through computer technology to obtain an industrial defect image to be denoised.

[0063] S34. Perform an encoding process on the industrial defect image to be denoised to obtain intermediate features; wherein the encoding process includes at least one first down-sampling process and at least one second down-sampling process, the first down-sampling process includes an operation of extracting context information, and the second down-sampling process is implemented based on a Transformer structure.

[0064] In an exemplary embodiment of the present disclosure, each first down-sampling process includes processing of a CoT module to implement the operation of extracting context information.

[0065] Taking the industrial defect image to be denoised as input, at least one first down-sampling process and at least one second down-sampling process are sequentially performed to obtain intermediate features.

[0066] According to some embodiments of the present disclosure, the output of the last second down-sampling process is taken as the intermediate features.

[0067] According to some other embodiments of the present disclosure, an encoding processing procedure is performed on the industrial defect image to be denoised, and an output of a last second down-sampling procedure is taken as an encoding feature. At least one global feature extraction procedure is performed on the encoding feature to obtain an intermediate feature.

[0068] Specifically, each global feature extraction procedure can be implemented based on a Transformer structure with invariant feature map resolution. Specifically, the Transformer structure is a non-sampling Transformer structure, and non-sampling means that parameters involved in the Transformer structure are set to be empty in the code.

[0069] S36. A decoding processing procedure is performed on the intermediate feature to generate a denoised industrial defect image corresponding to the industrial defect image to be denoised; wherein the decoding processing procedure includes at least one first up-sampling procedure and at least one second up-sampling procedure, the first up-sampling procedure is implemented based on a Transformer structure, and the second up-sampling procedure includes an operation of extracting context information.

[0070] In an exemplary embodiment of the present disclosure, each second up-sampling procedure includes processing of a CoT module to implement the operation of extracting context information.

[0071] The processing procedures involved in steps S34 and S36 will be described below in conjunction with the accompanying drawings.

[0072] Figure 4 A schematic diagram of the encoding processing procedure and the decoding processing procedure of the embodiments of the present disclosure is shown. Referring to Figure 4 , the encoding processing procedure includes at least one first down-sampling procedure and at least one second down-sampling procedure, and the decoding processing procedure includes at least one first up-sampling procedure and at least one second up-sampling procedure.

[0073] For the first down-sampling procedure and / or the second up-sampling procedure, a convolution block can be implemented, which can include two residual convolution units, each of which includes a linear layer, a convolution layer, and two context information extraction layers. The context information extraction layer is the CoT module described above.

[0074] For the second down-sampling procedure and / or the first up-sampling procedure, a Transformer structure can be implemented. The Transformer structure can include a MultiLayer Perceptron (MLP) unit, a Multi-head Self-Attention (MSA) unit, and a Layer Normalization (LN) operator.

[0075] For the encoding process and the decoding process, processes with the same feature map resolution can also be connected to improve the estimation accuracy.

[0076] Figure 5 A schematic diagram illustrating the encoding and decoding processes of the embodiments of the present disclosure in combination with the reverse denoising stage is shown. The encoding process and the decoding process shown in FIG. 1 can be referred to as an encoding and decoding process. The encoding and decoding process can be implemented by using the encoding and decoding processes shown in FIG. 2 and FIG. 3. Figure 4 Figure 5 For each denoising process, the above encoding and decoding processes can be used.

[0077] In addition, it should be noted that due to the intervention of the algorithm, the image in the reverse denoising process is usually not completely consistent with the image at the corresponding position in the forward diffusion in terms of image content, and therefore, Figure 5 The image generated in the reverse denoising process is a new image.

[0078] Figure 6 A structural schematic diagram of a convolutional block of the embodiments of the present disclosure is shown. The convolutional block includes two residual convolutional parts connected to each other, and each part includes a linear layer, a convolutional layer, two CoT modules, and two adders. For the convolutional layer, it can be a spatial filter with a size of 3x3 and a stride of 1, which is used to learn the local semantic information of the input features. The addition of the CoT module can make full use of the local context information and prevent the problem of feature loss. Figure 6

[0079] A structural schematic diagram of a Transformer module of the embodiments of the present disclosure is shown. The Transformer module is the Transformer structure as referred to in the present disclosure, and specifically can include a multi-head self-attention unit, a multi-layer perception unit, two normalization operators, and two adders. Figure 7 Figure 7

[0080] It should be noted that Figure 6 and Figure 7 are only exemplary descriptions, and the convolutional block and the Transformer module used in the present disclosure are not limited thereto.

[0081] Figure 8 A schematic diagram of the entire network structure including encoding and decoding of the embodiments of the present disclosure is shown.

[0082] Referring to FIG. 6, first, the industrial defect image to be denoised can enter the encoding process. Specifically, after passing through the input convolutional layer, two down-sampling convolutional blocks are used to extract early features, and the above convolutional blocks can learn early local features from the high-resolution input image. Figure 8

[0083] ​​​​Next, four consecutive down-sampling Transformer structures and three consecutive non-sampling Transformer structures are used to learn global features from low-resolution features, i.e., to generate the intermediate features as described in the present disclosure.

[0084] Then, the intermediate features enter the decoding process. Specifically, four consecutive up-sampling Transformer structures and two up-sampling convolution blocks are used to restore the original feature map resolution.

[0085] Subsequently, two parameter convolution layers are used to obtain noise parameters and variance coefficients, respectively, to recursively remove Gaussian noise to a noise-free image, thereby generating a new industrial defect image.

[0086] In this process, the same resolution encoding process and decoding process can be connected by a shortcut connection to better transfer high-resolution information from the encoding end to the decoding end to improve the estimation accuracy. The CoT-TransDDPM model estimates the parameters in the training process, including the estimated mean and the estimated variance.

[0087] In addition, the output of the convolution layer and the Timstep are transmitted to the subsequent convolution layer, so that each convolution block can learn the feature information on the Timestep. Each convolution layer can apply a group normalization (GN) and a SiLu function, and a residual connection is adopted between the convolution layer and the output.

[0088] In addition, for the training process of the above model, an alternating training method of generative adversarial can be used, and an AdamW optimizer can be used. Parameters such as the following are set: the image size is 256x256, the learning rate is 0.00002, the batch size is 4, the weight decay is 0.0001, and the time step, i.e., the number of noise adding rounds, is set to 3500, that is, Figure 8 The t in the Timestept in the above formula is set to 3500.

[0089] Through experiments, the CoT-TransDDPM model constructed by the embodiments of the present disclosure has better image generation effect than the DDPM model. According to the FID index evaluation, the FID index of the DDPM model under the same conditions is 81.96, while the FID index of the CoT-TransDDPM model of the embodiments of the present disclosure is 34.17.

[0090] Taking a PCB defect image as an example, Figure 9An effect comparison diagram of the generated industrial defect image of the embodiment of the present disclosure is schematically shown, where A is an original industrial defect image, B is an industrial defect image generated by applying a network without a context information extraction operation and a Transformer structure, and C is an industrial defect image generated by applying the image generation scheme of the embodiment of the present disclosure.

[0091] As shown in Figure 9 , compared with B, C is closer to A and the details of C are clearer, and it can be seen that the embodiment of the present disclosure has a better image generation effect.

[0092] It should be noted that although the various steps of the method in the present disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired result. In addition or alternatively, certain steps can be omitted, multiple steps can be combined into one step, and / or one step can be divided into multiple steps, etc.

[0093] Further, an industrial defect image generation apparatus is also provided in the present example embodiment.

[0094] Figure 10 A block diagram of the industrial defect image generation apparatus of the example embodiment of the present disclosure is schematically shown. Referring to Figure 10 , the industrial defect image generation apparatus 10 according to the example embodiment of the present disclosure can include an image acquisition module 101, an image encoding module 103, and an image generation module 105.

[0095] Specifically, the image acquisition module 101 can be configured to acquire a to-be-de-noised industrial defect image; the image encoding module 103 can be configured to perform an encoding processing process on the to-be-de-noised industrial defect image to obtain intermediate features; wherein the encoding processing process includes at least one first down-sampling process and at least one second down-sampling process, the first down-sampling process includes an operation of extracting context information, and the second down-sampling process is implemented based on a Transformer structure; the image generation module 105 can be configured to perform a decoding processing process on the intermediate features to generate a de-noised industrial defect image corresponding to the to-be-de-noised industrial defect image; wherein the decoding processing process includes at least one first up-sampling process and at least one second up-sampling process, the first up-sampling process is implemented based on a Transformer structure, and the second up-sampling process includes an operation of extracting context information.

[0096] According to the example embodiment of the present disclosure, the image acquisition module 101 can be configured to acquire an original industrial defect image; perform forward diffusion on the original industrial defect image to generate a plurality of Gaussian noise images; wherein the to-be-de-noised industrial defect image is any one of the plurality of Gaussian noise images.

[0097] According to the example embodiments of the present disclosure, the image encoding module 103 can be configured to perform an encoding processing procedure on the industrial defect image to be denoised to generate an encoding feature, and perform at least one global feature extraction procedure on the encoding feature to obtain an intermediate feature.

[0098] According to the example embodiments of the present disclosure, the global feature extraction procedure is implemented based on a Transformer structure that is invariant to feature map resolution.

[0099] According to the example embodiments of the present disclosure, the first down-sampling procedure and / or the second up-sampling procedure are implemented based on a convolution block, and the convolution block includes two residual convolution units, and each residual convolution unit includes one linear layer, one convolution layer and two context information extraction layers.

[0100] According to the example embodiments of the present disclosure, the Transformer structure includes a multi-layer perceptron unit, a multi-head self-attention unit and a normalization operator.

[0101] According to the example embodiments of the present disclosure, the encoding processing procedure is connected with a processing procedure having the same feature map resolution as that in the decoding processing procedure.

[0102] Since the various functional modules of the industrial defect image generation apparatus according to the embodiments of the present disclosure are the same as those in the above-mentioned method embodiments, no further description is given herein.

[0103] In the example embodiments of the present disclosure, a computer readable storage medium having stored thereon a program product capable of implementing the above-mentioned method of the present disclosure is also provided. In some possible embodiments, various aspects of the present disclosure can also be implemented in the form of a program product including program codes for causing an end device to perform the steps described in the above-mentioned “example method” section of the present disclosure according to various example embodiments of the present disclosure when the program product is run on the end device.

[0104] The program product for implementing the above-mentioned method according to the embodiments of the present disclosure can take a portable compact disc read-only memory (CD-ROM) and include program codes, and can be run on an end device such as a personal computer. However, the program product of the present disclosure is not limited thereto, and in the present document, the readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus or device.

[0105] The program product can employ any combination of one or more computer-readable media. The computer-readable media can be a computer-readable storage medium or a computer-readable signal medium. The computer-readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium include the following: an electrical connection having one or more wires, a portable disc, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical disc, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0106] The computer-readable signal medium can include a computer-readable storage medium that is propagated as a carrier wave. The computer-readable signal medium can further be any computer-readable medium that is not a storage medium. The computer-readable signal medium can be a computer-readable storage medium that is a propagated signal.

[0107] The program code embodied on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to wireless, wired, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0108] The program code can be executed by one or more programmable processors, which can be implemented using one or more microprocessors, microcontrollers, digital signal processors, central processing units, state machines, logic circuitries, and / or any other devices suitable for retrieval and execution of instructions. The program code can execute entirely on a user's computing device, partly on the user's computing device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device or entirely on the remote computing device or server. In the latter scenario, the remote computing device can be connected to the user's computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computing device, such as through the Internet using an Internet Service Provider (ISP).

[0109] In an exemplary embodiment of the present disclosure, an electronic device capable of implementing the above-described method is also provided.

[0110] Those skilled in the art can understand that various aspects of the present disclosure can be implemented as a system, a method or a program product. Therefore, various aspects of the present disclosure can be embodied as a whole hardware implementation, a whole software implementation (including firmware, microcode, etc.), or an implementation combining hardware and software aspects, which can be collectively referred to as "circuitry", "module" or "system" herein.

[0111] The electronic device 1100 according to this embodiment of the present disclosure will be described below with reference to Figure 11 Figure 11 The electronic device 1100 shown is merely an example and should not limit the function and usage range of the embodiments of the present disclosure.

[0112] As Figure 11 shown, the electronic device 1100 is in the form of a general computing device. The components of the electronic device 1100 can include, but are not limited to, the at least one processing unit 1110 described above, the at least one storage unit 1120 described above, a bus 1130 connecting different system components (including the storage unit 1120 and the processing unit 1110), and a display unit 1140.

[0113] The storage unit stores program code that can be executed by the processing unit 1110, so that the processing unit 1110 performs the steps according to various exemplary embodiments of the present disclosure described in the "Exemplary Method" section of the present specification. For example, the processing unit 1110 can perform each step of the industrial defect image generation method of the embodiments of the present disclosure.

[0114] The storage unit 1120 can include a readable medium in the form of a volatile storage unit, such as a random access memory (RAM) 11201 and / or a cache memory unit 11202, and can further include a read-only memory (ROM) 11203.

[0115] The storage unit 1120 can also include a program / utility 11204 having a set of program modules 11205, including but not limited to, an operating system, one or more application programs, other program modules, and program data, each of which or a combination thereof can include implementation of a network environment.

[0116] The bus 1130 can represent one or more of several types of bus structures, including a storage unit bus or storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of a variety of bus structures.

[0117] ​The electronic device 1100 can also communicate with one or more external devices 1200 such as a keyboard or pointing device, a Bluetooth device, or a device for reading media. Communication with one or more devices enabling user interaction with the electronic device 1100 and / or one or more devices enabling communication between the electronic device 1100 and other computing devices or systems (such as a router, modem, etc.) can be accomplished through input / output (I / O) interface 1150. Still yet, the electronic device 1100 can communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), or the Internet through network adapter 1160. As Figure 11 illustrated, network adapter 1160 can communicate with the other components of electronic device 1100 through bus 1130. It should be understood that although not shown, other hardware and / or software components could be used in conjunction with electronic device 1100. These include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0118] From the foregoing description, it will be apparent to those skilled in the art that a variety of modifications and changes can be made to the implementations described without departing from the scope of the disclosure. Thus, it is intended that the disclosure cover all such modifications and changes that fall within the scope of the disclosure. In addition, while the preferred embodiments of this disclosure have been described, it will be obvious to those skilled in the art that various changes can be made without departing from the

[0119] In addition, the above-described diagrams are only schematic illustrations of processes included in the method according to the exemplary embodiments of the disclosure, and are not intended for limiting purposes. It is easily understood that the processes shown in the above-described diagrams do not indicate or limit the time sequence of the processes. In addition, it is easily understood that the processes can be executed, for example, synchronously or asynchronously in a plurality of modules.

[0120] It should be noted that although several modules or units of a device for action execution are mentioned in the foregoing detailed description, such a division is not mandatory. Indeed, according to an embodiment of the disclosure, the features and functionalities of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functionalities of one module or unit described above can be further divided into a plurality of modules or units.

[0121] Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the concepts disclosed herein. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

[0122] It should be understood that the present disclosure is not limited to the precise structures as herein described and illustrated in the drawings, and that various modifications and changes can be made without departing from its scope. The scope of the present disclosure is limited only by the claims that follow.

Claims

1. An industrial defect image generation method characterized by comprising: The method comprises: obtaining an industrial defect image to be denoised; performing an encoding process on the industrial defect image to be denoised to generate encoded features; performing at least one global feature extraction process on the encoded features to obtain intermediate features; wherein the encoding process comprises at least one first down-sampling process and at least one second down-sampling process, the first down-sampling process comprises an operation of extracting context information, and the second down-sampling process is implemented based on a Transformer structure, and the global feature extraction process is implemented based on a Transformer structure with constant feature map resolution; performing a decoding process on the intermediate features to generate a denoised industrial defect image corresponding to the industrial defect image to be denoised; wherein the decoding process comprises at least one first up-sampling process and at least one second up-sampling process, the first up-sampling process is implemented based on a Transformer structure, and the second up-sampling process comprises an operation of extracting context information; wherein the first down-sampling process and / or the second up-sampling process is implemented based on a convolution block, the convolution block comprises two residual convolution units, each of the residual convolution units comprises a linear layer, a convolution layer, and two context information extraction layers; and the Transformer structure comprises a multi-layer perception unit, a multi-head self-attention unit, and a normalization operator. The method for obtaining an industrial defect image to be denoised comprises: obtaining an original industrial defect image; performing forward diffusion on the original industrial defect image to generate a plurality of Gaussian noise images; wherein the industrial defect image to be denoised is any one of the plurality of Gaussian noise images.

2. The industrial defect image generation method according to claim 1, characterized by, The method further comprises: connecting the encoding process and the decoding process with the same feature map resolution.

3. An industrial defect image generation apparatus characterized by comprising: The method comprises: an image acquisition module for obtaining an industrial defect image to be denoised; an image encoding module for performing an encoding process on the industrial defect image to be denoised to generate encoded features; performing at least one global feature extraction process on the encoded features to obtain intermediate features; wherein the encoding process comprises at least one first down-sampling process and at least one second down-sampling process, the first down-sampling process comprises an operation of extracting context information, and the second down-sampling process is implemented based on a Transformer structure, and the global feature extraction process is implemented based on a Transformer structure with constant feature map resolution; an image generation module for performing a decoding process on the intermediate features to generate a denoised industrial defect image corresponding to the industrial defect image to be denoised; wherein the decoding process comprises at least one first up-sampling process and at least one second up-sampling process, the first up-sampling process is implemented based on a Transformer structure, and the second up-sampling process comprises an operation of extracting context information; The first down-sampling process and / or the second up-sampling process are implemented based on a convolution block, the convolution block comprising two residual convolution units, each of the residual convolution units comprising a linear layer, a convolution layer and two context information extraction layers; the Transformer structure comprising a multi-layer perception unit, a multi-head self-attention unit and a normalization operator. The image acquisition module is configured to acquire an original industrial defect image; and perform forward diffusion on the original industrial defect image to generate a plurality of Gaussian noise images; wherein the industrial defect image to be de-noised is any one of the plurality of Gaussian noise images.

4. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program, when executed by a processor, implements the industrial defect image generation method of claim 1 or 2.

5. An electronic device, comprising: Comprise: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to implement the industrial defect image generation method of claim 1 or 2 via execution of the executable instructions.

Images