Neural network training method, image generation method, defect detection method and device

Abstract

The invention provides a training method and device for a generative neural network, an image generation method and device, a training method and device for a defect detection neural network, and a defect detection method and device. According to the embodiment of the invention, the method for training the generative neural network comprises the following steps: inputting a training image, and obtaining a feature representation graph of the training image according to the features of the training image; performing first enhancement processing on the feature representation graph of the training image to obtain a first enhanced training image; performing second enhancement processing on the training image to obtain a second enhanced training image, the second enhancement processing being different from the first enhancement processing; and at least according to the training image, the feature representation graph of the training image, the first enhanced training image and the second enhanced training image, performing image generation operation by using the generative neural network, and obtaining a training generated image, so as to train the generative neural network and adjust parameters of the generative neural network.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to a neural network training method and apparatus, as well as a method and apparatus for image generation and defect detection using neural networks. Background Technology

[0002] The generation of desired images from acquired images, and the subsequent training of neural networks for defect detection, are important research topics. Defect detection is a crucial factor in product quality control. Since various defects continuously arise during production, it is impossible to exhaustively list all possible causes. Therefore, how to robustly and automatically detect these defects is a pressing problem that needs to be solved.

[0003] Current defect detection methods typically train neural network models for defect detection using a large number of simulated defect samples. However, because the generated images are insufficient to simulate real product defects, it is difficult to accurately detect and locate defects in the images.

[0004] Therefore, there is a need for an improved method and apparatus for training neural networks, as well as a method and apparatus for generating more realistic images and detecting defects more accurately using neural networks. Summary of the Invention

[0005] To address the aforementioned technical problems, according to one aspect of the present invention, a neural network training method is provided, comprising: inputting a training image and obtaining a feature representation map of the training image based on the features of the training image; performing a first enhancement processing on the feature representation map of the training image to obtain a first enhanced training image; performing a second enhancement processing on the training image to obtain a second enhanced training image, wherein the second enhancement processing is different from the first enhancement processing; and performing an image generation operation using the generative neural network based at least on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image to obtain a training generated image, thereby training the generative neural network and adjusting the parameters of the generative neural network.

[0006] According to another aspect of the present invention, a method for training a defect detection neural network is provided, comprising: inputting a training image and obtaining a feature representation of the training image; selecting one or more semantic regions based on the feature representation of the training image and generating a semantic defect feature representation based on the one or more semantic regions; generating a training defect image based at least on the training image and the semantic defect feature representation; and performing defect detection based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the defect detection neural network and adjust the parameters of the defect detection neural network.

[0007] According to another aspect of the present invention, an image generation method is provided, comprising: acquiring a feature representation map of feature information for generating an image, and a texture representation map of texture information for generating an image; acquiring the generated image using a generative neural network based on the feature representation map and the texture representation map, wherein the generative neural network is trained by: inputting a training image and acquiring a feature representation map of the training image based on features of the training image; performing a first enhancement processing on the feature representation map of the training image to obtain a first enhanced training image; performing a second enhancement processing on the training image to obtain a second enhanced training image, the second enhancement processing being different from the first enhancement processing; and performing an image generation operation using the generative neural network based at least on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image to obtain a training generated image, thereby training the generative neural network and adjusting the parameters of the generative neural network.

[0008] According to another aspect of the present invention, a defect detection method is provided, comprising: inputting an image to be detected; reconstructing the image to be detected using a defect detection neural network to obtain a reconstructed image and obtaining a feature representation of the reconstructed image; performing defect detection on the image to be detected using the defect detection neural network based on the reconstructed image and the feature representation of the reconstructed image; wherein the defect detection neural network is trained by: inputting a training image and obtaining a feature representation of the training image; selecting one or more semantic regions based on the feature representation of the training image and generating a semantic defect feature representation based on the one or more semantic regions; generating a training defect image based at least on the training image and the semantic defect feature representation; and performing defect detection based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the defect detection neural network and adjust the parameters of the defect detection neural network.

[0009] According to another aspect of the present invention, a generative neural network training apparatus is provided, comprising: a processor; a memory storing computer program instructions, wherein when the computer program instructions are executed by the processor, the processor performs the following steps: inputting a training image and obtaining a feature representation map of the training image based on features of the training image; performing a first enhancement processing on the feature representation map of the training image to obtain a first enhanced training image; performing a second enhancement processing on the training image to obtain a second enhanced training image, the second enhancement processing being different from the first enhancement processing; performing an image generation operation using the generative neural network based at least on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image to obtain a training generated image, thereby training the generative neural network and adjusting the parameters of the generative neural network.

[0010] According to another aspect of the present invention, a defect detection neural network training apparatus is provided, comprising: a processor; and a memory storing computer program instructions, wherein, when the computer program instructions are executed by the processor, the processor performs the following steps: inputting a training image and obtaining a feature representation of the training image; selecting one or more semantic regions based on the feature representation of the training image and generating a semantic defect feature representation based on the one or more semantic regions; generating a training defect image based at least on the training image and the semantic defect feature representation; and performing defect detection based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the defect detection neural network and adjust the parameters of the defect detection neural network.

[0011] According to the above-described generative neural network training method and apparatus of the present invention, as well as the method and apparatus for image generation using a generative neural network, different information from the training image and the feature representation map of the training image can be comprehensively obtained to train the generative neural network by performing different enhancement processing on the feature representation map of the training image and the training image respectively, so as to make the generated image more realistic and meet various user needs.

[0012] Furthermore, according to the above-described defect detection neural network training method and apparatus of the present invention, as well as the method and apparatus for defect detection using the defect detection neural network, the sample data used to train the defect detection neural network can be enriched by generating semantic defect feature representations of semantic regions of feature representations of training images. This enables robust detection of semantic defects for different types of products or objects, avoids the collection of large amounts of defect data and manual annotation process, greatly reduces production costs, and improves user experience. Attached Figure Description

[0013] The above and other objects, features, and advantages of the present invention will become clearer from the detailed description of the embodiments of the present invention in conjunction with the accompanying drawings.

[0014] Figure 1 A flowchart of a generative neural network training method according to an embodiment of the present invention is shown;

[0015] Figure 2 A flowchart illustrating a method for image generation using a generative neural network according to an embodiment of the present invention is shown;

[0016] Figure 3 A flowchart of a method for training a defect detection neural network according to an embodiment of the present invention is shown;

[0017] Figure 4 A flowchart of a method for defect detection using a defect detection neural network according to an embodiment of the present invention is shown;

[0018] Figure 5 A schematic diagram of training images is shown as an example according to an embodiment of the present invention;

[0019] Figure 6 A feature representation map of a training image is shown as an example according to an embodiment of the present invention;

[0020] Figure 7 This illustrates an example of a first enhanced training image obtained by performing a first enhancement process on the contour map of a training image according to an embodiment of the present invention;

[0021] Figure 8 The illustration shows an example of a second enhanced training image obtained by performing a second enhancement process on a training image according to an embodiment of the present invention;

[0022] Figure 9 An example of obtaining a training-generated image from an intermediate training image and an anomaly probability map according to an embodiment of the present invention is shown;

[0023] Figure 10 A schematic diagram of an image generation method according to an embodiment of the present invention is shown;

[0024] Figure 11 A schematic diagram illustrating a semantic defect feature representation according to an embodiment of the present invention is shown;

[0025] Figure 12 An example of a training defect image generated according to an embodiment of the present invention is shown;

[0026] Figure 13 A block diagram of an apparatus for training a generative neural network according to an embodiment of the present invention is shown;

[0027] Figure 14 A block diagram of an image generation apparatus according to an embodiment of the present invention is shown;

[0028] Figure 15 A block diagram of a defect detection neural network training apparatus according to an embodiment of the present invention is shown;

[0029] Figure 16 A block diagram of an apparatus for defect detection using a defect detection neural network according to an embodiment of the present invention is shown. Detailed Implementation

[0030] The following description, with reference to the accompanying drawings, outlines a method and apparatus for training a generative neural network according to embodiments of the present invention, as well as a method and apparatus for generating images using a generative neural network. Furthermore, it also describes a method and apparatus for training a defect detection neural network according to embodiments of the present invention, and a method and apparatus for defect detection using a defect detection neural network. Throughout the drawings, the same reference numerals denote the same elements. It should be understood that the embodiments described herein are merely illustrative and should not be construed as limiting the scope of the invention.

[0031] Figure 1 A flowchart of a generative neural network training method 100 according to an embodiment of the present invention is shown. Referring below... Figure 1 A method for training a generative neural network according to an embodiment of the present invention is described.

[0032] In step S101, a training image is input, and a feature representation map of the training image is obtained based on the features of the training image.

[0033] In this embodiment of the invention, optionally, the feature representation map of the acquired training image may be, for example, a contour map of the training image. Furthermore, the feature representation map of the training image may also include images showing various relevant features such as color information, shape information, and classification information of the training image, without limitation.

[0034] In this embodiment of the invention, optionally, a feature detection network applicable to this embodiment can be trained to extract features from the training image and output a feature representation map corresponding to the training image. For example, when the feature representation map of the training image is the contour map of the training image, a lightweight edge detection network with fewer parameters can be trained by combining the training image with a pre-trained edge detection network with relatively more parameters. This allows the lightweight edge detection network to learn the output of the pre-trained edge detection network with more parameters and obtain edge detection capabilities similar to the pre-trained edge detection network in an unsupervised training manner. More specifically, for the same input training image, it is desirable to train the output of the lightweight edge detection network to be as close as possible to the output of the pre-trained edge detection network. Therefore, the goal of the above training process is to minimize the output difference between the lightweight edge detection network and the pre-trained edge detection network. After training the lightweight edge detection network, it can be used to perform edge detection on the input training image and output a contour map corresponding to the training image. The above describes a training example of an edge detection network where the feature representation map of the training image is the contour map of the training image. In practical applications, different feature representation maps of training images can be used to train and apply corresponding feature detection networks to obtain the feature representation maps of the corresponding training images.

[0035] In step S102, the feature representation map of the training image is subjected to a first enhancement process to obtain a first enhanced training image.

[0036] Image enhancement processing refers to various preprocessing techniques applied to images. Through image enhancement processing, new transformed images can be generated by transforming, modifying, and expanding existing images, thereby achieving diversity in image samples.

[0037] In this embodiment of the invention, the first enhancement process on the feature representation map of the training image may include one or more operations such as image distortion, image flipping, image rotation, image cropping, edge adjustment, image translation, and image scaling. For example, random local distortion deformation, flipping, cropping, zero-padding, and other operations may be performed on the feature representation map of the training image. Specifically, when performing local distortion deformation on the feature representation map of the training image, a local region may be selected in the feature representation map of the training image and divided into a grid. Several nodes in the grid may be randomly selected, and the selected nodes may be randomly translated horizontally or vertically. On this basis, different bending deformations, flipping, cropping, zero-padding, and other processes may be performed to obtain the first enhanced training image.

[0038] In step S103, the training image is subjected to a second enhancement process to obtain a second enhanced training image. The second enhancement process is different from the first enhancement process.

[0039] Similar to the steps described above, the second enhancement processing of the training image may also include one or more operations such as image distortion, image flipping, image rotation, image cropping, edge adjustment, image translation, and image scaling, and the second enhancement processing differs from the first enhancement processing. Optionally, a local region of the training image may be selected and subjected to bending deformation, flipping, cropping, zero-padding, or other processing different from the first enhancement processing to obtain a second enhanced training image.

[0040] The specific operation methods of the first enhancement processing and the second enhancement processing different from the first enhancement processing described above are only examples. In the embodiments of the present invention, different enhancement processing methods can be selected according to different training purposes and scenario requirements, and no restrictions are imposed here.

[0041] In step S104, at least based on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image, an image generation operation is performed using the generative neural network to obtain a training generated image, so as to train the generative neural network and adjust the parameters of the generative neural network.

[0042] In this step, optionally, the generative neural network can first be used to generate intermediate training images and anomaly probability maps representing weights; then the anomaly probability maps can be used to fuse the training images and the intermediate training images to obtain the training generated images.

[0043] Specifically, during the fusion process in the generative neural network, based on the input to the generative neural network, intermediate training images and anomaly probability maps are output during the intermediate process of the generative neural network. The anomaly probability map can be used as weights to fuse the input training image and the generated intermediate training image to obtain the final trained generative image.

[0044] In this embodiment of the invention, the training generated image generated by the generative neural network can be a training generated image that combines the feature representation map of the training image as feature information and the texture of the training image as texture information. When acquiring the training generated image, the generative neural network can be trained and its parameters adjusted accordingly to achieve convergence. For example, the result of performing a first enhancement process on the training image can be used as the ground truth value to compare with the training generated image, and the parameters of the generative neural network can be adjusted accordingly.

[0045] The generative neural network training method according to embodiments of the present invention can comprehensively acquire different information from the training image and its feature representation map by performing different enhancement processes on the feature representation map of the training image and the training image respectively, thereby training the generative neural network and making the generated image more realistic and meeting various user needs.

[0046] Figure 2 A flowchart of a method 200 for image generation using a generative neural network according to an embodiment of the present invention is shown. Referring below... Figure 2 An image generation method according to embodiments of the present invention is described. In embodiments of the present invention, methods such as... Figure 1 The steps shown illustrate the training of a generative neural network to perform image generation operations.

[0047] In step S201, a feature representation map for generating feature information of an image and a texture representation map for generating texture information of an image are obtained.

[0048] In this embodiment of the invention, optionally, the feature information can be contour information, and the feature representation map of the feature information used to generate the image can be a contour map representing the contour of the generated image. Furthermore, the feature information may also include information showing various related features of the generated image, such as color information, shape information, and classification information, without limitation.

[0049] Optionally, the texture representation map used to generate the texture information of the image may contain texture information that is expected to be presented in the generated image, so that the generative neural network combines the feature representation map to present the texture information in the texture representation map in the generated image.

[0050] In step S202, the generated image is obtained using a generative neural network based on the feature representation map and the texture representation map. The generative neural network is trained as follows: a training image is input, and a feature representation map of the training image is obtained based on the features of the training image; a first enhancement process is performed on the feature representation map of the training image to obtain a first enhanced training image; a second enhancement process is performed on the training image to obtain a second enhanced training image, the second enhancement process being different from the first enhancement process; at least based on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image, an image generation operation is performed using the generative neural network to obtain a training generated image, thereby training the generative neural network and adjusting its parameters.

[0051] In this step, the generated image is generated using the previously trained generative neural network. The generated image presents the feature information contained in the feature representation map and the texture information contained in the texture representation map. The specific training steps for the generative neural network in this embodiment are as follows: Figure 1 The details mentioned above will not be repeated here.

[0052] The image generation method according to embodiments of the present invention can utilize a well-trained generative neural network by performing different enhancement processes on the feature representation map of the training image and the training image respectively, so that the generated image is more realistic and meets various user needs.

[0053] After training the generative neural network used to generate images, in this embodiment of the invention, the trained generative neural network can also be used to generate samples for the defect detection network, making the defects in the samples richer and more diverse, so as to train a more robust defect detection neural network.

[0054] This invention proposes a robust defect detection neural network training and defect detection method, apparatus, and storage medium. The defect detection neural network training and defect detection method of this invention does not require manual annotation and can effectively simulate defects generated in semantic regions, thereby enabling the trained defect detection neural network to accurately detect defects in images. The method, apparatus, and medium of this invention can be applied to the detection and analysis of products such as breakfast boxes, screw toolkits, and cable plugs, and can also be applied to other computer vision tasks, such as defect detection on road surfaces, chip surfaces, or any other object surfaces. In the application of this invention, a convolutional neural network (CNN) capable of extracting more complex features at multiple semantic levels rather than limited low-level features can be used.

[0055] Figure 3 A flowchart of a method 300 for training a defect detection neural network according to an embodiment of the present invention is shown. Referring below... Figure 3 A method for training a defect detection neural network according to an embodiment of the present invention is described.

[0056] In step S301, a training image is input, and the feature representation of the training image is obtained.

[0057] In embodiments of the present invention, the feature representation of the training image may include at least one of the contour of the training image, image segmentation information of the training image, and multimodal information of the training image. In one example, the feature representation of the training image may include the contour of the training image, which can generally represent the contours of objects present in the training image. In another example, the feature representation of the training image may include the image segmentation information of the training image, which can be represented by object segmentation processes based on thresholding, graph theory, clustering, etc. In yet another example, the feature representation of the training image may include the multimodal information of the training image, which can be information from multiple modalities of the training image, such as textual descriptions and image descriptions of objects in the training image obtained using a multimodal large model, etc. In yet another example, according to a specific application scenario of the present invention, the feature representation of the training image may also include various related information that can reflect the features of objects in the training image, such as color information, shape information, and classification information, etc., without limitation.

[0058] In this embodiment of the invention, optionally, a feature detection network applicable to this embodiment can be trained using training images free of defects to extract features from the training images and output the feature extraction results corresponding to the training images. For example, when the features of the training images are represented as the contours of the training images, a lightweight edge detection network with fewer parameters can be trained by combining the training images with a pre-trained edge detection network with relatively more parameters. This allows the lightweight edge detection network to learn the output of the pre-trained edge detection network with more parameters and obtain edge detection capabilities similar to the pre-trained edge detection network in an unsupervised training manner. More specifically, for the same input training image, it is desirable to train the output of the lightweight edge detection network to be as close as possible to the output of the pre-trained edge detection network. Therefore, the goal of the above training process is to minimize the output difference between the lightweight edge detection network and the pre-trained edge detection network. After training the lightweight edge detection network, it can be used to perform edge detection on the input training images and output the contours corresponding to the training images. The above description is an example of using the contour of the training image as the feature representation of the training image. In practical applications, different feature representations of training images can be used to train and apply corresponding feature detection networks to obtain the feature representation detection results of the corresponding training images.

[0059] In step S302, one or more semantic regions are selected based on the feature representation of the training image, and a semantic defect feature representation is generated based on the one or more semantic regions.

[0060] In this step, optionally, one or more semantic regions may be selected from at least a portion of the feature representation of the training image; a third enhancement process may be performed on the feature representation of the training image within the selected one or more semantic regions; the feature representation of the one or more semantic regions after the third enhancement process may be fused with the feature representation of the training image to generate the semantic defect feature representation. The third enhancement process may include one or more operations such as adding, deleting, or modifying feature representations.

[0061] Specifically, semantic regions can be randomly selected first. These semantic regions can be obtained in various ways, such as using a pre-trained model for semantic region extraction; or, for example, selecting multiple large continuous regions or small discrete regions from the foreground region of the feature representation of the training image as semantic regions. Within the selected semantic regions, the feature representation of the training image can be modified, including but not limited to adding, deleting, or modifying feature representations, to obtain the feature representation of one or more semantic regions after the third enhancement processing. The specific operations for modifying the feature representation of the training image described above are merely examples. Different modifications can be made to semantic regions for different feature representations, and modifications can be made once or multiple times; no limitation is imposed here.

[0062] After obtaining the feature representations of the one or more semantic regions after the third enhancement process, they can be fused with the feature representations of the training image to obtain semantic defect feature representations. For example, these feature representations can be superimposed to obtain semantic defect feature representations; alternatively, a portion of each feature representation can be selected and concatenated to obtain semantic defect feature representations; still further, the feature representations of the one or more semantic regions after the third enhancement process can be overlapped, concatenated, or a portion of each can be selected and used to replace a portion of the feature representations of the corresponding training image to obtain fused semantic defect feature representations.

[0063] The above describes an example of the generation process for semantic defect feature representation. According to another embodiment of the present invention, the semantic defect feature representation of the training image may include not only logical defects, but also at least one structural defect of the training image, such as depressions, protrusions, distortions, etc., and these defect types are not limited herein.

[0064] In step S303, a training defect image is generated based at least on the training image and the semantic defect feature representation.

[0065] According to an embodiment of the present invention, a training defect image can be generated based on the trained generative neural network, the training image, and the semantic defect feature representation.

[0066] As mentioned above, the generative neural network can be trained in the following manner: inputting a training image and obtaining a feature representation map of the training image based on the features of the training image; performing a first enhancement process on the feature representation map of the training image to obtain a first enhanced training image; performing a second enhancement process on the training image to obtain a second enhanced training image, wherein the second enhancement process is different from the first enhancement process; and performing an image generation operation using the generative neural network based at least on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image to obtain a training generated image, thereby training the generative neural network and adjusting the parameters of the generative neural network.

[0067] After obtaining the trained generative neural network, the training images can be used as the texture information of the generated training defect images, and the semantic defect feature representations can be used as the feature information of the generated training defect images to generate the corresponding training defect images. Furthermore, during the generation of training defect images, the number of samples can be further enriched by, for example, adding random noise.

[0068] In step S304, defect detection is performed based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the neural network and adjust the parameters of the neural network.

[0069] According to an embodiment of the present invention, a neural network can be trained by receiving an input training image, a feature representation of the training image, and a training defect image and a corresponding semantic defect feature representation generated by the aforementioned method, and the parameters of the neural network can be adjusted so that the loss function of the neural network converges as much as possible.

[0070] Optionally, the input training image, its feature representation, the training defect image, and the semantic defect feature representation can be used as samples to train an anomaly localization network for defect detection in the neural network, and the parameters of the anomaly localization network can be adjusted. In one example, the anomaly localization network may include a reconstruction subnetwork and a localization subnetwork. The reconstruction subnetwork is used to transform the input training defect image containing the defect and its semantic defect feature representation into a reconstructed image without the defect and its corresponding feature representation. The localization subnetwork locates the defect by calculating the difference between the reconstructed image and its corresponding feature representation transformed by the reconstruction subnetwork and the original input image.

[0071] More specifically, this reconstruction sub-network can consist of an encoder and a decoder. The encoder extracts feature maps from the input image, and the decoder reconstructs the feature maps back to the original resolution. During feature map extraction by the encoder, different levels of feature maps can be extracted using convolution, normalization, and pooling processes with varying parameters. Specifically, the input image is first convolved with a convolution kernel to obtain a convolutional map; then, a conventional linear correction unit and batch normalization method are used to normalize the convolutional map, resulting in a normalized convolutional map; finally, max or average pooling is applied to the normalized convolutional map. To obtain rich multi-scale features, the reconstruction sub-network can adjust relevant parameters and repeat the above process multiple times to extract multi-scale feature maps through multiple downsampling processes. The decoder can restore the resolution of the image's feature maps through corresponding convolution, normalization, and upsampling processes. This reconstruction sub-network combines feature maps with the same resolution from both the encoder and decoder and utilizes multiple sets of convolution processes. The higher the similarity between the reconstructed image transformed by the reconstruction subnetwork and the original input training image, the higher the probability that the training image does not contain defects, and vice versa. The structure of the localization subnetwork of the anomaly localization network is similar to that of the reconstruction subnetwork described above, but it can further have cross-layer connection operations to fuse feature maps of the same scale from the encoder and decoder in the localization subnetwork.

[0072] Optionally, after the trained anomaly localization network performs defect detection on the image, it can output the specific location of the defect and also output an estimated value for the degree of defect.

[0073] The neural network training method described above according to embodiments of the present invention can enrich the sample data used to train the defect detection neural network by generating semantic defect feature representations of semantic regions of feature representations of training images, thereby achieving robust detection of semantic defects for different types of products or objects, avoiding the collection of a large amount of defect data and the process of manual annotation, greatly reducing production costs and improving user experience.

[0074] Figure 4 A flowchart of a method 400 for defect detection using a defect detection neural network according to an embodiment of the present invention is shown. In this embodiment, a method 400 for defect detection using a defect detection neural network can be employed. Figure 3 The defect detection neural network trained according to the process shown is used for defect detection. See below for reference. Figure 4 A method for defect detection using a defect detection neural network according to an embodiment of the present invention is described.

[0075] In step S401, the image to be detected is input, and the image to be detected is reconstructed using a defect detection neural network to obtain a reconstructed image and a feature representation of the reconstructed image.

[0076] In this embodiment of the invention, the image to be detected can be input through... Figure 3 The process shown involves reconstructing the anomaly localization subnetwork within the defect detection neural network trained to completion. Optionally, the image to be detected can be an image without logical and / or structural defects, or an image with one or more logical and / or structural defects. During the reconstruction of the image to be detected, the corresponding feature representation of the reconstructed image, after removing defects, can be obtained simultaneously for use in subsequent defect detection and localization processes.

[0077] As mentioned earlier, the reconstruction sub-network in the anomaly localization network can be used to transform the input image to be detected into a reconstructed image and its corresponding feature representation. More specifically, this reconstruction sub-network can consist of an encoder and a decoder. The encoder is used to extract feature maps from the input image to be detected, and the decoder is used to reconstruct the feature maps back to the original resolution. Specifically, when the encoder extracts feature maps from the image to be detected, it can utilize convolution, normalization, and pooling processes with different parameters to extract feature maps at different levels. Specifically, the image to be detected can first be convolved using convolution kernels to obtain a convolutional map; then, the convolutional map can be normalized using traditional linear correction units and batch normalization methods to obtain a normalized convolutional map; finally, max or average pooling can be applied to the normalized convolutional map. To obtain rich multi-scale features, the reconstruction sub-network can adjust relevant parameters and repeat the above process multiple times to extract multi-scale feature maps through multiple downsampling processes. The decoder can restore the resolution of the image's feature maps through corresponding convolution, normalization, and upsampling processes. This reconstruction sub-network can combine feature maps with the same resolution from both the encoder and decoder and uses multiple sets of convolution processes.

[0078] In step S402, defect detection is performed on the image to be detected using the defect detection neural network based on the reconstructed image and its feature representation. The defect detection neural network is trained using the following method: inputting a training image and obtaining its feature representation; selecting one or more semantic regions based on the feature representation and generating semantic defect feature representations based on the one or more semantic regions; generating a training defect image based at least on the training image and the semantic defect feature representation; and performing defect detection based on the training image, its feature representation, the training defect image, and the semantic defect feature representation to train and adjust the parameters of the defect detection neural network.

[0079] According to embodiments of the present invention, defects can be located using a localization subnetwork in an anomaly localization network based on the reconstructed image and its feature representation. The structure of the localization subnetwork in the anomaly localization network is similar to that of the reconstructed subnetwork described above, but it may further have cross-layer connection operations to fuse feature maps of the same scale from the encoder and decoder in the localization subnetwork.

[0080] During the defect detection process, the anomaly localization network can detect both logical and structural defects in the image to be detected. Optionally, after detecting defects in the image, the anomaly localization network can output the specific location of the defects and simultaneously output an estimated value for the degree of defect for user reference.

[0081] According to an embodiment of the present invention, the defect detection neural network in this defect detection method utilizes, for example... Figure 3 The training is performed according to the steps shown. The specific training process is already described in [the document / document / etc.]. Figure 3 The steps shown are explained in detail and will not be repeated here.

[0082] The defect detection method according to embodiments of the present invention can enrich the sample data used to train the defect detection neural network by generating semantic defect feature representations of semantic regions of feature representations of training images, thereby achieving robust detection of semantic defects for different types of products or objects, avoiding the collection of a large amount of defect data and the process of manual annotation, greatly reducing production costs and improving user experience.

[0083] The following illustrates a specific implementation process of a generative neural network training method and an image generation method according to an embodiment of the present invention.

[0084] According to one example of an embodiment of the present invention, a training image may be input first, and a feature representation map of the training image may be obtained based on the features of the training image. In this example, the feature representation map of the training image may be a contour map of the training image. Figure 5 A schematic diagram of training images is shown as an example according to an embodiment of the present invention. Figure 5 In this process, the input training image can be an image with texture features. Figure 6 A feature representation map of a training image is shown as an example according to an embodiment of the present invention. Figure 6In this invention, a feature detection network applicable to the embodiments of the invention can be trained to extract features from training images and output a feature representation map corresponding to the training images. Specifically, a lightweight edge detection network with fewer parameters can be trained by combining the training images with a pre-trained edge detection network with relatively more parameters. This lightweight edge detection network learns from the output of the pre-trained edge detection network with more parameters and obtains edge detection capabilities similar to the pre-trained edge detection network in an unsupervised training manner. More specifically, for the same input training image, it is desirable to train the output of the lightweight edge detection network to be as close as possible to the output of the pre-trained edge detection network. Therefore, the goal of the above training process is to minimize the output difference between the lightweight edge detection network and the pre-trained edge detection network. After training the lightweight edge detection network, it can be used to perform edge detection on the input training image and output a contour map corresponding to the training image.

[0085] Subsequently, a first enhancement process can be performed on the feature representation map of the training image to obtain a first enhanced training image. In an example of this embodiment, the first enhancement process on the feature representation map of the training image may include one or more operations such as image distortion, image flipping, image rotation, image cropping, edge adjustment, image translation, and image scaling. Specifically, when performing local distortion deformation on the feature representation map of the training image, a local region can be selected in the feature representation map of the training image and divided into a grid. Several nodes in the grid are randomly selected, and the selected nodes are randomly translated horizontally or vertically. On this basis, different bending deformations, flipping, cropping, zero-padding, and other processes can be further performed to obtain the first enhanced training image. Figure 7 An example of a first enhanced training image obtained by performing a first enhancement process on the contour map of a training image according to an embodiment of the present invention is shown.

[0086] After this, a second enhancement process can be performed on the training image to obtain a second enhanced training image, which is different from the first enhancement process.

[0087] The second enhancement processing of the training image may also include one or more operations such as image distortion, image flipping, image rotation, image cropping, edge adjustment, image translation, and image scaling, and the second enhancement processing is different from the first enhancement processing. Optionally, a local region of the training image may be selected and subjected to bending deformation, flipping, cropping, zero-padding, etc., which are different from the first enhancement processing, to obtain a second enhanced training image. Figure 8 An example of a second enhanced training image obtained by performing a second enhancement process on a training image according to an embodiment of the present invention is shown.

[0088] Finally, based on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image, the generative neural network can be used to perform image generation operations to obtain training generated images, so as to train the generative neural network and adjust the parameters of the generative neural network.

[0089] Optionally, the generative neural network can first be used to generate intermediate training images and anomaly probability maps representing weights; then, the anomaly probability maps can be used to fuse the training images and the intermediate training images to obtain the training generated images.

[0090] Specifically, during the fusion process in the generative neural network, based on the input to the generative neural network, intermediate training images and anomaly probability maps are output during the intermediate process of the generative neural network. The anomaly probability map can be used as weights to fuse the input training image and the generated intermediate training image to obtain the final trained generative image. Figure 9 An example of obtaining a training-generated image from an intermediate training image and an anomaly probability map according to an embodiment of the present invention is shown. In this example, the anomaly probability map can be used as a weight such that the result of multiplying it pointwise with the intermediate training image and the result of multiplying it pointwise with the training image after taking its inverse are added together to obtain the training-generated image.

[0091] In this embodiment of the invention, the training generated image generated by the generative neural network can be a training generated image that combines the feature representation map of the training image as feature information and the texture of the training image as texture information. When acquiring the training generated image, the generative neural network can be trained and its parameters adjusted accordingly to make the parameters of the generative neural network converge.

[0092] After training the generative neural network, it can optionally be used to obtain the desired image generation result. For example, the generative neural network can be applied to fields such as clothing design and virtual try-on. For instance, a user can input a sketch showing the clothing structure as a feature map and a desired texture map to obtain a clothing image that combines the design sketch and the texture image.

[0093] In one example of an embodiment of the present invention, a feature representation map for generating feature information of an image and a texture representation map for generating texture information of an image can be obtained.

[0094] Optionally, the feature information can be contour information, and the feature representation map used to generate the image can be a contour map representing the contour of the generated image.

[0095] Alternatively, the texture representation map used to generate the image may contain texture information that is expected to be presented in the generated image, such that the generative neural network combines the feature representation map to present the texture information in the texture representation map in the generated image.

[0096] Subsequently, the generated image can be obtained using a generative neural network based on the feature representation map and the texture representation map, wherein the generative neural network is trained in the manner described above.

[0097] The generated image is generated using the previously trained generative neural network, and the generated image presents the feature information contained in the feature representation map and the texture information contained in the texture representation map.

[0098] Figure 10 A schematic diagram of an example image generation method according to an embodiment of the present invention is shown. Figure 10 In this context, the same texture representation map is used. Figure 10 The first column on the left), and respectively through different contour maps ( Figure 10 Feature representations in the second and fourth columns from the left Figure 1 and 2 Different generated images were produced using this method. Figure 10 The generated images in the third and fifth columns from the left (1 and 2). Figure 10 As shown, the generated images obtained by the generative neural network can obtain different results based on the same texture representation map and different feature representation maps, and vice versa.

[0099] The following illustrates an example of a defect detection neural network training method and a specific implementation process of a defect detection method according to an embodiment of the present invention.

[0100] In this example of the embodiment of the invention, the scenario is applied to the detection and analysis of objects in a breakfast box. In this example, a training image is first input, and the feature representation of the training image is obtained. In this example, the feature representation of the training image can be the contour of the training image. Figure 5 and Figure 6 Similar to the example shown, they can be obtained separately. Figure 5 The training image shown is used to extract its contour as Figure 6 The outline of the training image shown.

[0101] Subsequently, one or more semantic regions can be selected based on the feature representation of the training image, and semantic defect feature representations can be generated based on the one or more semantic regions.

[0102] Optionally, one or more semantic regions may be selected from at least a portion of the feature representation of the training image; a third enhancement process may be performed on the feature representation of the training image within the selected one or more semantic regions; the feature representation of the one or more semantic regions after the third enhancement process may be fused with the feature representation of the training image to generate the semantic defect feature representation. The third enhancement process may include one or more operations such as adding, deleting, or modifying feature representations.

[0103] Figure 11 A schematic diagram illustrating a semantic defect feature representation according to an embodiment of the present invention is shown. Figure 11 As shown, it will Figure 6 The training image shown has a selected semantic region of contours, which has been deleted. Figure 11 In the text, the semantic regions of one orange and some dried fruit were selected and deleted as a third enhancement process, thereby obtaining, for example, the semantic regions of oranges and some dried fruit. Figure 11 The semantic defect features shown are represented.

[0104] Subsequently, a training defect image is generated based at least on the training image and the defect feature representation.

[0105] After obtaining the trained image generation network according to the aforementioned method, the trained image generation network can be used to generate the corresponding training defect image by using the training image as the texture information of the generated training defect image and the semantic defect feature representation as the feature information of the generated training defect image. Figure 12 An example of a training defect image generated according to an embodiment of the present invention is shown. Figure 12 In China, according to Figure 11 The semantic defect features shown are represented as feature information, such as... Figure 5 The training images shown are used as texture information to generate training defect images.

[0106] Finally, defect detection is performed based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the neural network and adjust its parameters.

[0107] According to embodiments of the present invention, a neural network can be trained using the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation, and the parameters of the neural network can be adjusted so that the loss function of the neural network converges as much as possible.

[0108] Optionally, the input images described above can be used as samples to train the anomaly localization network for defect detection in the defect detection neural network, and the parameters of the anomaly localization network can be adjusted. In one example, the anomaly localization network may include a reconstruction subnetwork and a localization subnetwork. The reconstruction subnetwork is used to convert the input training defect image containing defects into a reconstructed image without defects and its corresponding feature representation. The localization subnetwork locates the defects by calculating the difference between the reconstructed image and its corresponding feature representation converted by the reconstruction subnetwork and the original input image.

[0109] This anomaly localization network can detect not only logical defects in training defect images but also structural defects during the defect detection process. Optionally, after performing defect detection on an image, the trained anomaly localization network can output the specific location of the defect and simultaneously output an estimated value for the degree of defect.

[0110] In another example of this invention, after a neural network for defect detection has been trained, defect detection can be performed on an input image to be detected using this neural network. Specifically, the image to be detected can be input first, and the defect detection neural network can be used to reconstruct the image to obtain a reconstructed image, and the feature representation of the reconstructed image, such as the contour of the reconstructed image, can be obtained.

[0111] Subsequently, defect detection can be performed on the image to be detected based on the reconstructed image and its feature representation. The anomaly localization network in this defect detection neural network can detect not only logical defects but also structural defects in the image to be detected during the defect detection process. Optionally, after detecting defects in the image to be detected, the anomaly localization network can output the specific location of the defects and simultaneously output an estimated value for the degree of defect for user reference.

[0112] Below, refer to Figure 13 The apparatus 1300 for training a generative neural network according to an embodiment of the present invention will be described. Figure 13 A block diagram of an apparatus 1300 for training a generative neural network according to an embodiment of the present invention is shown. Figure 13 As shown, the device 1300 can be a computer or a server.

[0113] like Figure 13 As shown, device 1300 includes one or more processors 1310 and memory 1320. In addition, device 1300 may also include input devices, output devices (not shown), etc., and these components can be interconnected via a bus system and / or other forms of connection mechanisms. It should be noted that... Figure 13The components and structure of the device 1300 shown are merely exemplary and not limiting; the device 1300 may have other components and structures as needed.

[0114] The processor 1310 may be a central processing unit (CPU) or other processing unit with data processing and / or instruction execution capabilities, and may utilize computer program instructions stored in memory 1220 to perform desired functions, including: inputting a training image and obtaining a feature representation map of the training image based on the features of the training image; performing a first enhancement process on the feature representation map of the training image to obtain a first enhanced training image; performing a second enhancement process on the training image to obtain a second enhanced training image, the second enhancement process being different from the first enhancement process; and performing an image generation operation using the generative neural network based at least on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image to obtain a training generated image, so as to train the generative neural network and adjust the parameters of the generative neural network.

[0115] The memory 1320 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1310 may execute the program instructions to implement the functions of the apparatus described in the embodiments of the present invention above, and / or other desired functions, and / or to execute a generative neural network training method according to embodiments of the present invention. Various application programs and various data may also be stored in the computer-readable storage medium.

[0116] The following describes a computer-readable storage medium according to an embodiment of the present invention, having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, perform the following steps: inputting a training image and obtaining a feature representation map of the training image based on the features of the training image; performing a first enhancement process on the feature representation map of the training image to obtain a first enhanced training image; performing a second enhancement process on the training image to obtain a second enhanced training image, the second enhancement process being different from the first enhancement process; and performing an image generation operation using the generative neural network based at least on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image to obtain a training generated image, so as to train the generative neural network and adjust the parameters of the generative neural network.

[0117] Below, refer to Figure 14This invention describes an apparatus for image generation using a generative neural network according to an embodiment of the present invention. Figure 14 A block diagram of an image generation apparatus 1400 according to an embodiment of the present invention is shown. Figure 14 As shown, the device 1400 can be a computer or a server.

[0118] like Figure 14 As shown, device 1400 includes one or more processors 1410 and memory 1420. In addition, device 1400 may also include input devices, output devices (not shown), etc., and these components can be interconnected via a bus system and / or other forms of connection mechanisms. It should be noted that... Figure 14 The components and structure of the device 1400 shown are merely exemplary and not limiting; the device 1400 may also have other components and structures as needed.

[0119] The processor 1410 may be a central processing unit (CPU) or other processing unit with data processing capabilities and / or instruction execution capabilities, and may utilize computer program instructions stored in memory 1420 to perform desired functions, including: acquiring a feature representation map for generating feature information of an image, and a texture representation map for generating texture information of an image; acquiring the generated image using a generative neural network based on the feature representation map and the texture representation map, wherein the generative neural network is trained by: inputting a training image and acquiring a feature representation map of the training image based on the features of the training image; performing a first enhancement processing on the feature representation map of the training image to obtain a first enhanced training image; performing a second enhancement processing on the training image to obtain a second enhanced training image, the second enhancement processing being different from the first enhancement processing; and performing an image generation operation using the generative neural network based at least on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image to obtain a training generated image, so as to train the generative neural network and adjust the parameters of the generative neural network.

[0120] The memory 1420 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1410 may execute the program instructions to implement the functions of the image generation apparatus described in the embodiments of the present invention above, and / or other desired functions, and / or to execute the image generation method according to the embodiments of the present invention. Various application programs and various data may also be stored in the computer-readable storage medium.

[0121] The following describes a computer-readable storage medium according to embodiments of the present invention, having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, perform the following steps: obtaining a feature representation map of feature information for generating an image, and a texture representation map of texture information for generating an image; obtaining the generated image using a generative neural network based on the feature representation map and the texture representation map, wherein the generative neural network is trained by: inputting a training image and obtaining a feature representation map of the training image based on the features of the training image; performing a first enhancement processing on the feature representation map of the training image to obtain a first enhanced training image; performing a second enhancement processing on the training image to obtain a second enhanced training image, the second enhancement processing being different from the first enhancement processing; performing an image generation operation using the generative neural network based at least on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image to obtain a training generated image, thereby training the generative neural network and adjusting the parameters of the generative neural network.

[0122] Below, refer to Figure 15 This describes a defect detection neural network training apparatus according to an embodiment of the present invention. Figure 15 A block diagram of a defect detection neural network training apparatus 1500 according to an embodiment of the present invention is shown. Figure 15 As shown, the device 1500 can be a computer or a server.

[0123] like Figure 15 As shown, the defect detection neural network training device 1500 includes one or more processors 1510 and a memory 1520. In addition, the defect detection neural network training device 1500 may also include input devices, output devices (not shown), etc., and these components can be interconnected via a bus system and / or other forms of connection mechanisms. It should be noted that... Figure 15 The components and structure of the defect detection neural network training device 1500 shown are merely exemplary and not limiting. The defect detection neural network training device 1500 may also have other components and structures as needed.

[0124] The processor 1510 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may utilize computer program instructions stored in memory 1520 to perform desired functions, including: inputting a training image and obtaining a feature representation of the training image; selecting one or more semantic regions based on the feature representation of the training image and generating a semantic defect feature representation based on the one or more semantic regions; generating a training defect image based at least on the training image and the semantic defect feature representation; and performing defect detection based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the defect detection neural network and adjust the parameters of the defect detection neural network.

[0125] The memory 1520 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1510 may execute the program instructions to implement the functions of the defect detection neural network training apparatus described in the embodiments of the present invention above, and / or other desired functions, and / or execute the defect detection neural network training method according to the embodiments of the present invention. Various application programs and various data may also be stored in the computer-readable storage medium.

[0126] The following describes a computer-readable storage medium according to embodiments of the present invention, having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, perform the following steps: inputting a training image and obtaining a feature representation of the training image; selecting one or more semantic regions based on the feature representation of the training image and generating a semantic defect feature representation based on the one or more semantic regions; generating a training defect image based at least on the training image and the semantic defect feature representation; and performing defect detection based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the defect detection neural network and adjust the parameters of the defect detection neural network.

[0127] Below, refer to Figure 16 This invention describes an apparatus for defect detection using a defect detection neural network according to an embodiment of the present invention. Figure 16 A block diagram of an apparatus 1600 for defect detection using a defect detection neural network according to an embodiment of the present invention is shown. Figure 16 As shown, the device 1600 can be a computer or a server.

[0128] like Figure 16As shown, device 1600 includes one or more processors 1610 and memory 1620. In addition, device 1600 may also include input devices, output devices (not shown), etc., and these components can be interconnected via a bus system and / or other forms of connection mechanisms. It should be noted that... Figure 16 The components and structure of the device 1600 shown are merely exemplary and not limiting; the device 1600 may also have other components and structures as needed.

[0129] The processor 1610 may be a central processing unit (CPU) or other processing unit with data processing and / or instruction execution capabilities, and may utilize computer program instructions stored in memory 1620 to perform desired functions, including: inputting an image to be detected; reconstructing the image to be detected using a defect detection neural network; obtaining a reconstructed image and obtaining a feature representation of the reconstructed image; performing defect detection on the image to be detected using the defect detection neural network based on the reconstructed image and the feature representation of the reconstructed image; wherein the defect detection neural network is trained by: inputting a training image and obtaining a feature representation of the training image; selecting one or more semantic regions based on the feature representation of the training image and generating a semantic defect feature representation based on the one or more semantic regions; generating a training defect image based at least on the training image and the semantic defect feature representation; and performing defect detection based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train and adjust the parameters of the defect detection neural network.

[0130] The memory 1620 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 1610 may execute the program instructions to implement the functions of the apparatus for defect detection using a defect detection neural network according to the embodiments of the present invention described above, and / or other desired functions, and / or to execute the method for defect detection using a defect detection neural network according to the embodiments of the present invention. Various application programs and various data may also be stored in the computer-readable storage medium.

[0131] The following describes a computer-readable storage medium according to embodiments of the present invention, having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, perform the following steps: inputting an image to be detected; reconstructing the image to be detected using a defect detection neural network to obtain a reconstructed image and obtaining a feature representation of the reconstructed image; performing defect detection on the image to be detected using the defect detection neural network based on the reconstructed image and the feature representation of the reconstructed image; wherein the defect detection neural network is trained by: inputting a training image and obtaining a feature representation of the training image; selecting one or more semantic regions based on the feature representation of the training image and generating a semantic defect feature representation based on the one or more semantic regions; generating a training defect image based at least on the training image and the semantic defect feature representation; performing defect detection based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the defect detection neural network and adjust the parameters of the defect detection neural network.

[0132] Of course, the specific embodiments described above are merely examples and not limitations. Those skilled in the art can combine and integrate some steps and devices from the various embodiments described separately above to achieve the effects of the present invention. Such combined and integrated embodiments are also included in the present invention, but will not be described one by one here.

[0133] Note that the advantages, benefits, and effects mentioned in this invention are merely examples and not limitations, and should not be considered as essential features of every embodiment of the invention. Furthermore, the specific details described above are for illustrative and illustrative purposes only, and are not intended to limit the invention. These details do not limit the invention from being implemented solely by employing these specific details.

[0134] The block diagrams of devices, apparatuses, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0135] The flowcharts and method descriptions in this invention are merely illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the given order. As those skilled in the art will recognize, the steps in the above embodiments can be performed in any order. Words such as "then," "next," etc., are not intended to limit the order of steps; these words are only used to guide the reader through the description of these methods. Furthermore, any reference to a singular element, such as the use of the articles "a," "one," or "the," is not to be construed as limiting that element to the singular.

[0136] Furthermore, the steps and apparatus in the various embodiments herein are not limited to any one embodiment. In fact, new embodiments can be conceived by combining relevant steps and apparatus in the various embodiments herein with the concepts of the present invention, and these new embodiments are also included within the scope of the present invention.

[0137] Each operation described above can be performed by any suitable means capable of performing the corresponding function. Such means may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors.

[0138] The various exemplified logic blocks, modules, and circuits described herein can be implemented or performed using a general-purpose processor, digital signal processor (DSP), ASIC, field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, it may be any commercially available processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0139] The steps of the methods or algorithms described in this invention can be directly embedded in hardware, in a software module executed by a processor, or a combination of both. The software module can reside in any form of tangible storage medium. Some examples of usable storage media include random access memory (RAM), read-only memory (ROM), flash memory, EPROM, EEPROM, registers, hard disks, removable disks, CD-ROMs, etc. The storage medium can be coupled to the processor so that the processor can read information from and write information to the storage medium. Alternatively, the storage medium can be integral with the processor. The software module can be a single instruction or many instructions, and can be distributed across several different code segments, different programs, and across multiple storage media.

[0140] The method of this invention includes one or more actions for implementing the method. The methods and / or actions may be interchanged without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions may be modified without departing from the scope of the claims.

[0141] The described functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions on a tangible computer-readable medium. The storage medium can be any available tangible medium that can be accessed by a computer. By way of example, and not limitation, such a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other tangible medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. As used herein, a disc includes a compact disc (CD), a laser disc, an optical disc, a digital universal disc (DVD), a floppy disk, and a Blu-ray disc.

[0142] Therefore, a computer program product can perform the operations described herein. For example, such a computer program product can be a computer-readable tangible medium having instructions tangibly stored (and / or encoded) thereon, which can be executed by one or more processors to perform the operations described herein. The computer program product may include packaging materials.

[0143] Software or instructions can also be transmitted via a transmission medium. For example, software can be transmitted from a website, server, or other remote source using transmission media such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, or microwave.

[0144] Furthermore, modules and / or other suitable means for carrying out the methods and techniques described herein can be downloaded and / or obtained by user terminals and / or base stations as appropriate. For example, such a device can be coupled to a server to facilitate the transmission of means for carrying out the methods described herein. Alternatively, the various methods described herein can be provided via storage components (e.g., RAM, ROM, physical storage media such as CDs or floppy disks) so that user terminals and / or base stations can obtain the various methods when coupled to the device or when providing storage components to the device. Furthermore, any other suitable techniques for providing the methods and techniques described herein to the device can be utilized.

[0145] Other examples and implementations are within the scope and spirit of this invention and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Features implementing the functions can also be physically located in various places, including being distributed so that parts of the functions are implemented at different physical locations. Moreover, as used herein, including as used in the claims, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the described examples are preferred or better than other examples.

[0146] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims is not limited to the specific aspects of the processes, machines, manufacturing processes, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufacturing processes, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufacturing processes, events, means, methods, or actions within their scope.

[0147] The above description of aspects of the invention is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features of the invention herein.

[0148] The above description has been given for illustrative and descriptive purposes. Furthermore, this description is not intended to limit the embodiments of the invention to the forms described herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A method for training a generative neural network, comprising: Input a training image, and obtain a feature representation map of the training image based on its features; The feature representation map of the training image is subjected to a first enhancement process to obtain a first enhanced training image; The training image is subjected to a second enhancement process to obtain a second enhanced training image, wherein the second enhancement process is different from the first enhancement process; At least based on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image, the generative neural network is used to perform an image generation operation to obtain a training generated image, so as to train the generative neural network and adjust the parameters of the generative neural network.

2. The method as described in claim 1, wherein, Generating a feature representation map of the training image based on the features of the training image includes: Extract the contours of the training images and generate a contour map of the training images as a feature representation map of the training images.

3. The method as described in claim 1, wherein, The first enhancement process and / or the second enhancement process include: Image distortion, image flipping, image rotation, image cropping, edge adjustment, image translation, and image scaling are one or more of the following operations: image distortion, image flipping, image rotation, image cropping, image edge adjustment, image translation, and image scaling.

4. The method of claim 1, wherein, At least based on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image, the generative neural network is used to perform an image generation operation to obtain a training generated image, including: The generative neural network is used to generate intermediate training images and anomaly probability maps representing weights; The training image and the intermediate training image are fused using the anomaly probability map to obtain the training generated image.

5. A method for training a defect detection neural network, comprising: Input a training image and obtain its feature representation; Select one or more semantic regions based on the feature representation of the training image, and generate semantic defect feature representations based on the one or more semantic regions; At least based on the training image and the semantic defect feature representation, a training defect image is generated; Defect detection is performed based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the defect detection neural network and adjust the parameters of the defect detection neural network.

6. The method of claim 5, wherein, Generating training defect images based at least on the training images and the semantic defect feature representations includes: Based on the training image and the semantic defect feature representation, a generative neural network is used to generate the training defect image, wherein the generative neural network is trained in the following manner: Input a training image, and obtain a feature representation map of the training image based on its features; The feature representation map of the training image is subjected to a first enhancement process to obtain a first enhanced training image; The training image is subjected to a second enhancement process to obtain a second enhanced training image, wherein the second enhancement process is different from the first enhancement process; At least based on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image, the generative neural network is used to perform an image generation operation to obtain a training generated image, so as to train the generative neural network and adjust the parameters of the generative neural network.

7. The method of claim 5, wherein, Selecting one or more semantic regions based on the feature representation of the training image, and generating semantic defect feature representations based on the one or more semantic regions, includes: Select one or more semantic regions from at least a portion of the feature representation of the training image; A third enhancement process is performed on the feature representation of the training image in one or more selected semantic regions; The feature representations of the one or more semantic regions after the third enhancement process are fused with the feature representations of the training image to generate the semantic defect feature representation.

8. The method of claim 7, wherein, The third enhancement process includes: One or more operations among adding, deleting, and modifying features.

9. The method of claim 5, wherein, Defect detection is performed based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation. Training the defect detection neural network and adjusting its parameters includes: The defect detection neural network is trained by converting the training defect image and the semantic defect feature representation into the training image and the feature representation of the training image, respectively, and the parameters of the defect detection neural network are adjusted.

10. An image generation method, comprising: Obtain a feature representation map for generating the feature information of the image, and a texture representation map for generating the texture information of the image; The generated image is obtained using a generative neural network based on the feature representation map and the texture representation map, wherein the generative neural network is trained in the following manner: Input a training image, and obtain a feature representation map of the training image based on its features; The feature representation map of the training image is subjected to a first enhancement process to obtain a first enhanced training image; The training image is subjected to a second enhancement process to obtain a second enhanced training image, wherein the second enhancement process is different from the first enhancement process; At least based on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image, the generative neural network is used to perform an image generation operation to obtain a training generated image, so as to train the generative neural network and adjust the parameters of the generative neural network.

11. A defect detection method, comprising: Input the image to be detected, reconstruct the image using a defect detection neural network, obtain the reconstructed image, and obtain the feature representation of the reconstructed image; Based on the reconstructed image and its feature representation, the defect detection neural network is used to detect defects in the image to be detected. The defect detection neural network is trained using the following method: Input a training image and obtain its feature representation; Select one or more semantic regions based on the feature representation of the training image, and generate semantic defect feature representations based on the one or more semantic regions; At least based on the training image and the semantic defect feature representation, a training defect image is generated; Defect detection is performed based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the defect detection neural network and adjust the parameters of the defect detection neural network.

12. A generative neural network training device, comprising: processor; and a memory, in which computer program instructions are stored. When the computer program instructions are executed by the processor, the processor performs the following steps: Input a training image, and obtain a feature representation map of the training image based on its features; The feature representation map of the training image is subjected to a first enhancement process to obtain a first enhanced training image; The training image is subjected to a second enhancement process to obtain a second enhanced training image, wherein the second enhancement process is different from the first enhancement process; At least based on the training image, the feature representation map of the training image, the first enhanced training image, and the second enhanced training image, the generative neural network is used to perform an image generation operation to obtain a training generated image, so as to train the generative neural network and adjust the parameters of the generative neural network.

13. A defect detection neural network training device, comprising: processor; and a memory, in which computer program instructions are stored. When the computer program instructions are executed by the processor, the processor performs the following steps: Input a training image and obtain its feature representation; Select one or more semantic regions based on the feature representation of the training image, and generate semantic defect feature representations based on the one or more semantic regions; At least based on the training image and the semantic defect feature representation, a training defect image is generated; Defect detection is performed based on the training image, the feature representation of the training image, the training defect image, and the semantic defect feature representation to train the defect detection neural network and adjust the parameters of the defect detection neural network.

Images