Neural network training method, image generation method, defect detection method and apparatus, and computer program

Abstract

The present invention provides a neural network training method, an image generation method, a defect detection method and apparatus, and a computer program.SOLUTION: Inputting a training image and obtaining a feature map of the training image according to a feature of the training image, performing first enhancement processing on the feature map of the training image to obtain a first enhanced training image, performing second enhancement processing different from the first enhancement processing on the training image to obtain a second enhanced training image, and performing an image generation operation using the generative neural network based on at least the training image, the feature map of the training image, the first enhanced training image, and the second enhanced training image to obtain a generated training image, and training the generative neural network to adjust parameters of the generative neural network.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to the field of image processing, and in particular to neural network training methods, image generation methods, defect detection methods and apparatus, and computer programs. [Background technology]

[0002] One of the important research topics is the technology to generate desired images using acquired images and to train neural networks for defect detection based on the generated images to detect defects in products. Defect detection is an important factor in product quality control, among other things. Since various defects occur constantly during production, it is usually impossible to cover all the situations in which various defects occur. Therefore, how to detect these defects robustly and automatically is an issue that needs to be solved as soon as possible.

[0003] In current defect detection methods, neural network models for defect detection are typically trained by a large number of simulated defect samples, but the generated images are insufficient to simulate the actual defects of the product, making it difficult to accurately detect and locate the defects in the images.

[0004] Thus, there is a need for improved methods and apparatus for training neural networks, methods for generating more realistic images using neural networks, and more accurate methods and apparatus for detecting defects. Summary of the Invention [Problem to be solved by the invention]

[0005] SUMMARY OF THE INVENTION It is an object of the present invention to provide a method and apparatus for training a generative neural network, a method and apparatus for generating an image, a method and apparatus for training a defect detection neural network, a method and apparatus for detecting defects, and a computer program. [Means for solving the problem]

[0006] In order to solve the above problem, according to one aspect of the present invention, a generative neural network training method is provided, which includes the steps of inputting training images and obtaining a feature display diagram of the training images based on the features of the training images; performing a first enhancement process on the feature display diagram of the training images to obtain first enhanced training images; performing a second enhancement process different from the first enhancement process on the training images to obtain second enhanced training images; and performing an image generation operation using the generative neural network based on at least the training images, the feature display diagram of the training images, the first enhanced training images, and the second enhanced training images to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network.

[0007] According to another aspect of the present invention, there is provided a defect detection neural network training method, comprising the steps of inputting training images and obtaining a characterization of the training images; selecting one or more semantic domains based on the characterization of the training images and generating a semantic defect characterization based on the one or more semantic domains; generating training defect images based on at least the training images and the semantic defect characterization; and performing defect detection based on the training images, the characterization of the training images, the training defect images and the semantic defect characterization to train the defect detection neural network and adjust parameters of the defect detection neural network.

[0008] According to another aspect of the present invention, there is provided an image generation method, comprising the steps of: obtaining a feature representation map for generating feature information of an image and a texture representation map for generating texture information of the image; and obtaining the generated image using a generative neural network based on the feature representation map and the texture representation map, wherein the generative neural network is trained by a method including the steps of: inputting training images and obtaining a feature representation map of the training images based on features of the training images; performing a first enhancement process on the feature representation map of the training images to obtain first enhanced training images; performing a second enhancement process different from the first enhancement process on the training images to obtain second enhanced training images; and performing an image generation operation using the generative neural network based on at least the training images, the feature representation map of the training images, the first enhanced training images, and the second enhanced training images to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network.

[0009] According to another aspect of the present invention, there is provided a defect detection method, comprising the steps of: inputting an image to be detected and reconstructing the image to be detected using a defect detection neural network, obtaining the reconstructed image, and obtaining a characterization of the reconstructed image; and performing defect detection on the image to be detected using the defect detection neural network based on the reconstructed image and the characterization of the reconstructed image, wherein the defect detection neural network is trained in a manner including the steps of inputting a training image and obtaining a characterization of the training image; selecting one or more semantic regions based on the characterization of the training image and generating semantic defect characterizations based on the one or more semantic regions; generating training defect images based on at least the training images and the semantic defect characterizations; and performing defect detection based on the training images, the characterizations of the training images, the training defect images, and the semantic defect characterizations to train the defect detection neural network and adjust parameters of the defect detection neural network.

[0010] According to another aspect of the present invention, there is provided an apparatus for training a generative neural network, the apparatus comprising: a processor; and a memory having computer program commands stored therein, which, when executed by the processor, cause the processor to perform the following steps: inputting training images; and obtaining a feature display map of the training images based on features of the training images; performing a first enhancement process on the feature display map of the training images to obtain first enhanced training images; performing a second enhancement process on the training images, which is different from the first enhancement process, to obtain second enhanced training images; and performing an image generation operation using the generative neural network based on at least the training images, the feature display map of the training images, the first enhanced training images, and the second enhanced training images to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network.

[0011] According to another aspect of the present invention, there is provided an apparatus for training a defect detection neural network, comprising: a processor; and a memory having computer program commands stored therein, which, when executed by the processor, cause the processor to perform the following steps: inputting training images and obtaining characterizations of the training images; selecting one or more semantic domains based on the characterizations of the training images and generating semantic defect characterizations based on the one or more semantic domains; generating training defect images based on at least the training images and the semantic defect characterizations; and performing defect detection based on the training images, the characterizations of the training images, the training defect images, and the semantic defect characterizations to train the defect detection neural network and adjust parameters of the defect detection neural network. [Effects of the Invention]

[0012] According to the method and device for training a generative neural network and the method and device for generating images using a generative neural network of the present invention, by performing different enhancement processes on the feature display map of the training image and the training image, different information from the training image and the feature display map of the training image can be comprehensively obtained to train the generative neural network, and the generated images can be made more realistic and meet various user needs.

[0013] In addition, according to the above-mentioned defect detection neural network training method and apparatus, and the method and apparatus for performing defect detection using a defect detection neural network of the present invention, by generating semantic defect feature representations in the semantic domain of the feature representations of training images, it is possible to enrich the sample data for training the defect detection neural network, thereby realizing robust detection of semantic defects in different types of products and objects, avoiding the process of collecting large amounts of defect data and manual marking, significantly reducing production costs, and improving user experience. [Brief explanation of the drawings]

[0014] The above and other objects, features and advantages of the present invention will become more apparent from the detailed description of the embodiments of the present invention in combination with the drawings. [Figure 1] FIG. 1 is a flow diagram of a method for training a generative neural network according to an embodiment of the present invention. [Figure 2] FIG. 2 is a flow diagram of a method for image generation using a generative neural network according to an embodiment of the present invention. [Figure 3] FIG. 3 is a flow diagram of a method for training a defect detection neural network according to an embodiment of the present invention. [Figure 4] FIG. 4 is a flow diagram of a method for performing defect detection using a defect detection neural network according to an embodiment of the present invention. [Figure 5] FIG. 5 is a schematic diagram of a training image according to an example embodiment of the present invention. [Figure 6] FIG. 6 is a diagram illustrating a feature display of a training image according to an example embodiment of the present invention. [Figure 7] FIG. 7 is a first enhanced training image obtained by performing a first enhancement process on a contour map of a training image according to an example embodiment of the present invention. [Figure 8] FIG. 8 shows a second enhanced training image obtained by performing the second enhancement process on the training image according to an example of the embodiment of the present invention. [Figure 9] FIG. 9 is a diagram illustrating obtaining training-generated images through intermediate training images and anomaly probability maps according to an example embodiment of the present invention. [Figure 10] FIG. 10 is a schematic diagram of an image generation method according to an example embodiment of the present invention. [Figure 11] FIG. 11 is a schematic diagram of a semantic defect characterization according to an example embodiment of the present invention. [Figure 12] FIG. 12 is an illustration of a generated training defect image according to an example embodiment of the present invention. [Figure 13] FIG. 13 is a block diagram of a generative neural network training apparatus according to an embodiment of the present invention. [Figure 14] FIG. 14 is a block diagram of an image generating device according to an embodiment of the present invention. [Figure 15] FIG. 15 is a block diagram of a fault detection neural network training apparatus according to an embodiment of the present invention. [Figure 16] FIG. 16 is a block diagram of an apparatus for performing defect detection using a defect detection neural network according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, with reference to the drawings, a method and apparatus for training a neural network, and a method and apparatus for generating an image using a neural network according to embodiments of the present invention will be described. Also, a method and apparatus for training a defect detection neural network, and a method and apparatus for detecting defects using a defect detection neural network according to embodiments of the present invention will be described. In the drawings, like reference numerals refer to like elements throughout. It should be understood that the embodiments described herein are merely illustrative and should not be construed as limiting the scope of the present invention.

[0016] 1 is a flow diagram of a method 100 for training a generative neural network according to an embodiment of the present invention. The method 100 for training a generative neural network according to an embodiment of the present invention will now be described with reference to FIG.

[0017] In step S101, a training image is input, and a feature representation (expression) of the training image is obtained based on the features of the training image.

[0018] In an embodiment of the present invention, the feature display diagram of the acquired training image may optionally be, for example, a contour diagram of the training image, and may also include, but is not limited to, images showing various relevant features of the training image, such as color information, shape information, and classification information.

[0019] In embodiments of the present invention, a feature detection network applicable to embodiments of the present invention can be optionally trained to perform feature extraction on training images and output a feature display map corresponding to the training images. For example, if the feature display map of a training image is a contour map of the training image, a lightweight edge detection network with fewer parameters can be trained using the training images in combination with a pre-trained edge detection network with a relatively large number of parameters. The lightweight edge detection network can then learn the output of the pre-trained edge detection network with a relatively large number of parameters, thereby obtaining edge detection capabilities similar to those of the pre-trained edge detection network through unsupervised training. More specifically, for the same input training images, it is desirable to train the output of the lightweight edge detection network and the output of the pre-trained edge detection network as close as possible to each other. Therefore, the goal of the training process is to minimize the difference between the outputs of the lightweight edge detection network and the pre-trained edge detection network. After training the lightweight edge detection network, the trained lightweight edge detection network can be used to perform edge detection on the input training images and output a contour map corresponding to the training images. The above is an example of training an edge detection network in which the feature display map of a training image is a contour map of the training image. In practical applications, the feature displays of different training images can be used to train and apply corresponding feature detection networks to obtain the feature display maps of the corresponding training images.

[0020] In step S102, a first enhancement process is performed on the feature display diagram of the training image to obtain a first enhanced training image.

[0021] Image enhancement is a technique that performs various pre-processing operations on images. Image enhancement can transform, modify, enhance, etc. an image to generate a new transformed image, thereby achieving diversity in image samples.

[0022] In an embodiment of the present invention, the first enhancement process for the feature display map of the training images includes one or more of image distortion, image flip, image rotation, image crop, edge adjustment, image translation, and image scaling. For example, the feature display map of the training images may be subjected to randomly selected local distortion, flip, zero padding, and other operations. Specifically, when performing local distortion on the feature display map of the training images, a local region is selected from the feature display map of the training images, divided into a grid, multiple nodes in the grid are randomly selected, and the selected nodes are randomly translated horizontally or vertically. Based on this, different distortion, flip, zero padding, and other operations are successively performed to obtain a first enhanced training image.

[0023] In step S103, a second enhancement process different from the first enhancement process is performed on the training image to obtain a second enhanced training image.

[0024] Similar to the above steps, the second enhancement process for the training image also includes one or more operations of image distortion, image flip, image rotation, image crop, edge adjustment, image translation, and image scaling, and the second enhancement process is different from the first enhancement process. Optionally, a local region of the training image can be selected and subjected to a process such as curvature transformation, flipping, or zero padding that is different from the first enhancement process to obtain the second enhanced training image.

[0025] In step S104, based on at least the training image, the feature display diagram of the training image, the first enhanced training image, and the second enhanced training image, an image generation operation is performed using the generative neural network to obtain a training generated image, and the generative neural network is trained to adjust the parameters of the generative neural network.

[0026] In this step, optionally, the generative neural network may first be used to generate an anomaly probability map showing intermediate training images and weights, and then the anomaly probability map may be used to fuse the training images and the intermediate training images to obtain the training generative images.

[0027] Specifically, during the fusion process in the generative neural network, based on the input to the generative neural network, an intermediate training image and an abnormality probability map are output in the intermediate process of the generative neural network, and the input training image and the generated intermediate training image are fused using the abnormality probability map as a weight to obtain a final training generated image.

[0028] In an embodiment of the present invention, the training images generated by the generative neural network may be generated by combining a feature display map of the training images as feature information and the texture of the training images as texture information. When the training images are acquired, the generative neural network can be trained based on the training images and the parameters of the generative neural network can be adjusted to converge the parameters of the generative neural network. For example, the results of a first enhancement process on the training images can be used as true values, and the results can be compared with the training images generated by the generative neural network, and the parameters of the generative neural network can be adjusted accordingly.

[0029] The generative neural network training method according to the embodiment of the present invention performs different enhancement processes on the feature display map of the training image and the training image, respectively, to comprehensively obtain different information from the training image and the feature display map of the training image to train the generative neural network, so that the generated images are more realistic and can meet various user needs.

[0030] Figure 2 is a flow diagram of a method 200 for image generation using a generative neural network according to an embodiment of the present invention. The image generation method according to an embodiment of the present invention will now be described with reference to Figure 2. In an embodiment of the present invention, the image generation operation can be performed using a generative neural network trained according to the steps shown in Figure 1.

[0031] In step S201, a feature display diagram for generating feature information of an image and a texture display diagram for generating texture information of the image are obtained.

[0032] In an embodiment of the present invention, the feature information may optionally be contour information, and the feature display diagram for generating the feature information of the image may be a contour diagram representing the contour of the generated image. The feature information may also include, but is not limited to, information indicating various related features of the generated image, such as color information, shape information, and classification information.

[0033] Optionally, the texture display diagram for generating texture information for an image can include texture information that is desired to appear in the generated image, so that the texture information contained in the texture display diagram appears in the generated image by the generation neural network combining the feature display diagrams.

[0034] In step S202, the generated images are obtained using a generative neural network based on the feature display map and the texture display map, where the generative neural network is trained using a method including the steps of: inputting training images and obtaining a feature display map for the training images based on features of the training images; performing a first enhancement process on the feature display map for the training images to obtain first enhanced training images; performing a second enhancement process different from the first enhancement process on the training images to obtain second enhanced training images; and performing an image generation operation using the generative neural network based on at least the training images, the feature display map for the training images, the first enhanced training images, and the second enhanced training images to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network.

[0035] In this step, the generated image is generated using the trained neural network, and the feature information contained in the feature display map and the texture information contained in the texture display map appear in the generated image. The specific training steps of the neural network in this embodiment are shown in Figure 1, and will not be described here.

[0036] According to the image generation method of the embodiment of the present invention, by utilizing a trained neural network that performs different enhancement processes on the feature display diagram of the training image and the training image, the generated image can be made more realistic and meet various user needs.

[0037] After training the generative neural network to generate images, embodiments of the present invention can also utilize the trained generative neural network to generate samples for the defect detection network, thereby providing a richer and more diverse set of defects in the samples and training a more robust defect detection neural network.

[0038] In accordance with an embodiment of the present invention, a robust defect detection neural network, a defect detection method, an apparatus, and a storage medium are provided. The defect detection neural network training and defect detection method of the embodiment of the present invention does not require manual marking and can effectively simulate defects occurring in the semantic domain, so that the trained defect detection neural network can accurately detect defects in images. The method, apparatus, and medium of the embodiment of the present invention can be applied to product detection and analysis scenarios such as breakfast lunch boxes, screw tool kits, and cable plugs, as well as other computer vision tasks such as defect detection on road surfaces, chip surfaces, or any other object surfaces. The application process of the embodiment of the present invention can be based on a convolutional neural network (CNN), which can extract more complex features at multiple semantic levels rather than limited low-level features.

[0039] 3 is a flow diagram of a method for training a defect detection neural network according to an embodiment of the present invention, which will now be described with reference to FIG.

[0040] In step S301, a training image is input and a feature representation of the training image is obtained.

[0041] In an embodiment of the present invention, the characterization of the training images may include at least one of the contours of the training images, image segmentation information of the training images, and multimodal information of the training images. In one example, the characterization of the training images may include the contours of the training images, which may generally indicate the contours of objects present in the training images. In another example, the characterization of the training images may include image segmentation information of the training images, which may indicate image segmentation information in the training images by an object segmentation process based on thresholds, graph theory, clustering, etc. In yet another example, the characterization of the training images may include multimodal information of the training images, which may be multiple modal information of the training images, including information from various angles, such as text description information and image description information of objects in the training images obtained by using a multimodal big model. In yet another example, based on a specific application scenario of an embodiment of the present invention, the feature display of the training image may further include various related information such as color information, shape information, classification information, etc. of the training image, which can represent the features of objects in the training image, all of which are not limited herein.

[0042] In embodiments of the present invention, a feature detection network applicable to embodiments of the present invention can be trained by selectively using defect-free training images, performing feature extraction on the training images, and outputting feature extraction results corresponding to the training images. For example, if the feature representation of the training images is the contour of the training images, the training images can be combined with a pre-trained edge detection network with a relatively large number of parameters to train a lightweight edge detection network with a relatively small number of parameters. The lightweight edge detection network can then learn the output of the pre-trained edge detection network with a relatively large number of parameters, thereby obtaining edge detection capabilities similar to those of the pre-trained edge detection network through unsupervised training. More specifically, for the same input training images, it is desirable to train the output of the lightweight edge detection network and the output of the pre-trained edge detection network as close as possible to each other for good results. Therefore, the goal of the training process is to minimize the difference between the outputs of the lightweight edge detection network and the pre-trained edge detection network. After training the lightweight edge detection network, the trained lightweight edge detection network can be used to perform edge detection on the input training images and output a contour corresponding to the training image. The above is an example in which the feature representation of the training image is the contour of the training image. In practical applications, by using the feature representations of different training images, the corresponding feature detection networks can be trained and applied respectively to obtain the feature representation detection results of the corresponding training images.

[0043] In step S302, one or more semantic regions are selected based on the characterization of the training images, and a semantic defect characterization is generated based on the one or more semantic regions.

[0044] In this step, one or more semantic regions in at least a portion of the characterization of the training images may be selected, a third enhancement process may be performed on the characterization of the training images in the selected one or more semantic regions, and the characterization of the one or more semantic regions after the third enhancement process may be merged with the characterization of the training images to generate the semantic defect characterization, where the third enhancement process may include one or more operations of adding, deleting, and modifying characterizations.

[0045] Specifically, a semantic region can be randomly selected. The semantic region can be obtained by a variety of methods, including, for example, using a pre-trained model to extract the semantic region, or by selecting multiple large, continuous, or small, discrete regions in the foreground region of the feature representation of the training image as the semantic region. The selected semantic region can then be modified, including but not limited to, adding, deleting, and modifying the feature representation, to obtain the one or more semantic region feature representations after the third enhancement process. The specific operations for modifying the feature representation of the training image are merely examples, and different semantic regions can be modified for different feature representations, and the modifications can be performed once or multiple times. The present disclosure is not limited to these.

[0046] After obtaining the feature representation of the one or more semantic regions after the third enhancement process, the feature representation can be fused with the feature representation of the training image to obtain a semantic defect feature representation. For example, the feature representations can be superimposed to obtain a semantic defect feature representation. Furthermore, for example, portions of the feature representations can be selected and spliced ​​into the semantic defect feature representation. Furthermore, for example, the feature representations of the one or more semantic regions after the third enhancement process can also be superimposed and spliced, or portions of the feature representations can be selected and used to replace portions of the corresponding feature representations of the training image to obtain a fused semantic defect feature representation.

[0047] The above describes an example process for generating a semantic defect characterization. According to another embodiment of the present invention, the semantic defect characterization of the training images can include not only logical defects but also at least one structural defect of the training images, such as a depression, a bump, a distortion, etc., and the types of these defects are not limited.

[0048] In step S303, training defect images are generated based on at least the training images and the semantic defect characterization.

[0049] According to an embodiment of the present invention, training defect images may be generated based on the training images and the semantic defect characterization based on a trained generative neural network.

[0050] As described above, the generative neural network is trained by a method including the steps of inputting training images and obtaining a feature display diagram of the training images based on the features of the training images, performing a first enhancement process on the feature display diagram of the training images to obtain first enhanced training images, performing a second enhancement process different from the first enhancement process on the training images to obtain second enhanced training images, and performing an image generation operation using the generative neural network based on at least the training images, the feature display diagram of the training images, the first enhanced training images, and the second enhanced training images to obtain training generated images, training the generative neural network, and adjusting parameters of the generative neural network.

[0051] After obtaining a trained generation neural network, the training image can be used as the texture information of the generated training defect image, and the semantic defect feature representation can be used as the feature information of the generated training defect image to generate a corresponding training defect image. In addition, during the generation of the training defect image, the number of training defect image samples can be further increased by, for example, adding random noise.

[0052] In step S304, defect detection is performed based on the training images, the training image feature representations, the training defect images, and the semantic defect feature representations to train the defect detection neural network and adjust parameters of the defect detection neural network.

[0053] According to an embodiment of the present invention, by receiving input training images, feature representations of the training images, and training defect images and corresponding semantic defect feature representations generated by the above-mentioned method, a neural network can be trained and the parameters of the neural network can be adjusted to adjust the loss function of the neural network so that it converges as much as possible.

[0054] Optionally, input training images, feature representations of the training images, the training defect images, and the semantic defect feature representations can be used as samples to train an anomaly location network for defect detection within a neural network and adjust parameters of the anomaly location network. In one example, the anomaly location network can include one reconstruction subnetwork and one location subnetwork. The reconstruction subnetwork is used to convert input training defect images containing defects and semantic defect feature representations into reconstructed images and corresponding feature representations that do not contain defects, and the location subnetwork locates defects by calculating the difference between the reconstructed images and corresponding feature representations converted by the reconstruction subnetwork and the original input image.

[0055] More specifically, the reconstruction subnetwork can be composed of an encoder and a decoder. The encoder is used to extract feature maps of an input image, and the decoder is used to reconstruct the feature maps to their original resolution. When the encoder extracts the feature maps of an image, it can extract feature maps of different levels by using convolution, normalization, and pooling processes with different parameters. Specifically, it can first convolve the input image using a convolution kernel to obtain a convolution map, then normalize the convolution map using a conventional linear correction unit and batch normalization method to obtain a normalized convolution map, and finally apply a maximum or average pooling process to the normalized convolution map. To obtain rich multi-scale features, the reconstruction subnetwork can adjust related parameters and repeat the above process multiple times to extract multi-scale feature maps through multiple downsampling processes. The decoder can restore the resolution of the image feature maps through corresponding convolution, normalization, and upsampling processes. The reconstruction subnetwork can combine feature maps with the same resolution in both the encoder and decoder, and use multiple sets of convolution processes. The higher the similarity between the reconstructed image transformed by the reconstruction sub-network and the original input training image, the higher the probability that the training image does not contain defects, and vice versa. The structure of the location sub-network of the anomaly location network is similar to the reconstruction sub-network described above, but can further have a cross-layer connection operation to fuse the same-scale feature maps of the encoder and decoder in the location sub-network.

[0056] Optionally, after performing defect detection on the image, the trained anomaly location network may output the specific location of the defect and may simultaneously output an estimate of the extent of the defect.

[0057] According to the above-mentioned neural network training method of the embodiment of the present invention, by generating semantic defect representations in the semantic domain of the feature representation of the training image, sample data for training the defect detection neural network can be enriched, thereby realizing robust detection of semantic defects in different types of products and objects, avoiding the process of collecting large amounts of defect data and manually marking them, greatly reducing production costs, and improving user experience.

[0058] 4 is a flow diagram of a method 400 for performing defect detection using a defect detection neural network according to an embodiment of the present invention. In this embodiment, defect detection can be performed using the defect neural network trained according to the process shown in FIG. 3. The method for performing defect detection using a defect detection network according to an embodiment of the present invention will now be described with reference to FIG. 4.

[0059] In step S401, a detection target image is input, and the detection target image is reconstructed using a defect detection neural network to obtain a reconstructed image, and a feature representation of the reconstructed image is also obtained.

[0060] In an embodiment of the present invention, the detection target image can be input to a reconstruction subnetwork of the anomaly location network included in the defect detection neural network trained by the process shown in Figure 3 and reconstructed. Optionally, the detection target image may be an image that does not contain logical defects and / or structural defects, or an image that contains one or more logical defects and / or structural defects. In the reconstruction process for the search target image, a reconstructed image with the defects removed is obtained, and at the same time, a characteristic representation of the reconstructed image is obtained and can be used in the subsequent defect detection and location process.

[0061] As described above, the reconstruction subnetwork in the anomaly location network can be used to convert an input target image into a reconstructed image and corresponding feature representation. More specifically, the reconstruction subnetwork can be composed of an encoder and a decoder. The encoder is used to extract a feature map of the input image, and the decoder is used to reconstruct the feature map back to its original resolution. When the encoder extracts the feature map of the image, different levels of feature maps can be extracted by using convolution, normalization, and pooling processes with different parameters. Specifically, the input image can be convolved using a convolution kernel to obtain a convolution map. Then, the convolution map can be normalized using a conventional linear correction unit and batch normalization method to obtain a normalized convolution map. Finally, a maximum or average pooling process can be applied to the normalized convolution map. To obtain rich multi-scale features, the reconstruction subnetwork can adjust related parameters and repeat the above process multiple times to extract multi-scale feature maps through multiple downsampling processes. The decoder can restore the resolution of the image feature maps through appropriate convolution, normalization, and upsampling processes. The reconstruction sub-network can combine feature maps with the same resolution in both the encoder and decoder, and can use multiple sets of convolution processes.

[0062] In step S402, defect detection is performed on the target image using the defect detection neural network based on the reconstructed image and a characterization of the reconstructed image, where the defect detection neural network is trained by a method including the steps of inputting a training image and obtaining a characterization of the training image, selecting one or more semantic regions based on the characterization of the training image and generating a semantic defect characterization based on the one or more semantic regions, generating training defect images based on at least the training image and the semantic defect characterization, and performing defect detection based on the training image, the characterization of the training image, the training defect image, and the semantic defect characterization to train the defect detection neural network and adjust parameters of the defect detection neural network.

[0063] According to an embodiment of the present invention, a location subnetwork in an abnormal location network can be used to locate defects based on the reconstructed image and a feature representation of the reconstructed image. The structure of the location subnetwork in the abnormal location network is similar to the reconstruction subnetwork described above, but can further include a cross-layer connection operation to fuse feature maps of the same scale of the encoder and decoder in the location subnetwork.

[0064] In the defect detection process, the anomaly location network can detect both logical defects and structural defects in the target image. Optionally, after performing defect detection on the target image, the anomaly location network can output the specific location of the defect, or simultaneously output an estimated value of the degree of the defect, for reference.

[0065] According to an embodiment of the present invention, the defect detection neural network in the defect detection method is trained using the steps shown in Figure 3. The specific training process is described in detail in the steps shown in Figure 3, and will not be repeated here.

[0066] According to the defect detection method of an embodiment of the present invention, by generating semantic defect feature representations in the semantic domain of the feature representations of training images, sample data for training the defect detection neural network can be enriched, thereby realizing robust detection of semantic defects in different types of products and objects, avoiding the process of collecting and manually marking large amounts of defect data, greatly reducing production costs, and improving user experience.

[0067] The specific implementation process of the generative neural network training method and image generation method according to an embodiment of the present invention will be described below.

[0068] According to an exemplary embodiment of the present invention, training images are first input, and a feature display diagram for the training images is obtained based on the features of the training images. In this example, the obtained feature display diagram for the training images may be a contour diagram of the training images. FIG. 5 is a schematic diagram of training images according to an exemplary embodiment of the present invention. In FIG. 5, the input training images may be images having texture features. FIG. 6 is a feature display diagram for the training images according to an exemplary embodiment of the present invention. In FIG. 6, a feature detection network applicable to an embodiment of the present invention can be trained to perform feature extraction on the training images and output a feature display diagram corresponding to the training images. Specifically, by combining the training images with a pre-trained edge detection network having a relatively large number of parameters, a lightweight edge detection network having a relatively small number of parameters is trained, and the lightweight edge detection network is trained using the output of the pre-trained edge detection network having a relatively large number of parameters, thereby obtaining edge detection capabilities similar to those of the pre-trained edge detection network through an unsupervised training method. More specifically, for the same input training image, it is desirable to make the output of the lightweight edge detection network and the output of the pre-trained edge detection network as close as possible through training, and therefore the goal of the training process is to minimize the difference in output between the lightweight edge detection network and the pre-trained edge detection network. After training the lightweight edge detection network, the trained lightweight edge detection network can be used to perform edge detection on the input training image and output a contour map corresponding to the training image.

[0069] Then, a first enhancement process is performed on the feature display map of the training image to obtain a first enhanced training image. In one embodiment of the present invention, the first enhancement process on the feature display map of the training image can include one or more of image distortion, image flip, image rotation, image crop, edge adjustment, image translation, and image scaling. Specifically, when performing local distortion transformation on the feature display map of the training image, a local region is selected from the feature display map of the training image, divided into a grid, multiple nodes in the grid are randomly selected, and the selected nodes are randomly translated horizontally or vertically. Based on this, different curvature transformations, inversion, zero padding, and other processes are subsequently performed to obtain a first enhanced training image. Figure 7 shows a first enhanced training image obtained by performing a first enhancement process on a contour map of the training image according to one embodiment of the present invention.

[0070] Thereafter, a second enhancement process different from the first enhancement process may be performed on the training image to obtain a second enhanced training image.

[0071] The second enhancement process on the training image may also include one or more operations of image distortion, image flip, image rotation, image crop, edge adjustment, image translation, and image scaling, and the second enhancement process may be different from the first enhancement process. Optionally, a local region of the training image may be selected and subjected to a process different from the first enhancement process, such as warping, flipping, or zero padding, to obtain the second enhanced training image. Figure 8 illustrates a second enhanced training image obtained by performing the second enhancement process on the training image according to an example embodiment of the present invention.

[0072] Finally, based on at least the training image, the feature display diagram of the training image, the first enhanced training image, and the second enhanced training image, an image generation operation is performed using the generative neural network to obtain a training generated image, and the generative neural network can be trained to adjust parameters of the generative neural network.

[0073] Optionally, the generative neural network may first be used to generate an anomaly probability map showing intermediate training images and weights, and then the anomaly probability map may be used to fuse the training images with the intermediate training images to obtain the training generative images.

[0074] Specifically, during the fusion process in the generative neural network, intermediate training images and anomaly probability maps are output in the intermediate process of the generative neural network based on the input to the generative neural network, and the input training images and the generated intermediate training images are fused using the anomaly probability map as weights to obtain a final training image. Figure 9 is a diagram illustrating the acquisition of training images using intermediate training images and anomaly probability maps according to an embodiment of the present invention. In this example, the anomaly probability map is used as a weight, and the result of pointwise multiplication of the weight by the intermediate training images is added to the result of pointwise multiplication of the inverted weight by the training image to obtain a training image.

[0075] In an embodiment of the present invention, the training images generated by the generative neural network may be generated by combining a feature display map of the training images as feature information and a texture of the training images as texture information. When the training images are obtained, the generative neural network can be trained based on the training images and the parameters of the generative neural network can be adjusted to converge.

[0076] After training the generative neural network, the generative neural network can be selectively used to obtain a desired image generation result. For example, the generative neural network can be applied to fields such as clothing design, virtual try-on, etc. For example, a user can input a sketch showing the structure of the clothing as a feature map and a desired texture map to obtain a clothing image that combines the design sketch and a texture image.

[0077] In one example embodiment of the present invention, a feature representation for generating feature information of an image and a texture representation for generating texture information of the image can be obtained.

[0078] Optionally, the feature information may be contour information, and the feature representation for generating the feature information of the image may be a contour diagram representing the contour of the generated image.

[0079] Furthermore, optionally, the texture display diagram for generating texture information for an image can include texture information that is desired to appear in the generated image, so that the texture information contained in the texture display diagram appears in the generated image by the generation neural network combining the feature display diagrams.

[0080] The generated image can then be obtained based on the feature and texture representations using a generative neural network, the generative neural network being trained in the manner described above.

[0081] The generated image is generated using the trained neural network, and the feature information contained in the feature display map and the texture information contained in the texture display map appear in the generated image.

[0082] 10 is a schematic diagram of an image generation method according to an example embodiment of the present invention. In FIG. 10, the same texture representation (first column on the left side of FIG. 10) is used, but different contour representations (feature representations 1 and 2 in the second and fourth columns on the left side of FIG. 10) are used to generate different generated images (generated images 1 and 2 in the third and fifth columns on the left side of FIG. 10). As shown in FIG. 10, the generated images obtained by the generative neural network can obtain different results based on the same texture representation and different feature representations, and vice versa.

[0083] The specific implementation process of the defect detection neural network training method and defect detection method according to an embodiment of the present invention will be described below.

[0084] This example of an embodiment of the present invention is used in the context of detecting and analyzing objects in a breakfast lunch box. In this example, a training image is first input and a feature representation of the training image is obtained. In this example, the feature representation of the training image can be the contour of the training image. As shown in Figures 5 and 6, the training image shown in Figure 5 can be obtained and its contour can be obtained as a contour diagram of the training image shown in Figure 6.

[0085] One or more semantic regions may then be selected based on the characterization of the training images, and a semantic defect characterization may be generated based on the one or more semantic regions.

[0086] Selectably, one or more semantic regions of at least a portion of the characterization of the training images may be selected, a third enhancement process may be performed on the characterization of the training images in the selected one or more semantic regions, and the characterization of the one or more semantic regions after the third enhancement process may be merged with the characterization of the training images to generate the semantic defect characterization, where the third enhancement process may include one or more of adding, deleting, and modifying characterizations.

[0087] 11 is a schematic diagram of a semantic defect feature display according to an embodiment of the present invention. As shown in FIG. 11, the contours of the training image shown in FIG. 6 are extracted as part of the semantic region and then subjected to a deletion process. In FIG. 11, the semantic region of orange and some semantic regions of dried fruits are selected, and a deletion operation as a third enhancement process is performed, thereby obtaining the semantic defect feature display shown in FIG. 11.

[0088] A training defect image is then generated based on at least the training image and the defect characterization.

[0089] After obtaining a trained image generation network based on the aforementioned method, the trained image generation network can be used to generate corresponding training defect images by using the training images as texture information for the generated training defect images and the semantic defect characterization as feature information for the generated training defect images. Figure 12 is an illustrative diagram of a generated training defect image according to an example embodiment of the present invention. In Figure 12, the training defect image is generated based on the semantic defect characterization shown in Figure 11 as feature information and the training image shown in Figure 5 as texture information.

[0090] Finally, defect detection is performed based on the training images, the training image characterizations, the training defect images, and the semantic defect characterizations to train the neural network and adjust parameters of the neural network.

[0091] According to an embodiment of the present invention, a neural network is trained using the training images, the feature representations of the training images, the training defect images, and the semantic defect feature representation, and parameters of the neural network are adjusted to achieve as close convergence as possible to the loss function of the neural network.

[0092] Optionally, each of the input images can be used as a sample to train an anomaly location network for defect detection within the defect detection neural network and adjust parameters of the anomaly location network. In one example, the anomaly location network can include one reconstruction subnetwork and one location subnetwork. The reconstruction subnetwork is used to convert the input training defect images containing defects into reconstructed images and corresponding feature representations that do not contain defects, and the location subnetwork locates the defects by calculating the difference between the reconstructed images and corresponding feature representations converted by the reconstruction subnetwork and the original input image.

[0093] During the defect detection process, the anomaly location network can simultaneously detect not only logical defects but also structural defects in the training defect image. Optionally, after performing defect detection on the image, the trained anomaly location network can output the specific location of the defect, or can simultaneously output an estimate of the severity of the defect.

[0094] In another example of the embodiment of the present invention, after a neural network for defect detection has been trained, the neural network can be used to perform defect detection on an input image of a detection target. Specifically, the image of the detection target may be input, and the neural network for defect detection may be used to reconstruct the image of the detection target, and the reconstructed image may then be obtained, and a characteristic representation of the reconstructed image, such as the contour of the reconstructed image, may be obtained.

[0095] Subsequently, defect detection can be performed on the target image based on the reconstructed image and the characteristic display of the reconstructed image. The anomaly location network in the defect detection neural network can simultaneously detect not only logical defects but also structural defects in the target image during the defect detection process. Optionally, the anomaly location identification network can output the specific location of the defect or simultaneously output an estimated value for the severity of the defect after performing defect detection on the target image.

[0096] An apparatus 1300 for training a generative neural network according to an embodiment of the present invention will now be described with reference to Fig. 13. Fig. 13 is a block diagram of an apparatus 1300 for training a generative neural network according to an embodiment of the present invention. As shown in Fig. 13, the apparatus 1300 may be a computer or a server.

[0097] As shown in Figure 13, the device 1300 includes one or more processors 1310 and a memory 1320. Of course, the device 1300 may also include other input and output devices (not shown), which may be interconnected via a bus system and / or other type of connection mechanism. It should be noted that the components and structure of the device 1300 shown in Figure 13 are examples and not limiting, and the device 1300 may include other components and structures as needed.

[0098] Processor 1310 may be a central processing unit (CPU) or other type of processing device having data processing capabilities and / or command execution capabilities, and may use computer program commands stored in memory 1220 to perform desired functions, which may include: inputting training images and obtaining a feature display map of the training images based on features of the training images, performing a first enhancement process on the feature display map of the training images to obtain first enhanced training images, performing a second enhancement process different from the first enhancement process on the training images to obtain second enhanced training images, performing an image generation operation using the generative neural network based on at least the training images, the feature display map of the training images, the first enhanced training images, and the second enhanced training images to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network.

[0099] Memory 1320 may include one or more computer program products, which may include various types of computer-readable storage media, such as volatile and / or non-volatile memory. The computer-readable storage media may store one or more computer program commands, which processor 1310 may execute to perform the functions of the apparatus of the above-described embodiments of the present invention and / or other desired functions and / or to perform the generative neural network training method of the embodiments of the present invention. The computer-readable storage media may also store various application programs and various data.

[0100] The following describes a computer-readable storage medium storing computer program commands according to an embodiment of the present invention. When the computer program commands are executed by a processor, the computer program commands achieve the following steps: inputting a training image and obtaining a feature display diagram of the training image based on the features of the training image; performing a first enhancement process on the feature display diagram of the training image to obtain a first enhanced training image; performing a second enhancement process different from the first enhancement process on the training image to obtain a second enhanced training image; and performing an image generation operation using the generative neural network based on at least the training image, the feature display diagram of the training image, the first enhanced training image, and the second enhanced training image to obtain a training generated image, and training the generative neural network to adjust parameters of the generative neural network.

[0101] An apparatus for generating an image using a generative neural network according to an embodiment of the present invention will now be described with reference to Fig. 14. Fig. 14 is a block diagram of an image generating apparatus 1400 according to an embodiment of the present invention. As shown in Fig. 14, the apparatus 1400 may be a computer or a server.

[0102] As shown in Figure 14, the device 1400 includes one or more processors 1410 and a memory 1420. Of course, the device 1400 may also include other components, such as input and output devices (not shown), which may be interconnected via a bus system and / or other type of connection mechanism. It should be noted that the components and structure of the device 1400 shown in Figure 14 are exemplary and not limiting, and the device 1400 may include other components and structures as needed.

[0103] Processor 1410 may be a central processing unit (CPU) or other type of processing device having data processing and / or command execution capabilities, and may perform desired functions using computer program commands stored in memory 1420, including: obtaining a feature representation map for generating feature information of an image and a texture representation map for generating texture information of an image; and obtaining the generated images using a generative neural network based on the feature representation map and the texture representation map, wherein the generative neural network is trained using a method including the steps of: inputting training images and obtaining a feature representation map of the training images based on features of the training images; performing a first enhancement process on the feature representation map of the training images to obtain first enhanced training images; performing a second enhancement process on the training images, different from the first enhancement process, to obtain second enhanced training images; and performing an image generation operation using the generative neural network based on at least the training images, the feature representation map of the training images, the first enhanced training images, and the second enhanced training images to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network.

[0104] Memory 1420 may include one or more computer program products, which may include various types of computer-readable storage media, such as volatile and / or non-volatile memory. The computer-readable storage media may store one or more computer program commands, which processor 1410 may execute to perform image generation device functions and / or other desired functions of an apparatus according to an embodiment of the present invention and / or to perform an image generation method according to an embodiment of the present invention. The computer-readable storage media may also store various application programs and various data.

[0105] Hereinafter, based on a computer-readable storage medium storing computer program commands according to the present invention, when the computer program commands are executed by a processor, the following steps are realized: acquiring a feature display map for generating feature information of an image and a texture display map for generating texture information of an image; and acquiring the generated images using a generative neural network based on the feature display map and the texture display map. The generative neural network is trained using a method including the following steps: receiving training images and acquiring the feature display map for the training images based on features of the training images; performing a first enhancement process on the feature display map for the training images to obtain first enhanced training images; performing a second enhancement process on the training images that is different from the first enhancement process to obtain second enhanced training images; and performing an image generation operation using the generative neural network based on at least the training images, the feature display map for the training images, the first enhanced training images, and the second enhanced training images to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network.

[0106] An apparatus for training a defect detection neural network according to an embodiment of the present invention will now be described with reference to Fig. 15. Fig. 15 is a block diagram of an apparatus 1500 for training a defect detection neural network according to an embodiment of the present invention. As shown in Fig. 15, the apparatus 1500 may be a computer or a server.

[0107] As shown in Figure 15, apparatus 1500 includes one or more processors 1510 and memory 1520, although it should be understood that apparatus 1500 may also include other components such as input and output devices (not shown) that may be interconnected via a bus system and / or other types of connection mechanisms. It should be noted that the components and structure of apparatus 1500 for training a defect detection neural network shown in Figure 15 are exemplary and not limiting, and apparatus 1500 for training a defect detection neural network may include other components and structures as needed.

[0108] Processor 1510 may be a central processing unit (CPU) or other type of processing device having data processing and / or command execution capabilities, and may utilize computer program commands stored in memory 1520 to perform desired functions, which may include: inputting training images and obtaining characterizations of the training images; selecting one or more semantic regions based on the characterizations of the training images and generating semantic defect characterizations based on the one or more semantic regions; generating training defect images based on at least the training images and the semantic defect characterizations; and performing defect detection based on the training images, the characterizations of the training images, the training defect images, and the semantic defect characterizations to train the defect detection neural network and adjust parameters of the defect detection neural network.

[0109] Memory 1520 may include one or more computer program products, which may include various types of computer-readable storage media, such as volatile and / or non-volatile memory. The computer-readable storage media may store one or more computer program commands, which processor 1510 may execute to perform the functions of the defect detection neural network training apparatus and / or other desired functions of the defect detection neural network training method of the present invention, as described above. The computer-readable storage media may also store various application programs and various data.

[0110] Hereinafter, based on a computer-readable storage medium storing computer program commands according to the present invention, when the computer program commands are executed by a processor, the following steps are realized: inputting a training image and obtaining a characterization of the training image; selecting one or more semantic regions based on the characterization of the training image and generating a semantic defect characterization based on the one or more semantic regions; generating a training defect image based on at least the training image and the semantic defect characterization; and performing defect detection based on the training image, the characterization of the training image, the training defect image and the semantic defect characterization, thereby training the defect detection neural network and adjusting parameters of the defect detection neural network.

[0111] An apparatus for detecting defects using a defect detection neural network according to an embodiment of the present invention will now be described with reference to Fig. 16. Fig. 16 is a block diagram of an apparatus 1600 for detecting defects using a defect detection neural network according to an embodiment of the present invention. As shown in Fig. 16, the apparatus 1600 may be a computer or a server.

[0112] As shown in Figure 16, device 1600 includes one or more processors 1610 and memory 1620. Of course, device 1600 may also include other input and output devices (not shown), which may be interconnected via a bus system and / or other type of connection mechanism. It should be noted that the components and structure of device 1600 shown in Figure 16 are exemplary and not limiting, and device 1600 may include other components and structures as needed.

[0113] Processor 1610 may be a central processing unit (CPU) or other type of processing device having data processing and / or command execution capabilities and may utilize computer program commands stored in memory 1620 to perform desired functions, which may include: inputting a target image to be detected and utilizing a defect detection neural network to reconstruct the target image, acquiring the reconstructed image, and acquiring a characterization of the reconstructed image; and performing defect detection on the target image using the defect detection neural network based on the reconstructed image and the characterization of the reconstructed image, wherein the defect detection neural network is trained in a manner including inputting training images and acquiring a characterization of the training images, selecting one or more semantic regions based on the characterization of the training images and generating semantic defect characterizations based on the one or more semantic regions, generating training defect images based on at least the training images and the semantic defect characterizations, and performing defect detection based on the training images, the characterization of the training images, the training defect images, and the semantic defect characterizations to train the defect detection neural network and adjust parameters of the defect detection neural network.

[0114] Memory 1620 may include one or more computer program products, which may include various types of computer-readable storage media, such as volatile and / or non-volatile memory. The computer-readable storage media may store one or more computer program commands, which processor 1610 may execute to perform the functionality of an apparatus for performing defect detection using a neural network according to embodiments of the present invention and / or other desired functionality and / or to perform a method for performing defect detection using a neural network according to embodiments of the present invention. The computer-readable storage media may also store various application programs and various data.

[0115] Hereinafter, based on a computer-readable storage medium storing computer program commands according to the present invention, a computer program command is stored, and when the computer program command is executed by a processor, the computer program command realizes the steps of: inputting a target image to be detected, reconstructing the target image using a defect detection neural network, acquiring the reconstructed image, and acquiring a characterization of the reconstructed image, and performing defect detection on the target image using the defect detection neural network based on the reconstructed image and the characterization of the reconstructed image. The defect detection neural network is trained by a method including the steps of inputting a training image and acquiring a characterization of the training image, selecting one or more semantic regions based on the characterization of the training image and generating a semantic defect characterization based on the one or more semantic regions, generating training defect images based on at least the training images and the semantic defect characterization, and performing defect detection based on the training images, the characterization of the training images, the training defect images, and the semantic defect characterization to train the defect detection neural network and adjust parameters of the defect detection neural network.

[0116] Of course, the above specific embodiments are merely examples and are not limiting. Furthermore, a person skilled in the art may achieve the effects of the present invention by combining or integrating multiple steps or devices from each of the above-described embodiments based on the concept of the present invention, and such combined or integrated embodiments are also included in the present invention, and such combined or integrated embodiments will not be described one by one herein.

[0117] The advantages, benefits, effects, etc. mentioned in this specification are merely examples and are not limiting, and these advantages, benefits, effects, etc. should not be considered essential for each embodiment of the present invention. In addition, the specific details of the above invention are merely for illustrative purposes and for facilitating understanding, and are not limiting, and the above details do not limit the invention to be realized using the specific details.

[0118] Block diagrams of devices, apparatus, facilities, and systems referred to in the present invention are merely examples and do not require or imply that they must be connected, arranged, or configured as shown by the block diagrams. As will be understood by those skilled in the art, these devices, apparatus, facilities, and systems can be connected, arranged, or configured in any manner. Terms such as "include," "includes," and "having" are open-ended terms and mean "including but not limited to" and can be used interchangeably. As used herein, the terms "or" and "and" mean "and / or" and can be used interchangeably unless the context clearly indicates otherwise. As used herein, the term "such as" means "such as but not limited to" and can be used interchangeably.

[0119] The flow diagrams of the present invention and the above method descriptions are merely examples and do not require or imply that the steps of each example must be performed in the order presented. As one of ordinary skill in the art would understand, the order of steps in the above examples can be performed in any order. Terms such as "then," "then," and "next" are not intended to limit the order of steps; these terms are used solely to guide the reader in reading through the method descriptions. Furthermore, any reference to an element in the singular, for example, using the articles "a," "one," or "the," should not be construed as limiting the element to the singular.

[0120] Furthermore, the steps and devices in each embodiment of this specification are not limited to being implemented in a particular embodiment, and in fact, new embodiments can be conceived by combining some of the related steps and some of the devices in each embodiment of this specification based on the concept of the present invention, and these new embodiments are also included within the scope of the present invention.

[0121] The various operations of the methods described above may be performed by any suitable means capable of performing the corresponding functions, which may include various hardware and / or software components and / or modules, including, but not limited to, circuits, application specific integrated circuits (ASICs), or processors.

[0122] The various exemplary logic blocks, modules, and circuits described may be implemented or performed using general-purpose processors, digital signal processors (DSPs), ASICs, field-configurable logic arrays (FPGAs) or other programmable logic devices (PLDs), discrete gate or transistor logic, discrete hardware components, or any combination thereof, designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but alternatively, the processor may be any commercially available processor, controller, microcontroller, or state machine. A processor may also be a combination of computing devices, such as, for example, a combination of a DSP and a microprocessor, and may be implemented as multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0123] The steps of a method or algorithm described in connection with the present invention may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module may reside in any form of tangible storage medium. Some examples of storage media that may be used include random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, hard disk, removable disk, CD-ROM, etc. A storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium. Alternatively, the storage medium and the processor may be integral. A software module may be a single command, or many commands, and may be distributed among several different code segments, among different programs, and across multiple storage media.

[0124] The methods of the present invention include one or more actions for achieving the method. The methods and / or actions may be interchanged without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions may be modified without departing from the scope of the claims.

[0125] The functions described above may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more commands on a tangible computer-readable storage medium. The storage medium may be any tangible medium accessible by a computer. By way of example and not limitation, such computer-readable storage media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or other tangible media that can be used to carry or store program code in the form of computer-accessible commands or data structures. As used herein, a disc may include a compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, or Blu-ray disc.

[0126] Thus, a computer program product can perform the operations set forth in the specification. For example, such a computer program product may be a computer-readable storage medium having instructions tangibly stored (and / or encoded) thereon, the instructions being executable by one or more processors to perform the operations described herein. The computer program product may include packaging materials.

[0127] The software or commands may be transmitted over a transmission medium. For example, the software may be transmitted from a website, server, or other remote source using a transmission medium such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless techniques such as infrared, radio, or microwave.

[0128] Additionally, modules and / or other suitable means for performing the methods and techniques described herein may be downloaded and / or obtained by other means by a user terminal and / or base station, as appropriate. For example, such devices may be connected to a server to facilitate the transmission of means for performing the methods described herein. Alternatively, the various methods described herein may be provided via storage means (e.g., RAM, ROM, physical storage media such as CDs, floppy disks, etc.) such that the various methods can be obtained when a user terminal and / or base station is connected to the device or when the storage means is provided to the device. Any other suitable technique for providing the methods and techniques described herein to a device may also be used.

[0129] Other examples and implementations are within the scope and spirit of the present invention and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Features implementing the functions may also be physically located in various locations, including being distributed such that portions of the functions are implemented in different physical locations. Also, as used herein, including in the claims, "or" used in a list of items beginning with "at least one" indicates a separate list; for example, a list of "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the term "exemplary" does not imply that the described example is preferred or superior to other examples.

[0130] Various changes, substitutions, and alterations can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims is not limited to the specific aspects of the elements, means, methods, and acts of a process, machine, manufacture, or event described above. Any currently existing or later-developed elements, means, methods, or acts of a process, machine, manufacture, or event can be utilized to perform substantially the same function or achieve substantially the same result as those described herein in corresponding aspects. Accordingly, the appended claims include within their scope such elements, means, methods, or acts of a process, machine, manufacture, or event.

[0131] The above description of aspects of the present invention is provided to enable any person skilled in the art to make or use the present invention. Various modifications to these aspects will be apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present invention. Thus, the present invention is not intended to be limited to the aspects described herein but is to be accorded the widest scope consistent with the principles and novel features of the present invention as described herein.

[0132] The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of the invention to the precise form in which they are presented. While various exemplary aspects and embodiments have been described above, those skilled in the art will recognize certain variations, modifications, variations, additions, and combinations thereof.

Claims

1. 1. A computer-implemented method for training a generative neural network, comprising: inputting a training image and obtaining a feature display diagram of the training image based on the features of the training image; performing a first enhancement process on the feature display diagram of the training image to obtain a first enhanced training image; performing a second enhancement process different from the first enhancement process on the training image to obtain a second enhanced training image; and performing an image generation operation using the generative neural network based on at least the training images, the feature display diagram of the training images, the first enhanced training images, and the second enhanced training images to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network. method.

2. obtaining a feature display diagram of the training image based on the features of the training image, extracting contours from the training images and generating a contour map of the training images as a feature display map of the training images; The method of claim 1.

3. The first enhancement process and / or the second enhancement process including one or more of the following operations: image distortion, image flip, image rotation, image crop, edge adjustment, image translation, and image scaling; The method of claim 1.

4. a step of performing an image generation operation using the generative neural network to obtain training generated images based on at least the training images, the feature display diagram of the training images, the first enhanced training images, and the second enhanced training images, generating an anomaly probability map showing intermediate training images and weights using the generative neural network; and fusing the training images and the intermediate training images utilizing the anomaly probability map to obtain the training generated images. The method of claim 1.

5. 1. A computer-implemented method for training a defect detection neural network, comprising: inputting training images and obtaining feature representations of the training images; selecting one or more semantic regions based on the characterization of the training images, and generating a semantic defect characterization based on the one or more semantic regions; generating training defect images based on at least the training images and the semantic defect characterization; performing defect detection based on the training images, the characterization of the training images, the training defect images, and the semantic defect characterization to train the defect detection neural network and adjust parameters of the defect detection neural network. method.

6. generating training defect images based on at least the training images and the semantic defect characterization; generating the training images based on the training images and the semantic defect characterization utilizing a generative neural network; wherein the generating neural network is inputting a training image and obtaining a feature display diagram of the training image based on the features of the training image; performing a first enhancement process on the feature display diagram of the training image to obtain a first enhanced training image; performing a second enhancement process different from the first enhancement process on the training image to obtain a second enhanced training image; performing an image generation operation using the generative neural network based on at least the training image, the feature display map of the training image, the first enhanced training image, and the second enhanced training image to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network; are trained in a manner that includes The method of claim 5.

7. selecting one or more semantic regions based on the characterization of the training images; and generating a semantic defect characterization based on the one or more semantic regions; selecting one or more semantic regions in at least a portion of the feature representations of the training images; performing a third enhancement process on the feature representations of the training images in the one or more selected semantic regions; and fusing the characterization of the one or more semantic regions after the third enhancement process with the characterization of the training images to generate the semantic defect characterization. The method of claim 5.

8. The third emphasis processing including one or more of adding, deleting, and modifying the feature representation; The method of claim 7.

9. performing defect detection based on the training images, the training image characterizations, the training defect images, and the semantic defect characterizations to train the defect detection neural network and adjust parameters of the defect detection neural network; training the defect detection neural network and tuning parameters of the defect detection neural network by converting the training defect images and the semantic defect feature representations into the training images and feature representations of the training images, respectively; The method of claim 5.

10. 1. A computer-implemented method for generating an image, comprising: obtaining a feature representation diagram for generating feature information of the image and a texture representation diagram for generating texture information of the image; and obtaining a generated image based on the feature representation map and the texture representation map using a generative neural network; wherein the generating neural network is inputting a training image and obtaining a feature display diagram of the training image based on the features of the training image; performing a first enhancement process on the feature display diagram of the training image to obtain a first enhanced training image; performing a second enhancement process different from the first enhancement process on the training image to obtain a second enhanced training image; performing an image generation operation using the generative neural network based on at least the training image, the feature display map of the training image, the first enhanced training image, and the second enhanced training image to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network; are trained in a manner that includes Image generation method.

11. 1. A computer-implemented method for detecting defects, comprising: inputting a detection target image, reconstructing the detection target image using a defect detection neural network, obtaining a reconstructed image, and obtaining a feature representation of the reconstructed image; and performing defect detection on the detection target image using the defect detection neural network based on the reconstructed image and a characteristic display of the reconstructed image, wherein the defect detection neural network: inputting training images and obtaining feature representations of the training images; selecting one or more semantic regions based on the characterization of the training images, and generating a semantic defect characterization based on the one or more semantic regions; generating training defect images based on at least the training images and the semantic defect characterization; performing defect detection based on the training images, the training image characterizations, the training defect images, and the semantic defect characterizations to train the defect detection neural network and tune parameters of the defect detection neural network; are trained in a manner that includes Defect detection methods.

12. 1. A generative neural network training apparatus, comprising: a processor; a memory having computer program commands stored therein; The computer program instructions, when executed by the processor, cause the processor to: inputting a training image and obtaining a feature display diagram of the training image based on the features of the training image; performing a first enhancement process on the feature display diagram of the training image to obtain a first enhanced training image; performing a second enhancement process different from the first enhancement process on the training image to obtain a second enhanced training image; performing an image generation operation using a generative neural network based on at least the training images, the feature display map of the training images, the first enhanced training images, and the second enhanced training images to obtain training generated images, and training the generative neural network to adjust parameters of the generative neural network; Generative neural network training device.

13. 1. A defect detection neural network training apparatus, comprising: a processor; a memory having computer program commands stored therein; The computer program instructions, when executed by the processor, cause the processor to: inputting training images and obtaining feature representations of the training images; selecting one or more semantic regions based on the characterization of the training images, and generating a semantic defect characterization based on the one or more semantic regions; generating training defect images based on at least the training images and the semantic defect characterization; performing defect detection based on the training images, the training image characterizations, the training defect images, and the semantic defect characterizations to train a defect detection neural network and tune parameters of the defect detection neural network. Defect detection neural network training device.

14. A program for causing a computer to execute the method according to claim 1, 5, 10 or 11.

15. A computer-readable storage medium storing the program according to claim 14.

Images