Neural network training method, and defect detection method, device and storage medium

Abstract

Embodiments of the present application provide a neural network training method, and a method, device and computer readable storage medium for defect detection using a neural network. The neural network training method according to an embodiment of the present application comprises: inputting a training image and calculating a perceptible map of the training image; generating a training defect image according to at least the training image and the perceptible map of the training image, the training defect image comprising one or more defects generated based on the perceptible map of the training image; and training the neural network according to at least the generated training defect image, obtaining a defect detection result and adjusting parameters of the neural network.

Description

Technical Field

[0001] This invention relates to the field of image processing, and more particularly to a neural network training method, as well as a method, apparatus, and computer-readable storage medium for defect detection using a neural network. Background Technology

[0002] Defect detection is a crucial factor in product quality control. Since various defects are constantly generated during production, it is usually impossible to exhaustively list all possible causes. Therefore, how to robustly and automatically detect these defects is a pressing issue that needs to be addressed.

[0003] Current defect detection methods typically use a large amount of manually labeled data to train the neural network used for defect detection. However, since defects in defect images are often small regions or minor defects found in normal images, it is difficult to obtain accurate detection results using current defect detection neural networks. Therefore, this training method for neural networks and the defect detection methods using such networks cannot achieve accurate generation and robust detection of disordered defects. Summary of the Invention

[0004] To address the aforementioned technical problems, according to one aspect of the present invention, a neural network training method is provided, comprising: inputting a training image and calculating an observability map of the training image; generating a training defect image based at least on the training image and the observability map of the training image, the training defect image including one or more defects generated based on the observability map of the training image; training the neural network based at least on the generated training defect image to obtain defect detection results and adjusting the parameters of the neural network.

[0005] According to another aspect of the present invention, a method for defect detection using a neural network is provided, comprising: inputting an image to be detected; reconstructing the image to be detected through a first sub-network in the neural network to remove defects, thereby obtaining a reconstructed image and a perceptible map of the reconstructed image; and performing defect segmentation on the reconstructed image through a second sub-network based at least on the reconstructed image and the perceptible map of the reconstructed image, thereby obtaining a defect detection result.

[0006] According to another aspect of the present invention, a neural network training apparatus is provided, comprising: an input unit configured to input a training image and calculate an observability map of the training image; a generation unit configured to generate a training defect image based at least on the training image and the observability map of the training image, the training defect image including one or more defects generated by simulation; and a training unit configured to train the neural network based at least on the generated training defect image to obtain defect detection results and adjust the parameters of the neural network.

[0007] According to another aspect of the present invention, a neural network training apparatus is provided, comprising: a processor; and a memory storing computer program instructions, wherein, when the computer program instructions are executed by the processor, the processor performs the following steps: inputting a training image and calculating an observability map of the training image; generating a training defect image, at least based on the training image and the observability map of the training image, the training defect image including one or more defects generated by simulation; training the neural network at least based on the generated training defect image to obtain defect detection results and adjusting the parameters of the neural network.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, perform the following steps: inputting a training image and calculating an observability map of the training image; generating a training defect image based at least on the training image and the observability map of the training image, the training defect image including one or more defects generated by simulation; training the neural network based at least on the generated training defect image to obtain defect detection results and adjusting the parameters of the neural network.

[0009] According to another aspect of the present invention, an apparatus for defect detection using a neural network is provided, comprising: an input unit configured to input an image to be detected, reconstruct the image to be detected through a first sub-network in the neural network to remove defects, thereby obtaining a reconstructed image and a perceptible map of the reconstructed image; and a detection unit configured to perform defect segmentation on the reconstructed image through a second sub-network based at least on the reconstructed image and the perceptible map of the reconstructed image, thereby obtaining a defect detection result.

[0010] According to another aspect of the present invention, an apparatus for defect detection using a neural network is provided, comprising: a processor; and a memory storing computer program instructions, wherein, when the computer program instructions are executed by the processor, the processor performs the following steps: inputting an image to be detected; reconstructing the image to be detected through a first sub-network in the neural network to remove defects, obtaining a reconstructed image and a perceptible map of the reconstructed image; and performing defect segmentation on the reconstructed image through a second sub-network, at least based on the reconstructed image and the perceptible map of the reconstructed image, to obtain a defect detection result.

[0011] According to another aspect of the present invention, a computer-readable storage medium is provided having computer program instructions stored thereon, wherein the computer program instructions, when executed by a processor, perform the following steps: inputting an image to be detected; reconstructing the image to be detected through a first sub-network in a neural network to remove defects, thereby obtaining a reconstructed image and a perceptible map of the reconstructed image; and performing defect segmentation on the reconstructed image through a second sub-network, at least based on the reconstructed image and the perceptible map of the reconstructed image, thereby obtaining a defect detection result.

[0012] According to the above-described neural network training method, apparatus, and computer-readable storage medium of the present invention, as well as the method, apparatus, and computer-readable storage medium for defect detection using a neural network, defects in training images can be generated by introducing perceptual maps to train a neural network for defect detection. Furthermore, the accuracy of defect detection can be improved by further introducing perceptual maps of the training reconstructed images obtained by the defect-generated reconstruction network. This achieves accurate generation and robust detection of disordered defects, avoids the collection of large amounts of defect data and the manual annotation process, and improves the user experience. Attached Figure Description

[0013] The above and other objects, features, and advantages of the present invention will become clearer from the detailed description of the embodiments of the present invention in conjunction with the accompanying drawings.

[0014] Figure 1 A flowchart illustrating a neural network training method according to an embodiment of the present invention is shown;

[0015] Figure 2 A flowchart illustrating a method for defect detection using a neural network according to an embodiment of the present invention is shown.

[0016] Figure 3 A schematic diagram of the structure of a neural network for training is shown in an example according to an embodiment of the present invention;

[0017] Figure 4 (a)-(l) show schematic diagrams of the generation process of training defect images and defect detection results according to an embodiment of the present invention;

[0018] Figure 5 This illustrates a defect detection process for an image to be detected according to an embodiment of the present invention;

[0019] Figure 6 A block diagram of a neural network training apparatus according to an embodiment of the present invention is shown;

[0020] Figure 7 A block diagram of a neural network training apparatus according to an embodiment of the present invention is shown;

[0021] Figure 8 A block diagram of an apparatus for defect detection using a neural network according to an embodiment of the present invention is shown;

[0022] Figure 9 A block diagram of an apparatus for defect detection using a neural network according to an embodiment of the present invention is shown. Detailed Implementation

[0023] The following description, with reference to the accompanying drawings, outlines a neural network training method, apparatus, and computer-readable storage medium according to embodiments of the present invention, as well as a method, apparatus, and computer-readable storage medium for defect detection using a neural network. Throughout the drawings, the same reference numerals denote the same elements. It should be understood that the embodiments described herein are merely illustrative and should not be construed as limiting the scope of the invention.

[0024] When training a neural network for defect detection, it is often necessary to introduce a large number of training images containing various defects as samples to adjust the network parameters. However, in actual training, the collection of samples and the training effect of the neural network are often difficult to guarantee. To address the above problems, this invention introduces a perceptible map obtained through calculation as a factor in the generation of defects in training images and subsequent defect detection.

[0025] Figure 1 A flowchart of a neural network training method 100 according to an embodiment of the present invention is shown. Referring below... Figure 1 A neural network training method according to an embodiment of the present invention is described.

[0026] In step S101, a training image is input, and the perceptibility map of the training image is calculated.

[0027] In this step, the training images used to train the neural network can be images of products or objects without defects. To enrich the training samples as much as possible, the training images can be images of the same product or object taken from different angles, under different lighting conditions, and against different backgrounds, in order to further improve the defect detection accuracy of the trained neural network.

[0028] After inputting training images, a noticeable map of the training images can be calculated. This noticeable map can be a multi-scale noticeable map. The multi-scale noticeable map reflects the variations in the strength of various image features within the training image, that is, it reflects the differences in the visual saliency of each image feature. The multi-scale noticeable map of the training images makes the generated training defect images more natural and realistic, and also makes the subsequent defect detection and localization processes more accurate.

[0029] In one example, a multi-scale just-noticeable map of the training image can be calculated using the input training image. For instance, the just-noticeable map of the training image can be calculated based on a just-noticeable distortion (JND) model. The JND value is an important indicator of visual saliency, describing the degree to which humans perceive image quality. The JND value reveals the perceptual threshold of image intensity changes that the human visual system can notice. Any perceptible level of distortion must be greater than the JND value. In this embodiment of the invention, a defect training image sample set with diversity and multiple scales can be generated using the JND model and used in subsequent defect detection and localization processes.

[0030] Optionally, the process of calculating the multi-scale minimum perceptual map of the training image based on the minimum perceptual error model may include: firstly, calculating the minimum perceptual map of the training image based on the minimum perceptual error model; then, processing the minimum perceptual map of the training image using multiple coefficients (e.g., multiplying multiple coefficients by the minimum perceptual map of the training image respectively) to obtain the multi-scale minimum perceptual map of the training image. For example, three coefficients (such as 0.33, 0.66, and 1) can be selected and multiplied by the minimum perceptual map of the training image respectively to obtain the minimum perceptual maps of the training image at three different scales; or, for another example, four coefficients (such as 0.25, 1, 0.75, and 1) can be selected and multiplied by the minimum perceptual map of the training image respectively to obtain the minimum perceptual maps of the training image at four different scales. The above methods for generating the multi-scale minimum perceptual map of the training image are merely examples. In the embodiments of the present invention, various methods can be used to generate the multi-scale minimum perceptual maps of training images at various scales according to the needs of different application scenarios. As mentioned above, the multi-scale minimum perceptible map of the training image can reflect the range and intensity of various image features in the training image. Based on human perception, the distribution and changes of gray values ​​in the map reflect the differences in the visual saliency of various image features in the training image.

[0031] In step S102, a training defect image is generated based at least on the training image and the perceptual map of the training image, the training defect image including one or more defects generated based on the perceptual map of the training image.

[0032] In this step, the process of generating a training defect image may first include generating a defect mask image and generating one or more defect texture images based on the training image; then, the training image and the one or more defect texture images may be fused based on the defect mask image and using the multi-scale perceptual map of the training image to generate a training defect image.

[0033] Optionally, the defect mask image can be generated by selecting one or more defect regions of varying sizes and shapes from the training image. Furthermore, the defect texture image can optionally be randomly generated from any image, or it can be generated by processing the training image. In one example, to enhance the realism of the simulated defect, the result of enhancing the training image can be used to generate a defect texture image. For example, various random processing methods, including region cropping, color transformation, brightness variation, and exposure variation, can be applied to the training image to obtain defect texture images with various textures.

[0034] After obtaining the defect mask image and the defect texture image, optionally, based on the defect regions delineated by the defect mask image, the multi-scale perceptual map of the training image can be used as a weight map to fuse the training image and one or more defect texture images to generate a training defect image with various simulated defects. According to the principle of perceptual maps, the perceptual map of the training image can reveal regions with high color contrast and saturation in the training image. Regions with larger grayscale values ​​in this perceptual map can occupy more defect components in the training defect image, meaning they can be assigned more weight to the defect texture image. Conversely, regions with smaller grayscale values ​​in the perceptual map of the training image can be assigned relatively more weight to the input training image. Based on this, the application of the multi-scale perceptual map of the training image can obtain fused training defect images at various scales, making the obtained training defect images more realistic and further usable in subsequent defect detection processes.

[0035] Optionally, when a multi-scale perceptual map of the training image is generated, a corresponding multi-scale defect mask image can be generated to fuse the training image and the defect texture image, ultimately obtaining a training defect image. Specifically, the multi-scale defect mask image can be a defect mask image with defect regions of different sizes marked, corresponding to each scale in the multi-scale perceptual map of the training image. For example, referring to the aforementioned process of generating the multi-scale minimum perceptual map of the training image, the initially generated defect mask image can also be processed using multiple coefficients (e.g., multiplying multiple coefficients with the defect mask image respectively) to obtain the corresponding multi-scale defect mask image, thereby defining the range of different defect regions at different scales. In the subsequent defect generation process, during the fusion of the training image and the defect texture image using these multi-scale defect mask images, corresponding image processing can be performed on the defect regions marked by the multi-scale defect mask images. For example, Gaussian blur or similar methods can be used to process each defect region at multiple scales during the fusion process, thereby obtaining a more realistic and natural training defect image.

[0036] The above methods for generating training defect images are merely examples. In practical applications, various methods can be used to generate training defect images according to the corresponding scenario, and no restrictions are imposed here.

[0037] In step S103, the neural network is trained at least based on the generated training defect images to obtain defect detection results and adjust the parameters of the neural network.

[0038] Optionally, the training defect image can be reconstructed using a first sub-network, which serves as a reconstruction network in the neural network, to remove defects and obtain a training reconstructed image. Subsequently, based at least on the training reconstructed image, defect detection can be performed on the training reconstructed image using a second sub-network, which serves as a segmentation network in the neural network, to obtain a defect detection result. In one example, while generating the training reconstructed image using the first sub-network, an aperceptual map of the training reconstructed image (e.g., a multi-scale aperceptual map of the training reconstructed image) can also be obtained using the first sub-network. Based on this, defect detection can be performed on the training reconstructed image using the second sub-network, based on the training reconstructed image and its (multi-scale) aperceptual map.

[0039] Optionally, the first sub-network of the reconstructed network can consist of an encoder and a decoder. The encoder extracts features from the image input to the first sub-network. This input image can include the original training image or previously generated images with training defects, all of which can be used as training samples. In one example, the encoder can utilize convolution, normalization, and pooling processes with different parameters to extract features at different levels from the input image: for instance, firstly, the image input to the first sub-network can be convolved using convolution kernels to obtain a convolutional map; then, linear correction and batch normalization methods can be used to normalize the obtained convolutional map to obtain a normalized convolutional map; finally, max or average pooling can be applied to the normalized convolutional map. To obtain rich feature maps of various sizes, the above process can be repeated multiple times by adjusting relevant parameters to extract feature maps through multiple downsampling processes. After the encoder, the decoder can further process the feature maps through convolution, normalization, and upsampling to restore the resolution of the previously encoder-processed feature maps. During the process of passing through the first sub-network, which serves as the reconstruction network, the parameters of the first sub-network can be learned and updated under the supervision of the training image and its (multi-scale) perceptual map during the training phase of the neural network. This transforms the input training defect image into the corresponding defect-removed training reconstruction image. Furthermore, the (multi-scale) perceptual map of the training reconstruction image can be generated simultaneously through this first sub-network for use in subsequent defect detection and localization processes.

[0040] Optionally, after obtaining the training reconstructed image and the (multi-scale) perceptual map of the training reconstructed image through the first sub-network of the neural network, various training defect images, the generated training reconstructed image and the (multi-scale) perceptual map of the training reconstructed image can be input into the second sub-network of the neural network. The above images are combined and features are extracted to perform multi-scale defect region segmentation based on the extracted features, thereby outputting the multi-scale defect region segmentation result as the defect detection result.

[0041] In one example, during the training phase of the neural network, various training defect images, generated training reconstructed images, and (multi-scale) perceptual maps of the training reconstructed images can be concatenated and input into the second sub-network of the neural network. More specifically, when passing through the second neural network, it concatenates all the input images and outputs a new defect representation image, which is compared with the defect localization image obtained as the ground truth through the training images and simulated training defect images, and used to train the second sub-network. Optionally, the second sub-network can consist of convolutional layers, pooling layers, and softmax layers, and can have an encoder and decoder structure similar to the first sub-network, and can further have a cross-layer connection structure to fuse feature maps of the same scale in the encoder and decoder. The second sub-network can be used to output two different defect detection results: defect-free images and defective images, or it can obtain the localization of multi-scale defect regions through segmentation.

[0042] Optionally, when locating defects, the pixel values ​​in the obtained defect representation image can be segmented and categorized to more intuitively show the location of the defect. For example, when outputting the defect representation image, the pixels in the grayscale image can be divided into three parts according to their grayscale values, such as normalized results of 0, 0.5, and 1, to more clearly show the location of the defect. Of course, the above process of segmenting and categorizing defects is only an example. In practical applications, various pixel value classification and normalization operations can be performed on the defect representation image as needed, without limitation.

[0043] The neural network training method described above according to embodiments of the present invention can generate defects in training images by introducing perceptual maps to train a neural network for defect detection. Furthermore, the accuracy of defect detection can be improved by further introducing perceptual maps of the reconstructed training images obtained from the defect-generated reconstruction network. This achieves accurate generation and robust detection of disordered defects, avoiding the need for collecting large amounts of defect data and manual annotation, thus improving the user experience. The neural network training method described above according to embodiments of the present invention is applicable to various object detection tasks and computer vision tasks involving surface defect detection of objects such as road surfaces and chip surfaces.

[0044] Furthermore, according to another embodiment of the present invention, neural network training can be performed not only using the specific operational procedures of the above-described neural network training method, but also by leveraging the architecture and procedures of the neural network to meet other requirements in various application scenarios. For example, the architecture of the above-described neural network can be used to create diverse samples, such as human faces and animated characters; or it can be applied to other corresponding scenarios that require the introduction of image defects or noise to introduce various relatively realistic simulated defects or noise into the image. The method described above according to embodiments of the present invention can combine training images and introduce various noises to obtain diverse samples suitable for a wide variety of application scenarios.

[0045] Figure 2 A flowchart of a method 200 for defect detection using a neural network according to an embodiment of the present invention is shown. In this embodiment, a method for defect detection using a neural network can be employed. Figure 1 The neural network trained according to the process shown is used for defect detection. See below for reference. Figure 2 A method for defect detection using a neural network according to an embodiment of the present invention is described.

[0046] In step S201, an image to be detected is input, and the image to be detected is reconstructed through the first sub-network in the neural network to remove defects, thereby obtaining a reconstructed image and a perceptible map of the reconstructed image.

[0047] In this step, the image to be detected can be input through... Figure 1 The process shown involves reconstructing the first subnetwork in the trained neural network. Optionally, the image to be detected can be a defect-free image or an image with one or more defects. During the detection process on the image to be detected, an observable map of the reconstructed image with defects removed can be obtained simultaneously, or a multi-scale observable map of the reconstructed image can be further obtained for use in subsequent defect detection and localization processes.

[0048] The first sub-network of the reconstructed network can consist of an encoder and a decoder. The encoder extracts features from the input image to this first sub-network. In one example, the encoder can utilize convolution, normalization, and pooling processes with different parameters to extract features at different levels from the input image. For instance, firstly, the input image can be convolved with a convolution kernel to obtain a convolutional map; then, linear correction and batch normalization can be used to normalize the obtained convolutional map to obtain a normalized convolutional map; finally, max or average pooling can be applied to the normalized convolutional map. To obtain rich feature maps of various sizes, the above process can be repeated multiple times by adjusting relevant parameters to extract feature maps from the image through multiple downsampling processes. After the encoder, the decoder can further process the feature maps through convolution, normalization, and upsampling to restore the resolution of the previously encoder-processed feature maps.

[0049] In step S202, at least based on the reconstructed image and the perceptible map of the reconstructed image, the reconstructed image is segmented for defects through a second sub-network to obtain a defect detection result.

[0050] Optionally, after obtaining the reconstructed image and the (multi-scale) perceptual map of the reconstructed image through the first sub-network of the neural network, the generated reconstructed image and the (multi-scale) perceptual map of the reconstructed image can be input into the second sub-network of the neural network. The above images are combined and features are extracted to perform multi-scale defect region segmentation based on the extracted features, thereby outputting the multi-scale defect region segmentation result as the defect detection result.

[0051] In one example, various images to be detected, the generated reconstructed images, and (multi-scale) perceptual maps of the reconstructed images can be concatenated and input into a second sub-network of the neural network. More specifically, as the images pass through the second neural network, it concatenates all the input images and outputs the result as a defect detection. Optionally, the second sub-network can consist of convolutional layers, pooling layers, and softmax layers, and can have an encoder and decoder structure similar to the first sub-network. It can further have cross-layer connections to fuse feature maps of the same scale from the encoder and decoder. The second sub-network can be used to output two different defect detection results for either a defect-free image or a defective image, or it can locate multi-scale defect regions in the image to be detected as a defective image through segmentation.

[0052] Optionally, during defect localization, the pixel values ​​in the defect detection image obtained for the image to be detected can be segmented and categorized to more intuitively represent the location of the defect. For example, when outputting the defect detection image of the image to be detected, the pixels in the grayscale image of the defect detection image can be divided into three parts according to their grayscale values, such as represented by normalized results of 0, 0.5, and 1, respectively, to more clearly show the location of the defect. Of course, the above process of segmenting and categorizing defects is only an example. In practical applications, various pixel value classification and normalization operations can be performed on the defect detection image as needed, without limitation.

[0053] The method for defect detection using a neural network according to embodiments of the present invention can generate defects in training images by introducing perceptual maps to train the neural network for defect detection. Furthermore, the accuracy of defect detection can be improved by introducing perceptual maps of the reconstructed training images obtained from the defect-generated reconstruction network. This achieves accurate generation and robust detection of disordered defects, avoiding the collection of large amounts of defect data and manual annotation, thus improving the user experience. The neural network training method described in the embodiments of the present invention is applicable to various object detection tasks and computer vision tasks involving surface defect detection of objects such as road surfaces and chip surfaces.

[0054] The following describes an example of a neural network training method and a defect detection method using the neural network according to embodiments of the present invention.

[0055] Figure 3 A schematic diagram of the structure of a neural network used for training is shown in one example according to an embodiment of the present invention. Figure 3 As shown, firstly, a training image is input, and then the multi-scale minimum perceptible map of the training image is calculated using a multi-scale JND model. Figure 3 As shown, the training images can be images of products or objects without defects. To enrich the training samples as much as possible, the training images can be collected from the same product or object at different angles, under different lighting conditions, and against different backgrounds, in order to further improve the defect detection accuracy of the trained neural network. After inputting the training images, the multi-scale minimum perceptible map of the training images can be calculated using a multi-scale JND model.

[0056] Subsequently, a training defect image is generated based at least on the training image and the multi-scale minimum perceptible map of the training image, the training defect image including one or more defects generated based on the multi-scale minimum perceptible map of the training image.

[0057] Figure 4A schematic diagram illustrating the generation process of training defect images and the defect detection results according to an embodiment of the present invention is shown. Figure 4 middle, Figure 4 (a) represents a training image without defects. Figure 4 (b) represents one of the multi-scale minimum perceptible maps of the training image obtained by calculating the training image through the multi-scale JND model. Figure 4 (c) represents performing various random processing on the training image, including region cropping, color transformation, brightness variation, and exposure variation, to obtain defect texture images with various textures. Furthermore, Figure 4 (d)-(f) represent multi-scale defect mask images generated by selecting one or more defect regions of varying sizes and shapes from the training image, each labeled with a defect region of different dimensions. (The process of obtaining the defect mask image...) Figure 4 (d)-(f)) and defect texture images ( Figure 4 (c) Following this, optionally, the defect mask image can be used based on a multi-scale defect mask image. Figure 4 The defect regions marked by (d)-(f) are used as weight maps by employing the multi-scale minimum perceptible map of the training image to weight the training image ( Figure 4 (a)) and defect texture image ( Figure 4 (c) is fused to generate fused images at various scales. Figure 4 (g)-(i)), and finally fused to obtain the corresponding training defect image ( Figure 4 (j)).

[0058] Based on this, the generated training defect image can be reconstructed using the first sub-network in the neural network (serving as the reconstruction network) to remove defects, obtaining a training reconstructed image, and simultaneously obtaining a multi-scale minimum perceptible map of the training reconstructed image. During the training process of the first sub-network of the neural network, the training image can be compared with the training reconstructed image obtained through the first sub-network to adjust the loss function L of the first sub-network. R Furthermore, the loss function L of the first sub-network can be adjusted by comparing the multi-scale minimum perceptual map of the training image with the multi-scale minimum perceptual map of the reconstructed training image obtained through the first sub-network. J Therefore, the parameters in the first subnetwork are adjusted by using the output of the first subnetwork, so that each loss function converges.

[0059] Furthermore, based on the training reconstructed image and its multi-scale minimum perceptible map, defect detection can be performed on the training reconstructed image through the second sub-network. After obtaining the training reconstructed image and its multi-scale minimum perceptible map through the first sub-network of the neural network, during the training phase of the neural network, various training defect images, the generated training reconstructed image, and its multi-scale minimum perceptible map can be concatenated and input into the second sub-network of the neural network, and a new defect representation image (e.g., ...) can be output. Figure 4 (k) is shown, to compare with the defect localization image obtained by processing the training image and the training defect image through, for example, structural similarity (SSIM) as the ground truth, in order to utilize the loss function L S Train the second subnetwork.

[0060] Optionally, during defect localization, the obtained defect representation image (such as...) can also be used... Figure 4 The pixel values ​​in (k) are segmented and categorized to more intuitively represent the location of the defect. For example, when outputting the defect representation image, the pixels in the grayscale image of the defect representation image can be divided into three parts according to their grayscale values, such as representing them with normalized results of 0, 0.5, and 1 respectively (e.g., Figure 4 (l) is shown in order to make the location of the defect more obvious.

[0061] The above combination Figure 3 and Figure 4 A neural network training method according to an embodiment of the present invention has been described in detail. Furthermore, the neural network structure trained as described above can be used for... Figure 5 The diagram illustrates a defect detection process for an image to be detected. First, the image to be detected can be reconstructed using a first sub-network in a neural network to remove defects, resulting in a reconstructed image and a multi-scale minimum perceptible map of the reconstructed image. Subsequently, based at least on the reconstructed image and its multi-scale minimum perceptible map, the reconstructed image can be segmented for defects using a second sub-network to obtain a defect detection result. Optionally, the reconstructed image and its multi-scale minimum perceptible map can be input into the second sub-network of the neural network, combining these images and extracting features. Multi-scale defect region segmentation is then performed based on the extracted features, outputting a multi-scale defect region segmentation result as the defect detection result. The second sub-network can be used to output two different defect detection results for either a defect-free image or a defective image, or it can be used to locate multi-scale defect regions in the image to be detected, which is then considered a defective image, through segmentation.

[0062] Optionally, when locating the defect area, the pixel values ​​in the result of the defect detection image of the image to be detected can be segmented and categorized as described above, which will not be repeated here.

[0063] Below, refer to Figure 6 The present invention describes a neural network training apparatus according to an embodiment of the present invention. Figure 6 A block diagram of a neural network training apparatus 600 according to an embodiment of the present invention is shown. Figure 6 As shown, the neural network training apparatus 600 includes an input unit 610, a generation unit 620, and a training unit 630. Besides these units, the neural network training apparatus 600 may also include other components; however, since these components are not relevant to the content of this embodiment, their illustrations and descriptions are omitted here. Furthermore, since the specific details of the operations performed by the neural network training apparatus 600 according to this embodiment are the same as those described above... Figure 1 The details described are the same, so repeated descriptions of the same details are omitted here to avoid repetition.

[0064] Figure 6 The neural network training device 600 inputs a training image into its input unit 610 and calculates a perceptual map of the training image.

[0065] The input unit 610 uses training images of products or objects without defects as input for training the neural network. To enrich the training samples as much as possible, the training images can be images of the same product or object collected from different angles, under different lighting conditions, and against different backgrounds, in order to further improve the defect detection accuracy of the trained neural network.

[0066] After the training image is input into the input unit 610, a noticeable map of the training image can be calculated using the input training image. This noticeable map can be a multi-scale noticeable map. The multi-scale noticeable map of the training image reflects the changes in the strength of various image features in the training image, that is, it reflects the differences in the visual saliency of each image feature in the training image. The multi-scale noticeable map of the training image can make the generated training defect images more natural and realistic, and can also make the subsequent defect detection and localization processes more accurate.

[0067] In one example, a multi-scale just-noticeable map of the training image can be calculated using the input training image. For instance, the just-noticeable distortion (JND) model can be used to calculate the just-noticeable map of the training image. The JND value is an important indicator of visual saliency, describing the degree to which humans perceive image quality. The JND value reveals the perceptual threshold of image intensity changes that the human visual system can notice. Any perceptible level of distortion must be greater than the JND value. In this embodiment of the invention, a defect training image sample set with diversity and multiple scales can be generated using the JND model and used in subsequent defect detection and localization processes.

[0068] Optionally, the process of calculating the multi-scale minimum perceptual map of the training image based on the minimum perceptual error model may include: firstly, calculating the minimum perceptual map of the training image based on the minimum perceptual error model; then, processing the minimum perceptual map of the training image using multiple coefficients (e.g., multiplying multiple coefficients by the minimum perceptual map of the training image respectively) to obtain the multi-scale minimum perceptual map of the training image. For example, three coefficients (such as 0.33, 0.66, and 1) can be selected and multiplied by the minimum perceptual map of the training image respectively to obtain the minimum perceptual maps of the training image at three different scales; or, for another example, four coefficients (such as 0.25, 1, 0.75, and 1) can be selected and multiplied by the minimum perceptual map of the training image respectively to obtain the minimum perceptual maps of the training image at four different scales. The above methods for generating the multi-scale minimum perceptual map of the training image are merely examples. In the embodiments of the present invention, various methods can be used to generate the multi-scale minimum perceptual maps of training images at various scales according to the needs of different application scenarios. As mentioned above, the multi-scale minimum perceptible map of the training image can reflect the range and intensity of various image features in the training image. Based on human perception, the distribution and changes of gray values ​​in the map reflect the differences in the visual saliency of various image features in the training image.

[0069] The generation unit 620 generates a training defect image based at least on the training image and the perceptual map of the training image, the training defect image including one or more defects generated based on the perceptual map of the training image.

[0070] The process of generating training defect images by generation unit 620 may first include generating a defect mask image and generating one or more defect texture images based on the training image; then, the training image and the one or more defect texture images may be fused based on the defect mask image and using the multi-scale perceptual map of the training image to generate training defect images.

[0071] Optionally, the defect mask image can be generated by selecting one or more defect regions of varying sizes and shapes from the training image. Furthermore, the defect texture image can optionally be randomly generated from any image, or it can be generated by processing the training image. In one example, to enhance the realism of the simulated defect, the result of enhancing the training image can be used to generate a defect texture image. For example, various random processing methods, including region cropping, color transformation, brightness variation, and exposure variation, can be applied to the training image to obtain defect texture images with various textures.

[0072] After obtaining the defect mask image and the defect texture image, optionally, the generation unit 620 can fuse the training image and one or more defect texture images based on the defect regions defined by the defect mask image, using the multi-scale perceptual map of the training image as a weight map, to generate a training defect image with various simulated defects. According to the principle of perceptual maps, the perceptual map of the training image can reveal regions with high color contrast and saturation in the training image. Regions with larger grayscale values ​​in this perceptual map can occupy more defect components in the training defect image, meaning they can be assigned more weight to the defect texture image. Conversely, regions with smaller grayscale values ​​in the perceptual map of the training image can be assigned relatively more weight to the input training image. Based on this, the application of the multi-scale perceptual map of the training image can obtain fused training defect images at various scales, making the obtained training defect images more realistic and further usable in subsequent defect detection processes.

[0073] Optionally, when the generation unit 620 generates a multi-scale perceptible map of the training image, it can correspondingly generate multi-scale defect mask images to fuse the training image and the defect texture image, ultimately obtaining a training defect image. Specifically, the multi-scale defect mask images can be defect mask images that mark defect regions of different sizes corresponding to each scale in the multi-scale perceptible map of the training image. For example, referring to the aforementioned generation process of the multi-scale minimum perceptible map of the training image, the initially generated defect mask image can also be processed using multiple coefficients (e.g., multiplying multiple coefficients with the defect mask image respectively) to obtain corresponding multi-scale defect mask images, thereby defining the range of different defect regions at different scales. In the subsequent defect generation process, during the fusion of the training image and the defect texture image using these multi-scale defect mask images, corresponding image processing can be performed on the defect regions marked by the multi-scale defect mask images. For example, Gaussian blur or other methods can be used to process each defect region at multiple scales during the fusion process, thereby fusing to obtain a more realistic and natural training defect image.

[0074] The above methods for generating training defect images are merely examples. In practical applications, various methods can be used to generate training defect images according to the corresponding scenario, and no restrictions are imposed here.

[0075] The training unit 630 trains the neural network based at least on the generated training defect images to obtain defect detection results and adjust the parameters of the neural network.

[0076] Optionally, the training unit 630 can reconstruct the training defect image using a first sub-network (as a reconstruction network) in the neural network to remove defects, obtaining a training reconstructed image. Subsequently, based at least on the training reconstructed image, the training reconstructed image can be used to perform defect detection using a second sub-network (as a segmentation network) in the neural network to obtain a defect detection result. In one example, while generating the training reconstructed image using the first sub-network, the training unit 630 can also further obtain an aperceptual map of the training reconstructed image (e.g., a multi-scale aperceptual map of the training reconstructed image) using the first sub-network. Based on this, the training reconstructed image can be combined with its (multi-scale) aperceptual map to perform defect detection using the second sub-network.

[0077] Optionally, the first sub-network of the reconstructed network can consist of an encoder and a decoder. The encoder extracts features from the image input to the first sub-network. This input image can include the original training image or previously generated images with training defects, all of which can be used as training samples. In one example, the encoder can utilize convolution, normalization, and pooling processes with different parameters to extract features at different levels from the input image: for instance, firstly, the image input to the first sub-network can be convolved using convolution kernels to obtain a convolutional map; then, linear correction and batch normalization methods can be used to normalize the obtained convolutional map to obtain a normalized convolutional map; finally, max or average pooling can be applied to the normalized convolutional map. To obtain rich feature maps of various sizes, the above process can be repeated multiple times by adjusting relevant parameters to extract feature maps through multiple downsampling processes. After the encoder, the decoder can further process the feature maps through convolution, normalization, and upsampling to restore the resolution of the previously encoder-processed feature maps. During the process of passing through the first sub-network, which serves as the reconstruction network, the parameters of the first sub-network can be learned and updated under the supervision of the training image and its (multi-scale) perceptual map during the training phase of the neural network. This transforms the input training defect image into the corresponding defect-removed training reconstruction image. Furthermore, the (multi-scale) perceptual map of the training reconstruction image can be generated simultaneously through this first sub-network for use in subsequent defect detection and localization processes.

[0078] Optionally, after obtaining the training reconstructed image and the (multi-scale) perceptual map of the training reconstructed image through the first sub-network of the neural network, various training defect images, the generated training reconstructed image and the (multi-scale) perceptual map of the training reconstructed image can be input into the second sub-network of the neural network. The above images are combined and features are extracted to perform multi-scale defect region segmentation based on the extracted features, thereby outputting the multi-scale defect region segmentation result as the defect detection result.

[0079] In one example, during the training phase of the neural network, various training defect images, generated training reconstructed images, and (multi-scale) perceptual maps of the training reconstructed images can be concatenated and input into the second sub-network of the neural network. More specifically, when passing through the second neural network, it concatenates all the input images and outputs a new defect representation image, which is compared with the defect localization image obtained as the ground truth through the training images and simulated training defect images, and used to train the second sub-network. Optionally, the second sub-network can consist of convolutional layers, pooling layers, and softmax layers, and can have an encoder and decoder structure similar to the first sub-network, and can further have a cross-layer connection structure to fuse feature maps of the same scale in the encoder and decoder. The second sub-network can be used to output two different defect detection results: defect-free images and defective images, or it can obtain the localization of multi-scale defect regions through segmentation.

[0080] Optionally, when locating defects, the pixel values ​​in the obtained defect representation image can be segmented and categorized to more intuitively show the location of the defect. For example, when outputting the defect representation image, the pixels in the grayscale image can be divided into three parts according to their grayscale values, such as normalized results of 0, 0.5, and 1, to more clearly show the location of the defect. Of course, the above process of segmenting and categorizing defects is only an example. In practical applications, various pixel value classification and normalization operations can be performed on the defect representation image as needed, without limitation.

[0081] The neural network training apparatus described above according to embodiments of the present invention can generate defects in training images by introducing perceptual maps to train a neural network for defect detection. Furthermore, it can improve the accuracy of defect detection by further introducing perceptual maps of the reconstructed training images obtained from the defect-generated reconstruction network. This achieves accurate generation and robust detection of disordered defects, avoiding the need for collecting large amounts of defect data and manual annotation, thus improving the user experience. The neural network training apparatus described above according to embodiments of the present invention is applicable to various object detection tasks and computer vision tasks involving surface defect detection of objects such as road surfaces and chip surfaces.

[0082] Below, refer to Figure 7 The present invention describes a neural network training apparatus according to an embodiment of the present invention. Figure 7 A block diagram of a neural network training apparatus 700 according to an embodiment of the present invention is shown. Figure 7 As shown, the device 700 can be a computer or a server.

[0083] like Figure 7As shown, the neural network training device 700 includes one or more processors 710 and a memory 720. In addition, the neural network training device 700 may also include input devices, output devices (not shown), etc., and these components can be interconnected via a bus system and / or other forms of connection mechanisms. It should be noted that... Figure 7 The components and structure of the neural network training device 700 shown are merely exemplary and not limiting. The neural network training device 700 may also have other components and structures as needed.

[0084] The processor 710 may be a central processing unit (CPU) or other form of processing unit with data processing capabilities and / or instruction execution capabilities, and may utilize computer program instructions stored in the memory 820 to perform desired functions, including: inputting a training image and calculating an observability map of the training image; generating a training defect image based at least on the training image and the observability map of the training image, the training defect image including one or more defects generated by simulation; training the neural network based at least on the generated training defect image to obtain defect detection results and adjusting the parameters of the neural network.

[0085] The memory 720 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 710 may execute the program instructions to implement the functions of the neural network training apparatus of the embodiments of the present invention described above, and / or other desired functions, and / or to execute the neural network training method according to the embodiments of the present invention. Various application programs and various data may also be stored in the computer-readable storage medium.

[0086] The following describes a computer-readable storage medium according to an embodiment of the present invention, having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, perform the following steps: inputting a training image and calculating an observability map of the training image; generating a training defect image based at least on the training image and the observability map of the training image, the training defect image including one or more defects generated by simulation; training the neural network based at least on the generated training defect image to obtain defect detection results and adjusting the parameters of the neural network.

[0087] Below, refer to Figure 8 This invention describes an apparatus for defect detection using a neural network according to embodiments of the present invention. Figure 8 A block diagram of an apparatus 800 for defect detection using a neural network according to an embodiment of the present invention is shown. Figure 8 As shown, the device 800 for defect detection using a neural network includes an input unit 810 and a detection unit 820. Besides these units, the device 800 may also include other components; however, since these components are not relevant to the content of this embodiment, their illustrations and descriptions are omitted here. Furthermore, the specific details of the operations performed by the device 800 according to this embodiment are similar to those described above. Figure 2 The details described are the same, so repeated descriptions of the same details are omitted here to avoid repetition.

[0088] Figure 8 The device 800 for defect detection using a neural network inputs an image to be detected into its input unit 810. The image to be detected is reconstructed through a first sub-network in the neural network to remove defects, thereby obtaining a reconstructed image and a perceptible map of the reconstructed image.

[0089] The input unit 810 can input the image to be detected through... Figure 1 The process shown involves reconstructing the first subnetwork in the trained neural network. Optionally, the image to be detected can be a defect-free image or an image with one or more defects. During the detection process on the image to be detected, an observable map of the reconstructed image with defects removed can be obtained simultaneously, or a multi-scale observable map of the reconstructed image can be further obtained for use in subsequent defect detection and localization processes.

[0090] The first sub-network of the reconstructed network can consist of an encoder and a decoder. The encoder extracts features from the input image to this first sub-network. In one example, the encoder can utilize convolution, normalization, and pooling processes with different parameters to extract features at different levels from the input image. For instance, firstly, the input image can be convolved with a convolution kernel to obtain a convolutional map; then, linear correction and batch normalization can be used to normalize the obtained convolutional map to obtain a normalized convolutional map; finally, max or average pooling can be applied to the normalized convolutional map. To obtain rich feature maps of various sizes, the above process can be repeated multiple times by adjusting relevant parameters to extract feature maps from the image through multiple downsampling processes. After the encoder, the decoder can further process the feature maps through convolution, normalization, and upsampling to restore the resolution of the previously encoder-processed feature maps.

[0091] The detection unit 820 performs defect segmentation on the reconstructed image through a second sub-network based at least on the reconstructed image and the perceptible map of the reconstructed image, and obtains a defect detection result.

[0092] Optionally, after obtaining the reconstructed image and the (multi-scale) perceptual map of the reconstructed image through the first sub-network of the neural network, the detection unit 820 can input the generated reconstructed image and the (multi-scale) perceptual map of the reconstructed image into the second sub-network of the neural network, combine the above images together and extract features, perform multi-scale defect region segmentation based on the extracted features, and output the multi-scale defect region segmentation result as the defect detection result.

[0093] In one example, various images to be detected, the generated reconstructed images, and (multi-scale) perceptual maps of the reconstructed images can be concatenated and input into a second sub-network of the neural network. More specifically, as the images pass through the second neural network, it concatenates all the input images and outputs the result as a defect detection. Optionally, the second sub-network can consist of convolutional layers, pooling layers, and softmax layers, and can have an encoder and decoder structure similar to the first sub-network. It can further have cross-layer connections to fuse feature maps of the same scale from the encoder and decoder. The second sub-network can be used to output two different defect detection results for either a defect-free image or a defective image, or it can locate multi-scale defect regions in the image to be detected as a defective image through segmentation.

[0094] Optionally, during defect localization, the detection unit 820 can also segment and categorize the pixel values ​​in the obtained defect detection image of the image to be detected to more intuitively represent the location of the defect. For example, when outputting the defect detection image of the image to be detected, the pixels in the grayscale image of the defect detection image can be divided into three parts according to their grayscale values, such as represented by normalized results of 0, 0.5, and 1, respectively, to more clearly show the location of the defect. Of course, the above process of segmenting and categorizing defects is only an example. In practical applications, various pixel value classification and normalization operations can be performed on the defect detection image as needed, without limitation.

[0095] The apparatus for defect detection using a neural network according to embodiments of the present invention can generate defects in training images by introducing perceptual maps to train the neural network for defect detection. Furthermore, the accuracy of defect detection can be improved by introducing perceptual maps of the reconstructed training images obtained from the defect-generated reconstruction network. This achieves accurate generation and robust detection of disordered defects, avoiding the collection of large amounts of defect data and manual annotation, thus improving the user experience. The neural network training apparatus described in the embodiments of the present invention is applicable to various object detection tasks and computer vision tasks involving surface defect detection of objects such as road surfaces and chip surfaces.

[0096] Below, refer to Figure 9This invention describes an apparatus for defect detection using a neural network according to embodiments of the present invention. Figure 9 A block diagram of an apparatus 900 for defect detection using a neural network according to an embodiment of the present invention is shown. Figure 9 As shown, the device 900 can be a computer or a server.

[0097] like Figure 9 As shown, device 900 includes one or more processors 910 and memory 920. In addition, device 900 may also include input devices, output devices (not shown), etc., and these components can be interconnected via a bus system and / or other forms of connection mechanisms. It should be noted that... Figure 9 The components and structure of the device 900 shown are merely exemplary and not limiting; the device 900 may also have other components and structures as needed.

[0098] The processor 910 may be a central processing unit (CPU) or other processing unit with data processing and / or instruction execution capabilities, and may utilize computer program instructions stored in the memory 1020 to perform desired functions, including: inputting an image to be detected; reconstructing the image to be detected through a first sub-network to remove defects, obtaining a reconstructed image and a perceptible map of the reconstructed image; and at least based on the image to be detected and the perceptible map of the reconstructed image, performing defect segmentation on the reconstructed image through a second sub-network to obtain a defect detection result.

[0099] The memory 920 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 910 may execute the program instructions to implement the functions of the apparatus for defect detection using a neural network according to the embodiments of the present invention described above, and / or other desired functions, and / or to execute the method for defect detection using a neural network according to the embodiments of the present invention. Various application programs and various data may also be stored in the computer-readable storage medium.

[0100] The following describes a computer-readable storage medium according to an embodiment of the present invention, having stored thereon computer program instructions, wherein the computer program instructions, when executed by a processor, perform the following steps: inputting an image to be detected; reconstructing the image to be detected through a first sub-network to remove defects, thereby obtaining a reconstructed image and a perceptible map of the reconstructed image; and performing defect segmentation on the reconstructed image through a second sub-network, at least based on the image to be detected and the perceptible map of the reconstructed image, thereby obtaining a defect detection result.

[0101] Of course, the specific embodiments described above are merely examples and not limitations. Those skilled in the art can combine and integrate some steps and devices from the various embodiments described separately above to achieve the effects of the present invention. Such combined and integrated embodiments are also included in the present invention, but will not be described one by one here.

[0102] Note that the advantages, benefits, and effects mentioned in this invention are merely examples and not limitations, and should not be considered as essential features of every embodiment of the invention. Furthermore, the specific details described above are for illustrative and illustrative purposes only, and are not intended to limit the invention. These details do not limit the invention from being implemented solely by employing these specific details.

[0103] The block diagrams of devices, apparatuses, devices, and systems involved in this invention are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0104] The flowcharts and method descriptions in this invention are merely illustrative examples and are not intended to require or imply that the steps of the various embodiments must be performed in the given order. As those skilled in the art will recognize, the steps in the above embodiments can be performed in any order. Words such as "then," "next," etc., are not intended to limit the order of steps; these words are only used to guide the reader through the description of these methods. Furthermore, any reference to a singular element, such as the use of the articles "a," "one," or "the," is not to be construed as limiting that element to the singular.

[0105] Furthermore, the steps and apparatus in the various embodiments herein are not limited to any one embodiment. In fact, new embodiments can be conceived by combining relevant steps and apparatus in the various embodiments herein with the concepts of the present invention, and these new embodiments are also included within the scope of the present invention.

[0106] Each operation described above can be performed by any suitable means capable of performing the corresponding function. Such means may include various hardware and / or software components and / or modules, including but not limited to circuits, application-specific integrated circuits (ASICs), or processors.

[0107] The various exemplified logic blocks, modules, and circuits described herein can be implemented or performed using a general-purpose processor, digital signal processor (DSP), ASIC, field-programmable gate array (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The general-purpose processor may be a microprocessor, but alternatively, it may be any commercially available processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors cooperating with a DSP core, or any other such configuration.

[0108] The steps of the methods or algorithms described in this invention can be directly embedded in hardware, in a software module executed by a processor, or a combination of both. The software module can reside in any form of tangible storage medium. Some examples of usable storage media include random access memory (RAM), read-only memory (ROM), flash memory, EPROM, EEPROM, registers, hard disks, removable disks, CD-ROMs, etc. The storage medium can be coupled to the processor so that the processor can read information from and write information to the storage medium. Alternatively, the storage medium can be integral with the processor. The software module can be a single instruction or many instructions, and can be distributed across several different code segments, different programs, and across multiple storage media.

[0109] The method of this invention includes one or more actions for implementing the method. The methods and / or actions may be interchanged without departing from the scope of the claims. In other words, unless a specific order of actions is specified, the order and / or use of specific actions may be modified without departing from the scope of the claims.

[0110] The described functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions on a tangible computer-readable medium. The storage medium can be any available tangible medium that can be accessed by a computer. By way of example, and not limitation, such a computer-readable medium can include RAM, ROM, EEPROM, CD-ROM or other optical disc storage, disk storage or other magnetic storage devices, or any other tangible medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. As used herein, a disc includes a compact disc (CD), a laser disc, an optical disc, a digital universal disc (DVD), a floppy disk, and a Blu-ray disc.

[0111] Therefore, a computer program product can perform the operations described herein. For example, such a computer program product can be a computer-readable tangible medium having instructions tangibly stored (and / or encoded) thereon, which can be executed by one or more processors to perform the operations described herein. The computer program product may include packaging materials.

[0112] Software or instructions can also be transmitted via a transmission medium. For example, software can be transmitted from a website, server, or other remote source using transmission media such as coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, or microwave.

[0113] Furthermore, modules and / or other suitable means for carrying out the methods and techniques described herein can be downloaded and / or obtained by user terminals and / or base stations as appropriate. For example, such a device can be coupled to a server to facilitate the transmission of means for carrying out the methods described herein. Alternatively, the various methods described herein can be provided via storage components (e.g., RAM, ROM, physical storage media such as CDs or floppy disks) so that user terminals and / or base stations can obtain the various methods when coupled to the device or when providing storage components to the device. Furthermore, any other suitable techniques for providing the methods and techniques described herein to the device can be utilized.

[0114] Other examples and implementations are within the scope and spirit of this invention and the appended claims. For example, due to the nature of software, the functions described above can be implemented using software executed by a processor, hardware, firmware, hardwired, or any combination thereof. Features implementing the functions can also be physically located in various places, including being distributed so that parts of the functions are implemented at different physical locations. Moreover, as used herein, including as used in the claims, the "or" used in a list of items beginning with "at least one" indicates a separate list, such that a list of, for example, "at least one of A, B, or C" means A or B or C, or AB or AC or BC, or ABC (i.e., A and B and C). Furthermore, the word "exemplary" does not mean that the described examples are preferred or better than other examples.

[0115] Various changes, substitutions, and modifications can be made to the technology described herein without departing from the teachings defined by the appended claims. Furthermore, the scope of the claims is not limited to the specific aspects of the processes, machines, manufacturing processes, events, means, methods, and actions described above. Currently existing or later-developed processes, machines, manufacturing processes, events, means, methods, or actions that perform substantially the same function or achieve substantially the same result as the corresponding aspects described herein can be utilized. Therefore, the appended claims include such processes, machines, manufacturing processes, events, means, methods, or actions within their scope.

[0116] The above description of aspects of the invention is provided to enable any person skilled in the art to make or use the invention. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of the invention. Therefore, the invention is not intended to be limited to the aspects shown herein, but rather to be carried out within the widest scope consistent with the principles and novel features of the invention herein.

[0117] The above description has been given for illustrative and descriptive purposes. Furthermore, this description is not intended to limit the embodiments of the invention to the forms described herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations therein.

Claims

1. A neural network training method, comprising: Input a training image and calculate the perceptual map of the training image; A training defect image is generated based at least on the training image and the perceptual map of the training image, the training defect image including one or more defects generated based on the perceptual map of the training image; The neural network is trained at least on the generated training defect images to obtain defect detection results as training results, and the parameters of the neural network are adjusted based on the training results, including: The training defect image is reconstructed through the first sub-network in the neural network to remove defects, thereby obtaining a training reconstructed image; The perceptual map of the trained reconstructed image is further obtained through the first sub-network; Based on the training reconstructed image and the perceptual map of the training reconstructed image, defect detection is performed on the training reconstructed image through the second sub-network in the neural network to obtain the defect detection result.

2. The method as described in claim 1, wherein, Calculating the perceptual map of the training images includes: Calculate the multi-scale perceptual map of the training images.

3. The method as described in claim 2, wherein, Generating training defect images from at least the training images and the perceptual maps of the training images includes: Generate a defect mask image, and generate one or more defect texture images based on the training image; Based on the defect mask image, and by fusing the training image and the one or more defect texture images using the multi-scale perceptual map of the training image, a training defect image is generated.

4. A defect detection method, comprising: Input the image to be detected, and reconstruct the image through the first sub-network in the neural network to remove defects, thereby obtaining the reconstructed image and the perceptual map of the reconstructed image; Based at least on the reconstructed image and the perceptual map of the reconstructed image, the reconstructed image is segmented for defects through the second sub-network in the neural network to obtain defect detection results.

5. A neural network training device, comprising: An input unit is configured to input a training image and calculate the perceptual map of the training image; The generation unit is configured to generate a training defect image based at least on the training image and a perceptual map of the training image, the training defect image including one or more defects generated by simulation; The training unit is configured to train the neural network at least based on the generated training defect images, obtain defect detection results as training results, and adjust the parameters of the neural network based on the training results; The training unit reconstructs the training defect image through the first sub-network in the neural network to remove defects and obtain a training reconstructed image. The perceptual map of the trained reconstructed image is further obtained through the first sub-network; Based on the training reconstructed image and the perceptual map of the training reconstructed image, defect detection is performed on the training reconstructed image through the second sub-network in the neural network to obtain the defect detection result.

6. A neural network training device, comprising: processor; and a memory, in which computer program instructions are stored. When the computer program instructions are executed by the processor, the processor performs the following steps: Input a training image and calculate the perceptual map of the training image; At least a training defect image is generated based on the training image and a perceptual map of the training image, the training defect image including one or more defects generated by simulation; The neural network is trained at least on the generated training defect images to obtain defect detection results as training results, and the parameters of the neural network are adjusted based on the training results, including: The training defect image is reconstructed through the first sub-network in the neural network to remove defects, thereby obtaining a training reconstructed image; The perceptual map of the trained reconstructed image is further obtained through the first sub-network; Based on the training reconstructed image and the perceptual map of the training reconstructed image, defect detection is performed on the training reconstructed image through the second sub-network in the neural network to obtain the defect detection result.

7. A computer-readable storage medium having stored thereon computer program instructions, wherein, When the computer program instructions are executed by the processor, the following steps are performed: Input a training image and calculate the perceptual map of the training image; At least a training defect image is generated based on the training image and a perceptual map of the training image, the training defect image including one or more defects generated by simulation; The neural network is trained at least on the generated training defect images to obtain defect detection results as training results, and the parameters of the neural network are adjusted based on the training results, including: The training defect image is reconstructed through the first sub-network in the neural network to remove defects, thereby obtaining a training reconstructed image; The perceptual map of the trained reconstructed image is further obtained through the first sub-network; Based on the training reconstructed image and the perceptual map of the training reconstructed image, defect detection is performed on the training reconstructed image through the second sub-network in the neural network to obtain the defect detection result.

8. An apparatus for defect detection using a neural network, comprising: The input unit is configured to input an image to be detected, and to reconstruct the image to be detected through a first sub-network in a neural network to remove defects, thereby obtaining a reconstructed image and a perceptible map of the reconstructed image. The detection unit is configured to perform defect segmentation on the reconstructed image through a second sub-network in the neural network, based at least on the reconstructed image and the perceptual map of the reconstructed image, to obtain a defect detection result.

Images