An industrial defect data enhancement system and method based on a generative adversarial network

Abstract

The application provides an industrial defect data enhancement system and method based on a generative adversarial network, belonging to the field of data enhancement, and through defect labeling and background separation on surface image data of collected workpieces, a defect image of a source domain, a target domain image and a defect type are obtained; a generator extracts domain features of the defect image and the target domain image and aligns and maps them to a shared semantic space; mutual information response values between all aligned domain features are calculated, all mutual information response values are forgotten and screened, and common features are obtained; a joint loss function under physical constraints is constructed, the common features are input into the generator, and an adversarial game training is performed with a discriminator, parameters during the adversarial game training are adjusted through the joint loss function, and an enhanced defect image is generated. By adopting the scheme of the application, information between data can be balanced in combination with the constraint of physical features, so that the defect data is more in line with the actual situation and effectively supplements the boundary samples.

Description

Technical Field

[0001] This application relates to the field of data augmentation technology, and more specifically, to an industrial defect data augmentation system and method based on generative adversarial networks. Background Technology

[0002] In the field of industrial defect detection, the scarcity of high-quality defect samples is a common problem, and data augmentation has become a core technical means to solve this bottleneck. Generative adversarial network-based methods are one of the most widely used and fastest-growing technical directions in industrial defect data augmentation. The core idea is to train a generator that can generate high-quality defect images through adversarial game between the generator and the discriminator.

[0003] In existing technologies, typical generative adversarial network (GAN) frameworks for defect data augmentation primarily employ cycle consistency loss to establish mapping relationships between unpaired images. However, in industrial scenarios, there is a severe information imbalance between defective images and defect-free background images. Defective regions often occupy only a tiny portion of the image, and cycle consistency loss forces the network to hide background information during generation, resulting in unnatural integration of generated defects with the background. Furthermore, existing methods lack constraints on the physical laws governing defects, leading to generated defect morphologies that often do not conform to actual physical processes. Additionally, existing methods struggle to generate critical samples near the classification boundaries of detection models, resulting in insufficient fitting accuracy of downstream detection models for these boundary samples. Therefore, how to combine physical feature constraints to balance the information between data, thereby making defect data more realistic and effectively supplementing boundary samples, has become a challenging problem in the industry. Summary of the Invention

[0004] This application provides an industrial defect data enhancement system and method based on generative adversarial networks, which can combine the information between data constrained by physical features to make the defect data more realistic and effectively supplement boundary samples.

[0005] In a first aspect, this application provides a method for industrial defect data augmentation based on generative adversarial networks (GANs), wherein the GAN includes a generator and a discriminator, and the method includes the following steps:

[0006] The surface image data of the acquired workpiece is used to perform defect annotation and background separation to obtain the defect image of the source domain, the target domain image, and the defect type;

[0007] The generator extracts the domain features of the defect image and the target domain image and aligns and maps them to a shared semantic space;

[0008] The generator calculates the mutual information response values ​​between all aligned domain features channel by channel using mutual information, and performs forgetting filtering on all mutual information response values ​​to obtain the common features between domains after feature balancing.

[0009] Based on the defect type, a joint loss function under physical constraints is constructed. The common features are input into the generator and the discriminator for adversarial game training. The parameters during adversarial game training are adjusted through the joint loss function so that the generator can generate enhanced defect images.

[0010] The enhanced defect image and the defect image are merged to obtain the enhanced defect dataset.

[0011] In some embodiments, defect annotation and background separation are performed on the acquired surface image data of the workpiece to obtain a defect image, a target domain image, and a defect type, specifically including:

[0012] Surface image data of workpieces are acquired using industrial cameras deployed on the production line;

[0013] Semantic segmentation is performed on the surface image data to obtain multiple defect regions and background regions;

[0014] A surface image containing at least one defective region is used as a defective image, and a surface image that does not contain any defective regions is used as a target domain image.

[0015] Based on the matching results of all defect areas and the industry's standard defect library, the corresponding defect types are marked.

[0016] In some embodiments, the generator extracts domain features from the defect image and the target domain image and aligns them to a shared semantic space, specifically including:

[0017] The generator uses two domain encoders to convolve and downsample the defect image and the target domain image respectively to obtain the feature map of the source domain and the feature map of the target domain.

[0018] Both the feature map of the source domain and the feature map of the target domain contain multiple channels;

[0019] The feature maps of the source and target domains are input into the embedding layer with shared weights to obtain aligned defect embedding feature maps and target embedding feature maps.

[0020] Extract the global semantic information of the defect embedding feature map and the target embedding feature map in each channel to obtain the corresponding semantic vector;

[0021] The generator maps all semantic vectors through a projection layer with shared weights to obtain the domain features of the aligned defect image and the target domain image.

[0022] In some embodiments, the generator calculates the mutual information response values ​​between all aligned domain features channel by channel using mutual information, specifically including:

[0023] Align all the aligned domain features by channel, with each channel corresponding to a pair of feature activation values;

[0024] For any channel, calculate the KL divergence between the feature activation values ​​of the source domain and the feature activation values ​​of the target domain, and use it as the mutual information response value of the corresponding channel.

[0025] In some embodiments, forgetting filtering is performed on all mutual information response values ​​to obtain shared features between domains after feature balancing, specifically including:

[0026] The forgetting weight for the corresponding channel is determined based on all mutual information response values;

[0027] Obtain the preset forgetting threshold;

[0028] Set the feature maps of all channels whose forgetting weight is lower than the forgetting threshold to zero to remove them;

[0029] The feature maps corresponding to all channels whose forgetting weight is greater than or equal to the forgetting threshold are retained.

[0030] Multiply all retained feature maps by the forgetting weights of the corresponding channels one by one to obtain the common features between the domains after feature balancing.

[0031] In some embodiments, constructing a joint loss function under physical constraints based on the defect type specifically includes:

[0032] The physical model is called from the preset physical model library according to the defect type. The physical model library includes an elastic mechanical model for cracks, a geometric continuity model for scratches, and an optical reflection model for stains.

[0033] The physical constraint loss is defined as the deviation between the candidate defect image generated by the generator and the physical model.

[0034] The adversarial loss is defined as the difference in discrimination between the discriminator's judgment of the defect image in the source domain and the candidate defect image generated by the generator. The adversarial loss adopts the form of least squares loss plus gradient penalty.

[0035] The mean gradient of the defect region edge is defined as the response index when the generator generates an image, and the degree to which the response index deviates from the classification boundary is used as the critical degree loss.

[0036] The physical constraint loss, adversarial loss, and criticality loss are weighted to obtain the joint loss function.

[0037] In some embodiments, adjusting the parameters during adversarial game training using the joint loss function to enable the generator to generate enhanced defect images specifically includes:

[0038] Initialize the parameters in the generator before the adversarial game training begins;

[0039] In each training round, the common features are input into the generator to obtain candidate defect images, and the candidate defect images and the defect images are input into the discriminator for discrimination.

[0040] Based on the discrimination result of the discriminator and the candidate defect image, the joint loss function is instantiated and the joint loss value is calculated;

[0041] Calculate the gradient of the joint loss value with respect to each parameter in the generator, and then update the parameters of the generator based on the gradient;

[0042] Repeat the above process until the joint loss converges, and use the image output by the generator at this point as the enhanced defect image.

[0043] Secondly, this application provides an industrial defect data augmentation system based on generative adversarial networks, comprising:

[0044] The acquisition module is used to perform defect annotation and background separation on the surface image data of the acquired workpiece to obtain the defect image of the source domain, the target domain image, and the defect type;

[0045] The processing module is used to extract the domain features of the defect image and the target domain image from the generator and align and map them to a shared semantic space;

[0046] The processing module is also used by the generator to calculate the mutual information response value between all aligned domain features channel by channel using mutual information, and to perform forgetting filtering on all mutual information response values ​​to obtain the common features between domains after feature balancing.

[0047] The processing module is further configured to construct a joint loss function under physical constraints based on the defect type, input the common features into the generator and discriminator for adversarial game training, and adjust the parameters during adversarial game training through the joint loss function so that the generator can generate enhanced defect images.

[0048] The execution module is used to merge the enhanced defect image and the defect image to obtain the enhanced defect dataset.

[0049] Thirdly, this application provides a computer device including a memory and a processor, the memory storing code, and the processor being configured to acquire the code and execute the above-described industrial defect data enhancement method based on generative adversarial networks.

[0050] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described industrial defect data enhancement method based on generative adversarial networks.

[0051] The technical solutions provided by the embodiments disclosed in this application have the following beneficial effects:

[0052] This application provides an industrial defect data enhancement system and method based on generative adversarial networks. First, defect annotation and background separation are performed on the collected surface image data of the workpiece to obtain a source domain defect image, a target domain image, and defect types. The generator extracts domain features from the defect image and the target domain image and aligns them to a shared semantic space. The generator calculates the mutual information response values ​​between all aligned domain features channel-by-channel using mutual information, performs forgetting filtering on all mutual information response values, and obtains shared features between domains after feature balancing. A joint loss function under physical constraints is constructed based on the defect types. The shared features are input into the generator, and adversarial game training is performed with the discriminator. The parameters during adversarial game training are adjusted using the joint loss function, enabling the generator to generate enhanced defect images. The enhanced defect images and the original defect images are merged to obtain an enhanced defect dataset.

[0053] Therefore, in this application's method for industrial defect data augmentation based on generative adversarial networks, firstly, the generator extracts domain features from the defect image and the target domain image and aligns them to a shared semantic space. The generator then calculates the mutual information response values ​​between all aligned domain features channel-by-channel using mutual information, and performs forgetting filtering on all mutual information response values ​​to obtain shared features between the domains after feature balance. These shared features are weighted feature maps obtained by multiplying all retained channels by their corresponding forgetting weights channel-by-channel, used as input to the decoder. This facilitates emphasizing shared patterns between the two domains and suppressing their unique patterns in the shared semantic space, fundamentally solving the problem of forced information hiding caused by information imbalance. Secondly, a joint loss function under physical constraints is constructed based on the defect type. The shared features are input into the generator, and adversarial game training is performed with the discriminator. The parameters during adversarial game training are adjusted using the joint loss function, enabling the generator to generate augmented defect images. This scheme combines the constraints of physical features to balance the information between data, making the defect data more realistic and effectively supplementing boundary samples. Attached Figure Description

[0054] Figure 1 This is an exemplary flowchart of an industrial defect data augmentation method based on generative adversarial networks, according to some embodiments of this application;

[0055] Figure 2 This is a scene interaction diagram of acquiring surface images according to some embodiments of this application;

[0056] Figure 3 This is a schematic diagram illustrating adversarial game training according to some embodiments of this application;

[0057] Figure 4 This is a schematic diagram of the structure of an industrial defect data augmentation system based on generative adversarial networks, according to some embodiments of this application;

[0058] Figure 5 This is a schematic diagram of the structure of a computer device that implements an industrial defect data augmentation method based on generative adversarial networks, according to some embodiments of this application. Detailed Implementation

[0059] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0060] refer to Figure 1 The figure is an exemplary flowchart of an industrial defect data augmentation method based on generative adversarial networks according to some embodiments of this application. The industrial defect data augmentation method based on generative adversarial networks mainly includes the following steps:

[0061] In step 101, the surface image data of the acquired workpiece is subjected to defect annotation and background separation to obtain the source domain defect image, the target domain image, and the defect type.

[0062] In some embodiments, the defect annotation and background separation of the acquired workpiece surface image data can be achieved by the following steps:

[0063] Surface image data of workpieces are acquired using industrial cameras deployed on the production line;

[0064] Semantic segmentation is performed on the surface image data to obtain multiple defect regions and background regions;

[0065] A surface image containing at least one defective region is used as a defective image, and a surface image that does not contain any defective regions is used as a target domain image.

[0066] Based on the matching results of all defect areas and the industry's standard defect library, the corresponding defect types are marked.

[0067] In practice, acquiring surface image data of workpieces using industrial cameras deployed on the production line can be achieved in the following way: An industrial camera is fixedly installed directly above the conveyor belt of the production line, with the camera lens perpendicular to the workpiece detection surface and approximately 30 to 50 centimeters away from the workpiece surface. When the workpiece is triggered by a photoelectric sensor and reaches the center of the camera's field of view, the camera automatically takes a picture, capturing one surface image for each workpiece. Multiple workpieces are continuously photographed to obtain multiple surface images, forming a surface image data set. (Reference) Figure 2 As shown, this figure is a scene interaction diagram of acquiring surface images in some embodiments of this application. In the figure, an industrial camera is fixedly installed directly above the conveyor belt to acquire images of the workpiece in real time.

[0068] In a specific implementation, semantic segmentation of the surface image data to obtain multiple defect regions and background regions can be achieved in the following way: the surface image data is input into a pre-trained semantic segmentation neural network, which predicts the probability of each pixel belonging to the defect or the background; a segmentation label map of the same size as the input image is output, in which defect regions are represented by white pixels and background regions are represented by black pixels; all connected white pixel blocks are extracted from the segmentation label map as multiple defect regions, and the remaining black pixel parts are all regarded as background regions.

[0069] In specific implementation, the surface image containing at least one defective region is designated as the defective image, and the surface image without any defective regions is designated as the target domain image. This can be achieved in the following way: traverse all acquired surface image data and check all defective regions corresponding to each surface image; if a surface image contains at least one defective region, then mark the surface image as a defective image, and all surface images containing defects constitute a set of defective images, with each defective image corresponding to a workpiece; if a surface image does not contain any defective regions, then mark the surface image as a target domain image, and all surface images without defects constitute a set of target domain images.

[0070] In practice, the corresponding defect type can be labeled based on the matching results of all defect areas and the industrial standard defect library. This can be achieved in the following way: For each defect area, the scale-invariant feature transformation algorithm is used to extract numerical features of shape, texture, and size from the pixel matrix of each defect area to form a multi-dimensional feature vector; the cosine similarity or Euclidean distance calculation method is used to compare this feature vector with the template features of cracks, scratches, dents, stains, bubbles, etc., pre-stored in the industrial standard defect library one by one, and the similarity score is calculated; the defect name corresponding to the template with the highest similarity is selected as the type label of the defect area; the types of all defect areas are recorded, merged and deduplicated to obtain the defect type of the corresponding defect image.

[0071] It should be noted that the surface image data in this application is a set of original digital images used for subsequent defect annotation and background separation; the defect region and the background region are pixel regions used to distinguish abnormal and normal parts of the workpiece surface; the defect image is a sample image representing the presence of defects in the workpiece, and the target domain image is a reference image representing a defect-free background; the defect type is a label identifying the category of physical behavior that the generator should simulate.

[0072] In step 102, the generator extracts the domain features of the defect image and the target domain image and aligns them to a shared semantic space.

[0073] In some embodiments, the generator extracts domain features from the defect image and the target domain image and aligns them to a shared semantic space, which can be achieved by the following steps:

[0074] The generator uses two domain encoders to convolve and downsample the defect image and the target domain image respectively to obtain the feature map of the source domain and the feature map of the target domain.

[0075] Both the feature map of the source domain and the feature map of the target domain contain multiple channels;

[0076] The feature maps of the source and target domains are input into the embedding layer with shared weights to obtain aligned defect embedding feature maps and target embedding feature maps.

[0077] Extract the global semantic information of the defect embedding feature map and the target embedding feature map in each channel to obtain the corresponding semantic vector;

[0078] The generator maps all semantic vectors through a projection layer with shared weights to obtain the domain features of the aligned defect image and the target domain image.

[0079] In specific implementation, the generator uses two domain encoders to perform convolution and downsampling on the defect image and the target domain image respectively, and the feature maps of the source domain and the target domain can be obtained in the following way: The generator sets up two independent encoder networks, each encoder consisting of four convolutional layers and three pooling layers stacked alternately; a defect image is input into the first encoder, and after layer-by-layer convolution and downsampling, a feature map with a height and width of one-sixteenth of the original image and 256 channels is output, which is denoted as the feature map of the source domain; a target domain image is input into the second encoder, and after the same operation, a feature map with the same number of channels and the same spatial size is obtained, which is denoted as the feature map of the target domain; the feature maps of the source domain and the target domain are both single and each contains 256 channels, and each channel corresponds to an image mode response.

[0080] In specific implementation, the feature maps of the source and target domains are input into a shared-weight embedding layer to obtain aligned defect embedding feature maps and target embedding feature maps. This can be achieved as follows: An embedding layer is set in the generator, consisting of three residual blocks connected in series. The weight parameters of all residual blocks are completely shared between the source and target domains. The feature map of the source domain is input into the embedding layer, and after layer-by-layer nonlinear transformation of the residual blocks, a defect embedding feature map with the same input size is output. The feature map of the target domain is input into the same embedding layer, and after the residual block transformation, a target embedding feature map is output. The shared weights ensure that the features of the two domains are mapped to the same semantic embedding space, making the corresponding channels comparable. Since there is a severe information imbalance between the source domain (defect image) and the target domain (defect-free background image), directly matching the feature maps of the two domains would lead to many-to-one mapping and information hiding problems. Therefore, it is necessary to map them to a shared semantic space.

[0081] In specific implementation, the global semantic information of the defect embedding feature map and the target embedding feature map in each channel is extracted to obtain the corresponding semantic vector. This can be achieved in the following way: For the defect embedding feature map, a global average pooling operation is performed independently on each channel, that is, the arithmetic mean of the pixel values ​​at all spatial locations in that channel is calculated; the arithmetic mean of all 256 channels is arranged in channel order to form a 256-dimensional vector, called the defect semantic vector; the same operation is performed on the target embedding feature map to obtain the target semantic vector; the two semantic vectors respectively summarize the global response intensity of each channel of the corresponding feature map; wherein, global average pooling makes the spatial response intensity of each channel reflect the degree of existence of the semantic pattern represented by that channel in the whole image. By compressing the spatial dimension to obtain a scalar, subsequent mutual information comparison along the channel dimension can be performed, thereby solving the information imbalance problem.

[0082] In specific implementation, the generator maps all semantic vectors through a projection layer with shared weights to obtain the aligned domain features of the defect image and the target domain image. This can be achieved in the following way: a projection layer is set in the generator. This projection layer is a fully connected network containing a hidden layer and an output layer, and the weight parameters are shared between the source domain and the target domain. The defect semantic vector is input into the projection layer, and after linear transformation and nonlinear activation, another 256-dimensional vector is output, which is denoted as the domain feature of the defect image. The target semantic vector is input into the same projection layer to obtain the domain feature of the target domain image. The role of the projection layer is to align the distribution of the original semantic vectors to the same metric space, so that the semantic vectors from the two domains are comparable after projection, which facilitates the subsequent calculation of channel-level mutual information response values.

[0083] It should be noted that the feature map of the source domain in this application is a multi-channel two-dimensional response tensor extracted from the defect image, and the feature map of the target domain is a multi-channel two-dimensional response tensor extracted from the target domain image; the defect embedding feature map and the target embedding feature map are multi-channel feature maps located in the same semantic space obtained by transforming the original feature map through a shared weight embedding layer, which are used to unify the feature representation of the two domains and facilitate subsequent channel-by-channel comparison of their consistency; the domain feature is a pair of directly comparable feature vectors obtained by projecting the semantic vector through a shared weight projection layer, which are used to quantify the similarity of the two domains in the same channel, and facilitate subsequent provision of numerical basis for feature selection under information imbalance.

[0084] In step 103, the generator calculates the mutual information response values ​​between all aligned domain features channel by channel using mutual information, performs forgetting filtering on all mutual information response values, and obtains the common features between domains after feature balancing.

[0085] In some embodiments, the generator can calculate the mutual information response values ​​between all aligned domain features channel by channel using mutual information, which can be achieved by the following steps:

[0086] Align all the aligned domain features by channel, with each channel corresponding to a pair of feature activation values;

[0087] For any channel, calculate the KL divergence between the feature activation values ​​of the source domain and the feature activation values ​​of the target domain, and use it as the mutual information response value of the corresponding channel.

[0088] In practice, all aligned domain features are aligned by channel, with each channel corresponding to a pair of feature activation values. This can be achieved as follows: the domain features of the defect image and the domain features of the target image are treated as two vectors with the same dimension, each dimension corresponding to a channel. For the first channel, the first value in the defect domain feature and the first value in the target domain feature are extracted to form a pair of activation values. For the second channel, the second pair of values ​​is extracted, and so on until the 256th channel. This results in a total of 256 pairs of feature activation values, where the two values ​​in each pair come from the same channel but different domains. Since the features from different domains have been mapped to the same metric space in the projection layer, the two values ​​at the same channel position represent the activation intensity of the same semantic pattern in the two domains. Alignment by channel establishes a correspondence for subsequent channel-by-channel mutual information comparison, thereby enabling the location of specific channel positions where information is imbalanced.

[0089] In practice, for any channel, the KL divergence between the feature activation values ​​of the source domain and the target domain can be calculated as the mutual information response value for that channel. This can be achieved as follows: For any pair of activation values ​​in any channel, treat them as a single-point approximation of two probability distributions; according to the definition of KL divergence, calculate the source domain activation value multiplied by the natural logarithm of the source domain activation value divided by the target domain activation value; repeat this calculation process for each channel, resulting in 256 values, each ranging from 0 to positive infinity; smaller values ​​indicate greater inconsistency between the two domains in that channel, while larger values ​​indicate greater consistency; these values ​​are then used as the mutual information response value. Here, KL divergence is an indicator that measures the difference between two probability distributions. The KL divergence is defined as follows: For two distributions P and Q, the KL divergence is equal to the sum of the product of the probability of each event in P and the logarithm of that probability divided by the logarithm of the probability of that event in Q. In this step, since each channel has only one pair of values, the source domain activation value is considered as P and the target domain activation value is considered as Q. The natural logarithm of the source domain activation value multiplied by the target domain activation value is calculated. The KL divergence can quantify the similarity between two single-valued distributions. A low response value means that the channel features are significantly different in the two domains and belong to domain-specific information. A high response value indicates that the features are consistent and belong to inter-domain shared information. By calculating the KL divergence of each channel, a measure of feature consistency can be obtained, providing a quantitative basis for subsequent forgetting screening.

[0090] It should be noted that the feature activation values ​​in this application are two values ​​extracted from the same channel position of the defect domain features and the target domain features, which are used to represent the response intensity of the channel in the two domains; the mutual information response value is a scalar score obtained by calculating the KL divergence channel by channel, which is used to quantify the feature consistency of the two domains on the same channel, so as to identify the high response value channel as a common feature and the low response value channel as an inconsistent feature.

[0091] In some embodiments, the following steps can be used to perform forgetting filtering on all mutual information response values ​​to obtain the common features between domains after feature balancing:

[0092] The forgetting weight for the corresponding channel is determined based on all mutual information response values;

[0093] Obtain the preset forgetting threshold;

[0094] Set the feature maps of all channels whose forgetting weight is lower than the forgetting threshold to zero to remove them;

[0095] The feature maps corresponding to all channels whose forgetting weight is greater than or equal to the forgetting threshold are retained.

[0096] Multiply all retained feature maps by the forgetting weights of the corresponding channels one by one to obtain the common features between the domains after feature balancing.

[0097] In practice, determining the forgetting weight of a corresponding channel based on all mutual information response values ​​can be achieved as follows: Collect the mutual information response values ​​of all channels to form a response value vector; perform min-max normalization on this vector, linearly mapping each response value to the interval between 0 and 1, resulting in 256 forgetting weights. Channels with larger mutual information response values ​​have forgetting weights closer to 1, while channels with smaller mutual information response values ​​have forgetting weights closer to 0; use these forgetting weights as the basis for subsequent threshold filtering. The magnitude of the mutual information response value itself reflects the feature consistency between the source and target domains on that channel; a larger response value indicates more shared features, while a smaller response value indicates less inconsistent features. Min-max normalization compresses the original response values ​​to the 0-1 interval, making the normalized values ​​directly interpretable retention strength coefficients. That is, channels with high consistency receive high weights and should be retained, while channels with low consistency receive low weights and should be forgotten.

[0098] In specific implementation, setting the feature maps corresponding to all channels with forgetting weights lower than the forgetting threshold to zero for removal can be achieved as follows: Set a constant between 0 and 1 as the forgetting threshold, for example, 0.5; this threshold does not change during training and is used to compare with the forgetting weight of each channel; channels with forgetting weights greater than or equal to the threshold are determined as common feature channels, and channels with forgetting weights less than the threshold are determined as inconsistent feature channels; use this forgetting threshold as a filtering switch; extract the defect embedding feature map obtained in the previous steps, which has 256 channels; iterate through each channel and extract the corresponding forgetting weight; if the forgetting weight is less than 0.5, then... All pixel values ​​in this channel are forcibly set to 0; if the forgetting weight is greater than or equal to 0.5, the original value remains unchanged; after processing, all inconsistent channels are completely suppressed, and the remaining channels retain features that may be shared between domains; the core of the information imbalance problem lies in the existence of a large number of unique features in the source domain that are not present in the target domain, which can interfere with translation quality; by setting a forgetting threshold, low consistency channels are directly set to zero, which is equivalent to forcing the generator to ignore noise or redundant patterns that only exist in the defect domain and not in the background domain, thereby avoiding artifacts that do not blend with the background during subsequent decoding; the threshold is a balance point, and retaining channels with a weight greater than or equal to 0.5 ensures the integrity of shared features.

[0099] In specific implementation, the feature maps corresponding to all channels with forgetting weights greater than or equal to the forgetting threshold can be retained in the following way: for channels with forgetting weights greater than or equal to 0.5, no modifications are made, and all spatial pixel values ​​of the channel are completely retained; for those channels that have been set to zero, the retained channels become the non-zero components of the feature map.

[0100] In practice, all retained feature maps are multiplied channel by channel with the forgetting weights of the corresponding channels to obtain the shared features between the domains after feature balancing. This can be achieved as follows: for each retained channel, each pixel value on the feature map of that channel is multiplied by the forgetting weight corresponding to that channel; for channels that have been set to zero, the result after multiplying by the forgetting weight is still zero; after channel-by-channel multiplication, the response values ​​in the feature map are weighted and adjusted, with channels with higher mutual information contributing more; finally, a feature map with 256 channels and unchanged spatial size is obtained, which is called the shared feature; this shared feature emphasizes the patterns shared by the two domains and suppresses their unique patterns; this shared feature consists of two parts: one part is the effective channels from the defect embedding feature map that are retained and weighted by the forgetting weights, and the other part is the informationless channels that are set to zero; simple hard screening will lose the relative importance differences between channels. By using channel-by-channel multiplication with forgetting weights, even the contribution of the retained channels is scaled according to the size of the mutual information response value. By balancing the features through weighted averaging rather than binary selection, a more generalizable shared feature representation is obtained.

[0101] It should be noted that the forgetting weight in this application is a coefficient between 0 and 1 obtained by minimizing and maximizing the mutual information response value. It is used to represent the degree to which each channel should be retained, which facilitates the implementation of a soft attention mechanism through a threshold, so that the generator prioritizes common features and suppresses redundant information. The common features are the weighted feature maps obtained by multiplying all retained channels with the corresponding forgetting weights channel by channel. They are used as the input of the decoder, which facilitates the emphasis on the patterns shared by the two domains and the suppression of their unique patterns in the shared semantic space, thereby fundamentally solving the problem of forced information hiding caused by information imbalance.

[0102] In step 104, a joint loss function under physical constraints is constructed based on the defect type. The common features are input into the generator and the discriminator for adversarial game training. The parameters during adversarial game training are adjusted through the joint loss function so that the generator can generate enhanced defect images.

[0103] In some embodiments, constructing a joint loss function under physical constraints based on the defect type can be achieved using the following steps:

[0104] The physical model is called from the preset physical model library according to the defect type. The physical model library includes an elastic mechanical model for cracks, a geometric continuity model for scratches, and an optical reflection model for stains.

[0105] The physical constraint loss is defined as the deviation between the candidate defect image generated by the generator and the physical model.

[0106] The adversarial loss is defined as the difference in discrimination between the discriminator's judgment of the defect image in the source domain and the candidate defect image generated by the generator. The adversarial loss adopts the form of least squares loss plus gradient penalty.

[0107] The mean gradient of the defect region edge is defined as the response index when the generator generates an image, and the degree to which the response index deviates from the classification boundary is used as the critical degree loss.

[0108] The physical constraint loss, adversarial loss, and criticality loss are weighted to obtain the joint loss function.

[0109] In practice, calling a pre-set physical model from a physical model library based on the defect type can be achieved in the following way: A physical model library is pre-stored in the system, where crack defects correspond to an elasticity model describing the morphology of crack propagation under stress; scratch defects correspond to a geometric continuity model describing the width consistency and directional smoothness of the scratch; and stain defects correspond to an optical reflection model describing the light intensity absorption and scattering distribution in the stain area. The type of the current defect image is read from the defect type; if the type is crack, the elasticity model is called; if it is scratch, the geometric continuity model is called; and if it is stain, the optical reflection model is called. The called physical model is instantiated into a computable objective function for subsequent loss calculation.

[0110] In specific implementation, the physical constraint loss is defined as the deviation between the candidate defect image generated by the generator and the physical model. It can be implemented in the following way: The physical constraint loss is a quantitative index that measures the degree of inconsistency between the defect image output by the generator and the ideal defect shape specified by the physical model. The calculation method is as follows: For the ideal defect shape image, i.e. the subsequently generated candidate defect image, and the defect image output by the generator, at each corresponding pixel position in the defect region, calculate the absolute value of the difference between the two pixel values, then sum the absolute values ​​of all pixel positions, and divide by the total number of pixels in the defect region. The average value obtained is used as the physical constraint loss. The larger the value of the physical constraint loss, the less the generated defect conforms to the physical laws.

[0111] In specific implementation, the adversarial loss is defined as the difference in discrimination between the discriminator's judgment of the defect image in the source domain and the candidate defect image generated by the generator. This can be achieved as follows: the adversarial loss is a quantitative indicator used to force the generator's output image to approximate the distribution of real defect images. The discriminator is a binary classification neural network that takes an image as input and outputs a scalar between 0 and 1, representing the probability that the image is a real defect image. The adversarial loss is calculated by subtracting the square of 1 from the output value obtained by inputting the defect image into the discriminator, and then adding the square of subtracting 0 from the output value obtained by inputting the defect image output by the generator into the discriminator. The sum of these two squares is the adversarial loss value. A smaller adversarial loss indicates a more realistic generated image. Additionally, during discriminator training, the magnitude of the gradient of its output with respect to the input image is calculated, and this magnitude is forced to be close to 1 as a gradient penalty term added to the adversarial loss. A smaller adversarial loss indicates a more realistic generated image.

[0112] In specific implementation, the mean gradient of the defect region edge is defined as the response index when the generator generates the image. The degree to which the response index deviates from the classification boundary can be used as the critical loss in the following way: After obtaining the candidate defect image, the edge map of the defect region is first extracted by the edge detection operator; the gradient of all edge pixels on the edge map is calculated, and the average of all gradients is taken as the response index; before training, the distribution of the response index of all real defect images is statistically analyzed, and a threshold is found to separate stable samples and unstable samples. This threshold is the classification boundary; the absolute value of the difference between the response index of the current candidate defect image and the threshold is calculated as the critical loss; the smaller this loss is, the closer the generated defect is to the classification boundary. This loss is added to the joint loss function.

[0113] In practice, the physical constraint loss, adversarial loss, and criticality loss are weighted to obtain the joint loss function. This can be achieved by assigning a weight coefficient to each of the physical constraint loss, adversarial loss, and criticality loss, for example, a weight of 5 for the physical constraint loss, a weight of 1 for the adversarial loss, and a weight of 2 for the criticality loss. The three are then added together according to their weights to obtain a scalar value, which is called the joint loss function value. This value will be used for backpropagation to update the parameters of the generator and discriminator, and the joint loss function will be used as the sole optimization objective during the training phase.

[0114] It should be noted that the physical constraint loss in this application is the absolute error between the defect image output by the generator and the output of the physical model; the adversarial loss is the error of the discriminator during adversarial training of real defect images and generated defect images; the criticality loss is the absolute difference between the response index of the generated defect image and the preset classification boundary threshold during training; the joint loss function is the overall optimization objective obtained by weighted summation of the physical constraint loss, adversarial loss and criticality loss, which is used to simultaneously optimize the physical realism and boundary closeness of the generator in a single training process, so as to facilitate multi-objective collaborative convergence and improve the overall quality of the augmented samples.

[0115] In specific implementation, the common features are input into the generator, and adversarial game training with the discriminator can be implemented in the following way: the common features between the domains after feature balancing are used as the decoder input of the generator. The decoder gradually restores the spatial resolution through upsampling and convolution, and outputs a candidate defect image with the same size as the input image. The candidate defect image and the real defect image are simultaneously fed into the discriminator, and the discriminator outputs two true and false probability scores. The adversarial game training adopts an alternating update method: in each iteration, the gradient calculation flag of all trainable parameters of the generator is first set to false, so that the parameters of the generator remain unchanged in this update. The adversarial loss is calculated only for the discriminator. Then, the backpropagation algorithm is called to calculate the gradient of the adversarial loss with respect to each parameter of the discriminator. Finally, the gradient of the adversarial loss with respect to each parameter of the discriminator is calculated. The adaptive moment estimation optimizer updates the discriminator parameters by subtracting the gradient of each parameter multiplied by a preset learning rate. Next, it sets the gradient calculation flag for all trainable parameters of the discriminator to false, calculates the joint loss for the generator, calls the backpropagation algorithm to calculate the gradient of the joint loss with respect to each generator parameter, and then uses the same adaptive moment estimation optimizer to update the generator parameters with the same learning rate (i.e., subtracting the gradient of each generator parameter multiplied by the learning rate). This process is repeated until the joint loss converges. Through the alternating optimization of the generator and discriminator, the generator continuously learns the distribution of real defects, while the discriminator provides refined gradient signals to guide the generator to avoid pattern collapse and improve generation quality, thereby eliminating the training instability caused by information imbalance.

[0116] It should be noted that the joint loss value in this application is a scalar calculated by instantiating the joint loss function, used to comprehensively reflect the realism of the generated image.

[0117] In some embodiments, adjusting the parameters during adversarial game training using the joint loss function to enable the generator to generate enhanced defect images can be achieved through the following steps:

[0118] Initialize the parameters in the generator before the adversarial game training begins;

[0119] In each training round, the common features are input into the generator to obtain candidate defect images, and the candidate defect images and the defect images are input into the discriminator for discrimination.

[0120] Based on the discrimination result of the discriminator and the candidate defect image, the joint loss function is instantiated and the joint loss value is calculated;

[0121] Calculate the gradient of the joint loss value with respect to each parameter in the generator, and then update the parameters of the generator based on the gradient;

[0122] Repeat the above process until the joint loss converges, and use the image output by the generator at this point as the enhanced defect image.

[0123] In specific implementation, the parameters of the generator can be initialized before the adversarial game training begins in the following way: the weight parameters of all convolutional layers, fully connected layers and residual blocks in the generator are randomly initialized using a Gaussian distribution with a mean of 0 and a standard deviation of 0.02; all bias parameters are initialized to 0; the shared weights of the embedding layer and the projection layer are also initialized in the same way; after initialization, the generator has basic feature extraction and generation capabilities, and the initialized parameters are used as the starting point for training.

[0124] In specific implementation, in each training round, the common features are input into the generator to obtain candidate defect images. The candidate defect images and the defect images are input into the discriminator for discrimination, which can be achieved in the following way: the common features are input into the decoder of the generator, and the decoder outputs candidate defect images of the same size as the defect images; the candidate defect images and the defect images are input into the discriminator in pairs; the discriminator outputs a realism score for each image, and the closer the score is to 1, the closer it is to the real defect.

[0125] In specific implementation, the joint loss function instantiated based on the discriminator's discrimination result and the candidate defect image, and the joint loss value calculated, can be achieved in the following way: The adversarial loss is calculated according to the definition of the joint loss function based on the discriminator's output scores for the candidate defect image and the real defect image; the physical constraint loss is calculated according to the definition of the joint loss function based on the difference between the candidate defect image and the physical model; the criticality loss is calculated according to the definition of the joint loss function based on the mean edge gradient of the candidate defect image; and the three are weighted and summed to obtain the joint loss value. The joint loss value reflects the realism, physical consistency, and boundary proximity of the generated image, and is used as the basis for optimization.

[0126] In specific implementation, calculating the gradient of the joint loss value with respect to each parameter in the generator, and then updating the generator parameters based on the gradient, can be achieved in the following way: The automatic differentiation function of the deep learning framework is invoked, and backpropagation is performed starting from the joint loss value to calculate the partial derivative of each trainable parameter in the generator, obtaining the gradient vector; an adaptive moment estimation optimizer is used to multiply the gradient by a preset learning rate and then add it to the existing parameters to complete one parameter update; the updated generator parameters will slightly reduce the joint loss value in the next round; this process is repeated to gradually converge the generator, and the updated parameters are used as the basis for the next round of training.

[0127] In practice, the above process is repeated until the joint loss converges. The image output by the generator at this point can be used as the enhanced defect image in the following way: repeat the above steps, referring to... Figure 3 As shown in the figure, this figure is a schematic diagram of adversarial game training in some embodiments of this application. In the figure, the joint loss value is judged every time it is calculated, and the joint loss value is recorded after each training round. When the change of the joint loss value is less than one-thousandth of the preset value for 10 consecutive rounds, it is determined that the model has converged. Training is stopped, and a batch of common features are input into the converged generator to obtain the corresponding output images. These output images are used as enhanced defect images for subsequent merging with the original defect images to obtain enhanced defect images.

[0128] It should be noted that the enhanced defect image in this application is the final synthetic defect sample output by the generator after the joint loss converges. It is used to expand the number of defect samples in the original training set, so that the downstream detection model can learn the classification boundary from more diverse, more realistic and more critical samples, thereby improving the detection performance.

[0129] In step 105, the enhanced defect image and the defect image are merged to obtain the enhanced defect dataset.

[0130] In some embodiments, merging the enhanced defect image and the defect image to obtain the enhanced defect dataset can be achieved by the following steps:

[0131] The enhanced defect images are collected in batches and mixed with the defect images according to a preset ratio.

[0132] The mixed dataset is deduplicated and verified. All the verified enhanced defect images are stored together with the defect images to obtain the enhanced defect dataset.

[0133] In specific implementation, the enhanced defect images are collected in batches and mixed with the original defect images according to a preset ratio. This can be achieved as follows: All generated enhanced defect images are stored in a temporary folder, with 256 images generated in each batch; the total number of original defect images is counted and set to N; the preset ratio is set to 1:1, meaning the number of enhanced defect images is equal to the number of original defect images; N images are randomly selected from the enhanced defect image set and merged with the original N defect images to form a dataset containing 2N images; this mixed dataset is used for subsequent training of downstream detection models; the mixed dataset is deduplicated, and all verified enhanced defect images are stored together with the original defect images to obtain the enhanced defect dataset. This can be achieved as follows: The structural similarity index between each enhanced defect image in the mixed dataset and all original defect images is calculated; if the similarity between an enhanced image and any original image exceeds 0.95, it is considered a duplicate and deleted; the remaining enhanced defect images after deduplication are stored together with the original defect images in a standard image folder format, with each image file accompanied by a tag file with the same name recording the defect type; the entire folder is output as the enhanced defect dataset.

[0134] Furthermore, in another aspect of this application, in some embodiments, this application provides an industrial defect data enhancement system based on generative adversarial networks, referencing... Figure 4 The figure is a schematic diagram of the structure of an industrial defect data augmentation system according to some embodiments of this application. The industrial defect data augmentation system includes: an acquisition module 401, a processing module 402, and an execution module 403, which are described below:

[0135] The acquisition module 401 in this application is mainly used to perform defect annotation and background separation on the surface image data of the acquired workpiece to obtain the defect image of the source domain, the target domain image and the defect type;

[0136] Processing module 402, in this application, is used by the generator to extract domain features of the defect image and the target domain image and align and map them to a shared semantic space;

[0137] It should be noted that the processing module 402 in this application is also used for the generator to calculate the mutual information response value between all aligned domain features through mutual information channel by channel, and to perform forgetting filtering on all mutual information response values ​​to obtain the common features between domains after feature balancing.

[0138] Additionally, it should be noted that the processing module 402 in this application is also used to construct a joint loss function under physical constraints based on the defect type, input the common features into the generator and the discriminator for adversarial game training, and adjust the parameters during adversarial game training through the joint loss function so that the generator can generate enhanced defect images.

[0139] The execution module 403 in this application is mainly used to merge the enhanced defect image and the defect image to obtain the enhanced defect dataset.

[0140] In addition, this application also provides a computer device, the computer device including a memory and a processor, the memory storing code, the processor being configured to acquire the code and execute the above-described industrial defect data augmentation method based on generative adversarial networks.

[0141] In some embodiments, reference Figure 5 The figure is a schematic diagram of the structure of a computer device implementing a generative adversarial network-based industrial defect data augmentation method according to some embodiments of this application. The generative adversarial network-based industrial defect data augmentation method in the above embodiments can... Figure 5 The computer device shown is used to implement this, and the computer device includes at least one processor 501, a communication bus 502, a memory 503, and at least one communication interface 504.

[0142] Processor 501 can be a general-purpose central processing unit (CPU) or an application-specific integrated circuit (ASIC).

[0143] The communication bus 502 can be used to transmit information between the aforementioned components.

[0144] Memory 503 may be a read-only memory (ROM) or other type of static storage device capable of storing static information and instructions, random access memory (RAM) or other type of dynamic storage device capable of storing information and instructions, or electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile optical discs, Blu-ray discs, etc.), magnetic disks or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto. Memory 503 may exist independently and be connected to processor 501 via communication bus 502. Memory 503 may also be integrated with processor 501.

[0145] The memory 503 stores program code for executing the scheme of this application, and its execution is controlled by the processor 501. The processor 501 executes the program code stored in the memory 503. The program code may include one or more software modules. The method used in the above embodiments can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0146] Communication interface 504 uses any transceiver-like device to communicate with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0147] In a specific implementation, as one example, a computer device may include multiple processors, each of which may be a single-core (single-CPU) processor or a multi-core (multi-CPU) processor. Here, a processor may refer to one or more devices, circuits, and / or processing cores used to process data (e.g., computer program instructions).

[0148] The aforementioned computer device can be a general-purpose computer device or a special-purpose computer device. In specific implementations, the computer device can be a desktop computer, a portable computer, a network server, a handheld digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device, or an embedded device. This application does not limit the type of computer device.

[0149] In addition, this application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described industrial defect data enhancement method based on generative adversarial networks.

[0150] Although preferred embodiments of this application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of this application.

[0151] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for augmenting industrial defect data based on generative adversarial networks, wherein, The generative adversarial network includes a generator and a discriminator, and the method is characterized by the following steps: The surface image data of the acquired workpiece is used to perform defect annotation and background separation to obtain the defect image of the source domain, the target domain image, and the defect type; The generator extracts the domain features of the defect image and the target domain image and aligns and maps them to a shared semantic space; The generator calculates the mutual information response values ​​between all aligned domain features channel by channel using mutual information, and performs forgetting filtering on all mutual information response values ​​to obtain the common features between domains after feature balancing. Based on the defect type, a joint loss function under physical constraints is constructed. The common features are input into the generator and the discriminator for adversarial game training. The parameters during adversarial game training are adjusted through the joint loss function so that the generator can generate enhanced defect images. The enhanced defect image and the defect image are merged to obtain the enhanced defect dataset; Specifically, constructing the joint loss function under physical constraints based on the aforementioned defect type includes: The physical model is called from the preset physical model library according to the defect type. The physical model library includes an elastic mechanical model for cracks, a geometric continuity model for scratches, and an optical reflection model for stains. The physical constraint loss is defined as the deviation between the candidate defect image generated by the generator and the physical model. The adversarial loss is defined as the difference in discrimination between the discriminator's judgment of the defect image in the source domain and the candidate defect image generated by the generator. The adversarial loss adopts the form of least squares loss plus gradient penalty. The mean gradient of the defect region edge is defined as the response index when the generator generates an image, and the degree to which the response index deviates from the classification boundary is used as the critical loss. The classification boundary is a preset threshold that distinguishes stable samples from unstable samples. The physical constraint loss, adversarial loss, and criticality loss are weighted to obtain the joint loss function.

2. The method as described in claim 1, characterized in that, The surface image data of the acquired workpiece is subjected to defect annotation and background separation to obtain defect images, target domain images, and defect types, specifically including: Surface image data of workpieces are acquired using industrial cameras deployed on the production line; Semantic segmentation is performed on the surface image data to obtain multiple defect regions and background regions; A surface image containing at least one defective region is used as a defective image, and a surface image that does not contain any defective regions is used as a target domain image. Based on the matching results of all defect areas and the industry's standard defect library, the corresponding defect types are marked.

3. The method as described in claim 1, characterized in that, The generator extracts domain features from the defect image and the target domain image and aligns and maps them to a shared semantic space, specifically including: The generator uses two domain encoders to convolve and downsample the defect image and the target domain image respectively to obtain the feature map of the source domain and the feature map of the target domain. Both the feature map of the source domain and the feature map of the target domain contain multiple channels; The feature maps of the source and target domains are input into the embedding layer with shared weights to obtain aligned defect embedding feature maps and target embedding feature maps. Extract the global semantic information of the defect embedding feature map and the target embedding feature map in each channel to obtain the corresponding semantic vector; The generator maps all semantic vectors through a projection layer with shared weights to obtain the domain features of the aligned defect image and the target domain image.

4. The method as described in claim 1, characterized in that, The generator calculates the mutual information response values ​​between all aligned domain features channel-by-channel using mutual information, specifically including: Align all the aligned domain features by channel, with each channel corresponding to a pair of feature activation values; For any channel, calculate the KL divergence between the feature activation values ​​of the source domain and the feature activation values ​​of the target domain, and use it as the mutual information response value of the corresponding channel.

5. The method as described in claim 1, characterized in that, After performing forgetting filtering on all mutual information response values, the common features between domains after feature balancing specifically include: The forgetting weight for the corresponding channel is determined based on all mutual information response values; Obtain the preset forgetting threshold; Set the feature maps of all channels whose forgetting weight is lower than the forgetting threshold to zero to remove them; The feature maps corresponding to all channels whose forgetting weight is greater than or equal to the forgetting threshold are retained. Multiply all retained feature maps by the forgetting weights of the corresponding channels one by one to obtain the common features between the domains after feature balancing.

6. The method as described in claim 1, characterized in that, Adjusting the parameters during adversarial game training using the joint loss function to enable the generator to generate enhanced defect images specifically includes: Initialize the parameters in the generator before the adversarial game training begins; In each training round, the common features are input into the generator to obtain candidate defect images, and the candidate defect images and the defect images are input into the discriminator for discrimination. The joint loss function is instantiated based on the discrimination results and candidate defect images, and the joint loss value is calculated. Calculate the gradient of the joint loss value with respect to each parameter in the generator, and then update the parameters of the generator based on the gradient; Repeat the above process until the joint loss converges, and use the image output by the generator at this point as the enhanced defect image.

7. An industrial defect data augmentation system based on generative adversarial networks, wherein the system employs the method described in any one of claims 1 to 6 to perform industrial defect data augmentation based on generative adversarial networks, characterized in that, include: The acquisition module is used to perform defect annotation and background separation on the surface image data of the acquired workpiece to obtain the defect image of the source domain, the target domain image, and the defect type; The processing module is used to extract the domain features of the defect image and the target domain image from the generator and align and map them to a shared semantic space; The processing module is also used by the generator to calculate the mutual information response value between all aligned domain features channel by channel using mutual information, and to perform forgetting filtering on all mutual information response values ​​to obtain the common features between domains after feature balancing. The processing module is further configured to construct a joint loss function under physical constraints based on the defect type, input the common features into the generator and discriminator for adversarial game training, and adjust the parameters during adversarial game training through the joint loss function so that the generator can generate enhanced defect images. The execution module is used to merge the enhanced defect image and the defect image to obtain the enhanced defect dataset.

8. A computer device, characterized in that, The computer device includes a memory and a processor, the memory storing code, and the processor being configured to retrieve the code and execute the industrial defect data augmentation method based on generative adversarial networks as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the industrial defect data augmentation method based on generative adversarial networks as described in any one of claims 1 to 6.

Images