Small sample slab defect data enhancement and identification method based on RADS model

Abstract

The present invention relates to the field of metallurgy and steelmaking technology, and aims to provide a method for enhancing and identifying small sample slab defect data based on a RADS model. The model includes an encoder module, a decoder module, and a discriminator module connected in sequence, and each module has a neural network structure composed of a convolutional layer and a connection layer; after the defect sample image is input into the RADS model, each module extracts the feature vector of the image in sequence and outputs it backward; in the output of the discriminator module, the RGB image generated based on the normal distribution noise will be encoded and decoded again and input into the discriminator module, in this way, a cyclic iteration is achieved to enhance the number of samples. The present invention can solve the defect that the number of samples of different categories in the neural network is inconsistent, which makes it difficult to converge together and thus requires repeated training. Only one training is needed to simultaneously obtain the relevant data enhancement model of samples of different categories; it can generate images and realize defect recognition, thereby improving the efficiency of slab defect recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of metallurgy and steelmaking, and in particular relates to a small sample slab defect data enhancement and recognition method based on a Reconstructing the Adversarial Denoising Self-Encoder (RADS) model. Background Art

[0002] Hot rolling is a key step in slab production and has rapidly developed due to its excellent dimensional and performance adaptability. Whether using traditional hot rolling or thin steel plate continuous casting and rolling, slab production can be subject to various defects, including cracks, inclusions, spots, mill scale, and scratches, caused by factors such as the raw materials, rolling process, and production environment. The presence of slab surface defects is a key indicator of product quality, and surface defects in various types of hot-rolled steel strip have a significant impact on product quality. Therefore, surface defect detection of hot-rolled steel strip is essential in modern industrial production.

[0003] Accurately detecting and identifying slab surface defects is an important prerequisite for improving slab quality, but detecting slab surface defects is not easy. First, the temperature in the slab production process is extremely high, and workers cannot get close to the slab to confirm the surface conformity. Therefore, workers can only use monitoring equipment to manually classify defects. If defects that meet the requirements are to be manually labeled and classified, it is necessary to search frame by frame in the 24-hour video recorded every day, which requires a lot of manpower and time. Due to the difficulty of manually labeling and classifying all samples, the actual number of defect samples in the smelter is too small. Secondly, during the hot rolling operation, the steam present on the slab at high temperature will cause uneven lighting. At the same time, there are many impurities in the air that interfere with defect detection, making the detection process more complicated.

[0004] Slab surface defect detection has long been a research hotspot in the manufacturing industry. In recent years, with the continuous improvement of automated inspection technology and the vigorous development of machine vision inspection and deep learning, research and application in the field of imaging has become an important branch. Combining digital images with deep learning algorithms provides a new means for slab surface defect detection. In actual production, sample collection takes a long time, the number of defects of different categories is uneven, the number of samples obtained is small, and manual data labeling is very laborious, making it difficult to collect a large number of labeled samples. As a result, the main problems faced in slab surface defect detection are the small number of samples and the difficulty of sample labeling; the inconsistent amount of defect data for different categories makes it impossible for the neural network to converge uniformly using a single objective function for defects of different categories; and the harsh environment has a significant impact on defect recognition.

[0005] Currently, there are two main image enhancement algorithms: VAE and GAN. Both algorithms have their own advantages and disadvantages: VAE cannot be parallelized, resulting in low efficiency and less realistic and blurry generated images; VAE's loss function is a reconstruction loss, which often deviates from the data manifold, resulting in blurry generated images. Furthermore, the KL divergence used in VAEs essentially only makes the encoder output as close to a normal distribution as possible and does not directly improve the quality of the generated image. The neural network in VAEs is only used to fit the assumed normal distribution, thus limiting its capabilities. GANs are highly efficient due to their parallel computing, but the training process is irreversible, and the discriminator can only distinguish "real" and "fake" images when evaluating generated images.

[0006] The loss function of GAN is a loss function that measures the loss between distributions. The generator uses a neural network to learn a mapping function from a normal distribution to a feature image. However, the overall network is not stable enough and often fails to converge the objective function. The loss functions of traditional data-enhanced neural networks use KL divergence or JS divergence to seek the reconstruction error between real and fake images. The better the discriminator, the more serious the generator gradient disappears; the generated model diversity is not enough, and the training will be more difficult. The disadvantage of KL divergence and JS divergence is that they are unbounded. When there are two probability distributions P and Q, when θ≠0, when the two probability distributions do not intersect, When θ = 0, P and Q completely overlap, D JS (P, Q) = 0, but when the two distributions do not intersect, D KL (P, Q) is infinite, D JS The (P, Q) values ​​​​are abrupt and non-differentiable. Therefore, KL divergence and JS divergence have their own shortcomings in calculating two disjoint probability distributions, and neither can provide a loss function for stable gradient descent.

[0007] Since there are many categories of actual slab defects and the number of samples in each category is not consistent, when the samples are input into the neural network, when the loss function of the sample with a large amount of data converges, it is often impossible to perform sufficient iterations and training on the sample images with a small amount of data. In addition, for neural network training of sample data with different proportions, people often choose a mode of training different samples separately, but for deeper neural networks, separate training means it will take longer. Therefore, a neural network model that can train multiple types and different numbers of samples at the same time is needed to improve training efficiency and reduce time costs. In addition, the different categories of defects in slabs in harsh environments are not similar to each other, and due to the harsh environment, factors such as water vapor and dust will cause great interference to the training of the neural network. Therefore, the objective function and loss function of the traditional data enhancement neural network cannot provide sufficiently real, clear and diverse images for slab defect detection in actual harsh environments. Therefore, the algorithm of the neural network model is required to be able to identify image samples after stably removing impurities. Summary of the Invention

[0008] The technical problem to be solved by the present invention is to overcome the deficiencies in the prior art and provide a small sample slab defect data enhancement and identification method based on a reconstruction adversarial denoising autoencoder (RADS) model.

[0009] To solve the technical problem, the solution of the present invention is:

[0010] A RADS model for small sample data enhancement and recognition is provided. The model includes an encoder module, a decoder module, and a discriminator module connected in sequence, each module having a neural network structure consisting of a convolutional layer and a connection layer. After a defect sample image is input into the RADS model, each module extracts the feature vector of the image in sequence and outputs it backward. The input of the encoder module includes the original sample image and a preprocessed RGB image without impurities and randomly erased. After the two are mixed Poisson distribution encoded, they are output to the decoder module. The input of the decoder module also includes standard normal distribution noise, on which an RGB image is generated. The input of the discriminator module includes the original sample image, the image obtained by mixed Poisson distribution encoding, the RGB image generated based on normal distribution noise, and the image repeatedly decoded and encoded after recognition. The output of the discriminator module includes the feature values ​​obtained by the one-dimensional Dense layer and the RGB image generated based on normal distribution noise. The latter will be returned to the encoder module, and after being processed again by the encoder module and decoder module, it becomes the image repeatedly decoded and encoded after recognition and input into the discriminator module, in this way, a cyclic iteration is achieved to enhance the number of samples.

[0011] As a preferred solution of the present invention, the Stein smoothing method is used in the encoder module to achieve smooth transition by applying the quality of the trigonometric function instead of the total variation distance;

[0012] The total loss function of the RADS model is:

[0013]

[0014] Where: L prior is the prior loss of the latent variable (feature) z; is the high-level feature loss, the feature is extracted in the discriminator, assuming it is in the first layer of the discriminator; L gan is the discriminator loss of the decoder.

[0015] As a preferred embodiment of the present invention, the neural networks in the encoder module, decoder module and discriminator module respectively have the following structural features: in the encoder module, first all use 4*4 convolution kernels for convolution operation, Relu function activation, each layer is 8, 16, 32, 64, 128 depth, and finally a 100-dimensional Dense layer is used to reduce the latitude, and a 0.2 Dropout layer is used to prevent overfitting; in the decoder module, first use an 8*8*128 Dense layer and use the Relu function to activate the generated data, and then all use 4 *4 convolution kernel for deconvolution operation, Relu function activation, each layer has a depth of 128, 64, 32, 16, and 8 respectively, and the last layer uses a 4*4*1 convolution kernel and a sigmoid activation function; in the discriminator module, first all use 8*8 convolution kernels for dimensionality reduction convolution operation, BNorm for batch normalization, Relu function activation, each layer has a depth of 16, 32, 64, and 128 respectively, then pass through the global average pooling layer, 256-dimensional Dense layer and use Relu function activation, and finally the 1-dimensional Dense layer feature evaluation is performed.

[0016] The present invention further provides a method for data enhancement and identification of small sample slab defects using the aforementioned RADS model, comprising the following steps:

[0017] (1) Construct the RADS model according to the above content;

[0018] (2) Pre-training the RADS model

[0019] (2.1) First, the defect sample image is preliminarily processed to remove impurity features caused by water vapor and dust. Then, a random erasure algorithm is used to obtain a random erased image area.

[0020] (2.2) Transmit and process image data in accordance with the above, including:

[0021] The encoder module converts the RGB image into a Poisson distribution in high dimensions. The decoder module obtains the discrete variables of the low-dimensional Poisson distribution and reconstructs the pure RGB image information through the neural network deconvolution layer. The discriminator simultaneously recognizes the original sample image, the image encoded by the mixed Poisson distribution, the RGB image generated based on normal distribution noise, and the image that has been repeatedly decoded and encoded after recognition.

[0022] (2.3) Feedback the recognition results to the neural network of each module to adjust the respective loss function and objective function, and distinguish the weights of different parts; use the Adam optimizer to optimize the convolutional layer of each module to obtain the optimized RADS model;

[0023] (3) Obtain slab monitoring video and preprocess the video image

[0024] Real-time video of steel slabs being produced is captured, and each frame is preprocessed by an image processing unit to adjust the image pixel size. This includes adjusting the image's width and height pixel values, stacking them into multiple long vectors in rows or columns, calculating the image gradient along the horizontal X-axis and vertical Y-axis, and synthesizing them into a two-dimensional vector. Vector masks or filters are then used to remove unnecessary parameters.

[0025] (4) Identifying defective slab monitoring images

[0026] The enhanced image is input into the pre-trained RADS model, and the discriminator module performs recognition and probability classification. By comparing it with existing data stored in the database, if the calculated probability of defects in the image is greater than the set threshold, the image's timestamp, defect location, and defect category are recorded in the database. If the probability is less than the set threshold, it is considered to be defect-free and no relevant information is recorded.

[0027] As a preferred embodiment of the present invention, in step (2), when the defect sample image is preliminarily processed, it also includes performing a geometric transformation on the image to reduce the position deviation existing in the training data; the geometric transformation specifically refers to processing the image by rotation, flipping or translation.

[0028] As a preferred solution of the present invention, in step (4), it further includes: transmitting the defect information obtained by the identifier module to the steelmaking secondary control system, and performing operations according to the preset process disposal plan.

[0029] The present invention further provides a data enhancement and identification system for small sample slab defects, which is characterized by comprising:

[0030] Slab monitoring probe, used to collect real-time images of slab samples during the conveying process;

[0031] An image processing unit, used for preliminarily processing sample images and controlling the operation of the slab monitoring module;

[0032] The communication control module is connected to the image processing unit, deep learning host and steelmaking secondary control system via Ethernet;

[0033] A deep learning host with the aforementioned RADS model built in, used to perform quantitative enhancement and recognition processing on small sample slab images;

[0034] The converter steelmaking secondary control system performs operations according to the preset process disposal plan based on the slab defect information received and identified.

[0035] Compared with the prior art, the present invention has the following beneficial effects:

[0036] 1. The RADS model provided by the present invention can improve the disadvantage of reduced recognition of slab defect features with small samples when using neural network training, provides a data enhancement model, and meets the sample size requirements of other neural networks;

[0037] 2. The present invention can solve the problem of difficulty in convergence due to inconsistent numbers of samples of different categories in the neural network, which requires repeated training. The RADS model can obtain relevant data enhancement models of samples of different categories at the same time after only one training. It can also obtain a sample generation module and a sample recognition module at the same time, which can generate images and realize defect recognition, thereby improving the efficiency of slab defect recognition.

[0038] 3. The RADS model of the present invention can generate samples on demand. It can generate relevant defect samples without impurities in harsh environments, and can also generate defect samples in different environments, which is convenient for different smelters to establish different sample sets.

[0039] 4. The RADS model of the present invention can be embedded in the video tracking and recognition algorithm at no cost, and the slab defect recognition is replicable and traceable with high security;

[0040] 5. The method of the present invention can reduce the number of operators for manual identification of slab defects, thereby reducing the labor intensity and operational risks of workers. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 Schematic diagram of the monitoring and identification system of the present invention.

[0042] The reference numerals in the figure are: 1 slab monitoring probe; 2 image processing unit; 3 communication control module; 4 deep learning host; 5 slab conveyor belt.

[0043] Figure 2 Schematic diagram of the network structure of the RADS model in the present invention.

[0044] In the figure: x includes the original sample image and the preprocessed RGB image without impurities and randomly erased; z is the mixed Poisson distribution obtained by encoding the original sample image and the image with impurities removed and randomly erased; z′ is the noise of the standard normal distribution; x2 includes the image encoded by the mixed Poisson distribution, the RGB image generated based on the normal distribution noise, and the image repeatedly decoded and encoded after recognition; x′ is the image repeatedly decoded and encoded after recognition, and the feature quantity is the feature value output by the 1D Dense layer. DETAILED DESCRIPTION

[0045] First, it should be noted that the present invention involves the application of deep learning neural network technology, an application of artificial intelligence technology in the field of industrial control. The applicant believes that after carefully reading the application documents and accurately understanding the implementation principles and objectives of the present invention, and in combination with existing known technologies, those skilled in the art can fully utilize their mastery of deep learning neural network technology to implement the present invention.

[0046] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0047] 1. System composition

[0048] like Figure 1 As shown, the present invention first provides a data enhancement and identification system for realizing small sample slab defects, comprising: a slab monitoring probe 1, for collecting slab sample images in the transmission process in real time. An image processing unit 2, for preliminarily processing the sample image, and controlling the operation of the slab monitoring module. A communication control module 3, connected to the image processing unit 2, a deep learning host 4 and a steelmaking secondary control system (not shown in the figure) via Ethernet. The deep learning host 4, having a built-in RADS model originally designed by the present invention, is used for quantitative enhancement and identification processing of small sample slab images. The steelmaking secondary control system performs operations according to a preset process disposal plan based on the slab defect information received and identified.

[0049] The hardware and control devices used in this invention are all existing equipment or technologies. For example, the converter steelmaking secondary control system is a general-purpose device (or system) that is widely used by large steelmaking enterprises at home and abroad. The slab monitoring probe 1 can be a combination of monitoring probes, such as a Hikvision DS-NXCN3A204 high-temperature camera, to monitor the slab in the visible light region, providing live data for the deep learning host 4 to analyze slab defect information and classification. The deep learning host 4 is a learning computer with integrated multi-GPU processors, such as the Dell T7920 dual-GPU server. The RADS model, implemented in built-in software, analyzes and processes image data captured by the slab monitoring module. Iterative training is performed by reconstructing an adversarial denoising autoencoder model to generate slab defect image data, providing image samples for subsequent training and recognition. The communication control module 3 connects to other components via Ethernet, enabling bidirectional communication of data and control signals. After transmitting the recognition results to the converter steelmaking secondary control system, it executes actions according to the preset treatment plan based on the defect situation.

[0050] 2. Construction of RADS model

[0051] The RADS model is composed of a combination of software functional modules, including an encoder module, a decoder module and a discriminator module connected in sequence. Each module has a neural network structure consisting of a convolutional layer and a connection layer. After the defect sample image is input into the RADS model, each module extracts the feature vector of the image in turn and outputs it backward.

[0052] The input of the encoder module includes the original sample image and the preprocessed RGB image without impurities and randomly erased. After encoding the two RGB images with a mixed Poisson distribution, the converted Poisson distribution in high latitude is output to the decoder module.

[0053] The decoder module takes the discrete variables of the mixed Poisson distribution and reconstructs the clean RGB image information through the neural network deconvolution layer. In addition, the module also inputs standard normal distribution noise and generates an RGB image based on this.

[0054] The inputs to the discriminator module include the original sample image, an image encoded using a mixed Poisson distribution, an RGB image generated using normal distribution noise, and an image that has been repeatedly decoded and encoded after recognition. The outputs of the discriminator module include the feature values ​​obtained from the one-dimensional Dense layer and an RGB image generated using normal distribution noise. The latter is returned to the encoder module, where it is processed again by the encoder and decoder modules to become an image that has been repeatedly decoded and encoded after recognition, and then input into the discriminator module. This loop iteration is implemented to increase the number of samples. Once the overall network of the RADS model converges, the discriminator can be embedded as a separate recognition module in a real-time video tracking algorithm.

[0055] In the design of the RADS model, we hope that the image reconstructed by the encoder is as close to the original image as possible, and at the same time, the distribution of P(Z`|X) is as close as possible to the distribution of P(Z) (using L1 or L2 distance, L1 is Manhattan distance, L2 is Euclidean distance), where P(Z) is a normal distribution. We hope that the image reconstructed by the decoder / generator is as close to the original image as possible and can deceive the discriminator, that is, the generated image can be judged as real by the discriminator (including reconstructed images and generated images). We hope that the discriminator can distinguish between real images and fake images (including reconstructed images and generated images). After training, the decoder module and discriminator module can be used directly to generate and recognize images, or the encoder module can be used to compress and reduce the dimensionality of samples.

[0056] In traditional data augmentation networks, the loss function of the encoder module mainly uses KL divergence or JS divergence to find the reconstruction error between real and fake images. However, the better the discriminator is, the more serious the generator gradient disappears, and the generated model diversity is insufficient, which makes training more difficult. As shown in Formula 1, D JS (p||q) represents the JS divergence, DKL represents the KL divergence. p and q are two different discrete distributions, and x is a data point in the discrete distribution. JS divergence measures the similarity between two probability distributions and has a value range of [0, 1]. Neural networks use the JS divergence loss function to measure the similarity between the Poisson distribution Pg generated by the encoder and the true sampling distribution Pr. Formula 2 shows the model loss function calculated using JS divergence.

[0057]

[0058] L(G,D * )=2D JS (pr||pg)-2log2 (2)

[0059] The present invention innovatively proposes to use the Stein smoothing method instead of the JS divergence in the encoder module of the RADS model.

[0060] As shown in Formula 3, Represents a discrete variable, x and y represent the two-dimensional value of the point, y belongs to the data value between 0 and 1, assuming there are two probability distributions P and Q, when θ≠0, that is, when the two probability distributions do not intersect, When θ = 0, that is, P and Q completely overlap, D JS (P, Q) = 0; but when the two distributions do not intersect, D KL (P, Q) is infinite, D JS (P, Q) value mutation, non-differentiable. D KL (P, Q) represents the KL divergence between P and Q, D JS (P, Q) represents the JS divergence between P and Q. The Stein smoothing method provides a smoothing metric that is very effective for stabilizing the learning process of gradient descent in machine learning.

[0061]

[0062]

[0063] The RADS model adopts the Stein method and can achieve smooth transition by applying the quality of trigonometric functions instead of the total variation distance. As shown in formula (4), z represents all possible joint probability distributions of pr and pg, sup represents the supremum, E represents the expected value, and d rV (pr, pg) represents the Stein method distance between pr and pg, P(pr∈A) represents the probability that pr belongs to the set A, and f(pr) and f(pg) represent the distribution of pr and pg with probability less than 1. After topological transformation of formula (4), we get formula (5), where the encoder is defined as f w, K is a constant threshold, W(pr, pg) is the Stein method distance, x~Pr represents a point x within Pr, and θ is the difference variable between the two discrete distributions. Formula (5) is used to optimize the encoder parameters and calculate the Stein smoothing method between the true distribution and the generated distribution. The encoder minimizes Formula (5) to optimize the generator parameters, and the loss function L(θ) is Formula (6).

[0064]

[0065]

[0066] L(θ)=-E x～Pg [f w (x)] (6)

[0067] After using the Stein smoothing method instead of KL divergence and JS divergence, the images generated by the RADS model are clearer and more realistic than those generated by traditional data augmentation networks, the gradient descent is more reasonable, and the generated images are more diverse.

[0068] As shown in formula (7), the total loss function of the RADS model is:

[0069]

[0070] L prior is the prior loss of the latent variable (feature) z, where x is the input original image and the preprocessed image, x r is the reconstructed image, z r is the Poisson distribution of the reconstructed image, f w Represents the encoder:

[0071]

[0072] It is the "high-level feature" loss. The features are extracted in the discriminator. Assume that in the first layer of the discriminator, x is the original image and the preprocessed image, z is the mixed Poisson distribution based on the original image and the preprocessed image, x r To reconstruct the image, z r is the mixed Poisson distribution of the reconstructed image, E is the expected value, and Dis represents the eigenvalue generated by the discriminator:

[0073]

[0074] The characteristics are assumed to conform to the Poisson distribution, where is the image generated by the standard normal distribution, x is the original image and the preprocessed image, z is the mixed Poisson distribution based on the original image and the preprocessed image, Dis represents the eigenvalue generated by the discriminator, Represents the discrete distribution Dis using Stein's method l (x) and The expected value at the shortest distance, I represents the variance:

[0075]

[0076] L gan is the discriminator loss of the decoder, where X rp To reconstruct the Poisson distribution of the image, Dis represents the eigenvalue generated by the discriminator, Gen represents the activation value of the generator, z is a mixed Poisson distribution based on the original image and the preprocessed image, and x is the original image and the preprocessed image:

[0077] L gan =log(Dis(x))+log(1-Dis(Gen(z)))+log(1-Dis(Gen(X rp ))) (11)

[0078] Table 1 RADS model algorithm flow

[0079]

[0080] Table 1 shows the algorithm flow of the RADS model. During the training process, the encoder is trained to minimize the Kullback-Leibler divergence latent loss and the mean square error (MSE) reconstruction error. The generator is trained to minimize the Kullback-Leibler divergence latent loss, the MSE reconstruction error and -1*log(X p ), the discriminator is trained to minimize

[0081] The network structure of the RADS model is shown in Table 2. In the RADS model, each of the three modules has a neural network structure consisting of convolutional layers and convolutional layers. In the encoder, convolution operations are performed using 4*4 convolutional kernels and activated by the Relu function. Each layer has a depth of 8, 16, 32, 64, and 128, respectively. Finally, a 100-dimensional Dense layer is used to reduce the dimensionality, and a 0.2-degree Dropout layer is used to prevent overfitting. In the decoder, an 8*8*128 Dense layer is first used and activated by the Relu function to generate data. Deconvolution operations are then performed using 4*4 convolutional kernels and activated by the Relu function. Each layer has a depth of 128, 64, 32, 16, and 8, respectively. The final layer uses a 4*4*1 convolutional kernel and a sigmoid activation function. In the discriminator module, first all 8*8 convolution kernels are used for dimensionality reduction convolution operations, BNorm is used for batch normalization, and the Relu function is activated. The depth of each layer is 16, 32, 64, and 128 respectively. Then it goes through the global average pooling layer, the 256-dimensional Dense layer and uses the Relu function for activation, and finally the 1-dimensional Dense layer feature evaluation is performed.

[0082] Table 2 Network structure design of RADS model

[0083]

[0084] 3. Pre-training of RADS model

[0085] The example RADS model training process is as follows:

[0086] First, the RADS model is constructed according to the above design; the encoder, decoder and discriminator modules are used to extract feature vectors, and the prediction values ​​are obtained through the connection layer; sample images are used for pre-training, the objective function and loss function are adjusted, and the weights of different parts are distinguished; then the Adam optimizer is used to optimize the convolutional layers of the three modules to obtain the optimal convolutional layer parameters, and each neural network is optimized so that it finally meets the preset conditions.

[0087] Before training, the sample images need preliminary processing to remove impurities caused by water vapor and dust. Because geometric transformations of images are easy to implement and can effectively overcome positional deviations in the training data, the images are further processed using denoising, rotation, flipping, and translation. A random erase algorithm is then used to obtain randomly erased image regions.

[0088] The random erasing algorithm described above is shown in Table 3 and is used to reduce noise and outliers in the dataset before training the model.

[0089] Table 3 Random erase algorithm

[0090]

[0091] After processing, the combination x of the impurity-free and randomly erased RGB image and the original sample image is input into the encoder of the model for training.

[0092] During the training process, in addition to transmitting and processing image data according to the description of the aforementioned RADS model, it also includes: the encoder module converts the RGB image into a Poisson distribution in high latitude; the decoder module obtains the discrete variables of the low-dimensional Poisson distribution and reconstructs the impurity-free RGB image information through the neural network deconvolution layer; the discriminator simultaneously recognizes the original sample image, the image encoded by the mixed Poisson distribution, the RGB image generated based on the normal distribution noise, and the image that has been repeatedly decoded and encoded after recognition; the recognition results are fed back to the neural network of each module to adjust their respective loss functions and objective functions, and the weights of different parts are distinguished; and the convolution layer of each module is optimized using the Adam optimizer to obtain the optimized RADS model.

[0093] 4. Practical application of the RADS model

[0094] Based on the construction and training of the aforementioned system and RADS model, small sample slab defect data enhancement and identification can be achieved in actual production. The method includes the following steps:

[0095] (1) Obtain slab monitoring video and preprocess the video image

[0096] The video of the produced steel slab is acquired in real time, and the image processing unit pre-processes each frame of the image to adjust the image pixel size for subsequent recognition.

[0097] The specific contents of preprocessing include: adjusting the width and height pixel values ​​of the image, stacking them into multiple long vectors by rows or columns; calculating the differences along the horizontal X-axis and vertical Y-axis in the image to calculate the image gradient, synthesizing them into a two-dimensional vector, and then using a vector mask or filter to remove unnecessary parameters;

[0098] When the RADS model is actually used, it is only necessary to perform pixel enhancement processing on the video image to meet the recognition requirements. There is no need to perform operations such as removing image impurities and randomly erasing local areas according to the model pre-training process.

[0099] (2) Identifying defective slab monitoring images

[0100] The enhanced image is input into the pre-trained RADS model, and the discriminator module performs recognition and probability classification. By comparing it with existing data stored in the database, if the calculated probability of defects in the image is greater than the set threshold, the image's timestamp, defect location, and defect category are recorded in the database. If the probability is less than the set threshold, it is considered to be defect-free and no relevant information is recorded.

[0101] 5. Specific examples of practical applications

[0102] (1) After the slab generated in the harsh field environment enters the conveyor belt, the defect detection control slab monitoring probe starts working;

[0103] (2) After the image processing unit obtains the sample image of the slab, it preprocesses the image information and then sends it to the deep learning host 4. The deep learning host 4 uses the pre-trained RADS model to combine the discriminator module with the video tracking algorithm to track and identify the slab in the video image, analyze whether the slab has defects, and determine the classification of the defects;

[0104] (3) When the deep learning host 4 determines that the defective image of the current slab is an abnormal target (the slab has defects), the corresponding defective image data is input as a new sample into the neural network learning model for iterative training to improve the subsequent recognition accuracy and wait for it to be used when the next slab is generated.

[0105] 6. Summary

[0106] In the present invention, the deep learning host 4 is mainly used for the machine vision part. By analyzing and processing the data of videos containing different categories of specific state features in harsh environments, the RADS model can autonomously identify and mark the specified target or state. By directly transferring parameters between modules within the RADS model, the purpose of reducing unnecessary manual operations is achieved. After the RADS model generates a certain number of required (including impurities and no impurities) target samples and continuously accumulates samples of different forms to complete the construction of the neural network, the target recognition accuracy is as high as more than 97%. At the same time, during the operation of the system, it is also possible to continuously collect new target samples, continue to iterate the model training, optimize the neural network, and improve the accuracy of target state recognition.

[0107] Taking the judgment of whether there are defects in slabs in harsh environments as an example, during the slab transmission process, the steel-outlet video SPV transmitted back in real time from the slab monitoring module is used as input information. The video tracking and recognition subsystem MIS of the discriminator module embedded in the RADS model calculates the defect confidence of the area where the pixel value function changes rapidly by generating recognition boxes of different sizes, and can automatically mark the defect location of the slab.

[0108] In reality, the smelting environment, lighting, and air quality of different smelters are all different, so different smelters often do not have more targeted recognition algorithms. The general defect recognition network has obvious differences in defect recognition in different environments. Therefore, smelters have a great demand to establish their own defect sample sets for smelting operations. However, due to the harsh environment of smelting itself, a large number of defect sample sets cannot be generated quickly. Since the system and method proposed in the present invention can be well applied to the harsh environment of converter steelmaking, it is of great significance for slab defect sample data enhancement and identification, evaluation and improvement of slab quality; it can not only improve the errors caused by manual defect identification, but also solve the overfitting problem caused by small samples in deep learning. In addition, this technology can not only classify the relevant types according to the defects and then optimize the steel production process, but also perform fault diagnosis on related equipment in the factory according to the defect type. Therefore, this technology is also of great significance for the realization of smart factories,

[0109] Finally, it should be noted that the above examples are merely specific embodiments of the present invention. Obviously, the present invention is not limited to the above embodiments and is subject to numerous variations. All variations that can be directly derived or conceived by a person of ordinary skill in the art from the disclosure of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A method for constructing a RADS model for small sample data enhancement and recognition, characterized in that: The RADS model is formed by connecting the encoder module, decoder module and discriminator module in sequence. Each module has a neural network structure composed of a convolutional layer and a connection layer. After the defect sample image is input into the RADS model, each module extracts the feature vector of the image in sequence and outputs it backward. The input of the encoder module includes the original sample image and the preprocessed RGB image without impurities and randomly erased; after encoding the two with mixed Poisson distribution, the output is sent to the decoder module; The input of the decoder module also includes standard normally distributed noise, and RGB images are generated based on this; The input of the discriminator module includes the original sample image, the image encoded by the mixed Poisson distribution, the RGB image generated based on the normal distribution noise, and the image that has been repeatedly decoded and encoded after identification; the output of the discriminator module includes the feature values ​​obtained by the one-dimensional Dense layer, and the RGB image generated based on the normal distribution noise; the latter will be returned to the encoder module, and after being processed again by the encoder module and the decoder module, it becomes the image that has been repeatedly decoded and encoded after identification and input into the discriminator module, in this way, a cyclic iteration is realized to increase the number of samples.

2. The method according to claim 1, characterized in that Using the Stein smoothing method in the encoder module to achieve smooth transition by applying the quality of the trigonometric function instead of the total variation distance; The total loss function of the RADS model is: Where: L prior is the prior loss of the latent variable feature z; is the high-level feature loss, the feature is Extraction within the discriminator, assuming it is in the lth layer of the discriminator; L gan is the discriminator loss of the decoder.

3. The method according to claim 1, characterized in that The neural networks in the encoder module, decoder module and discriminator module respectively have the following structural features: In the encoder module, convolution operations are first performed using 4*4 convolution kernels and ReLU activation functions. The depth of each layer is 8, 16, 32, 64, and 128 respectively. Finally, a 100-dimensional Dense layer is used to reduce the latitude, and a 0.2-degree Dropout layer is used to prevent overfitting. In the decoder module, the 8*8*128 Dense layer is first used and the Relu function is used to activate the generated data. Then all 4*4 convolution kernels are used for deconvolution operations and the Relu function is activated. The depths of each layer are 128, 64, 32, 16, and 8 respectively. The last layer uses a 4*4*1 convolution kernel and a sigmoid activation function. In the discriminator module, first all 8*8 convolution kernels are used for dimensionality reduction convolution operations, BNorm is used for batch normalization, and the Relu function is activated. The depth of each layer is 16, 32, 64, and 128 respectively. Then it goes through the global average pooling layer, the 256-dimensional Dense layer and uses the Relu function for activation, and finally the 1-dimensional Dense layer feature evaluation is performed.

4. A method for enhancing and identifying small sample slab defect data based on the RADS model constructed by the method of claim 1, characterized in that: The following steps are involved: (1) constructing a RADS model according to the method of claim 1; (2) Pre-training the RADS model (2.1) First, the defect sample image is preliminarily processed to remove impurity features caused by water vapor and dust. Then, a random erasure algorithm is used to obtain a random erased image area. (2.2) Transmitting and processing image data according to claim 1, including: The encoder module converts the RGB image into a Poisson distribution in high dimensions. The decoder module obtains the discrete variables of the low-dimensional Poisson distribution and reconstructs the pure RGB image information through the neural network deconvolution layer. The discriminator simultaneously recognizes the original sample image, the image encoded by the mixed Poisson distribution, the RGB image generated based on normal distribution noise, and the image that has been repeatedly decoded and encoded after recognition. (2.3) Feedback the recognition results to the neural network of each module to adjust the respective loss function and objective function, and distinguish the weights of different parts; use the Adam optimizer to optimize the convolutional layer of each module to obtain the optimized RADS model; (3) Obtain slab monitoring video and preprocess the video image Real-time video of steel slabs being produced is captured, and each frame is preprocessed by an image processing unit to adjust the image pixel size. This includes adjusting the image's width and height pixel values, stacking them into multiple long vectors in rows or columns, calculating the image gradient along the horizontal X-axis and vertical Y-axis, and synthesizing them into a two-dimensional vector. Vector masks or filters are then used to remove unnecessary parameters. (4) Identifying defective slab monitoring images The enhanced image is input into the pre-trained RADS model, and the discriminator module performs recognition and probability classification. By comparing it with existing data stored in the database, if the calculated probability of defects in the image is greater than the set threshold, the image's timestamp, defect location, and defect category are recorded in the database. If the probability is less than the set threshold, it is considered to be defect-free and no relevant information is recorded.

5. The method according to claim 4, characterized in that In the step (2), when the defect sample image is preliminarily processed, the image is also geometrically transformed to reduce the position deviation existing in the training data; the geometric transformation specifically refers to processing the image by rotation, flipping or translation.

6. The method according to claim 4, characterized in that In step (4), it further includes: transmitting the defect information obtained by the identifier module to the steelmaking secondary control system, and performing operations according to the preset process disposal plan.

7. A data enhancement and identification system for small sample slab defects, characterized by: include: Slab monitoring probe, used to collect real-time images of slab samples during the conveying process; An image processing unit, used for preliminarily processing sample images and controlling the operation of the slab monitoring module; The communication control module is connected to the image processing unit, deep learning host and steelmaking secondary control system via Ethernet; A deep learning host, having a built-in RADS model constructed by the method of claim 1, for performing quantitative enhancement and recognition processing on a small sample of slab images; The converter steelmaking secondary control system performs operations according to the preset process disposal plan based on the slab defect information received and identified.

Images