Intelligent visual monitoring method for oily water of iron and steel enterprises

Abstract

The invention discloses an intelligent visual monitoring method for oily water of iron and steel enterprises, and belongs to the field of oil stain detection. The method comprises the following steps: S1, acquiring an initial water surface image, generating abnormity for non-abnormity in the initial water surface image, labeling abnormity in the initial water surface image, and obtaining a final water surface image; s2, an image reconstructor is constructed, and the purpose is to reconstruct a greasy dirt part in the image with greasy dirt into a part without greasy dirt; and S3, constructing an image discriminator for discriminating the difference degree between the input image and the output image of the reconstructor, and setting a detection index to represent the occurrence probability of image sewage. According to the method, the oil stain is regarded as an anomaly of the image, that is, the abnormal oil stain appears in the normal water surface, so that proxy task image reconstruction is introduced, that is, how to reconstruct the abnormal image into the normal image, and then the abnormal part, that is, the oil stain, is found out by comparing the difference between the abnormal image and the normal image.

Description

Technical Field

[0001] The invention relates to oil pollution detection, and in particular to an intelligent visual monitoring method for oily wastewater in a steel enterprise. Background Art

[0002] In fact, the amount of wastewater discharged from steel mills is not much, but this kind of wastewater is highly polluting to the environment, and the discharge of wastewater also indicates that there are problems in the production process that lead to wastewater generation, so it is very important to detect whether wastewater is discharged for steel production. The small amount of wastewater discharged makes this work unsuitable for manual completion. Not only does it require high labor costs, but manual detection of wastewater discharge may also result in a series of problems such as false positives and false negatives. Therefore, it is best to use visual methods for oil pollution detection, but since wastewater occurs relatively rarely, the available training samples are very limited, and the existing visual methods are not suitable for small sample steel mill wastewater detection in the current environment, with many false positives and false negatives, and the overall accuracy is not high. Summary of the invention

[0003] Purpose of the invention: The purpose of the present invention is to provide an intelligent visual monitoring method for oily wastewater in a steel enterprise.

[0004] Technical solution: An intelligent visual monitoring method for oily wastewater in a steel enterprise, comprising the following steps:

[0005] S1. Obtain an initial water surface image, generate anomalies for those without anomalies in the initial water surface image, label those with anomalies, and obtain a final water surface image;

[0006] S2, construct an image reconstructor, the purpose of which is to reconstruct the oily part of the oily image into the part without oily part;

[0007] S3, construct an image discriminator, the purpose of which is to identify the difference between the input and output images of the reconstructor, and set a detection index to indicate the probability of sewage appearing in the image;

[0008] S4, connect the two networks in steps S2 and S3 in series for training;

[0009] S5. Input the data to be detected into the water surface oil pollution detection model to obtain the water surface oil pollution detection result.

[0010] Preferably, step S1 is specifically implemented as follows:

[0011] S11, acquiring real-time video data of the water surface, and performing frame interception on the real-time video data to obtain an initial water surface image;

[0012] S12. For images with oil stains on the water surface, these images can be labeled and divided into two parts, one for training and the other for testing;

[0013] S13. For images without oil stains, directly use the anomaly generation module to generate anomalies and labels for them, and superimpose the real anomaly data used for training in the previous step and the generated anomaly data in this step together as a training data set;

[0014] S14, performing offline image enhancement on the water surface image and the corresponding label file together, so that the trained model has higher robustness;

[0015] S15. The real sewage data used for testing is used as a test data set for testing.

[0016] Preferably, in step S14, the enhancement methods used include: affine scaling, brightness adjustment, horizontal flipping, simulated fog generation, Gaussian noise generation, rotation, and shearing, and a sewage data set is obtained after the enhancement is completed.

[0017] Preferably, the implementation method of step S2 is as follows: the image reconstructor is a U-Net type network, a self-attention mechanism is added to the convolutional network in the U-Net, and in the final output image link, the initial image is added to the final output image, and finally, a reconstructed, sewage-free image is obtained.

[0018] Preferably, the self-attention mechanism is a local attention mechanism, and the self-attention calculation is performed on non-overlapping windows, setting a two-dimensional mapping as X∈R C×H×W , H and W are the height and width of the mapping respectively; split X into non-overlapping windows of size M×M, assuming that the number of heads is K and the head dimension is dK=C / k; then calculate the self-attention of the kth head in the non-overlapping window can be formulated as follows:

[0019] X={X 1 ,X 2 ,...,X N},N=HW / M 2

[0020]

[0021] in, The projection matrices representing the query, key, and value of the k-th head respectively; X k is the output of the kth head; N is the number of feature blocks; Attention is the attention mechanism used to calculate weights and aggregate input features; is the output representation of the i-th feature block calculated by the k-th attention head; is the output representation of the i-th feature block calculated by the k-th attention head;

[0022] We also apply relative position encoding to the attention module, so the attention calculation can be formulated as:

[0023]

[0024] Where Q, K, and V represent the input matrices of query, key, and value, respectively; is a scaling factor that prevents the inner product value from being too large and helps stabilize the gradient; B is an optional additive bias term used to adjust the similarity matrix to enhance feature representation. SoftMax normalizes the similarity matrix and generates attention weights so that the sum of the weights of each row is 1; V is the final value matrix, which is multiplied by the attention weight to obtain the output of the attention mechanism.

[0025] Preferably, the implementation method of step S3 is as follows: the image discriminator is a U-Net type network, and the image input to the network is the input image to be detected and the reconstructed, sewage-free image; the purpose of the network is to extract the features between the two input images, and finally obtain a feature map, which represents the probability that the corresponding pixel points of the input image are sewage. By setting a suitable indicator, a good detection accuracy can be obtained through the feature map.

[0026] Preferably, the implementation method of step S4 is as follows: when training the image reconstructor, two loss functions, L2 loss and SSIMloss, are used; after the reconstruction is completed, the input image and the output image are cascaded and sent to the image discriminator together, and the image discriminator is further trained, and the image discriminator uses Focal Loss as the loss function; after the loss functions are calculated, the network gradient is back-propagated, and the above steps are repeated to complete the network training, and a suitable network model is obtained, and then the model is evaluated using the test data set; wherein:

[0027]

[0028] Where N is the number of samples, x i is the true value, y i is the predicted value;

[0029]

[0030] Among them, y and are the real image and the predicted image, μ y and is the mean of the real image and the predicted image; and is the variance between the real image and the predicted image; is the covariance between the real image and the predicted image; C1 and C2 are constants used for stable calculation, usually C1 = (0.01 × L) 2 、C1=(0.03×L) 2, L is the maximum value of the pixel value range (for example, for an 8-bit image, L = 255);

[0031]

[0032] Where N is the number of samples; α is the balance factor, which is used to adjust the impact of positive and negative samples; is the predicted probability of the i-th sample (for the correct category); γ is an adjustment factor used to adjust the influence of difficult and easy samples. Usually α is set to 0.25 and γ is set to 2.0.

[0033] Beneficial effects: The present invention regards oil pollution as an abnormality of the image, that is, abnormal oil pollution appears in the normal water surface, and thus introduces the proxy task image reconstruction, that is, how to reconstruct the abnormal image into a normal image, and then find out the abnormal part, that is, the oil pollution, by comparing the difference between the two. In addition, in order to solve the problem of fewer oil pollution samples, the present invention introduces an unsupervised learning framework, that is, constructing an anomaly generation module to generate anomalies in normal water surfaces. The form of this anomaly is not important, because the purpose is to know the characteristic distribution of normal samples. In this way, no matter what form of anomaly appears, it can be accurately detected. After combining these two parts, the model can achieve higher accuracy with small samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 It is a schematic flow chart of the oil pollution monitoring method of the present invention. DETAILED DESCRIPTION

[0035] In order to make the technical solution of the present invention clearer, the present invention is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0036] like Figure 1 As shown, an intelligent visual monitoring method for oily wastewater in a steel enterprise comprises the following steps:

[0037] (1) Obtain real-time video data of the water surface and perform frame capture on the real-time video data to obtain an initial water surface image; for images with oil stains on the water surface, label these images and divide them into two parts, one for training and the other for testing; for images without oil stains, directly use the anomaly generation module to generate anomalies and labels for them, and superimpose the real anomaly data used for training in the previous step and the generated anomaly data in this step together as a training data set; perform offline image enhancement on the water surface image and the corresponding label file, and the enhancement methods include: affine scaling, brightness adjustment, horizontal flipping, simulated fog generation, Gaussian noise generation, rotation, and shearing, so that the trained model has higher robustness; the real sewage data used for testing is used as a test data set for testing.

[0038] (2) Build an image reconstructor, which is a U-Net type network. Its purpose is to reconstruct the oily part of the oily image into the oil-free part. Due to the complex detection environment, we added a self-attention mechanism to the convolutional network in the U-Net, and added the initial image to the final output image in the final output image link. This is equivalent to adding more parameters to the network, which can make the reconstructed image closer to the input image. Finally, a reconstructed image without sewage is obtained.

[0039] The self-attention mechanism is a local attention mechanism, which only performs self-attention calculations on non-overlapping windows and sets a two-dimensional mapping as X∈R C×H×W (X is 256), H and W are the height and width of the mapping respectively; split X into non-overlapping windows of size M×M (M is 8), assuming that the number of heads is K and the head dimension is dK=C / k; then calculate the self-attention of the kth head in the non-overlapping window can be formulated as follows:

[0040] X={X 1 ,X 2 ,...,X N},N=HW / M 2

[0041]

[0042] The projection matrices representing the query, key, and value of the k-th head respectively; X k is the output of the kth head; N is the number of feature blocks; Attention is the attention mechanism used to calculate weights and aggregate input features; is the output representation of the i-th feature block calculated by the k-th attention head; is the output representation of the i-th feature block calculated by the k-th attention head; then, the outputs of all heads {1,2,...,k} are concatenated and then linearly projected to get the final result;

[0043] We also apply relative position encoding to the attention module, so the attention calculation can be formulated as:

[0044]

[0045] Where, Q, K, and V represent the input matrices of query, key, and value, respectively; is a scaling factor that prevents the inner product value from being too large and helps stabilize the gradient; B is an optional additive bias term (such as position encoding) used to adjust the similarity matrix to enhance feature representation. SoftMax normalizes the similarity matrix and generates attention weights so that the sum of the weights of each row is 1; V is the final value matrix, which is multiplied by the attention weight to obtain the output of the attention mechanism.

[0046] (3) Construct an image discriminator, which is also a U-Net type network. Its purpose is to identify the difference between the input and output images of the reconstructor and set a detection index to indicate the probability of sewage in the image. The images input to the network are the input image to be detected and the reconstructed image without sewage. The purpose of the network is to extract the features between the two input images and finally obtain a feature map. This feature map represents the probability that the corresponding pixel point of the input image is sewage. By setting a suitable index, a good detection accuracy can be obtained through the feature map.

[0047] (4) Connect the two networks in steps (2) and (3) in series for training: Two loss functions are used to train the image reconstructor: L2 loss and SSIM loss. The former is a commonly used loss function for image reconstruction, and the latter is used to ensure the similarity between the reconstructed image and the input image. After the reconstruction is completed, the input image and the output image are cascaded and sent to the image discriminator to continue training the image discriminator. The image discriminator uses Focal Loss as the loss function, which can improve the robustness of the discriminator. After the loss functions are calculated, the network gradient is back-propagated. Repeat the above steps to complete the network training and obtain a suitable network model. The model will then be evaluated using a test dataset. Among them:

[0048]

[0049] Where N is the number of samples, x i is the true value, y i For the predicted value

[0050]

[0051] Among them, y and are the real image and the predicted image, μ y and is the mean of the real image and the predicted image; and is the variance between the real image and the predicted image; is the covariance between the real image and the predicted image; C1 and C2 are constants used for stable calculation, usually C1 = (0.01 × L) 2 、C1=(0.03×L)2 , L is the maximum value of the pixel value range.

[0052]

[0053] Where N is the number of samples; α is the balance factor, which is used to adjust the impact of positive and negative samples; is the predicted probability of the i-th sample; γ is the adjustment factor used to adjust the influence of difficult and easy samples. Usually α is set to 0.25 and γ is set to 2.0.

[0054] (5) Input the data to be detected into the water surface oil pollution detection model to obtain the water surface oil pollution detection results.

[0055] The above-mentioned embodiments only express several implementation methods of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the patent of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. An intelligent visual monitoring method for oily wastewater in a steel enterprise, characterized in that: The following steps are involved: S1. Obtain an initial water surface image, generate anomalies for those without anomalies in the initial water surface image, label those with anomalies, and obtain a final water surface image; S2, construct an image reconstructor, the purpose of which is to reconstruct the oily part of the oily image into the part without oily part; S3, construct an image discriminator, the purpose of which is to identify the difference between the input and output images of the reconstructor, and set a detection index to indicate the probability of sewage appearing in the image; S4, connect the two networks in steps S2 and S3 in series for training; S5. Input the data to be detected into the water surface oil pollution detection model to obtain the water surface oil pollution detection result.

2. The intelligent visual monitoring method for oily wastewater in a steel enterprise according to claim 1 is characterized in that: The specific implementation of step S1 is as follows: S11, acquiring real-time video data of the water surface, and performing frame interception on the real-time video data to obtain an initial water surface image; S12. For images with oil stains on the water surface, these images can be labeled and divided into two parts, one for training and the other for testing; S13. For images without oil stains, directly use the anomaly generation module to generate anomalies and labels for them, and superimpose the real anomaly data used for training in the previous step and the generated anomaly data in this step together as a training data set; S14, performing offline image enhancement on the water surface image and the corresponding label file together, so that the trained model has higher robustness; S15. The real sewage data used for testing is used as a test data set for testing.

3. The intelligent visual monitoring method for oily wastewater in steel enterprises according to claim 2 is characterized in that: In step S14, the enhancement methods used include: affine scaling, brightness adjustment, horizontal flipping, simulated fog generation, Gaussian noise generation, rotation, and shearing. After the enhancement is completed, the sewage data set is obtained.

4. The intelligent visual monitoring method for oily wastewater in a steel enterprise according to claim 1 is characterized in that: The implementation method of step S2 is as follows: the image reconstructor is a U-Net type network, a self-attention mechanism is added to the convolutional network in the U-Net, and in the final output image link, the initial image is added to the final output image, and finally, a reconstructed, non-polluted image is obtained.

5. The intelligent visual monitoring method for oily wastewater in a steel enterprise according to claim 4 is characterized in that: The self-attention mechanism is a local attention mechanism, which performs self-attention calculations on non-overlapping windows and sets a two-dimensional mapping as X∈R C×H×W , H and W are the height and width of the mapping respectively; split X into non-overlapping windows of size M×M, assuming that the number of heads is K and the head dimension is dK=C / k; then calculate the self-attention of the kth head in the non-overlapping window can be formulated as follows: X={X 1 ,X 2 ,...,X N },N=HW / M 2 in, The projection matrices representing the query, key, and value of the k-th head respectively; X k is the output of the kth head; N is the number of feature blocks; Attention is the attention mechanism used to calculate weights and aggregate input features; is the output representation of the i-th feature block calculated by the k-th attention head; is the output representation of the i-th feature block calculated by the k-th attention head; We also apply relative position encoding to the attention module, so the attention calculation can be formulated as: Where Q, K, and V represent the input matrices of query, key, and value, respectively; is a scaling factor that prevents the inner product value from being too large and helps stabilize the gradient; B is an optional additive bias term used to adjust the similarity matrix to enhance feature representation. SoftMax normalizes the similarity matrix and generates attention weights so that the sum of the weights of each row is 1; V is the final value matrix, which is multiplied by the attention weight to obtain the output of the attention mechanism.

6. The intelligent visual monitoring method for oily wastewater in a steel enterprise according to claim 1 is characterized in that: The implementation method of step S3 is as follows: the image discriminator is a U-Net type network, and the image input to the network is the input image to be detected and the reconstructed, sewage-free image; the purpose of the network is to extract the features between the two input images, and finally obtain a feature map, which represents the probability that the corresponding pixel points of the input image are sewage. By setting a suitable indicator, a good detection accuracy can be obtained through the feature map.

7. The intelligent visual monitoring method for oily wastewater in a steel enterprise according to claim 1 is characterized in that: The implementation method of step S4 is as follows: when training the image reconstructor, two loss functions, L2 loss and SSIM loss, are used. After the reconstruction is completed, the input image and the output image are cascaded and sent to the image discriminator together to continue training the image discriminator. The image discriminator uses Focal Loss as the loss function. After the loss functions are calculated, the network gradient is back-propagated. The above steps are repeated to complete the network training and obtain a suitable network model. The model is then evaluated using the test data set. Among them: Where N is the number of samples, x i is the true value, y i is the predicted value; Among them, y and are the real image and the predicted image, μ y and is the mean of the real image and the predicted image; and is the variance between the real image and the predicted image; is the covariance between the real image and the predicted image; C1 and C2 are constants used for stable calculation, usually C1 = (0.01 × L) 2 、C1=(0.03×L) 2 , L is the maximum value of the pixel value range; Where N is the number of samples; α is the balance factor, which is used to adjust the impact of positive and negative samples; is the predicted probability of the i-th sample; γ is the adjustment factor used to adjust the influence of difficult and easy samples. Usually α is set to 0.25 and γ is set to 2.0.

Images