Defect image enhancement method based on traditional data enhancement
By combining traditional data augmentation technology and reinforcement learning algorithms, the visual effects and feature extraction of defective images are optimized to address the problem of insufficient image quality, and the defect detection difficulties in image recognition and analysis are solved, achieving more accurate and stable image processing .
Patent Information
- Application Number
- CN202411031481.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-30
- Publication Date
- 2024-11-15
AI Technical Summary
The prior art has not been effectively solved in defect detection difficulties caused by insufficient image quality in image recognition, detection or analysis, especially defect detection difficulties caused by insufficient image quality in image recognition, detection or analysis.
Using a defective image enhancement method based on traditional data enhancement technology, the visual effect of the image is optimized through image preprocessing, feature extraction and improved automatic data enhancement algorithm, combined with low-pass filtering, high-pass filtering, image smoothing and feature extraction technologies. Feature extraction, using reinforcement learning algorithm to automatically search for the best data augmentation strategy, enhance the diversity and robustness of images.
It significantly improves the visual effect and robustness of the image, reduces noise interference, enhances the contrast and clarity of the image, reduces the risk of false detection or missed detection, improves the generalization ability and adaptability of the model, and is suitable for different applications. Scene.
Smart Images

Figure CN118967474A_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the field of image processing and enhancement, and in particular is a defect image enhancement method based on traditional data enhancement. Background Art
[0002] As an image processing technology, defect image enhancement methods aim to improve the visibility and recognition of defect areas in images, thereby ensuring accurate detection, identification and analysis of defects. Traditional data enhancement methods rely on techniques such as image filtering, histogram equalization and contrast adjustment, and increase sample diversity by rotating, scaling, cropping, flipping and other operations on the original image, thereby enhancing the generalization and robustness of the model. However, how to effectively use data enhancement technology to optimize defect images and the difficulty of defect detection caused by insufficient image quality in image recognition, detection or analysis are still urgent problems to be solved. Summary of the invention
[0003] The present invention aims to solve the problem of difficulty in defect detection due to insufficient image quality in image recognition, detection or analysis, and proposes a defect image enhancement method based on traditional data enhancement technology. This method combines and improves traditional image enhancement technology, and performs targeted enhancement processing on defective images, aiming to improve the clarity, contrast and overall visual quality of defects. After the defect image is enhanced by this method, the subsequent target detection algorithm can identify defects in the image more quickly and accurately, effectively reducing the risk of false detection or missed detection due to poor image quality.
[0004] The present invention adopts the following technical solutions to achieve the above-mentioned purpose:
[0005] A defect image enhancement method based on a traditional data enhancement method comprises the following steps:
[0006] Collect and preprocess image data of product surfaces and interiors in industrial production, label and classify image data of normal products and defective products, and build a defective product image dataset;
[0007] Perform image enhancement on images in the defective product image dataset to improve the visual effect of the image and highlight the overall or local features;
[0008] Extracting features from the enhanced defect image data; learning features using a pattern recognition algorithm and identifying multiple groups representing potential pattern features; the potential patterns are used to distinguish features of each group;
[0009] Based on the feature extraction and clustering grouping results of the defect image, an improved automatic data enhancement algorithm is adopted to learn the data enhancement method through the training strategy search algorithm, and the best combination and sequence of data enhancement methods are selected to achieve the best enhancement effect on the image data; the enhancement effect is to increase the diversity of the image while maintaining the integrity of its key features;
[0010] The product images collected in real time during industrial production are enhanced using the best data enhancement strategy to obtain enhanced image data.
[0011] The image data of the product is captured by a sensor or a camera; the preprocessing includes cleaning and formatting to filter out interference data and unify the size to ensure the quality and consistency of the data.
[0012] The image enhancement comprises the following steps:
[0013] Analyze the grayscale distribution and background of defective product images to determine the locations to be enhanced;
[0014] Low-pass filtering is used to remove high-frequency noise generated during image acquisition and transmission;
[0015] High-pass filtering is used to enhance the high-frequency signals at the edge of the image, making the blurred image clear;
[0016] Calculate the average value of each pixel in the image and its neighboring pixels to smooth the image;
[0017] Take the median value of the pixel and its neighboring pixels as the new pixel value to remove salt and pepper noise;
[0018] The enhanced images are evaluated by combining quantitative metrics and qualitative observations to ensure that defect information is enhanced while avoiding the introduction of new noise or distortion.
[0019] The feature extraction method uses one of principal component analysis PCA, linear discriminant analysis LDA, and local sensitive hashing LSH to extract the most relevant features from the original data.
[0020] The pattern recognition algorithm includes a clustering algorithm and a classification algorithm for dividing data according to features to identify potential patterns or structures; the clustering algorithm uses K-means and hierarchical clustering, and the classification algorithm uses support vector machines, decision trees, and neural networks.
[0021] The improved automatic data enhancement algorithm includes:
[0022] Based on the feature extraction and clustering results of the training set train data, the strategy search algorithm is used to learn the best data enhancement strategy, and the generalization ability of the target model is used as a reward signal;
[0023] Apply the learned data enhancement strategy to process the validation set val data to generate new samples, and evaluate the performance of the sub-model on the training set containing the generated data to determine the quality of the generated new sample data;
[0024] Based on the accuracy performance feedback of the sub-model, the data augmentation strategy is continuously adjusted and optimized, and the data augmentation strategy that best suits the current data set and pattern recognition task is found through repeated iterations.
[0025] The strategy search algorithm includes a modeling controller and a training algorithm;
[0026] The controller is a recurrent neural network (RNN), which is a single-layer LSTM with 100 hidden neurons in each layer. The output decision S is predicted by softmax in each neural layer and embedded as the input of the next neural layer. The controller has 30 softmax outputs for predicting 5 sub-strategies, each of which contains two image operations applied in sequence, and each operation is associated with two hyperparameters: probability and size of the operation.
[0027] Train the sub-models using augmented data generated on the training set using the 5 sub-strategies;
[0028] The algorithm for training the controller RNN uses a proximal strategy to optimize PPO;
[0029] After the search is completed, the strategy search algorithm connects several sub-strategies that meet the accuracy requirements as an overall strategy for subsequent model training, so that the sub-model performs data enhancement on the input defect image according to the overall strategy.
[0030] The best data augmentation strategy is a combination of data augmentation methods and the order of operations; data augmentation methods include horizontal flipping, vertical flipping, random image rotation, random cropping, image scaling, nonlinear transformation, and one or more of the methods for adjusting image brightness, contrast, and saturation.
[0031] A system for a defective image enhancement method based on traditional data enhancement technology comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement the defective image enhancement method based on traditional data enhancement technology as described above when executing the computer program.
[0032] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, a defect image enhancement method based on traditional data enhancement technology as described above is implemented.
[0033] The present invention has the following beneficial effects and advantages:
[0034] 1. The present invention uses an image enhancement method to improve the robustness of the image, effectively reducing the impact of external noise and interference on the image quality, so that the image can remain clear and stable in various complex environments. At the same time, the present invention significantly improves the visual effect of the image through a sophisticated image processing algorithm, making the image brighter, clearer, and with higher contrast. Finally, the present invention has excellent adaptability, and it can be customized and optimized according to different application scenarios to meet the personalized needs of different application scenarios.
[0035] 2. The present invention uses a reinforcement learning algorithm to automatically search and select the best data enhancement strategy and sequence, without the need for researchers to manually design or adjust, which significantly saves researchers' time and energy, while avoiding performance losses caused by improper human selection of strategies. Implementing the best data enhancement strategy searched out can not only increase the amount and diversity of data, but also balance the category distribution by enhancing the data of a few categories, thereby optimizing the model's ability to handle category imbalance problems and improving the overall performance of the model. The enhanced data can play a role in different application scenarios, showing a wide range of applicability. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 is a flow chart of the method of the present invention;
[0037] Figure 2 It is a schematic diagram of the model structure of the strategy search algorithm used in the present invention;
[0038] Figure 3 It is a schematic diagram of the RNN model structure used in the present invention;
[0039] Figure 4 It is a schematic diagram of the structure of a single-layer LSTM model used in the present invention;
[0040] Figure 5 is a comparison diagram of the image enhanced by the method used in the present invention and the original image; DETAILED DESCRIPTION
[0041] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the invention, so the present invention is not limited by the specific implementation disclosed below.
[0042] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.
[0043] like Figure 1 As shown, a method for detecting air quality levels based on deep transfer learning includes the following steps:
[0044] Step 1: Collect image data of the surface and interior of products in industrial production. The data of the present invention comes from a parts processing factory, and the data includes real-time images, videos and radar data collected by cameras and sensors.
[0045] Step 2: First, clean and format the collected data, filter out dirty data and modify the image data to a uniform size. Then, classify the image data into two categories: normal images and defective images. Then, label the data to construct a defective product image dataset.
[0046] Among them, the collected industrial product data is cleaned, distorted or duplicated data is deleted, missing and abnormal data is processed, and the size is unified. Then the collected image data is classified into normal image data and defective image data.
[0047] The collected industrial product data is scaled to a uniform size of 224*224 pixels.
[0048] Among them, the image data is labeled, specifically, the product images in industrial production obtained by using sensors or cameras are labeled as labels of the industrial product images, that is, the image data is renamed using a combination of image categories and Arabic numerals.
[0049] Among them, the defective product image dataset contains a total of 26,232 images and labels. The labeled data is made into a dataset. The data is divided into training set, validation set and test set in a ratio of 3:1:1, and put into three folders: train, val and test respectively. These three folders are placed in the same directory.
[0050] Step 3: Use the tool to perform histogram analysis on the defect image data in the train folder to understand the grayscale distribution and background influence of the image, and determine the specific method of image enhancement based on the analysis results.
[0051] Step 4: According to the results of image analysis, select appropriate image enhancement technology to enhance the images in the defective product image data set, such as: low-pass filtering, high-pass filtering, Gaussian filtering, median filtering and other methods.
[0052] 4.1) Use low-pass filtering to remove high-frequency noise generated during image acquisition and transmission;
[0053] 4.2) Use high-pass filtering to enhance high-frequency signals such as image edges to make blurry images clear;
[0054] 4.3) Calculate the average value of each pixel in the image and its neighboring pixels, and use the average value to replace the original pixel value to achieve image smoothing;
[0055] 4.4) Take the median of the pixel and its neighborhood pixels as the new pixel value to remove salt and pepper noise.
[0056] Step 5: Evaluate the enhanced image to ensure that the image data is enhanced and no new noise is introduced.
[0057] The enhanced image is evaluated by combining quantitative indicators and qualitative observations (for example, the statistical grayscale histogram can quantify the sharpness of the histogram to determine whether it meets the requirements) to ensure that the defect information is enhanced while avoiding the introduction of new noise or distortion.
[0058] Step 6: Use feature extraction methods on the image data enhanced in step 5 to extract the most useful features from the original data, such as principal component analysis (PCA), linear discriminant analysis (LDA), local sensitive hashing (LSH), etc.
[0059] Step 7: Use clustering or classification algorithms to divide the data into multiple groups, each group represents a potential pattern, and identify the pattern of the data.
[0060] Among them, clustering algorithms (such as K-means, hierarchical clustering) or classification algorithms (such as support vector machines, decision trees, neural networks) classify data to identify underlying patterns or structures.
[0061] Step 8: Then, based on the results of feature extraction and pattern recognition, the enhanced image data in step 5 is used as input to the training strategy search algorithm to learn the best data enhancement strategy.
[0062] Step 9: Use the best data enhancement strategy to apply the best data enhancement strategy learned in step 6 to the images in the defective product image dataset in the val folder for further data enhancement.
[0063] Step 10: Use the sub-model to validate the augmented data to judge the quality of the generated data.
[0064] Step 11: Repeat steps 6 to 10, iterate according to the performance feedback of the model, and continuously adjust and optimize the data enhancement strategy until the data enhancement method that best suits the current data set is found or the preset number of iterations is reached. At this point, the best data enhancement strategy is obtained. The best data enhancement strategy can be one or a combination of methods such as horizontal flipping, vertical flipping, random image rotation, random cropping, image scaling, nonlinear transformation, and adjusting the brightness, contrast, and saturation of the image.
[0065] Among them, in steps 6-11, the strategy search algorithm learns the best data enhancement strategy, including a controller and a training algorithm. A single-layer LSTM is used as a recurrent neural network, which outputs a decision through softmax prediction at each step, and the output decision is input into a neural network as input for evaluation, and then the recurrent neural network is iteratively adjusted.
[0066] The controller is a recurrent neural network (RNN), a single-layer LSTM with 100 hidden neurons per layer. It predicts the output decision S at each step via softmax and embeds it as the input for the next step. The controller has 30 softmax outputs to predict 5 sub-policies, each of which consists of two image operations applied sequentially, and each operation is associated with two hyperparameters, the probability and size of the operation. We train the sub-models using augmented data generated on the training set using the 5 sub-policies. For each mini-batch sample, we randomly select a sub-policy to augment the image. We evaluate the accuracy (i.e., generalization ability) of the sub-model on the validation set and use this as a reward signal (validation accuracy R) to train the RNN controller. On each dataset, the controller samples approximately 15,000 policies.
[0067] The training algorithm of the training controller RNN adopts proximal policy optimization (PPO), which updates the policy in a progressive manner by comparing the old and new policies, while introducing a clipping term to control the update size and avoid excessive policy changes. The loss of PPO consists of two parts: the advantage-weighted policy loss, which measures the improvement of the new policy over the old policy; and the clipping term, which limits the magnitude of a single update. The learning rate of PPO is set to 0.0003, supplemented by an entropy penalty with a weight of 0.00001.
[0068] After the search is completed, the strategy search algorithm connects the sub-strategies of the best 5 strategies to finally obtain an overall strategy containing 25 sub-strategies for subsequent model training.
[0069] like Figure 2 As shown, the specific process of learning the best data enhancement strategy is:
[0070] Step 1: Select RNN as the controller, and then the controller samples the data enhancement strategy S, which includes the type of data enhancement operation and the probability of using the operation.
[0071] Step 2: Input the selected strategy S into a neural network to obtain the verification accuracy R, and back-propagate R to the RNN to update the RNN. The update strategy adopts the proximal strategy optimization strategy.
[0072] Step 3: Continue iterating until you find the data augmentation method that best suits the current dataset.
[0073] like Figure 3 As shown, the specific process of iterative updating of the recurrent neural network RNN is as follows:
[0074] Step 1: At each time step, the RNN receives the input data and the hidden state of the previous time step, and then calculates the hidden state and output of the previous time step.
[0075] Step 2: Calculate the loss function based on the RNN output and the true label.
[0076] Step 3: Calculate the gradient of the loss function with respect to the model parameters through the back-propagation algorithm.
[0077] Step 4: Use an optimization algorithm to update the model parameters based on the calculated gradients to minimize the loss function.
[0078] Step 5: Repeat the above steps until the preset number of iterations is reached or a certain stopping condition is met.
[0079] like Figure 4 As shown, the specific process of the long short-term memory network LSTM used by the recurrent neural network RNN is:
[0080] Step 1: For each time step, calculate the forget gate, input gate, output gate, and candidate memory cells based on the input data and the hidden state of the previous time step.
[0081] Step 2: Put the memory cell of the previous moment in as the state, and then calculate the real memory cell of the current moment based on the forget gate and the candidate memory cell.
[0082] Step 3: Calculate the hidden state at the current moment based on the actual memory unit and output gate at the current moment.
[0083] Step 12: Apply the best data enhancement strategy generated by the final iteration and learned to the defective product image dataset in the test folder for enhancement testing to obtain the final enhanced data.
[0084] When the user sends a picture that needs data enhancement, it is first resized to 224*224, then passed to the image enhancement module, and then the final data enhancement method (that is, the best data enhancement strategy) is applied to the final optimization model of the image enhancement module to perform data enhancement and output the final enhanced data.
[0085] like Figure 5 As shown, a is the original image and bo is the image enhanced by the method of the present invention. By comparing the original image with the enhanced image, it can be found that the image processed by the image enhancement method proposed in the present invention not only enhances the diversity and richness of the data, but also helps the model learn more robust and universal feature representations, thereby improving the generalization ability and adaptability of the model. This enables the model to better adapt to complex scenes in the real world and provide more accurate and reliable solutions for various computer vision tasks.
[0086] The method of the present invention combines the advantages of traditional methods and designs a defect image enhancement strategy based on traditional enhancement according to the characteristics of defect images. By selecting appropriate transformations and parameter settings, the defect image is customized to enhance the contrast, reduce noise, and highlight features, thereby providing more accurate image data for subsequent defect identification and analysis.
[0087] The above is a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should be regarded as within the scope of protection of the present invention.
Claims
1. A defect image enhancement method based on a traditional data enhancement method, characterized in that: The following steps are involved: Collect and preprocess image data of product surfaces and interiors in industrial production, label and classify image data of normal products and defective products, and build a defective product image dataset; Perform image enhancement on images in the defective product image dataset to improve the visual effect of the image and highlight the overall or local features; Extracting features from the enhanced defect image data; learning features using a pattern recognition algorithm and identifying multiple groups representing potential pattern features; the potential patterns are used to distinguish features of each group; Based on the feature extraction and clustering grouping results of the defect image, an improved automatic data enhancement algorithm is adopted to learn the data enhancement method through the training strategy search algorithm, and the best combination and sequence of data enhancement methods are selected to achieve the best enhancement effect on the image data; the enhancement effect is to increase the diversity of the image while maintaining the integrity of its key features; The product images collected in real time during industrial production are enhanced using the best data enhancement strategy to obtain enhanced image data.
2. The defect image enhancement method based on the traditional data enhancement method according to claim 1, characterized in that: The image data of the product is captured by a sensor or a camera; the preprocessing includes cleaning and formatting to filter out interference data and unify the size to ensure the quality and consistency of the data.
3. The defect image enhancement method based on the traditional data enhancement method according to claim 1, characterized in that: The image enhancement comprises the following steps: Analyze the grayscale distribution and background of defective product images to determine the locations to be enhanced; Low-pass filtering is used to remove high-frequency noise generated during image acquisition and transmission; High-pass filtering is used to enhance the high-frequency signals at the edge of the image, making the blurred image clear; Calculate the average value of each pixel in the image and its neighboring pixels to smooth the image; Take the median value of the pixel and its neighboring pixels as the new pixel value to remove salt and pepper noise; The enhanced images are evaluated by combining quantitative metrics and qualitative observations to ensure that defect information is enhanced while avoiding the introduction of new noise or distortion.
4. The defect image enhancement method based on the traditional data enhancement method according to claim 1, characterized in that: The feature extraction method uses one of principal component analysis PCA, linear discriminant analysis LDA, and local sensitive hashing LSH to extract the most relevant features from the original data.
5. The defect image enhancement method based on the traditional data enhancement method according to claim 1, characterized in that: The pattern recognition algorithm includes a clustering algorithm and a classification algorithm for dividing data according to features to identify potential patterns or structures; the clustering algorithm uses K-means and hierarchical clustering, and the classification algorithm uses support vector machines, decision trees, and neural networks.
6. The defect image enhancement method based on the traditional data enhancement method according to claim 1, characterized in that: The improved automatic data enhancement algorithm includes: Based on the feature extraction and clustering results of the training set train data, the strategy search algorithm is used to learn the best data enhancement strategy, and the generalization ability of the target model is used as a reward signal; Apply the learned data enhancement strategy to process the validation set val data to generate new samples, and evaluate the performance of the sub-model on the training set containing the generated data to determine the quality of the generated new sample data; Based on the accuracy performance feedback of the sub-model, the data augmentation strategy is continuously adjusted and optimized, and the data augmentation strategy that best suits the current data set and pattern recognition task is found through repeated iterations.
7. The defect image enhancement method based on the traditional data enhancement method according to claim 6, characterized in that: The strategy search algorithm includes a modeling controller and a training algorithm; The controller is a recurrent neural network (RNN), which is a single-layer LSTM with 100 hidden neurons in each layer. The output decision S is predicted by softmax in each neural layer and embedded as the input of the next neural layer. The controller has 30 softmax outputs for predicting 5 sub-strategies, each of which contains two image operations applied in sequence, and each operation is associated with two hyperparameters: probability and size of the operation. Train the sub-models using augmented data generated on the training set using the 5 sub-strategies; The algorithm for training the controller RNN uses a proximal strategy to optimize PPO; After the search is completed, the strategy search algorithm connects several sub-strategies that meet the accuracy requirements as an overall strategy for subsequent model training, so that the sub-model performs data enhancement on the input defect image according to the overall strategy.
8. The defect image enhancement method based on the traditional data enhancement method according to claim 1 or 7, characterized in that: The best data augmentation strategy is a combination and sequence of data augmentation methods; data augmentation methods include one or more of horizontal flipping, vertical flipping, random image rotation, random cropping, image scaling, nonlinear transformation, and adjusting image brightness, contrast, and saturation.
9. A defect image enhancement method system based on traditional data enhancement technology, characterized in that: It comprises a memory and a processor; the memory is used to store a computer program; the processor is used to implement a defect image enhancement method based on traditional data enhancement technology as described in any one of claims 1 to 8 when executing the computer program.
10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the defect image enhancement method based on traditional data enhancement technology as described in any one of claims 1 to 8 is implemented.
Citation Information
Patent Citations
Industrial product surface defect adaptive detection method based on deep learning model AGLNet
CN113808079A
Point cloud data processing method and device, equipment and storage medium
CN115457500A
Display device
KR1020210092351A
Cited By
Student classroom learning state evaluation method and system based on artificial intelligence
CN120032300A
Visual optimization processing method and system based on real-time image processing
CN120634893A
A visual optimization processing method and system based on real-time image processing
CN120634893B
Self-adaptive image enhancement method and system based on physical modeling and dynamic prediction
CN122134574A
