Method and system for automatically detecting surface flaws of printed matter by utilizing machine vision

Abstract

The invention relates to the field of flaw detection, in particular to a printed matter surface flaw automatic detection method and system utilizing machine vision, and the method comprises the following steps: uniformly irradiating the surface of a printed matter by using a light source module, and capturing a detailed image of the surface of the printed matter by adopting an imaging module in combination with multispectral imaging; the image processing module is used for carrying out preprocessing and image feature extraction on the image, carrying out non-contact measurement on the surface of the printed matter, collecting physical parameters of the surface of the printed matter, integrating the image features and the physical parameters into time sequence data, preprocessing the time sequence data, constructing an RNN model, and using the preprocessed time sequence data to carry out image feature extraction on the surface of the printed matter; according to the method, the RNN model is trained and applied, final quality prediction and flaw classification are carried out according to output data of the RNN model and the fused feature parameters, and through the multispectral imaging technology and the fine image preprocessing step, tiny flaws on the surface of the printed matter can be captured, and the detection accuracy is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of defect detection, and in particular to a method and system for automatically detecting surface defects of printed matter using machine vision. Background Art

[0002] With the rapid development of the modern printing industry, the quality and accuracy of printed products have become one of the important factors determining product quality. Various defects may occur in the production process of printed products:

[0003] 1. Existing print defect detection systems mostly use traditional image processing algorithms. These algorithms often have difficulty achieving high-precision detection results when dealing with complex defects such as tiny defects and color gradient defects. The light source modules in the existing technology usually do not have the function of adaptive lighting adjustment, which leads to a significant reduction in the defect detection effect when the reflection on the printed surface is uneven or the defect features are not obvious;

[0004] 2. Traditional defect detection methods require manual adjustment of detection parameters, and the detection process is usually cumbersome, resulting in a slow overall detection speed, which is difficult to meet the high-efficiency requirements of modern printing production lines. Although some existing technologies use automated detection methods, the real-time detection is poor due to the high algorithm complexity and large amount of calculation;

[0005] 3. The historical defect data in the existing technology is usually simply stored and queried, lacking in-depth data analysis and mining capabilities, resulting in low data utilization and difficulty in discovering potential defect patterns and trends. Although some existing technologies have established data warehouses, the data management and query functions are relatively simple and cannot support complex data analysis and decision support needs. Summary of the invention

[0006] The purpose of the present invention is to overcome the above technical defects and provide a method and system for automatically detecting surface defects of printed products using machine vision. The method and system use adaptive lighting adjustment technology, monitor the reflection situation through a sensor module, automatically adjust the brightness and color temperature of the light source, ensure that the defect features are clearly visible, adopt multispectral imaging technology, configure multiple light sources of different bands, separate multispectral images of different bands, obtain the characteristics of defects under different spectra, and improve the detection accuracy. The image processing module preprocesses and extracts features of the image, applies denoising algorithm, histogram equalization, morphological operation and deep learning model CNN, extracts high-dimensional features that can accurately reflect the characteristics and types of defects, constructs an RNN model, especially an LSTM architecture, captures complex relationships in time series data, and realizes fast and accurate defect detection and quality prediction. Combined with the output data of the RNN model and the fused feature parameters, the classification algorithm random forest is applied to classify the defects, improves the classification accuracy, and notifies the operator of the defect location and type in real time. The production line parameters are automatically adjusted according to the defect type and frequency, reduce the generation of defects, and improve the overall production efficiency. A data warehouse is established to store historical defect data, provide efficient data storage and query functions, and support subsequent analysis and mining.

[0007] The present invention provides a method for automatically detecting surface defects of printed products using machine vision, which is applied to an automatic detection system for surface defects of printed products using machine vision. The system includes a light source module, an imaging module, an image processing module, a sensor module, a physical analysis module, an RNN deep learning module, a defect recognition module, a feedback optimization module and a decision support module. The method includes the following steps:

[0008] Step S1, using a light source module to evenly illuminate the surface of the printed product to ensure that defect features are clearly visible in the image;

[0009] Step S2, using an imaging module to capture detailed images of the printed surface in combination with multispectral imaging, setting appropriate exposure time, shutter speed and other parameters to ensure that the image is clear and blur-free;

[0010] Step S3: using an image processing module to preprocess the image and extract image features, the extracted image features can accurately reflect the characteristics and types of defects;

[0011] Step S4: Use the sensor module to perform non-contact measurement on the surface of the printed matter to collect physical parameters of the surface of the printed matter;

[0012] Step S5, integrating the image features and physical parameters into time series data, preprocessing the time series data, building an RNN model, using the preprocessed time series data to train and apply the RNN model to obtain output data, fusing the output data of the RNN model with the image features and physical parameters, and further processing the fused feature parameters through a fully connected layer;

[0013] Step S6: Perform final quality prediction and defect classification based on the output data of the RNN model and the fused feature parameters;

[0014] Step S7, the operator is notified of the defect location and type in real time through the interface display or sound and light alarm, and the production is automatically adjusted according to the result of the defect classification in step S6. Adjusting the production can reduce the occurrence of defects and improve the overall production efficiency. A data warehouse is established to store historical defect data. The data warehouse provides efficient data storage and query functions to facilitate subsequent analysis and mining.

[0015] Furthermore, in step S1, adaptive lighting adjustment is performed during the process of uniformly irradiating the surface of the printed matter using the light source module. The specific method includes the following steps:

[0016] Step S11: using a sensor module to monitor the reflection of the printed matter surface to obtain sensor data;

[0017] Step S12: the light source module automatically adjusts the brightness and color temperature of the light source according to the sensor data;

[0018] Step S13: Design a feedback loop to ensure that the light source output is stable and the defect features are clearly visible.

[0019] As a further improvement of this solution, in step S2, the multispectral imaging process specifically includes the following steps:

[0020] Step S21: configuring a plurality of light sources of different wavelength bands to obtain a multi-spectral image of the printed matter surface;

[0021] Step S22: Use filters to separate multispectral images of different wavelength bands to obtain the characteristics of defects under different spectra.

[0022] As a further improvement of this solution, in step S3, the image is preprocessed and the image feature is extracted. The specific method includes the following steps:

[0023] Step S31: Image preprocessing: applying a denoising algorithm to remove image noise using bilateral filtering, using histogram equalization to enhance image contrast, performing image correction, and eliminating lens distortion and perspective distortion;

[0024] Step S32: Image feature extraction: Use the edge detection algorithm Canny to extract the defect edge, apply morphological operations to dilate, separate and highlight the defect area, and use the deep learning model CNN to extract high-dimensional features.

[0025] Furthermore, in step S31, the process of applying the denoising algorithm bilateral filtering to remove image noise is that the bilateral filter considers spatial proximity and pixel value similarity, removes noise by weighted average, and the formula for calculating the filtered image is as follows: ;

[0026] in, is the filtered image at position The pixel value of For the original image at position The pixel value of is the position of the original image in the neighborhood The pixel value of is a spatial Gaussian function used to measure the spatial proximity between pixels. is the standard deviation of the spatial Gaussian function, is the intensity Gaussian function, which is used to measure the similarity between pixel values. is the standard deviation of the intensity Gaussian function, For location and The Euclidean distance between The total number of pixels in the neighborhood;

[0027] Furthermore, in step S31, the process of enhancing the image contrast by using histogram equalization is to adjust the grayscale distribution of the image so that the histogram of the image is as flat as possible, thereby enhancing the contrast;

[0028] Furthermore, in step S32, the process of applying the morphological operation to dilate, separate and highlight the defective area is to use the structural element to dilate the image, and the formula used is as follows: ;

[0029] in, is the input image, is a structural element used to define the scope and shape of the dilation operation. is the expansion operator, is the pixel position in the output image, is the pixel position in the structural element;

[0030] Furthermore, in step S32, the process of extracting high-dimensional features using the deep learning model CNN is to extract high-dimensional features through convolutional layers, pooling layers and fully connected layers.

[0031] As a further improvement of this solution, in step S5, the RNN model is constructed, and the specific method includes the following steps:

[0032] Step S51: data preprocessing: integrating image features and physical parameters into time series data, and standardizing the time series data to have zero mean and unit variance;

[0033] Step S52: Model construction: Select the LSTM architecture, design the number of network layers and neurons, and ensure that the RNN model can capture the complex relationships in the time series data;

[0034] Step S53: Model training: Use the cross-validation method to divide the time series data into a training set and a validation set, apply the gradient descent algorithm to optimize the RNN model parameters, and use the early stopping method to prevent overfitting.

[0035] Furthermore, in step S51, the goal of data preprocessing is to integrate image features and physical parameters into time series data and perform standardization processing. The specific method includes the following steps:

[0036] Step S511: Integrate data: Integrate the image features extracted by the image processing module and the physical parameters collected by the sensor module into time series data in chronological order;

[0037] Step S512: Standardization processing: To speed up the training process and improve the performance of the RNN model, the time series data is standardized. The formula used to calculate the standardized time series data is as follows: ;

[0038] in, is the time step, is the standardized time series data, For time series data, is the mean of the time series data, is the standard deviation of the time series data.

[0039] Furthermore, in step S52, the specific method of constructing the RNN model includes the following steps:

[0040] Step S521: Select LSTM architecture: LSTM unit can learn long-term dependency information, and its structure includes a forget gate, an input gate, and an output gate;

[0041] Step S522: Design network structure: determine the number of network layers and the number of neurons in each layer , the network structure can be expressed as: ;

[0042] in, is the input latitude, is the number of hidden neurons, is the output latitude;

[0043] Furthermore, in step S53, the specific method of RNN model training includes the following steps:

[0044] Step S531: Data division: divide the preprocessed time series data into a training set and a validation set, and use the k-fold cross-validation method for verification;

[0045] Step S532: Gradient descent algorithm: Use the gradient descent algorithm to minimize the loss function , the formula used is as follows: ;

[0046] in, are the RNN model parameters, is the learning rate, is the gradient of the loss function with respect to the parameters;

[0047] Step S533: Early stopping method: Stop training when the performance on the validation set no longer improves to prevent overfitting. The specific method is to monitor the loss on the validation set and stop training when there is no improvement for multiple consecutive epochs.

[0048] As a further improvement of this solution, in step S6, the quality prediction and defect classification process specifically includes the following steps:

[0049] Step S61: Quality prediction: Use the trained RNN model to predict the time series data, analyze the prediction results, and determine the existence and severity of defects;

[0050] Step S62: defect classification: applying the classification algorithm random forest to classify defects, combined with the high-dimensional features output by the deep learning model CNN, to improve classification accuracy;

[0051] Furthermore, in step S61, the specific method of quality prediction includes the following steps:

[0052] Step S611: Model prediction: Use the trained RNN model to predict the new time series data. The prediction formula is as follows: ;

[0053] in, is the output predicted by the RNN model, For the trained RNN model, For time series data, is the RNN model parameter;

[0054] Step S612: Analyze prediction results: To determine the presence and severity of defects, and set thresholds to determine whether defects exist;

[0055] Furthermore, in step S62, the specific method of defect classification includes the following steps:

[0056] Step S621: Feature extraction: Use the deep learning model CNN to extract high-dimensional features of the image. The high-dimensional feature extraction formula is as follows: ;

[0057] in, To extract high-dimensional features, For the deep learning model CNN, is the processed input image, are the parameters of the deep learning model CNN;

[0058] Step S622: Random forest classification, high-dimensional features extracted by CNN The output of the RNN model Combine to form a new feature vector , and then use the random forest classifier for classification;

[0059] Furthermore, in step S622, the specific method of random forest classification includes the following steps:

[0060] Step K1: Feature vector merging: ;

[0061] Step K2: Random forest training: Random forest consists of multiple decision trees, and the training formula for each decision tree is as follows: ;

[0062] in, For decision tree, is the label in the training set;

[0063] Step K3: Classification prediction: Use the trained random forest for classification prediction. The formula is as follows: ;

[0064] in, is the predicted defect category, is the decision tree in the random forest;

[0065] Furthermore, in step S62, through ensemble learning, random forest can reduce overfitting and improve the robustness and accuracy of classification. Each decision tree randomly selects a feature subset and a sample subset during training to form a diversified prediction model, and finally determines the final classification result by voting or averaging.

[0066] The beneficial effects achieved by the present invention using the above scheme are as follows:

[0067] (1) Through multispectral imaging technology and sophisticated image preprocessing steps, it is possible to capture tiny defects on the surface of printed products and improve the accuracy of detection. Combined with the RNN deep learning module, it is able to process time series data, capture the dynamic characteristics of defects, and further improve detection accuracy.

[0068] (2) The light source module can adaptively adjust the brightness and color temperature according to the reflective conditions on the printed surface, ensuring that the defect features are clearly visible under different lighting conditions. The system can automatically adjust the production line parameters to reduce defects according to the defect type and frequency, thereby improving overall production efficiency, showing a high degree of adaptability and flexibility;

[0069] (3) The RNN model with LSTM architecture can efficiently process time series data and capture complex relationships in the data, providing strong support for quality prediction and defect classification. The establishment of a data warehouse enables efficient storage and query of historical defect data, facilitating subsequent data analysis and mining, and providing a data basis for the continuous improvement of detection algorithms.

[0070] (4) The high-dimensional features of the image are extracted by combining the CNN deep learning model with the output of the RNN model and classified by the random forest classifier, which improves the accuracy and robustness of defect classification. This method can identify multiple types of defects and provide comprehensive support for quality control in the production process.

[0071] (5) The system can inform operators of the location and type of defects in real time, so that problems can be dealt with quickly, reducing the risk of production interruptions. The function of automatically adjusting production line parameters can continuously optimize the production process based on real-time feedback, reduce the occurrence of defects, and improve overall production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 It is a module schematic diagram of a system for automatically detecting surface defects of printed matter using machine vision according to the present invention;

[0073] Figure 2 is a schematic flow chart of step S3;

[0074] Figure 3 is a schematic flow chart of step S5;

[0075] Figure 4 is a flow chart of step S51;

[0076] Figure 5 is a schematic diagram of the classification of data mining techniques in step S52;

[0077] Figure 6 It is a schematic diagram of the RNN model structure;

[0078] Figure 7 is a schematic flow chart of step S6;

[0079] Figure 8 is a schematic flow chart of step S61;

[0080] Fig. 9 is a schematic flow chart of step S62;

[0081] Fig.10 It is a flowchart of the random forest classification method.

[0082] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention. DETAILED DESCRIPTION

[0083] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments.

[0084] Embodiment 1, as Figure 1 As shown, in this embodiment, a method for automatically detecting surface defects of printed products using machine vision is applied to a system for automatically detecting surface defects of printed products using machine vision. The system includes a light source module, an imaging module, an image processing module, a sensor module, a physical analysis module, an RNN deep learning module, a defect recognition module, a feedback optimization module, and a decision support module. Through the coordinated work of the light source module, the imaging module, and the image processing module, the defect characteristics of the printed product surface can be accurately captured. The method includes the following steps:

[0085] Step S1, using a light source module to evenly illuminate the surface of the printed product to ensure that the defect features are clearly visible in the image;

[0086] Step S2, using an imaging module to capture detailed images of the printed surface in combination with multispectral imaging, setting appropriate parameters such as exposure time and shutter speed to ensure that the image is clear and blur-free;

[0087] Step S3: using an image processing module to preprocess the image and extract image features, the extracted image features can accurately reflect the characteristics and types of defects;

[0088] Step S4: Use the sensor module to perform non-contact measurement on the surface of the printed matter to collect physical parameters of the surface of the printed matter;

[0089] Step S5: Integrate the image features and physical parameters into time series data, preprocess the time series data, build an RNN model, use the preprocessed time series data to train and apply the RNN model to obtain output data, fuse the output data of the RNN model with the image features and physical parameters, further process the fused feature parameters through a fully connected layer, and use the RNN deep learning module to analyze the time series data, thereby realizing automatic identification and classification of defects, reducing manual intervention, and improving detection efficiency;

[0090] Step S6: Perform final quality prediction and defect classification based on the output data of the RNN model and the fused feature parameters;

[0091] Step S7, the operator is notified of the defect location and type in real time through the interface display or sound and light alarm, and the production is automatically adjusted according to the defect classification result in step S6. Adjusting the production can reduce the occurrence of defects and improve the overall production efficiency. A data warehouse is established to store historical defect data. The data warehouse provides efficient data storage and query functions to facilitate subsequent analysis and mining. The operator is notified of the defect location and type in real time through the interface display or sound and light alarm, and the production is automatically adjusted according to the defect classification result, which effectively reduces the occurrence of defects and improves the overall production efficiency.

[0092] Embodiment 2: This embodiment is based on the above embodiment. In step S1, adaptive lighting adjustment is performed during the process of uniformly irradiating the surface of the printed matter using the light source module. The specific method includes the following steps:

[0093] Step S11: using a sensor module to monitor the reflection of the printed matter surface to obtain sensor data;

[0094] Step S12: the light source module automatically adjusts the brightness and color temperature of the light source according to the sensor data;

[0095] Step S13: designing a feedback loop to ensure that the light source output is stable and the defect features are clearly visible;

[0096] The sensor module monitors the reflection situation, and the light source module automatically adjusts the brightness and color temperature to ensure that the defect features are clearly visible and improve detection accuracy;

[0097] In step S2, the multispectral imaging process specifically includes the following steps:

[0098] Step S21: configuring a plurality of light sources of different wavelength bands to obtain a multi-spectral image of the printed matter surface;

[0099] Step S22: using filters to separate multispectral images of different wavelength bands to obtain characteristics of defects under different spectra;

[0100] By configuring multiple light sources of different wavelengths and separating multispectral images, the characteristics of defects under different spectra are obtained, which enhances the detection capability and helps to identify more types of defects.

[0101] Embodiment three, as Figure 2 As shown, this embodiment is based on the above embodiment. In step S3, the image is preprocessed and the image feature is extracted. The specific method includes the following steps:

[0102] Step S31: Image preprocessing: Apply denoising algorithm bilateral filtering to remove image noise, use histogram equalization to enhance image contrast, perform image correction, eliminate lens distortion and perspective distortion, apply denoising algorithm and histogram equalization to improve image quality, reduce the impact of noise and distortion on detection, and provide a better basis for subsequent feature extraction and defect recognition;

[0103] Step S32: Image feature extraction: Use the edge detection algorithm Canny to extract the defect edge, apply morphological operation to dilate, separate and highlight the defect area, and use the deep learning model CNN to extract high-dimensional features. Use the edge detection algorithm and morphological operation to accurately extract the defect edge and area, and use the deep learning model CNN to extract high-dimensional features, which further improves the detection accuracy;

[0104] In step S31, the process of applying the denoising algorithm bilateral filtering to remove image noise is that the bilateral filter considers spatial proximity and pixel value similarity, removes noise by weighted average, and the formula for calculating the filtered image is as follows: ;

[0105] in, is the filtered image at position The pixel value of For the original image at position The pixel value of is the position of the original image in the neighborhood The pixel value of is a spatial Gaussian function used to measure the spatial proximity between pixels. is the standard deviation of the spatial Gaussian function, is the intensity Gaussian function, which is used to measure the similarity between pixel values. is the standard deviation of the intensity Gaussian function, For location and The Euclidean distance between The total number of pixels in the neighborhood;

[0106] In step S31, the process of enhancing the image contrast by using histogram equalization is to adjust the grayscale distribution of the image so that the histogram of the image is as flat as possible, thereby enhancing the contrast;

[0107] In step S32, the process of applying morphological operation to dilate, separate and highlight the defective area is to use the structural element to dilate the image, and the formula used is as follows: ;

[0108] in, is the input image, is a structural element used to define the scope and shape of the dilation operation. is the expansion operator, is the pixel position in the output image, is the pixel position in the structural element;

[0109] In step S32, the process of extracting high-dimensional features using the deep learning model CNN is to extract high-dimensional features through convolutional layers, pooling layers and fully connected layers.

[0110] Embodiment 4, as Figure 3-Figure 6 As shown, this embodiment is based on the above embodiment. In step S5, an RNN model is constructed. The specific method includes the following steps:

[0111] Step S51: Data preprocessing: Integrate image features and physical parameters into time series data, standardize the time series data to make it have zero mean and unit variance, integrate image features and physical parameters into time series data and perform standardization, so as to provide high-quality data input for subsequent RNN model training;

[0112] Step S52: Model construction: Select the LSTM architecture, design the number of network layers and neurons, and ensure that the RNN model can capture the complex relationships in the time series data;

[0113] Step S53: Model training: Use the cross-validation method to divide the time series data into a training set and a validation set, apply the gradient descent algorithm to optimize the RNN model parameters, and use the early stopping method to prevent overfitting;

[0114] Selecting the LSTM architecture and designing a reasonable network structure ensures that the RNN model can capture the complex relationships in time series data. At the same time, the model parameters are optimized using cross-validation and gradient descent algorithms to improve the training efficiency and performance of the model.

[0115] Embodiment 5, as Figure 4 As shown, this embodiment is based on the above embodiment. In step S51, the goal of data preprocessing is to integrate image features and physical parameters into time series data and perform standardization processing to provide a basis for subsequent analysis. The specific method includes the following steps:

[0116] Step S511: Integrate data: Integrate the image features extracted by the image processing module and the physical parameters collected by the sensor module into time series data in chronological order;

[0117] Step S512: Standardization processing: To speed up the training process and improve the performance of the RNN model, the time series data is standardized. The formula used to calculate the standardized time series data is as follows: ;

[0118] in, is the time step, is the standardized time series data, For time series data, is the mean of the time series data, is the standard deviation of the time series data;

[0119] The image features and physical parameters are integrated into time series data in chronological order and standardized, which provides a basis for subsequent analysis and ensures the consistency and comparability of the data.

[0120] Embodiment six, as Figure 5-Figure 6 As shown, this embodiment is based on the above embodiment, and in step S52, the specific method of constructing the RNN model includes the following steps:

[0121] Step S521: Selecting LSTM architecture: LSTM unit can learn long-term dependency information, and its structure includes a forget gate, an input gate, and an output gate. By selecting LSTM architecture and designing a reasonable network structure, it is ensured that the RNN model can learn long-term dependency information and improve the model's ability to process time series data;

[0122] Step S522: Design network structure: determine the number of network layers and the number of neurons in each layer , the network structure can be expressed as: ;

[0123] in, is the input latitude, is the number of hidden neurons, is the output latitude;

[0124] In step S53, the specific method of RNN model training includes the following steps:

[0125] Step S531: Data division: divide the preprocessed time series data into a training set and a validation set, and use the k-fold cross-validation method for verification;

[0126] Step S532: Gradient descent algorithm: Use the gradient descent algorithm to minimize the loss function , the formula used is as follows: ;

[0127] in, are the RNN model parameters, is the learning rate, is the gradient of the loss function with respect to the parameters;

[0128] Step S533: Early stopping method: stop training when the performance on the validation set no longer improves to prevent overfitting. The specific method is to monitor the loss on the validation set and stop training when there is no improvement for multiple consecutive epochs.

[0129] The cross-validation method was used to divide the data set, the gradient descent algorithm was applied to optimize the model parameters, and the early stopping method was used to prevent overfitting, ensuring the stability and generalization ability of the model.

[0130] Embodiment 7, as Figure 7-Figure 9 As shown, this embodiment is based on the above embodiment, and in step S6, the quality prediction and defect classification process, the specific method includes the following steps:

[0131] Step S61: Quality prediction: Use the trained RNN model to predict the time series data, analyze the prediction results, and determine the existence and severity of defects;

[0132] Step S62: defect classification: applying the classification algorithm random forest to classify defects, combined with the high-dimensional features output by the deep learning model CNN, to improve classification accuracy;

[0133] In step S61, the specific method of quality prediction includes the following steps:

[0134] Step S611: Model prediction: Use the trained RNN model to predict the new time series data. The prediction formula is as follows: ;

[0135] in, is the output predicted by the RNN model, For the trained RNN model, For time series data, is the RNN model parameter;

[0136] Step S612: Analyze prediction results: To determine the presence and severity of defects, and set thresholds to determine whether defects exist;

[0137] The trained RNN model is used to predict time series data, and the defects are classified using the classification algorithm Random Forest, which enables accurate judgment of the existence, severity, and type of defects. The existence and severity of defects are quickly determined through model prediction and analysis of prediction results. At the same time, the random forest classifier is combined with the high-dimensional features extracted by CNN for classification, which improves the accuracy and robustness of classification.

[0138] Embodiment 8: Figure 9-10 As shown, this embodiment is based on the above embodiment, and the step S621: feature extraction: using the deep learning model CNN to extract high-dimensional features of the image, the high-dimensional feature extraction formula is as follows: ;

[0139] in, To extract high-dimensional features, For the deep learning model CNN, is the processed input image, are the parameters of the deep learning model CNN;

[0140] Step S622: Random forest classification, high-dimensional features extracted by CNN Output of the RNN model Combine to form a new feature vector , and then use the random forest classifier for classification;

[0141] In step S622, the specific method of random forest classification includes the following steps:

[0142] Step K1: Feature vector merging: ;

[0143] Step K2: Random forest training: Random forest consists of multiple decision trees, and the training formula for each decision tree is as follows: ;

[0144] in, For decision tree, is the label in the training set;

[0145] Step K3: Classification prediction: Use the trained random forest for classification prediction. The formula is as follows: ;

[0146] in, is the predicted defect category, is the decision tree in the random forest;

[0147] In step S62, through ensemble learning, random forest can reduce overfitting and improve the robustness and accuracy of classification. Each decision tree randomly selects a feature subset and a sample subset during training, thereby forming a diversified prediction model, and finally determines the final classification result by voting or averaging.

[0148] The high-dimensional features extracted by CNN are combined with the output of the RNN model to form a new feature vector, and the random forest classifier is used for classification, which improves the accuracy and efficiency of classification. The integrated learning characteristics of random forest are used to reduce overfitting and improve the robustness and accuracy of classification. At the same time, the diversified training methods of each decision tree help to form a more comprehensive prediction model.

[0149] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. A method for automatically detecting surface defects of printed matter using machine vision, applied to an automatic detection system for surface defects of printed matter using machine vision, characterized in that: The system includes a light source module, an imaging module, an image processing module, a sensor module, a physical analysis module, an RNN deep learning module, a defect recognition module, a feedback optimization module and a decision support module, and the method includes the following steps: Step S1, using a light source module to evenly illuminate the surface of the printed matter; Step S2, using an imaging module in combination with multispectral imaging to capture a detailed image of the printed product surface; Step S3: using an image processing module to preprocess the image and extract image features; Step S4: Use the sensor module to perform non-contact measurement on the surface of the printed matter to collect physical parameters of the surface of the printed matter; Step S5, integrating the image features and physical parameters into time series data, preprocessing the time series data, building an RNN model, using the preprocessed time series data to train and apply the RNN model to obtain output data, fusing the output data of the RNN model with the image features and physical parameters, and further processing the fused feature parameters through a fully connected layer; Step S6: Perform final quality prediction and defect classification based on the output data of the RNN model and the fused feature parameters; Step S7, notifying the operator of the defect location and type in real time through interface display or sound and light alarm, automatically adjusting production according to the defect classification result in step S6, establishing a data warehouse, and storing historical defect data; In step S5, the RNN model is constructed, and the specific method includes the following steps: Step S51: data preprocessing: integrating image features and physical parameters into time series data, and standardizing the time series data to have zero mean and unit variance; Step S52: Model construction: select LSTM architecture, design the number of network layers and neurons; Step S53: Model training: Use the cross-validation method to divide the time series data into a training set and a validation set, apply the gradient descent algorithm to optimize the RNN model parameters, and use the early stopping method to prevent overfitting.

2. The method for automatically detecting surface defects of printed matter using machine vision according to claim 1, characterized in that: In step S51, the goal of data preprocessing is to integrate image features and physical parameters into time series data and perform standardization processing. The specific method includes the following steps: Step S511: Integrate data: Integrate the image features extracted by the image processing module and the physical parameters collected by the sensor module into time series data in chronological order; Step S512: Standardization processing: To speed up the training process and improve the performance of the RNN model, the time series data is standardized. The formula used to calculate the standardized time series data is as follows: ； in, is the time step, is the standardized time series data, For time series data, is the mean of the time series data, is the standard deviation of the time series data; In step S52, the specific method of constructing the RNN model includes the following steps: Step S521: Select LSTM architecture: The LSTM unit structure includes a forget gate, an input gate, and an output gate; Step S522: Design network structure: determine the number of network layers and the number of neurons in each layer , the network structure can be expressed as: ； in, is the input latitude, is the number of hidden neurons, is the output latitude; In step S53, the specific method of RNN model training includes the following steps: Step S531: Data division: divide the preprocessed time series data into a training set and a validation set, and use the k-fold cross-validation method for verification; Step S532: Gradient descent algorithm: Use the gradient descent algorithm to minimize the loss function , the formula used is as follows: ； in, are the RNN model parameters, is the learning rate, is the gradient of the loss function with respect to the parameters; Step S533: Early stopping method: Stop training when the performance on the validation set no longer improves to prevent overfitting. The specific method is to monitor the loss on the validation set and stop training when there is no improvement for multiple consecutive epochs.

3. The method for automatically detecting surface defects of printed matter using machine vision according to claim 1, characterized in that: In step S3, the image is preprocessed and image features are extracted. The specific method includes the following steps: Step S31: Image preprocessing: applying a denoising algorithm to remove image noise using bilateral filtering, using histogram equalization to enhance image contrast, performing image correction, and eliminating lens distortion and perspective distortion; Step S32: Image feature extraction: Use the edge detection algorithm Canny to extract the defect edge, apply morphological operations to dilate, separate and highlight the defect area, and use the deep learning model CNN to extract high-dimensional features.

4. The method for automatically detecting surface defects of printed matter using machine vision according to claim 3, characterized in that: In step S31, the process of applying the denoising algorithm bilateral filtering to remove image noise is that the bilateral filter considers spatial proximity and pixel value similarity, removes noise by weighted average, and the formula for calculating the filtered image is as follows: ； in, is the filtered image at position The pixel value of For the original image at position The pixel value of is the position of the original image in the neighborhood The pixel value of is a spatial Gaussian function used to measure the spatial proximity between pixels. is the standard deviation of the spatial Gaussian function, is the intensity Gaussian function, which is used to measure the similarity between pixel values. is the standard deviation of the intensity Gaussian function, For location and The Euclidean distance between The total number of pixels in the neighborhood.

5. The method for automatically detecting surface defects of printed matter using machine vision according to claim 3, characterized in that: In step S31, the process of enhancing the image contrast by using histogram equalization is to adjust the grayscale distribution of the image so that the histogram of the image is as flat as possible, thereby enhancing the contrast.

6. The method for automatically detecting surface defects of printed matter using machine vision according to claim 3, characterized in that: In step S32, the process of applying morphological operation to dilate, separate and highlight the defective area is to use the structural element to dilate the image, and the formula used is as follows: ； in, is the input image, is a structural element used to define the scope and shape of the dilation operation. is the expansion operator, is the pixel position in the output image, is the pixel position in the structure element.

7. The method for automatically detecting surface defects of printed matter using machine vision according to claim 1, characterized in that: In step S6, the quality prediction and defect classification process specifically includes the following steps: Step S61: Quality prediction: Use the trained RNN model to predict the time series data, analyze the prediction results, and determine the existence and severity of defects; Step S62: Defect classification: Apply the classification algorithm Random Forest to classify the defects, and combine the high-dimensional features output by the deep learning model CNN to improve the classification accuracy.

8. The method for automatically detecting surface defects of printed matter using machine vision according to claim 7, characterized in that: In step S61, the specific method of quality prediction includes the following steps: Step S611: Model prediction: Use the trained RNN model to predict the new time series data. The prediction formula is as follows: ； in, is the output predicted by the RNN model, For the trained RNN model, For time series data, is the RNN model parameter; Step S612: Analyze prediction results: To determine the presence and severity of defects, and set a threshold to determine whether a defect exists.

9. The method for automatically detecting surface defects of printed matter using machine vision according to claim 7, characterized in that: In step S62, the specific method of defect classification includes the following steps: Step S621: Feature extraction: Use the deep learning model CNN to extract high-dimensional features of the image. The high-dimensional feature extraction formula is as follows: ； in, To extract high-dimensional features, For the deep learning model CNN, is the processed input image, are the parameters of the deep learning model CNN; Step S622: Random forest classification, high-dimensional features extracted by deep learning model CNN Output of the RNN model Combine to form a new feature vector , and then classified using a random forest classifier.

10. The method for automatically detecting surface defects of printed matter using machine vision according to claim 9, characterized in that: In step S622, the specific method of random forest classification includes the following steps: Step K1: Feature vector merging: ； Step K2: Random forest training: Random forest consists of multiple decision trees, and the training formula for each decision tree is as follows: ； in, For decision tree, is the label in the training set; Step K3: Classification prediction: Use the trained random forest for classification prediction. The formula is as follows: ； in, is the predicted defect category, is a decision tree in a random forest.

Images