Industrial automation quality inspection device based on image recognition

Abstract

The present invention relates to the field of automated detection, and discloses an industrial automated quality detection device based on image recognition, which is used to realize quality detection of continuous image sequences in the production process of industrial products. The industrial automated quality detection method based on image recognition includes: using multi-channel image acquisition, LSTM model, GAN-EIS model and EIS model, through a series of technical means, to realize efficient and accurate quality detection of continuous image sequences in the production process of industrial products. The present invention uses multi-channel image data and fuse processing technology to obtain industrial product image sequences; then, the LSTM model is used to capture the change characteristics in the time dimension and predict potential quality problems; then, a large number of subtle defect images are generated using the GAN-EIS model to enhance the training data set; finally, the EIS model is combined to perform defect recognition, and the quality status of the product is comprehensively judged in combination with the time series change recognition results.

Description

Technical Field

[0001] The present invention relates to the field of automated detection, and in particular to an industrial automated quality detection device based on image recognition. Background Art

[0002] In modern manufacturing, quality control is an important part of ensuring that products meet specifications, improving customer satisfaction and maintaining market competitiveness. However, with the expansion of production scale and the increase in product complexity, traditional manual quality inspection methods can no longer meet the needs of efficiency and accuracy. Manual inspection is not only inefficient, but also easily affected by human factors, making it difficult to ensure the accuracy and consistency of the inspection results. In addition, in the face of large-scale, high-speed production lines, the cost of manual inspection has also risen sharply, becoming an important factor restricting the development of enterprises.

[0003] On the other hand, with the rapid development of computer vision and artificial intelligence technology, image recognition technology has gradually shown great potential in industrial production quality inspection. Image recognition technology can obtain high-definition images of product surfaces through high-resolution cameras and image acquisition systems, and use advanced algorithms to analyze and process images, thereby realizing automatic identification and classification of product defects. This non-contact detection method not only improves detection efficiency, but also avoids human errors and physical damage in manual detection, providing a new solution for industrial production quality inspection.

[0004] However, the existing image recognition technology still has some shortcomings in industrial automation quality inspection. First, the recognition accuracy and robustness of traditional image recognition methods need to be improved when dealing with products with complex shapes, structures and textures. Secondly, with the expansion of production scale and the increase in product types, higher requirements are placed on the real-time and stability of image recognition algorithms. In addition, factors such as lighting, background, and angle in industrial scenes will also have a significant impact on the accuracy of image recognition technology, and the algorithm needs to be further optimized to adapt to the complex and changing production environment.

[0005] To address the above shortcomings, this paper proposes an industrial automation quality inspection method based on image recognition. Summary of the invention

[0006] The present invention provides an industrial automation quality inspection device based on image recognition, which is used to realize efficient and accurate quality inspection of continuous image sequences in the production process of industrial products.

[0007] The first aspect of the present invention provides an industrial automation quality inspection method based on image recognition, the industrial automation quality inspection method based on image recognition comprising: acquiring multi-channel image data of industrial products, performing fuse processing according to the multi-channel image data, and obtaining an industrial product image sequence; processing a continuous image sequence of the industrial product image sequence according to an LSTM model, capturing change characteristics in the time dimension, analyzing the industrial product image sequence through a trained LSTM model, predicting potential quality problems, and extracting time series change recognition results; presetting a GAN framework and a generator network according to a real defect image data set in the industrial product image sequence, so that it can generate a defect image The invention discloses a method for detecting a defective image of an industrial product by using a GAN-EIS model to detect a defective image of an industrial product, including slight defects, and presetting a discriminator network so that it can distinguish between real defective images and generated images, and optimizing the performance of the generator and the discriminator through continuous adversarial training to obtain a trained GAN-EIS model; using the trained GAN-EIS model to generate a large number of slight defective images as training data, and using the generated slight defective images and the real defective image data set in the industrial product image sequence to train the EIS model to obtain a trained EIS model; and obtaining an EIS model defect recognition result based on the industrial product image sequence in combination with the trained EIS model; and obtaining the quality inspection status of the product based on the combination of the time series change recognition result and the EIS model defect recognition result.

[0008] Optionally, in a first implementation method of the first aspect of the present invention, the multi-channel image data of industrial products is acquired, and fuse processing is performed based on the multi-channel image data to obtain an industrial product image sequence, including: using a camera combination to capture image data of different angles and different resolutions on the production line to obtain multi-channel image data of industrial products; using GPU acceleration and distributed computing architecture to perform multi-channel parallel processing on the acquired multi-channel image data of industrial products: automatically dividing each image into multiple small blocks or areas to ensure that each block or area can be processed in parallel; using the parallel computing capability of the GPU to simultaneously process feature extraction tasks of multiple image blocks or areas; summarizing the results of parallel processing to form an industrial product image sequence; in multi-channel parallel processing, a dynamic load balancing mechanism is introduced to dynamically adjust the image segmentation strategy and task allocation according to the real-time load conditions of each processing node to ensure that all nodes work evenly and avoid idle or overloaded resources, and the number of tasks j assigned to node i is set to TA i，j ,but;

[0009]

[0010] Among them, TA i，j represents the number of tasks j assigned to node i; N represents the total number of nodes; TT j represents the total number of tasks j; CL iObtained in real time through the node monitoring system, including CPU usage, memory usage, and disk I / O; MC i Estimate based on the hardware configuration of the node and determine the maximum processing capacity of the node through performance testing; TT j Determine according to the amount of tasks in the actual application scenario; and / or create a thread pool to manage multiple threads through a multi-threaded programming library or thread pool technology, and preset a task queue to store the image tasks to be processed. Each image processing task is encapsulated as a task object, including image data and processing functions; preset one or more image processing functions, which will be called by threads in the thread pool to process image data, and set the pixel value of the output image at the position (x, y) in the image processing function to g(x, y), then:

[0011]

[0012] Among them, f(x+s,y+t) represents the pixel value of the input image at the position (x+s,y+t); w(s,t) is the value of the convolution kernel at the position (s,t); a and b define the size of the convolution kernel; the image processing task is assigned to multiple threads for simultaneous execution, and the images are aggregated to form an industrial product image sequence; in multi-threaded processing, a cross-thread data prefetching mechanism is introduced. Before the image processing task starts, the next image block to be processed is preloaded into the memory through another thread or coroutine to reduce the thread waiting time; and / or a hierarchical detection strategy is used: a camera is used to capture real-time images on the production line, and a first-level image processing algorithm is used for rapid screening to exclude obviously defective products. For the suspected defective products that are initially screened, a high-resolution camera and a second-level image recognition algorithm are used for fine detection to form an industrial product image sequence.

[0013] Optionally, in a second implementation of the first aspect of the present invention, the continuous image sequence of the industrial product image sequence is processed according to the LSTM model to capture the change characteristics in the time dimension, and the industrial product image sequence is analyzed through the trained LSTM model to predict potential quality problems and extract the time series change recognition results, including: further processing based on the obtained industrial product image sequence to ensure the continuity and consistency of the sequence, performing data enhancement on the image sequence, including rotation, scaling, and translation, and using optical flow method and dynamic texture analysis technology to extract dynamic features in the image sequence, that is, features that change over time; presetting the structure of the LSTM model, including the number of LSTM layers and the number of hidden units, and introducing regularization technology to prevent overfitting during the model training process; or presetting the structure of the LSTM model as a bidirectional LSTM (Bi -LSTM), while utilizing the previous and next information to better understand the context of the entire sequence and improve the accuracy of prediction; or use a more complex and powerful Transformer model as the teacher model, distilling its knowledge into a lighter LSTM student model, while maintaining performance, reducing the complexity of the model and improving its real-time and deployability; adopt a sequence-to-sequence learning framework, through an encoder-decoder structure, encode continuous image sequences into vectors of fixed length, and then decode them into predicted quality issues or time series change recognition results; and / or in order to capture the change characteristics at different time scales, introduce a multi-scale time feature fusion strategy, through parallel multiple LSTM networks, each network processes image sequences at different time scales, and then fuse their outputs to obtain more comprehensive time series change recognition results.

[0014] Optionally, in a third implementation manner of the first aspect of the present invention, the GAN framework is preset based on the real defect image data set in the industrial product image sequence, the generator network is preset so that it can generate defect images, including subtle defects, and the discriminator network is preset so that it can distinguish between real defect images and generated images. Through continuous adversarial training, the performance of the generator and the discriminator is optimized to obtain a trained GAN-EIS model, including: preprocessing the real defect image data set, including image standardization, denoising, and contrast enhancement, and implementing data enhancement strategies, including rotation, scaling, and translation; GAN framework preset: selecting a GAN architecture, using a deep convolutional generative adversarial network (DCGAN) as the basic framework; or using a conditional GAN ​​(cGAN) structure, so that the model can generate corresponding defect images according to conditions, including defect type, size, and position; presetting a deep convolutional neural network as a generator, whose goal is to generate images with subtle defects, introducing residual blocks and upsampling layer technology , to enhance the generator's ability to capture detailed features; in order to enable the generator to generate diverse defect images, random noise or conditional variables are introduced into the generator's input; a discriminator network matching the generator is preset to distinguish between real defect images and generated images, a deep convolutional neural network structure is adopted, and the discriminator is constructed using downsampling layers and fully connected layers; in order to improve the discriminator's discrimination ability, an attention mechanism or spectral normalization technology is introduced; the parameters of the generator and the discriminator are initialized, and the learning rate and optimizer are set to perform adversarial training, and the model performance is optimized by alternately updating the parameters of the generator and the discriminator. During the training process, regularization technology is introduced to prevent overfitting, and the changes in the loss function and the quality of the generated images during the training process are monitored, and the training strategy is adjusted in time; and / or refer to the progressive growth GAN, starting from low-resolution images and gradually increasing the resolution of the generated images, so that the model first learns coarse-grained features, and then gradually refines them to improve the quality and stability of the generated images; use Inception The Score evaluation indicator is used to evaluate the trained GAN-EIS model, and the model is improved according to the evaluation results, including adjusting the network structure and increasing training data; thus, a trained GAN-EIS model is obtained.

[0015] Optionally, in a fourth implementation manner of the first aspect of the present invention, a large number of subtle defect images are generated using the trained GAN-EIS model as training data, the generated subtle defect images and a real defect image data set in the industrial product image sequence are used to train the EIS model to obtain a trained EIS model, and an EIS model defect recognition result is obtained based on the industrial product image sequence in combination with the trained EIS model, including: using the trained GAN-EIS model to generate a large number of subtle defect images, including subtle defects of various types, sizes and positions, performing quality assessment on the generated subtle defect images, screening out high-quality and representative images, and constructing a first-level subtle defect image data set; fusing the generated first-level subtle defect image data set with the real defect image data set to form a second-level subtle defect image data set, and performing data enhancement on the fused data set, including rotation, scaling, translation, and color transformation; selecting a deep learning model of a convolutional neural network as the basis of the EIS model According to the characteristics of subtle defects, the attention mechanism and multi-scale feature fusion are introduced to improve and optimize the EIS model; the fused secondary subtle defect image dataset and the real defect image dataset are used to train the EIS model, and the loss function and optimizer are used to update the parameters of the model through the back propagation algorithm. During the training process, the early stopping method is implemented to prevent the model from overfitting. The trained EIS model is evaluated using an independent validation set, and the accuracy, recall, and F1 score indicators are calculated. According to the evaluation results, the model is adjusted and optimized, including modifying the network structure and adjusting the hyperparameters; and / or the subtle defect images generated by GAN-EIS are used as adversarial samples to verify the robustness of the EIS model. By continuously adjusting the generation strategy of GAN-EIS, more and more challenging defect images are created to test and improve the performance of the EIS model; a trained EIS model is obtained; the industrial product image sequence is input into the trained EIS model for defect recognition, and the defect recognition result of the EIS model is obtained.

[0016] Optionally, in a fifth implementation of the first aspect of the present invention, the quality inspection status of the product is obtained based on the combination of the time series change identification results and the EIS model defect identification results, including: selecting timestamp synchronization or image frame number synchronization as a data synchronization mechanism, preprocessing the time series change identification results and the EIS model defect identification results to ensure that they have the same time range or image frame range, and adding a corresponding timestamp or image frame number to each data, and matching the time series data and defect identification data by comparing the timestamps or image frame numbers to ensure their synchronization in time and space; presetting a weighted scoring system, in which time The mutation of the time series and the defects identified by the EIS model will increase the doubt score of the product. Different weights are assigned to the time series mutation and defect identification results to reflect their different impacts on product quality. The weighted scoring system is used to calculate the doubt score of the product. The threshold is set according to the doubt score to classify the product into different quality levels or qualified / unqualified categories. The mutation score set M = {m1, m2, ..., mn} of the time series data and the score set D = {d1, d2, ..., dm} of the defect identification results are set, where mi and di represent the mutation score of the time series at the i-th time point and the score of the defect identification result at the i-th time point, respectively. Then:

[0017]

[0018] Among them, w m and w d are the weights of time series mutation and defect recognition results, respectively. and are the sum of the time series mutation score and the defect recognition result score, n and m are the number of time series data and defect recognition results, respectively; the quality inspection status of the product is obtained; or the time series change recognition result and the EIS model defect recognition result are fused using a serial fusion mechanism, the time series change recognition result is used as part of the feature vector, the EIS model defect recognition result is used as another part of the feature vector, they are serialized into a complete feature vector, the fused feature vector is used as input, a quality inspection model is preset to calculate the quality inspection status of the product, and the product is divided into different quality grades or qualified / unqualified categories according to the output of the quality inspection model; the feature vector X with the time series change recognition result is set to [x1, x2, ..., xn] and EIS model defect recognition result is set to [x1, x2, ..., xn]. The feature vector Y=[y1,y2,...,ym] of the S model defect recognition result, where xi and yi represent the i-th feature of the time series change recognition result and the EIS model defect recognition result respectively, and the feature vector after serial fusion is Z: Z=[x1,x2,...,xn,y1,y2,...,ym]; wherein Z is a feature vector containing n+m features, the first n features come from the time series change recognition result, and the last m features come from the EIS model defect recognition result; the quality inspection model f is used to calculate the quality inspection status Q of the product: Q=f(Z); wherein f is a machine learning model, including a support vector machine (SVM) and a neural network, Z is the input feature vector, and Q is the output quality inspection status; the quality inspection status of the product is obtained.

[0019] Optionally, in a sixth implementation of the first aspect of the present invention, it also includes: setting a dynamic threshold mechanism, which dynamically adjusts the standard for determining product quality based on historical data and real-time data. If a certain type of product has frequently had a certain minor defect in history, the system automatically adjusts its sensitivity to this type of defect; and / or developing an intuitive visualization interface to display time series changes, defect identification results, and comprehensive quality inspection status, so that quality inspectors can quickly understand the overall quality of the product through the interface; and / or establishing a feedback loop to allow quality inspectors to verify and correct the system's judgment. These feedbacks will be used to continuously optimize the EIS model, time series analysis algorithm, and result fusion mechanism; and / or integrating data from different sensors or detection methods, including temperature, pressure, and vibration, to form a multimodal quality inspection system, providing more comprehensive product quality information, and revealing defects that are difficult to find in a single mode; and / or based on the fused quality inspection status, presetting an early warning system to automatically trigger an alarm when a potential quality problem is detected, integrating automatic intervention measures, including suspending the production line and isolating suspected defective products; and / or uploading quality inspection data to a cloud platform, and using big data analysis tools for long-term trend analysis and fault prediction.

[0020] The second aspect of the present invention provides an industrial automation quality inspection device based on image recognition, and the industrial automation quality inspection device based on image recognition includes: an acquisition module, which is used to acquire multi-channel image data of industrial products, and perform fuse processing based on the multi-channel image data to obtain an industrial product image sequence; a processing module, which is used to process the continuous image sequence of the industrial product image sequence based on the LSTM model, capture the change characteristics in the time dimension, analyze the industrial product image sequence through the trained LSTM model, predict potential quality problems, and extract the time series change recognition result; a setting module, which is used to preset the GAN framework based on the real defect image data set in the industrial product image sequence, preset the generator network, so that it can generate Defect images, including subtle defects, are preset with a discriminator network, so that it can distinguish between real defect images and generated images, and the performance of the generator and the discriminator is optimized through continuous adversarial training to obtain a trained GAN-EIS model; an allocation module is used to generate a large number of subtle defect images using the trained GAN-EIS model as training data, and the generated subtle defect images and the real defect image data set in the industrial product image sequence are used to train the EIS model to obtain a trained EIS model, and the EIS model defect recognition result is obtained based on the industrial product image sequence combined with the trained EIS model; a calculation module is used to obtain the product quality inspection status based on the time series change recognition result and the EIS model defect recognition result.

[0021] The third aspect of the present invention provides an industrial automation quality inspection device based on image recognition, comprising: a memory and at least one processor, wherein the memory stores instructions; the at least one processor calls the instructions in the memory so that the industrial automation quality inspection device based on image recognition executes the above-mentioned industrial automation quality inspection method based on image recognition.

[0022] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute the above-mentioned industrial automation quality inspection method based on image recognition.

[0023] In the technical solution provided by the present invention, multi-channel image acquisition, LSTM model, GAN-EIS model and EIS model are comprehensively used, and through a series of innovative technical means, efficient and accurate quality inspection of continuous image sequences in the production process of industrial products is achieved.

[0024] Beneficial effects:

[0025] Improve detection accuracy and efficiency: Through multi-channel image acquisition and parallel processing technology, the acquisition and processing speed of image data is improved; the combination of LSTM model and EIS model significantly improves the accuracy of quality detection and reduces missed detection and false detection.

[0026] Reduce detection costs: Automated detection replaces traditional manual detection, reducing labor costs; at the same time, defect images are generated through the GAN-EIS model, reducing the demand for actual defect samples and further reducing detection costs.

[0027] Enhance system robustness: The introduction of dynamic load balancing, regularization technology, dynamic threshold mechanism and feedback loop enables the system to adapt to different production environments and product characteristics, improving the robustness and stability of the system.

[0028] Provide comprehensive quality information: By integrating multimodal data (such as temperature, pressure, vibration, etc.), the system can provide more comprehensive product quality information, helping to reveal potential problems that are difficult to detect under a single modality.

[0029] Support real-time early warning and intervention: The early warning system and automatic intervention measures based on quality inspection results can trigger alarms in time when potential quality problems are detected, and take corresponding intervention measures to prevent unqualified products from entering the market and ensure product quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of an embodiment of an industrial automation quality inspection method based on image recognition in an embodiment of the present invention;

[0031] Figure 2 It is a schematic diagram of another embodiment of the industrial automation quality inspection method based on image recognition in an embodiment of the present invention;

[0032] Figure 3 A schematic diagram of an embodiment of an industrial automation quality inspection device based on image recognition in an embodiment of the present invention;

[0033] Figure 4 It is a schematic diagram of an embodiment of an industrial automation quality inspection device based on image recognition in an embodiment of the present invention. DETAILED DESCRIPTION

[0034] The embodiment of the present invention provides an industrial automation quality inspection device based on image recognition, which is used to efficiently and accurately inspect the quality of a continuous image sequence of industrial products. The terms "including" or "having" and their variations are intended to cover non-exclusive inclusions, that is, the process, method, system, product or device may include other steps or units that are not listed or inherent. For ease of understanding, the specific process of the embodiment is described below (see Figure 1), an embodiment of an industrial automation quality inspection method based on image recognition in an embodiment of the present invention includes:

[0035] 101. Acquire multi-channel image data of industrial products, and perform fuse processing according to the multi-channel image data to obtain an industrial product image sequence;

[0036] It is understandable that the execution subject of the present invention may be an industrial automation quality inspection device based on image recognition, or a terminal or a server, which is not specifically limited here. The embodiment of the present invention is described by taking a server as the execution subject as an example.

[0037] It should be noted that we are performing quality inspection on an automotive parts production line, with the goal of detecting surface defects of parts. Due to the complex shapes of parts and high inspection requirements, we need to capture image data from multiple angles and process them to form a complete image sequence.

[0038] Implementation steps: 1. Obtain multi-channel image data: Equipment configuration: Install multiple high-definition industrial cameras on the production line, each camera is responsible for capturing images at different angles. For example, 4 cameras can be set up, located above, below, on the left and right of the parts, and the resolution of each camera is 4096x3072 pixels. Image acquisition: When the parts pass through the inspection area, trigger all cameras to shoot at the same time to obtain images of the parts from four different perspectives. Set the camera shooting rate to 30 frames per second, then for each part, we will obtain 4 perspectives and a total of 120 images per second. Data storage: The collected image data is transmitted to the server's hard disk array in real time for storage to ensure data integrity and accessibility. 2. Perform fuse processing: Time synchronization: First, ensure that the timestamps of all cameras are synchronized so that images from different cameras can be accurately matched during subsequent processing. Image screening: Since the speed of the production line is not constant and the time when the parts pass through the inspection area will vary, it is necessary to screen the collected images and only retain images where the parts are completely in the camera's field of view. This can be achieved through image processing algorithms, such as detecting specific markers or edges in the image to determine the location of the parts. Image stitching: For each component, stitch its images from four different perspectives. This involves algorithms such as feature point matching, affine transformation or perspective transformation to ensure that the stitched images are geometrically consistent. Sequence generation: Arrange the stitched images in chronological order to form a complete industrial product image sequence. If the time it takes for a component to pass through the inspection area is 1 second (i.e. 30 frames of images), the final image sequence generated will contain 30 stitched images. In the above embodiment, "fuse processing" actually refers to the process of screening, stitching and serializing multi-channel image data.

[0039] Through the above embodiments, we can clearly see how to obtain image data of industrial products from multiple angles, and generate a complete image sequence through a series of processing steps to provide reliable data support for subsequent quality inspection.

[0040] 102. Process the continuous image sequence of the industrial product image sequence according to the LSTM model to capture the change characteristics in the time dimension, analyze the industrial product image sequence through the trained LSTM model, predict potential quality problems, and extract the time series change recognition result;

[0041] It should be noted that we have obtained a continuous image sequence of industrial products, which consists of image frames arranged in time order, and each frame is a snapshot of the product on the production line. The goal is to use the LSTM model to capture the changing characteristics of the image in the time dimension and predict potential quality problems. Implementation steps: Image preprocessing: Normalization: Adjust the image frame size to 256x256 pixels, and normalize the pixel value to between 0 and 1. Feature extraction: Convert the image into a feature vector that can be processed by LSTM through CNN. Construct LSTM model: Model structure: Design an LSTM network with one or more LSTM layers, each of which can be followed by a fully connected layer. Finally, the prediction results are output through the output layer. Parameter setting: Set the LSTM layer parameters according to the task complexity and data set size, such as two layers of LSTM, 128 units per layer, and a dropout rate of 0.2. Data preparation and sequence partitioning: Sequence partitioning: Divide the subsequences according to the time window size (such as 10 frames of images per sequence) as the input samples of LSTM. Label generation: Generate corresponding labels based on the product quality reflected by the images in the subsequence, which can be binary classification, multi-classification or regression values. Model training: Training data: Use the divided subsequences and their labels. Loss function and optimizer: Select the appropriate loss function and optimizer according to the prediction task. Training process: Iteratively train the LSTM model to minimize the loss function. Use the validation set to monitor performance and stop early to prevent overfitting. Prediction and result extraction: Model evaluation: Use the test set to evaluate the model and calculate indicators such as accuracy, recall, and F1 score. Prediction: Input the new image sequence into the trained LSTM model and output the prediction result. Result extraction: Extract the time series change recognition results, including the starting point, duration, severity, and other information of the quality problem.

[0042] Specific data example: We have an image sequence dataset containing 100 industrial products, and the image sequence of each product consists of 50 frames. We divide each sequence into subsequences containing 10 frames of images (with an overlap rate of 5 frames), so each product will generate 41 subsequences as training samples. Our task is a binary classification prediction (qualified / unqualified), so each subsequence will correspond to a label (0 or 1). During the training process, we will find that the model reaches the best performance (highest accuracy on the validation set) at the 20th epoch, and then begins to overfit. Therefore, we select the model weights of the 20th epoch as the final model.

[0043] 103. Preset a GAN framework based on a real defect image data set in the industrial product image sequence, preset a generator network so that it can generate defect images including subtle defects, preset a discriminator network so that it can distinguish between real defect images and generated images, and optimize the performance of the generator and the discriminator through continuous adversarial training to obtain a trained GAN-EIS model;

[0044] It should be noted that we have a dataset of 10,000 real industrial product defect images, covering subtle defects such as scratches, pits, and uneven colors. The goal is to build a GAN-EIS model to generate realistic defect images and maintain the ability to capture subtle features. Implementation steps: Data preprocessing: Data cleaning: Remove noise and duplicate images to ensure the purity of the dataset. Normalization: The image is scaled to 256x256 pixels, and the pixel values ​​are normalized to [0,1]. Data partitioning: Divided into training set (80%), validation set (10%), and test set (10%). GAN framework presets: Generator network: Design a deep convolutional network, input random noise (can contain conditional vectors), and output a generated defect image. Contains transposed convolutional layer, residual block, and batch normalization. Discriminator network: Design a deep convolutional network, input real or generated images, and output true probability. Contains convolutional layer, fully connected layer, and can introduce spectral normalization and Leaky ReLU. Model training: Loss function: Generator loss includes discriminator "fake" label loss and content loss; discriminator loss is cross entropy loss. Optimizer: Select Adam optimizer and set learning rate. Adversarial training: Alternately optimize the generator and discriminator, use the validation set to monitor performance, and adjust parameters when appropriate. Training process example: Number of iterations: 100,000 times in total. Learning rate adjustment: Initial 0.0002, halved every 10,000 times. Model saving: Save model weights every 5,000 times. Model evaluation and selection: Inception Score: Evaluate the quality and diversity of generated images. FID Score: Calculate the Fréchet Inception Distance between generated and real images. Manual evaluation: Experts check the realism and subtle defects of generated images. Select the best model: Based on Inception Score, FID Score and manual evaluation results.

[0045] Specific data example: After the above training process, we obtained a GAN-EIS model with excellent performance. The model is able to generate realistic images with subtle defects (such as 0.1mm wide scratches, 0.2mm deep pits, etc.). By calculating the FID Score, we found that the FID value of the generated image and the real image is 12.34, indicating that the two are very close in feature space. At the same time, the manual evaluation results show that the generated defect images are visually difficult to distinguish from the real defect images.

[0046] 104. Generate a large number of subtle defect images using the trained GAN-EIS model as training data, use the generated subtle defect images and a real defect image dataset in the industrial product image sequence to train the EIS model to obtain a trained EIS model, and obtain an EIS model defect recognition result based on the industrial product image sequence combined with the trained EIS model;

[0047] It should be noted that we already have a dataset of 10,000 real industrial product defect images and a trained GAN-EIS model that can generate realistic subtle defect images. The goal is to use GAN-EIS to generate a large number of defect images, train the EIS model together with real images, and improve the accuracy and robustness of defect recognition. Implementation steps: Generate subtle defect images: Use the GAN-EIS model: Input random noise (can contain conditional vectors) to generate a large number of subtle defect images. Quality screening: Remove blurry, repeated or non-compliant images. Dataset integration: Merge datasets: Merge the filtered generated images with the original dataset. Data enhancement (optional): Improve the generalization ability of the model, such as rotation, scaling, etc. Train the EIS model: Model selection: Select a deep learning model, such as CNN. Data preprocessing: Normalization, cropping, resizing, etc. Model training: Set the loss function, optimizer, training parameters, and use the validation set to monitor performance. Model evaluation and selection: Evaluation indicators: Calculate accuracy, recall, F1 score, etc. Model selection: Select the best model based on the evaluation results. Defect recognition: Input image sequence: Input the image sequence to be detected into the EIS model. Defect recognition: The model outputs the defect recognition results of each frame of image. Result summary: Analyze and generate defect recognition report.

[0048] Specific data example: Number of generated images: 90,000 images of subtle defects were generated using the GAN-EIS model. Data enhancement: The merged dataset was subjected to data enhancement processing such as rotation (±10 degrees), scaling (0.9 to 1.1 times), and flipping (horizontal or vertical). Model training: The Adam optimizer was used, the initial learning rate was set to 0.001, the batch size was 64, and a total of 50 rounds were trained. During the training process, the early stopping method was used to prevent overfitting. Evaluation results: On the test set, the EIS model achieved an accuracy of 95%, a recall of 92%, and an F1 score of 93%.

[0049] Defect recognition report: For the input industrial product image sequence, the EIS model successfully identified all obvious defects and had a high recognition rate for subtle defects (such as scratches as wide as 0.2 mm). The recognition results are displayed in a chart form, which is convenient for quality inspectors to quickly view and process.

[0050] 105. Obtaining a quality inspection status of a product based on the time series change recognition result and the EIS model defect recognition result;

[0051] It should be noted that the background of the implementation example: time series change recognition (LSTM model analyzes image sequence and predicts quality problems) and EIS model defect recognition (recognizes defects in each frame of image) have been completed. Implementation steps: Data synchronization and preprocessing: Data synchronization: ensure that the two types of recognition results are synchronized in time and space. Preprocessing: Quantify change characteristics and organize defect information. Result fusion strategy: Weighted scoring system: Quantify and score the two types of recognition results and assign different weights (0.6 for time series changes and 0.4 for EIS model defects). Assign scores based on the degree of change and the type and size of the defect. Comprehensive evaluation: Calculate the total score: Combine the weighted scoring system to calculate the total score for each time period or product. Set thresholds: Set thresholds based on historical data and needs to judge product quality. Output quality inspection status: Based on the total score and threshold, output the product quality inspection status and display it in the form of a table, chart or report.

[0052] The specific data examples are shown in Table 1:

[0053] Table 1

[0054]

[0055]

[0056] In this example, we combined the time series change recognition results and the EIS model defect recognition results, and obtained the quality inspection status of the product in each time period through a weighted scoring system and threshold judgment. This method comprehensively considers the product's change characteristics over time and specific defect information, and improves the accuracy and comprehensiveness of quality inspection.

[0057] In the embodiment of the present invention, the application of multi-channel image acquisition and LSTM model can fully capture the multi-angle images and time series change characteristics of the product, and significantly improve the accuracy of quality inspection. At the same time, the automated inspection process reduces manual intervention and improves the inspection efficiency; automated inspection replaces the traditional manual inspection method and reduces labor costs. In addition, the GAN-EIS model can generate a large number of defect images as training data, reducing the demand for actual defect samples and further reducing the inspection cost; the training data set is expanded by the defect images generated by GAN, and the generalization ability of the EIS model is improved. At the same time, the combined use of the LSTM model and the EIS model enables the system to more comprehensively identify different types of defects and enhance the robustness of the system; the time series change recognition function can monitor the changes in product quality in real time. Once a potential quality problem is found, an alarm can be triggered immediately and corresponding intervention measures can be taken to prevent unqualified products from entering the market; it promotes the intelligent upgrading of industrial production and provides strong support for the digital transformation of enterprises. By continuously optimizing algorithms and models, the intelligence level of the detection system can be further improved, and the competitiveness and market position of enterprises can be improved. In summary, the technical solution of the present invention has significant advantages and broad application prospects in the field of industrial automation quality inspection.

[0058] See also Figure 2 Another embodiment of the industrial automation quality inspection method based on image recognition in the embodiment of the present invention includes:

[0059] 201. Acquire multi-channel image data of industrial products, and perform fuse processing according to the multi-channel image data to obtain an industrial product image sequence;

[0060] Specifically, a camera combination is used to capture image data of different angles and resolutions on the production line to obtain multi-channel image data of industrial products; GPU acceleration and distributed computing architecture are used to perform multi-channel parallel processing on the acquired multi-channel image data of industrial products: each image is automatically divided into multiple small blocks or regions to ensure that each block or region can be processed in parallel; the parallel computing capability of the GPU is used to simultaneously process feature extraction tasks of multiple image blocks or regions; the results of parallel processing are summarized to form an industrial product image sequence; in multi-channel parallel processing, a dynamic load balancing mechanism is introduced to dynamically adjust the image segmentation strategy and task allocation according to the real-time load of each processing node to ensure that all nodes work evenly and avoid idle or overloaded resources. The number of tasks j assigned to node i is set to TA i，j ,but;

[0061]

[0062] Among them, TA i，jrepresents the number of tasks j assigned to node i; N represents the total number of nodes; TT j represents the total number of tasks j; CL i Obtained in real time through the node monitoring system, including CPU usage, memory usage, and disk I / O; MC i Estimate based on the hardware configuration of the node and determine the maximum processing capacity of the node through performance testing; TT j Determine according to the amount of tasks in the actual application scenario; and / or create a thread pool to manage multiple threads through a multi-threaded programming library or thread pool technology, and preset a task queue to store the image tasks to be processed. Each image processing task is encapsulated as a task object, including image data and processing functions; preset one or more image processing functions, which will be called by threads in the thread pool to process image data, and set the pixel value of the output image at the position (x, y) in the image processing function to g(x, y), then:

[0063]

[0064] Among them, f(x+s,y+t) represents the pixel value of the input image at the position (x+s,y+t); w(s,t) is the value of the convolution kernel at the position (s,t); a and b define the size of the convolution kernel; the image processing task is assigned to multiple threads for simultaneous execution, and the industrial product image sequence is aggregated; in multi-threaded processing, a cross-thread data pre-fetching mechanism is introduced, and before the image processing task starts, the next image block to be processed is pre-loaded into the memory through another thread or coroutine to reduce the thread waiting time; and / or a hierarchical detection strategy is used: a camera is used to capture real-time images on the production line, and a first-level image processing algorithm is used for rapid screening to exclude obviously defective products. For the suspected defective products that are initially screened, a high-resolution camera and a second-level image recognition algorithm are used for fine detection to form an industrial product image sequence;

[0065] It should be noted that in industrial automation quality inspection based on image recognition, the acquisition and parallel processing of multi-channel image data is a complex but crucial process. The following is a specific embodiment, including a detailed description of camera combination, GPU acceleration, distributed computing architecture and dynamic load balancing mechanism, with specific data attached.

[0066] Six high-definition industrial cameras (5472x3648 pixels, 20 frames per second) are deployed on the production line, and the data is stored in a RAID hard disk array. NVIDIA Tesla V100 GPU (32GB HBM2 memory) is used to accelerate image processing, and a distributed computing cluster with 10 nodes is built, with 2 GPUs and Intel Xeon Gold processors on each node. Using the Hadoop / Spark framework, the task is decomposed into subtasks for parallel execution. The image is automatically segmented into 64x64 pixel blocks, with overlapping areas (16 pixels), and GPU parallel processing is used to extract features. Nodes communicate through high-speed networks, monitor performance indicators in real time, and dynamically adjust task allocation. The processing results are summarized to the central server, reassembled into complete component images, and arranged in time sequence to generate industrial product image sequences.

[0067] In industrial automated quality inspection based on image recognition, it is an effective method to use multithreading or thread pool technology to achieve parallel processing: Create thread pool and task queue: Use Java's ExecutorService or C++'s std::thread library to create a fixed-size thread pool, the size of which is determined by the number of CPU cores. Use Java's BlockingQueue or C++'s std::queue to create a thread-safe task queue. Encapsulate image processing tasks: Define the ImageProcessingTask class and implement the Runnable interface or std::function / lambda in C++. The task contains image data and processing functions, such as image filtering functions, which accept input images and convolution kernels and output processed images. Multithreaded processing and task allocation: Encapsulate image processing tasks into ImageProcessingTask instances and add them to the task queue. Threads in the thread pool continuously take out tasks to execute, and wait for new tasks when the queue is empty. Cross-thread data prefetching: Create a prefetch thread to preload the next image block before image processing. Use progress monitoring or signal mechanisms to ensure that prefetched data does not interfere with current processing data, and use cache or memory pool management. Aggregate to form an image sequence: After processing is completed, the results are stored in a shared data structure or output file / database. After all tasks are completed, the processed image blocks are read sequentially and aggregated to form a complete image sequence.

[0068] Specific data example: thread pool size: 8; task queue capacity: 100; image size: 1024x768 pixels; convolution kernel size: 3x3; prefetch buffer size: sufficient to store the amount of data for the next image block to be processed (for example, for a 1024x768 image, the prefetch buffer size is the number of bytes occupied by the image); processing time: a single image processing task takes about 100 milliseconds on a single-core CPU. After using 8 threads for parallel processing, the overall processing time can be reduced to about 12.5 milliseconds (assuming that the processing time decreases linearly, and ignoring the overhead of thread creation and destruction).

[0069] Through this embodiment, we can efficiently process a large number of image processing tasks in parallel, reduce processing time, and improve the efficiency and accuracy of industrial automation quality inspection.

[0070] In industrial automation quality inspection based on image recognition, the use of a hierarchical inspection strategy can improve efficiency and accuracy. Implementation steps: 1. Camera configuration: The first-level inspection uses multiple standard-resolution industrial cameras to cover the production line and capture images in real time; the second-level inspection uses a high-resolution camera to take detailed photos of suspected defective parts. 2. Hierarchical inspection process: The first-level inspection quickly screens defective products and reduces the number of fine inspections required. The production line is set to pass 100 parts per minute, and the first-level inspection processes each image within a few milliseconds, eliminating about 80% of defective products. The second-level inspection uses high-resolution cameras and deep learning models to finely identify defects. 3. Data aggregation and analysis: Summarize the inspection results, form image sequences and reports, and regularly analyze and evaluate performance and optimize. The specific data is as shown in Table 2:

[0071] Table 2

[0072]

[0073] Through this embodiment, we can effectively reduce the number of products that need to be carefully inspected, improve inspection efficiency, and ensure the accuracy of inspection results. At the same time, the hierarchical inspection strategy can also be flexibly adjusted and optimized according to actual production conditions.

[0074] 202. Processing the continuous image sequence of the industrial product image sequence according to the LSTM model to capture the change characteristics in the time dimension, analyzing the industrial product image sequence through the trained LSTM model, predicting potential quality problems, and extracting the time series change recognition result;

[0075] Specifically, further processing is performed based on the obtained industrial product image sequence to ensure the continuity and consistency of the sequence, and data enhancement is performed on the image sequence, including rotation, scaling, and translation. The dynamic features in the image sequence, that is, features that change over time, are extracted using the optical flow method and dynamic texture analysis technology; the structure of the LSTM model is preset, including the number of LSTM layers and the number of hidden units. During the model training process, regularization technology is introduced to prevent overfitting; or the structure of the LSTM model is preset to be a bidirectional LSTM (Bi-LSTM), and the previous and next information is used at the same time to better understand the context of the entire sequence and improve the accuracy of the prediction; or a more complex and powerful Tra The nsformer model is used as a teacher model to distill its knowledge into a lighter LSTM student model, while maintaining performance, reducing the complexity of the model and improving its real-time and deployability; a sequence-to-sequence learning framework is adopted, and a continuous image sequence is encoded into a fixed-length vector through an encoder-decoder structure, and then decoded into a predicted quality problem or time series change recognition result; and / or in order to capture the change characteristics at different time scales, a multi-scale time feature fusion strategy is introduced, through parallel multiple LSTM networks, each network processes image sequences at different time scales, and then their outputs are fused to obtain a more comprehensive time series change recognition result;

[0076] It should be noted that in industrial automation quality inspection based on image recognition, image sequence processing and data enhancement include ensuring sequence continuity and consistency, performing data enhancement (rotation ±10°, scaling 0.9 to 1.1 times, translation ±5% image width / height) to increase model generalization capabilities, and extracting dynamic features using optical flow and dynamic texture analysis. LSTM model configuration and optimization include presetting standard LSTM and bidirectional LSTM structures, using L2 regularization to prevent overfitting, migrating complex model knowledge to lightweight LSTM through Transformer distillation, using encoder-decoder structure for sequence-to-sequence learning, and running multiple LSTM networks in parallel and fusing their outputs to obtain more comprehensive time series change recognition results.

[0077] Specific data: LSTM model parameters: standard LSTM or Bi-LSTM, 2 layers, 128 hidden units. Transformer distillation results: teacher model accuracy 98%, student model (LSTM) accuracy 95%, but the computational complexity is significantly reduced. Multi-scale time feature fusion performance: Compared with single-scale LSTM, the multi-scale fusion strategy improves the F1 score of time series change recognition by about 5%.

[0078] Through this step, we have demonstrated how to further process image sequences, perform data augmentation, and configure and optimize LSTM models in industrial automated quality inspection based on image recognition. Together, these steps improve the accuracy and robustness of the quality inspection system while taking into account the requirements of real-time and deployability.

[0079] 203. Preset a GAN framework based on a real defect image data set in the industrial product image sequence, preset a generator network so that it can generate defect images including subtle defects, preset a discriminator network so that it can distinguish between real defect images and generated images, and optimize the performance of the generator and the discriminator through continuous adversarial training to obtain a trained GAN-EIS model;

[0080] Specifically, the real defect image dataset is preprocessed, including image standardization, denoising, contrast enhancement, and data enhancement strategies are implemented, including rotation, scaling, and translation; GAN framework presets: Select the GAN architecture and use the deep convolutional generative adversarial network (DCGAN) as the basic framework; or adopt the conditional GAN ​​(cGAN) structure so that the model can generate corresponding defect images according to conditions, including defect type, size, and location; preset a deep convolutional neural network as a generator, whose goal is to generate images with subtle defects, introduce residual blocks and upsampling layer technology to enhance the generator's ability to capture detailed features; in order to enable the generator to generate diverse defect images, introduce random noise or conditional variables into the generator's input; preset a discriminator network that matches the generator to distinguish between real defect images and generated images, adopt a deep convolutional neural network structure, and use downsampling to obtain the real defect image. The discriminator is constructed by using the sample layer and the fully connected layer; in order to improve the discrimination ability of the discriminator, the attention mechanism or spectral normalization technology is introduced; the parameters of the generator and the discriminator are initialized, and the learning rate and the optimizer are set to perform adversarial training, and the model performance is optimized by alternately updating the parameters of the generator and the discriminator. During the training process, regularization technology is introduced to prevent overfitting, and the loss function changes and the quality of the generated images during the training process are monitored, and the training strategy is adjusted in time; and / or the progressive growth GAN is used to gradually increase the resolution of the generated images from low-resolution images, so that the model first learns coarse-grained features and then gradually refines them to improve the quality and stability of the generated images; the trained GAN-EIS model is evaluated using the InceptionScore evaluation indicator, and the model is improved according to the evaluation results, including adjusting the network structure and increasing the training data; a trained GAN-EIS model is obtained;

[0081] It should be noted that in industrial automation quality inspection based on image recognition, data preprocessing and GAN framework presetting and training are key processes. Data preprocessing includes image standardization (converted to 256x256 pixels, pixel values ​​normalized to [0,1]), denoising (median filtering, Gaussian filtering, etc.), contrast enhancement (histogram equalization, etc.) and data enhancement (rotation ±15°, scaling 0.8 to 1.2 times, translation ±10% image width / height). The GAN framework selects DCGAN, and cGAN can be used to generate defective images according to conditions. The generator design includes deep convolutional neural networks, transposed convolutional layers, residual blocks, and introduces random noise or conditional variables. The discriminator design matches the generator, including convolutional layers and downsampling layers, and can introduce attention mechanisms or spectral normalization techniques. The training process includes adversarial loss and content loss functions, selects the Adam optimizer, alternately updates the generator and discriminator parameters, monitors the training process and adjusts it in time.

[0082] Initialize model parameters and set training configuration: Use random initialization method to initialize generator and discriminator weights, and set bias term to 0. Set initial learning rate to 0.0002, and use Adam optimizer (β1=0.5, β2=0.999) for training, which can be adjusted according to performance. Alternately update generator and discriminator to minimize their respective loss functions. Introduce L2 regularization to prevent overfitting, and the regularization coefficient is adjustable. Monitor the loss function and generated image quality during training. Use progressive growth GAN strategy, start training from low resolution, and gradually increase the number of layers and convolution kernels. Evaluation and improvement: Use InceptionScore to evaluate the quality and diversity of generated images, and adjust the model according to the evaluation results, including modifying the network structure, adding convolutional layers or residual blocks, adjusting the regularization coefficient, etc. If the model performs poorly on certain defects, you can add relevant types of training data.

[0083] Specific data and results: A total of 50 epochs were trained, and each epoch traversed the entire training set. The quality of the generated images improved with training. The Inception Score reached a peak of 8.5 at the 30th epoch, and then the risk score slightly decreased due to overfitting. The 30th epoch model was selected as the best. The GAN-EIS model can generate high-quality images with subtle defects that are difficult to distinguish from real images. The FID score is 15.2, indicating that the generated images have high similarity with the real image features.

[0084] Through the above examples, we can obtain a trained GAN-EIS model, which can be used to generate realistic industrial defect images and provide strong support for industrial automation quality inspection.

[0085] 204. Generate a large number of subtle defect images using the trained GAN-EIS model as training data, use the generated subtle defect images and a real defect image dataset in the industrial product image sequence to train the EIS model to obtain a trained EIS model, and obtain an EIS model defect recognition result based on the industrial product image sequence combined with the trained EIS model;

[0086] Specifically, a large number of subtle defect images containing subtle defects of various types, sizes and positions are generated using the trained GAN-EIS model. The quality of the generated subtle defect images is evaluated, and high-quality and representative images are screened out to construct a first-level subtle defect image dataset. The generated first-level subtle defect image dataset is fused with the real defect image dataset to form a second-level subtle defect image dataset, and the fused dataset is enhanced, including rotation, scaling, translation, and color transformation. The deep learning model of the convolutional neural network is selected as the basis of the EIS model, and the attention mechanism and multi-scale feature fusion are introduced to improve and optimize the EIS model according to the characteristics of subtle defects. The fused second-level subtle defect image dataset and the real defect image dataset are used to train the EIS. The model adopts a loss function and an optimizer, updates the parameters of the model through a back-propagation algorithm, implements an early stopping method during the training process to prevent the model from overfitting, uses an independent validation set to evaluate the trained EIS model, calculates the accuracy, recall, and F1 score indicators, and adjusts and optimizes the model according to the evaluation results, including modifying the network structure and adjusting the hyperparameters; and / or uses subtle defect images generated by GAN-EIS as adversarial samples to verify the robustness of the EIS model, and creates increasingly challenging defect images by continuously adjusting the generation strategy of GAN-EIS to test and improve the performance of the EIS model; obtains a trained EIS model; inputs an industrial product image sequence into the trained EIS model to perform defect recognition, and obtains the defect recognition result of the EIS model.

[0087] It should be noted that: 1. Generate subtle defect images and construct a primary dataset: Use the GAN-EIS model to generate 100,000 subtle defect images of various types, sizes and locations. Through quality assessment and screening, retain 80,000 high-quality images to construct a primary dataset. 2. Construct a secondary dataset and perform data enhancement: Fuse the primary dataset with 20,000 real defect images to form a secondary dataset of 100,000 images. Perform data enhancement processing, including rotation, scaling, translation, color transformation, etc., to increase data diversity and model generalization capabilities. 3. Select and optimize the EIS model: Select CNN as the basic architecture, introduce attention mechanism and multi-scale feature fusion technology, and improve the model's detection accuracy for subtle defects. Perform hyperparameter tuning and regularization processing to prevent overfitting and improve model generalization capabilities.

[0088] The specific data examples are shown in Table 3:

[0089] Table 3

[0090]

[0091] In industrial automation quality inspection based on image recognition, the specific embodiments of training and optimizing the EIS model and its use are as follows: Training the EIS model: Data preparation: Dataset: The fused secondary subtle defect image dataset (including GAN-EIS generated images and real defect images) and the real defect image dataset are used as training sets, totaling 100,000 images. Validation set and test set: Divided from the original real defect image dataset, 5,000 validation sets and 5,000 test sets are used for model evaluation and performance testing. Model training: Loss function and optimizer: Select the cross entropy loss function and Adam optimizer to measure the difference between the predicted and real labels, and adjust the model parameters to minimize the loss. Back propagation: Update the model weights and biases according to the gradient of the loss function to optimize performance. Early stopping method: Monitor the performance of the validation set to prevent overfitting, and stop training in advance when the performance no longer improves or decreases. Model evaluation and adjustment: Evaluation indicators: Use accuracy, recall and F1 score to evaluate model performance. Adjustment and optimization: adjust the network structure (such as adding convolutional layers, introducing attention mechanisms) and hyperparameters (such as learning rate, batch size) according to the evaluation results. 2. Verify the robustness of the EIS model: Adversarial sample generation: Use the GAN-EIS model to generate challenging images with subtle defects as adversarial samples, and diversify the generation strategy. Robustness test: Input the adversarial samples into the trained EIS model, observe the recognition results, and evaluate the robustness of the model. 3. Use the trained EIS model for defect recognition: Input image sequence: Input the image sequence of the industrial product to be inspected into the trained EIS model. Defect recognition: The model identifies defects in each frame of the image and outputs information such as location, type, and size. Result summary and analysis: Summarize and analyze the recognition results of each frame of the image to obtain a defect detection report for the entire image sequence, which helps the production line to detect and handle defective products in a timely manner.

[0092] The specific data are as follows in Table 4:

[0093] Table 4

[0094]

[0095] Through the above embodiments, we have trained an EIS model with excellent performance.

[0096] 205. Obtaining a quality inspection status of a product based on the time series change recognition result and the EIS model defect recognition result;

[0097] Specifically, timestamp synchronization or image frame number synchronization is selected as a data synchronization mechanism, the time series change recognition result and the EIS model defect recognition result are preprocessed to ensure that they have the same time range or image frame range, and a corresponding timestamp or image frame number is added to each data. By comparing the timestamp or image frame number, the time series data and the defect recognition data are matched to ensure their synchronization in time and space; a weighted scoring system is preset, in which the mutation of the time series and the defects identified by the EIS model will increase the question score of the product, and different weights are assigned to the time series mutation and the defect recognition result to reflect their different degrees of influence on product quality. The weighted scoring system is used to calculate the question score of the product, and a threshold is set according to the question score to divide the product into different quality grades or qualified / unqualified categories; the mutation score set M of the time series data is set to be {m1, m2, ..., mn} and the score set D of the defect recognition result is set to be {d1, d2, ..., dm}, where mi and di represent the mutation score of the time series at the i-th time point and the score of the defect recognition result at the i-th time point, respectively, then:

[0098]

[0099] Among them, w m and w d are the weights of time series mutation and defect recognition results, respectively. and are the sum of the time series mutation score and the defect recognition result score, n and m are the number of time series data and defect recognition results, respectively; the quality inspection status of the product is obtained;

[0100] Or use a serial fusion mechanism to fuse the time series change recognition result and the EIS model defect recognition result, use the time series change recognition result as one part of the feature vector, use the EIS model defect recognition result as another part of the feature vector, and concatenate them into a complete feature vector. Use the fused feature vector as input, preset a quality inspection model to calculate the quality inspection status of the product, and classify the products into different quality grades or qualified / unqualified categories according to the output of the quality inspection model; set the feature vector X=[x1,x2,...,xn] of the time series change recognition result and the feature vector Y=[y1,y2,...,ym] of the EIS model defect recognition result , where xi and yi represent the i-th feature of the time series change recognition result and the EIS model defect recognition result respectively, and the feature vector after serial fusion is Z: Z = [x1, x2, ..., xn, y1, y2, ..., ym]; where Z is a feature vector containing n + m features, the first n features come from the time series change recognition result, and the last m features come from the EIS model defect recognition result; the quality inspection model f is used to calculate the product's quality inspection status Q: Q = f(Z); where f is a machine learning model, including support vector machine (SVM) and neural network, Z is the input feature vector, and Q is the output quality inspection status; the product's quality inspection status is obtained;

[0101] It should be noted that in industrial automation quality inspection based on image recognition, the synchronization and comprehensive evaluation of time series changes and EIS model defect recognition results can be achieved by following the following steps: Data synchronization and processing: Select a timestamp synchronization mechanism and require the two systems to add an accurate timestamp for each recognition result. Preprocess the results of the two systems to ensure that the time range is consistent, including deleting data points out of the range and interpolating missing data. Add an accurate timestamp to each result to match corresponding points in the two data streams. Accurately match time series and defect recognition data by comparing timestamps. Weighted scoring system: Assign weights to time series mutations and defect recognition results based on the degree of impact, such as w_m=0.6, w_d=0.4. Set the time series mutation scoring set M and the defect recognition scoring set D, and note that the number of data points is different (n and m may not be equal). Calculate the questioning score: If n and m are not equal, they can be made equal through interpolation or sampling, or the calculation method can be adjusted according to the actual situation (for example, only data points with matching timestamps are considered).

[0102] Specific data example: Time series mutation score M = 3, 2, 5, 1 (4 time points); defect recognition score D = 2, 2, 4 (the original 3 time points have been set to be expanded to 4 through interpolation, but the original data is still shown here for simplification). Weight: w_m = 0.6, w_d = 0.4. Calculate the question score (using the simplified D set, the actual application requires the expanded set): Question_Score = (0.6·(3+2+5+1) / 4)+(0.4·(2+2+4) / 3)≈2.7. Note: This calculation is based on the original 3 points and needs to be adjusted for actual application. Set the threshold to 2.5. Since Question_Score = 2.7 exceeds the threshold, the product is judged to be "unqualified" or "has quality problems".

[0103] Through the above embodiments, we can effectively combine the time series change recognition results with the EIS model defect recognition results, and conduct a comprehensive evaluation of the product quality status through a weighted scoring system.

[0104] In industrial automation quality inspection based on image recognition, the implementation of the serial fusion mechanism to combine time series changes with the EIS model defect recognition results is as follows: Feature vector construction: Time series change recognition result feature vector X: The change score constitutes the feature vector, such as 10 time points, it is expressed as [x1,...,x10]. EIS model defect recognition result feature vector Y: Set the EIS model to output 5 defect features and take the average of the 10-frame image sequence, then Y is expressed as [y1,...,y5]. Feature vector Z after serial fusion: X and Y are concatenated to form a complete feature vector Z. If X has 10 features and Y has 5, then Z has 15 features. Quality inspection model construction: Model selection: Select a classification or regression model suitable for processing multi-feature inputs, such as SVM or neural network. Model training: Train the model with a data set with known quality levels or pass / fail labels. Each sample is a feature vector Z and its corresponding quality inspection result Q. Model evaluation and application: Model evaluation: Evaluate performance on the validation set or test set, and calculate indicators such as accuracy, recall, and F1 score. Application model: Deploy to the production environment, process new image sequences, construct feature vector Z, input it into the model, and obtain the quality inspection status Q.

[0105] Specific data examples: Feature vector X example (time series change score): [0.2, 0.3, 0.1, 0.5, 0.4, 0.6, 0.7, 0.8, 0.9, 1.0]; Feature vector Y example (EIS model defect recognition result): Defect recognition results include the number of defects, average defect size, maximum defect size, defect type encoding (set as one-hot encoding), etc. Here it is simplified to [2, 0.5, 1.2, 1] (number of defects, average defect size, maximum defect size, encoding of specific defect types). The concatenated feature vector Z: [0.2, 0.3, 0.1, 0.5, 0.4, 0.6, 0.7, 0.8, 0.9, 1.0, 2, 0.5, 1.2, 1]; Quality inspection model output Q: Set the model output to "qualified" or "unqualified". In this example, if the model judges it as "unqualified", it means that the product has quality problems.

[0106] Through the above embodiments, we can effectively combine the time series change recognition results with the EIS model defect recognition results, and use the machine learning model to comprehensively evaluate the quality inspection status of industrial products.

[0107] 206. Set up a dynamic threshold mechanism that dynamically adjusts the criteria for determining product quality based on historical data and real-time data. If a certain type of product has a history of frequent minor defects, the system automatically adjusts its sensitivity to that type of defect; and / or develop an intuitive visual interface that displays time series changes, defect identification results, and comprehensive quality inspection status, so that quality inspectors can quickly understand the overall quality of the product through the interface; and / or establish a feedback loop that allows quality inspectors to verify and correct the system's judgment. These feedbacks will be used to continuously optimize the EIS model, time series analysis algorithm, and result fusion mechanism; and / or integrate data from different sensors or detection methods, including temperature, pressure, and vibration, to form a multi-modal quality inspection system that provides more comprehensive product quality information and reveals defects that are difficult to detect under a single modality; and / or pre-set an early warning system based on the fused quality inspection status, automatically trigger an alarm when a potential quality problem is detected, and integrate automatic intervention measures, including suspending the production line and isolating suspected defective products; and / or upload quality inspection data to a cloud platform and use big data analysis tools to conduct long-term trend analysis and fault prediction;

[0108] It should be noted that:

[0109] 1. Dynamic threshold mechanism implementation: The system regularly analyzes historical data, especially defect records for specific types of products. Use statistical methods (such as mean, standard deviation) or machine learning models to identify which defects are common or critical. Based on the analysis results, dynamically adjust the threshold for determining product quality. For example, if a certain type of product has a history of frequent minor scratches, reduce the tolerance for such scratches and increase detection sensitivity.

[0110] Data example: After analyzing the data from the past year, it was found that 30% of the batches of type A products had tiny scratches during the production process, and these scratches were considered acceptable under the previous inspection standards. Therefore, the system adjusted the threshold for scratch detection from 0.2mm to 0.1mm.

[0111] 2. Visual interface implementation: Develop an interactive interface based on the Web or desktop to display time series changes, defect identification results, and comprehensive quality inspection status in real time. Use charts, heat maps, animations, etc. to intuitively present data, so that quality inspectors can quickly understand the product status. Interface example: The left side of the interface displays a time series change graph to show the trend of key quality indicators over time. The middle part displays the defect identification results of each batch of products in a grid format, including information such as defect location, type, and severity. The right side is the comprehensive quality inspection report area, which summarizes the overall quality evaluation of the current batch of products and gives improvement suggestions.

[0112] 3. Feedback loop implementation: Set feedback buttons or forms in the visual interface to allow quality inspectors to directly verify and correct system judgments. The collected feedback data is automatically recorded and used for subsequent model training and algorithm optimization. Data flow: Quality inspectors submit feedback on the defect judgment of a batch of products through the interface. The system integrates these feedbacks into the training data set for weight adjustment during the next model training.

[0113] 4. Implementation of multimodal quality inspection system: Integrate data from different sensors (such as temperature, pressure, vibration sensors) with image recognition results. Use multimodal fusion algorithms to combine information from multiple data sources to improve the comprehensiveness and accuracy of defect detection.

[0114] Data fusion example: When inspecting automobile engine parts, in addition to image recognition, temperature sensor data is also introduced. When image recognition detects a tiny crack on the surface of the part, the temperature sensor data shows that the temperature in that area is abnormally high. After combining this information, the multimodal system determines that the part has serious quality problems.

[0115] 5. Early warning system and automatic intervention measures: Set early warning rules based on the integrated quality inspection status. When potential quality problems are detected, the alarm is automatically triggered and sent to relevant personnel through the network. Integrate automatic intervention measures, such as suspending the production line, isolating suspected defective products, etc.

[0116] Warning example: The system detects that three consecutive batches of products have the same type of serious defects, triggering a high-level warning. An alarm email is automatically sent to the production manager and the head of the quality inspection department. At the same time, the production line is automatically suspended and the suspected defective products are isolated for further processing.

[0117] 6. Cloud platform data analysis implementation method: Upload quality inspection data to the cloud platform for storage in real time. Use big data analysis tools to mine and analyze long-term accumulated data to identify trends and patterns of quality problems. Perform fault prediction and preventive maintenance based on the analysis results.

[0118] Data analysis example: By analyzing the quality inspection data of the past five years through the cloud platform, it was found that type B products are more prone to material aging problems in high temperature environments in summer. Based on this finding, special quality control measures were formulated for summer production, and maintenance of the production line was carried out in advance to reduce the failure rate.

[0119] In summary, this embodiment significantly improves the efficiency, accuracy and response speed of industrial automation quality inspection by comprehensively using dynamic threshold mechanism, visual interface, feedback loop, multimodal quality inspection system, early warning system and cloud platform data analysis.

[0120] In the embodiment of the present invention, multi-channel image data acquisition and parallel processing technology significantly reduce image processing time and improve detection efficiency; the LSTM model can capture the changing characteristics in the time dimension, combined with the EIS model's ability to identify subtle defects, greatly improving the accuracy of detection; the defect images generated by the GAN-EIS model are used as training data, which enhances the generalization ability and robustness of the EIS model; the dynamic threshold mechanism and feedback loop mechanism enable the system to self-adjust and optimize according to actual conditions; the visual interface provides intuitive and comprehensive quality inspection information for quality inspectors, which is convenient for quickly understanding the product status; the multimodal quality inspection system and early warning system realize multi-dimensional monitoring and early warning of product quality, and improve the response speed; cloud platform data analysis provides strong support for long-term quality trend analysis and fault prediction, and helps to formulate more scientific quality control strategies; automated detection reduces manual intervention and reduces labor costs; the defect images generated by the GAN-EIS model are used as training data, which reduces dependence on real defect samples and reduces data collection costs; by continuously collecting and analyzing quality inspection data, enterprises can continuously optimize production processes and quality control strategies and improve the overall quality level of products; the feedback loop mechanism encourages quality inspectors to actively participate in the system optimization process, forming a good atmosphere for continuous improvement. In summary, the present invention not only improves the accuracy and efficiency of industrial automation quality inspection by comprehensively applying a variety of advanced technologies and methods, but also realizes intelligent management and continuous optimization of the inspection process.

[0121] The above describes the industrial automation quality inspection method based on image recognition in the embodiment of the present invention. The following describes the industrial automation quality inspection device based on image recognition in the embodiment of the present invention. Figure 3In one embodiment of the present invention, an industrial automation quality inspection device based on image recognition includes: an acquisition module 301, which is used to acquire multi-channel image data of industrial products, and perform fuse processing according to the multi-channel image data to obtain an industrial product image sequence; a processing module 302, which is used to process the continuous image sequence of the industrial product image sequence according to the LSTM model, capture the change characteristics in the time dimension, analyze the industrial product image sequence through the trained LSTM model, predict potential quality problems, and extract the time series change recognition result; a setting module 303, which is used to preset the GAN framework according to the real defect image data set in the industrial product image sequence, preset the generator network, so that it can generate defect images, including fine Micro-defects, a discriminator network is preset to enable it to distinguish between real defect images and generated images, and the performance of the generator and the discriminator is optimized through continuous adversarial training to obtain a trained GAN-EIS model; an allocation module 304 is used to use the trained GAN-EIS model to generate a large number of subtle defect images as training data, and use the generated subtle defect images and the real defect image data set in the industrial product image sequence to train the EIS model to obtain a trained EIS model, and the EIS model defect recognition result is obtained based on the industrial product image sequence combined with the trained EIS model; a calculation module 305 is used to obtain the product quality inspection status based on the time series change recognition result and the EIS model defect recognition result.

[0122] In the embodiment of the present invention, the application of multi-channel image acquisition and LSTM model enables the device to capture subtle changes and dynamic features on the product surface, significantly improving the accuracy of defect detection; the defect images generated by the GAN-EIS model greatly enrich the training data set, making the EIS model more robust and generalizable when facing complex and changeable defect types; the automated detection process reduces manual intervention, shortens the detection cycle, and improves the overall efficiency of the production line; reduces dependence on manual inspectors and reduces labor costs. At the same time, by generating defect images through generative adversarial networks, the dependence on real defect samples is reduced, further reducing the cost of data collection; the feedback loop mechanism allows the system to self-optimize according to the actual detection effect, ensuring the continuous improvement of detection performance. In addition, cloud platform data analysis provides data support for long-term quality monitoring and preventive maintenance; by integrating time series changes and defect recognition results, the device can give a more comprehensive and accurate product quality inspection report, providing strong support for production decisions. In summary, the industrial automation quality inspection device achieves efficient and accurate detection of industrial product quality by integrating advanced image recognition and deep learning technologies.

[0123] above Figure 3The industrial automation quality inspection device based on image recognition in the embodiment of the present invention is described in detail from the perspective of modular functional entities. The industrial automation quality inspection device based on image recognition in the embodiment of the present invention is described in detail from the perspective of hardware processing.

[0124] Figure 4 4 is a schematic diagram of the structure of an industrial automation quality inspection device based on image recognition provided by an embodiment of the present invention. The industrial automation quality inspection device 400 based on image recognition may have relatively large differences due to different configurations or performances, and may include one or more processors (central processing units, CPU) 410 (for example, one or more processors) and a memory 420, and one or more storage media 430 (for example, one or more mass storage devices) storing application programs 433 or data 432. Among them, the memory 420 and the storage medium 430 can be short-term storage or permanent storage. The program stored in the storage medium 430 may include one or more modules (not shown in the figure), and each module may include a series of instruction operations in the industrial automation quality inspection device 400 based on image recognition. Furthermore, the processor 410 can be configured to communicate with the storage medium 430 to execute a series of instruction operations in the storage medium 430 on the industrial automation quality inspection device 400 based on image recognition.

[0125] The industrial automation quality inspection device 400 based on image recognition may also include one or more power supplies 440, one or more wired or wireless network interfaces 450, one or more input and output interfaces 460, and / or one or more operating systems 431, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. It can be understood by those skilled in the art that Figure 4 The structure of the industrial automation quality inspection equipment based on image recognition shown does not constitute a limitation of the industrial automation quality inspection equipment based on image recognition, and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0126] The present invention also provides an industrial automation quality inspection device based on image recognition, which includes a memory and a processor. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, the processor executes the steps of the industrial automation quality inspection method based on image recognition in the above-mentioned embodiments.

[0127] The present invention also provides a computer-readable storage medium, which may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. Instructions are stored in the computer-readable storage medium. When the instructions are executed on a computer, the computer executes the steps of the industrial automation quality inspection method based on image recognition.

[0128] Those skilled in the art may refer to the aforementioned method embodiments to understand the specific working processes of the systems, devices and units, which will not be described in detail here.

[0129] If the integrated unit is implemented in the form of a software functional unit and sold or used independently, it can be stored in a computer-readable storage medium. The technical solution of the present invention or its contribution to the prior art can be embodied in the form of a software product, stored in a medium, and contains instructions for executing the method steps described in the embodiment of the present invention. The storage medium includes a USB flash drive, a mobile hard disk, a ROM, a RAM, a magnetic disk, an optical disk, and other media that can store program codes.

[0130] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An industrial automation quality inspection method based on image recognition, characterized in that: The industrial automation quality inspection method based on image recognition includes: Acquire multi-channel image data of industrial products, and perform fuse processing according to the multi-channel image data, wherein the fuse processing includes: adopting GPU acceleration and distributed computing architecture to perform multi-channel parallel processing on the acquired multi-channel image data of industrial products: automatically dividing each image into multiple small blocks or regions to ensure that each block or region can be processed in parallel; using the parallel computing capability of GPU to simultaneously process feature extraction tasks of multiple image blocks or regions; and summarizing the results after parallel processing to form an industrial product image sequence; Processing the continuous image sequence of the industrial product image sequence according to the LSTM model to capture the change characteristics in the time dimension, analyzing the industrial product image sequence through the trained LSTM model, predicting potential quality problems, and extracting the time series change recognition result; According to the real defect image data set in the industrial product image sequence, a GAN framework is preset, a generator network is preset so that it can generate defect images including subtle defects, and a discriminator network is preset so that it can distinguish between real defect images and generated images. Through continuous adversarial training, the performance of the generator and the discriminator is optimized. The steps of building a GAN-EIS model include: Preprocess the real defect image dataset, including image normalization, denoising, contrast enhancement, and implement data enhancement strategies, including rotation, scaling, and translation; Preset the GAN framework, select the deep convolutional generative adversarial network as the basic framework, or adopt the conditional GAN ​​structure; The preset generator network is a deep convolutional neural network that contains transposed convolutional layers and residual blocks, which can generate images with subtle defects; A preset discriminator network, which is a deep convolutional neural network, includes a convolutional layer and a downsampling layer, and can distinguish between real defect images and generated images; Initialize the parameters of the generator and discriminator, set the learning rate and optimizer, perform adversarial training, and optimize the model performance by alternately updating the parameters of the generator and discriminator to obtain a trained GAN-EIS model; A large number of subtle defect images are generated using the trained GAN-EIS model as training data, and the generated subtle defect images and a real defect image dataset in the industrial product image sequence are used to train the EIS model. The steps of constructing the EIS model include: Perform quality assessment on the generated subtle defect images, select high-quality images, and construct a first-level subtle defect image dataset; The generated first-level subtle defect image dataset is fused with the real defect image dataset to form a second-level subtle defect image dataset, and data enhancement is performed on the fused dataset; The convolutional neural network is selected as the basic architecture of the EIS model. According to the characteristics of subtle defects, the attention mechanism and multi-scale feature fusion technology are introduced to improve and optimize the EIS model. The EIS model is trained using the fused secondary subtle defect image dataset and the real defect image dataset, and the model parameters are updated through the back propagation algorithm to obtain the trained EIS model. Obtaining an EIS model defect recognition result based on the industrial product image sequence and the trained EIS model; The quality inspection status of the product is obtained based on the time series change recognition result and the EIS model defect recognition result, including: Selecting timestamp synchronization or image frame number synchronization as a data synchronization mechanism to ensure the synchronization of the time series change recognition result and the EIS model defect recognition result in time and space; A weighted scoring system is preset to assign different weights to the mutations in the time series and the defects identified by the EIS model, and the doubt score of the product is calculated. A threshold is set according to the doubt score to classify the product into different quality grades or qualified / unqualified categories; Or use a serial fusion mechanism to fuse the time series change recognition results and the EIS model defect recognition results to form a feature vector, which is input into a preset quality inspection model, and the quality inspection status of the product is obtained based on the output of the quality inspection model.

2. The industrial automation quality inspection method based on image recognition according to claim 1 is characterized in that: The step of acquiring multi-channel image data of industrial products and performing fuse processing according to the multi-channel image data to obtain an industrial product image sequence includes: Using camera combinations, we can capture image data at different angles and resolutions on the production line to obtain multi-channel image data of industrial products. In multi-channel parallel processing, a dynamic load balancing mechanism is introduced to dynamically adjust the image segmentation strategy and task allocation according to the real-time load of each processing node to ensure that all nodes work evenly and avoid idle or overloaded resources. The number of tasks j assigned to node i is set to TA i，j ,but; Among them, TA i，j represents the number of tasks j assigned to node i; N represents the total number of nodes; TT j represents the total number of tasks j; CL i Obtained in real time through the node monitoring system, including CPU usage, memory usage, and disk I / O; MC i Estimate based on the hardware configuration of the node and determine the maximum processing capacity of the node through performance testing; TT j Determine based on the amount of tasks in the actual application scenario; And / or through a multi-threaded programming library or thread pool technology, create a thread pool to manage multiple threads, and preset a task queue to store the image tasks to be processed. Each image processing task is encapsulated as a task object, including image data and processing functions; preset one or more image processing functions, which will be called by threads in the thread pool to process image data, and set the pixel value of the output image at the position (x, y) in the image processing function to g(x, y), then: Among them, f(x+s,y+t) represents the pixel value of the input image at the position (x+s,y+t); w(s,t) is the value of the convolution kernel at the position (s,t); a and b define the size of the convolution kernel; Assign image processing tasks to multiple threads for simultaneous execution, and aggregate them to form an industrial product image sequence; In multi-threaded processing, a cross-thread data prefetching mechanism is introduced. Before the image processing task starts, the next image block to be processed is preloaded into the memory through another thread or coroutine to reduce thread waiting time. And / or use a graded inspection strategy: use cameras to capture real-time images on the production line, use the first-level image processing algorithm to quickly screen and exclude obviously defective products, and use high-resolution cameras and second-level image recognition algorithms to conduct detailed inspections on the suspected defective products that have been initially screened out, thus forming an industrial product image sequence.

3. The industrial automation quality inspection method based on image recognition according to claim 1 is characterized in that: The continuous image sequence of the industrial product image sequence is processed according to the LSTM model to capture the change characteristics in the time dimension, and the industrial product image sequence is analyzed through the trained LSTM model to predict potential quality problems and extract the time series change recognition results, including: Further processing is performed based on the obtained industrial product image sequence to ensure the continuity and consistency of the sequence, and data enhancement is performed on the image sequence, including rotation, scaling, and translation, and the dynamic features in the image sequence, i.e., features that change over time, are extracted using optical flow method and dynamic texture analysis technology; Preset the structure of the LSTM model, including the number of LSTM layers and hidden units. During the model training process, introduce regularization technology to prevent overfitting. Or the structure of the preset LSTM model is bidirectional LSTM (Bi-LSTM), which uses the previous and next information at the same time to better understand the context of the entire sequence and improve the accuracy of the prediction; Or use a more complex and powerful Transformer model as the teacher model and distill its knowledge into a lighter LSTM student model, reducing the complexity of the model while maintaining performance and improving its real-time and deployability; Adopting a sequence-to-sequence learning framework, the continuous image sequence is encoded into a fixed-length vector through an encoder-decoder structure, and then decoded into the predicted quality problem or time series change recognition result; And / or in order to capture the changing features at different time scales, a multi-scale time feature fusion strategy is introduced. By running multiple LSTM networks in parallel, each network processes image sequences at different time scales, and then fusing their outputs, a more comprehensive time series change recognition result is obtained.

4. The industrial automation quality inspection method based on image recognition according to claim 1 is characterized in that: The GAN framework is preset based on the real defect image data set in the industrial product image sequence, the generator network is preset so that it can generate defect images including subtle defects, and the discriminator network is preset so that it can distinguish between real defect images and generated images. Through continuous adversarial training, the performance of the generator and the discriminator is optimized to obtain a trained GAN-EIS model, including: Preprocess the real defect image dataset, including image normalization, denoising, contrast enhancement, and implement data enhancement strategies, including rotation, scaling, and translation; GAN framework preset: Select the GAN architecture and use the deep convolutional generative adversarial network (DCGAN) as the basic framework; Or use a conditional GAN ​​(cGAN) structure to enable the model to generate corresponding defect images based on conditions, including defect type, size, and location; A deep convolutional neural network is preset as the generator, whose goal is to generate images with subtle defects. Residual blocks and upsampling layer technologies are introduced to enhance the generator's ability to capture detailed features. In order to enable the generator to generate diverse defect images, random noise or conditional variables are introduced into the input of the generator; A discriminator network matching the generator is preset to distinguish between real defect images and generated images. A deep convolutional neural network structure is adopted, and the discriminator is constructed using downsampling layers and fully connected layers. In order to improve the discriminator’s ability to distinguish, the attention mechanism or spectral normalization technology is introduced; Initialize the parameters of the generator and discriminator, set the learning rate and optimizer, conduct adversarial training, optimize the model performance by alternately updating the parameters of the generator and discriminator, introduce regularization technology to prevent overfitting during training, monitor the changes in the loss function and the quality of the generated images during training, and adjust the training strategy in time; And / or learn from the progressive growth GAN, starting from low-resolution images and gradually increasing the resolution of the generated images, so that the model first learns coarse-grained features and then gradually refines them, improving the quality and stability of the generated images; Use the Inception Score evaluation indicator to evaluate the trained GAN-EIS model, and improve the model based on the evaluation results, including adjusting the network structure and increasing training data; Get the trained GAN-EIS model.

5. The industrial automation quality inspection method based on image recognition according to claim 1 is characterized in that: The method uses the trained GAN-EIS model to generate a large number of subtle defect images as training data, uses the generated subtle defect images and a real defect image dataset in the industrial product image sequence to train the EIS model to obtain a trained EIS model, and obtains an EIS model defect recognition result based on the industrial product image sequence combined with the trained EIS model, including: Using the trained GAN-EIS model, a large number of subtle defect images are generated, including subtle defects of various types, sizes, and locations. The quality of the generated subtle defect images is evaluated, high-quality and representative images are screened out, and a first-level subtle defect image dataset is constructed. The generated first-level subtle defect image dataset is fused with the real defect image dataset to form a second-level subtle defect image dataset, and the fused dataset is enhanced, including rotation, scaling, translation, and color transformation; The deep learning model of convolutional neural network is selected as the basis of EIS model. According to the characteristics of subtle defects, attention mechanism and multi-scale feature fusion are introduced to improve and optimize the EIS model. The EIS model is trained using the fused secondary subtle defect image dataset and the real defect image dataset. The loss function and optimizer are used to update the model parameters through the back propagation algorithm. During the training process, the early stopping method is implemented to prevent the model from overfitting. The trained EIS model is evaluated using an independent validation set, and the accuracy, recall, and F1 score indicators are calculated. The model is adjusted and optimized based on the evaluation results, including modifying the network structure and adjusting the hyperparameters. And / or use the subtle defect images generated by GAN-EIS as adversarial samples to verify the robustness of the EIS model, and create increasingly challenging defect images by continuously adjusting the generation strategy of GAN-EIS to test and improve the performance of the EIS model; Get the trained EIS model; The industrial product image sequence is input into the trained EIS model for defect recognition, and the EIS model defect recognition result is obtained.

6. The industrial automation quality inspection method based on image recognition according to claim 1 is characterized in that: The quality inspection status of the product is obtained by combining the time series change recognition result and the EIS model defect recognition result, including: Select timestamp synchronization or image frame number synchronization as the data synchronization mechanism, pre-process the time series change recognition result and the EIS model defect recognition result to ensure that they have the same time range or image frame range, and add a corresponding timestamp or image frame number to each data, and match the time series data and the defect recognition data by comparing the timestamps or image frame numbers to ensure their synchronization in time and space; A weighted scoring system is preset, in which the mutation of the time series and the defects identified by the EIS model will increase the doubt score of the product. Different weights are assigned to the time series mutation and defect identification results to reflect their different degrees of influence on product quality. The weighted scoring system is used to calculate the doubt score of the product. A threshold is set according to the doubt score to classify the product into different quality levels or qualified / unqualified categories. The mutation score set M = {m1, m2, ..., mn} of the time series data and the score set D = {d1, d2, ..., dm} of the defect identification results are set, where mi and di represent the mutation score of the time series at the i-th time point and the score of the defect identification result at the i-th time point, respectively. Then: Among them, w m and w d are the weights of time series mutation and defect recognition results, respectively. and are the sum of the time series mutation score and the defect identification result score, respectively. n and m are the number of time series data and defect identification results, respectively. Get the quality inspection status of the product; Or use the serial fusion mechanism to fuse the time series change recognition result and the EIS model defect recognition result, use the time series change recognition result as one part of the feature vector, use the EIS model defect recognition result as the other part of the feature vector, and connect them into a complete feature vector. Use the fused feature vector as input, preset a quality inspection model to calculate the quality inspection status of the product, and classify the product into different quality grades or qualified / unqualified categories according to the output of the quality inspection model; set the feature vector X = [x1, x2, ..., xn] of the time series change recognition result and the feature vector Y = [y1, y2, ..., ym] of the EIS model defect recognition result, where xi and yi represent the i-th feature of the time series change recognition result and the EIS model defect recognition result, respectively, and the feature vector after serial fusion is Z: Z=[x1,x2,...,xn,y1,y2,...,ym]; Among them, Z is a feature vector containing n+m features, the first n features come from the time series change recognition results, and the last m features come from the EIS model defect recognition results; Use the quality inspection model f to calculate the product's quality inspection status Q: Q = f(Z); Among them, f is a machine learning model, including support vector machine (SVM) and neural network, Z is the input feature vector, and Q is the output quality detection status; Get the quality inspection status of the product.

7. The industrial automation quality inspection method based on image recognition according to claim 6 is characterized in that: Also includes: Set up a dynamic threshold mechanism, which will dynamically adjust the standard for judging product quality based on historical data and real-time data. If a certain type of product has a history of frequent minor defects, the system will automatically adjust the sensitivity to that type of defect. And / or develop an intuitive visualization interface to display time series changes, defect identification results, and comprehensive quality inspection status, so that quality inspectors can quickly understand the overall quality of the product through the interface; and / or establish a feedback loop that allows quality control personnel to verify and correct the judgments of the system, and this feedback will be used to continuously optimize the EIS model, time series analysis algorithm, and result fusion mechanism; and / or integrate data from different sensors or detection methods, including temperature, pressure, and vibration, to form a multi-modal quality detection system that provides more comprehensive product quality information and reveals defects that are difficult to detect using a single modality; And / or based on the integrated quality inspection status, preset an early warning system to automatically trigger an alarm when a potential quality problem is detected, and integrate automatic intervention measures, including suspending the production line and isolating suspected defective products; And / or upload quality inspection data to the cloud platform and use big data analysis tools to conduct long-term trend analysis and fault prediction.

8. An industrial automation quality inspection device based on image recognition, characterized in that: The industrial automation quality detection device based on image recognition comprises: An acquisition module is used to acquire multi-channel image data of industrial products and perform a fuse process according to the multi-channel image data. The fuse process includes: using GPU acceleration and distributed computing architecture to perform multi-channel parallel processing on the acquired multi-channel image data of industrial products: automatically dividing each image into multiple small blocks or regions to ensure that each block or region can be processed in parallel; using the parallel computing capability of the GPU to simultaneously process feature extraction tasks of multiple image blocks or regions; and summarizing the results of the parallel processing to form an industrial product image sequence; A processing module is used to process the continuous image sequence of the industrial product image sequence according to the LSTM model, capture the change characteristics in the time dimension, analyze the industrial product image sequence through the trained LSTM model, predict potential quality problems, and extract the time series change recognition result; A setting module is used to preset a GAN framework based on a real defect image data set in the industrial product image sequence, preset a generator network so that it can generate defect images including subtle defects, preset a discriminator network so that it can distinguish between real defect images and generated images, and optimize the performance of the generator and the discriminator through continuous adversarial training. The steps of building a GAN-EIS model include: Preprocess the real defect image dataset, including image normalization, denoising, contrast enhancement, and implement data enhancement strategies, including rotation, scaling, and translation; Preset the GAN framework, select the deep convolutional generative adversarial network as the basic framework, or adopt the conditional GAN ​​structure; The preset generator network is a deep convolutional neural network that contains transposed convolutional layers and residual blocks, which can generate images with subtle defects; A preset discriminator network, which is a deep convolutional neural network, includes a convolutional layer and a downsampling layer, and can distinguish between real defect images and generated images; Initialize the parameters of the generator and discriminator, set the learning rate and optimizer, perform adversarial training, and optimize the model performance by alternately updating the parameters of the generator and discriminator to obtain a trained GAN-EIS model; The allocation module is used to generate a large number of subtle defect images using the trained GAN-EIS model as training data, and use the generated subtle defect images and the real defect image data set in the industrial product image sequence to train the EIS model. The steps of constructing the EIS model include: Perform quality assessment on the generated subtle defect images, select high-quality images, and construct a first-level subtle defect image dataset; The generated first-level subtle defect image dataset is fused with the real defect image dataset to form a second-level subtle defect image dataset, and data enhancement is performed on the fused dataset; The convolutional neural network is selected as the basic architecture of the EIS model. According to the characteristics of subtle defects, the attention mechanism and multi-scale feature fusion technology are introduced to improve and optimize the EIS model. The EIS model is trained using the fused secondary subtle defect image dataset and the real defect image dataset, and the parameters of the model are updated by a back propagation algorithm to obtain a trained EIS model, and the EIS model defect recognition result is obtained based on the industrial product image sequence combined with the trained EIS model; A calculation module, used to obtain the quality inspection status of the product based on the time series change recognition result and the EIS model defect recognition result, including: Selecting timestamp synchronization or image frame number synchronization as a data synchronization mechanism to ensure the synchronization of the time series change recognition result and the EIS model defect recognition result in time and space; A weighted scoring system is preset to assign different weights to the mutations in the time series and the defects identified by the EIS model, and the doubt score of the product is calculated. A threshold is set according to the doubt score to classify the product into different quality grades or qualified / unqualified categories; Or use a serial fusion mechanism to fuse the time series change recognition results and the EIS model defect recognition results to form a feature vector, which is input into a preset quality inspection model, and the quality inspection status of the product is obtained based on the output of the quality inspection model.

9. An industrial automation quality inspection device based on image recognition, characterized in that: The industrial automation quality inspection device based on image recognition comprises: a memory and at least one processor, wherein instructions are stored in the memory; The at least one processor calls the instructions in the memory so that the industrial automation quality inspection device based on image recognition executes the industrial automation quality inspection method based on image recognition as described in any one of claims 1-7.

10. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, an industrial automation quality detection method based on image recognition as described in any one of claims 1 to 7 is implemented.

Images