Paster abnormity detection method and system based on flexible circuit board
Through the combination of high-resolution camera and multimodal sensor combined with deep learning model, real-time detection and classification of flexible circuit board patch anomalies is achieved, solving the problems of low efficiency and poor robustness of traditional methods, and improving detection accuracy and system flexibility Awesome, providing comprehensive exception information and production line optimization suggestions.
Patent Information
- Application Number
- CN202411129774.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Traditional flexible circuit board patch abnormality detection methods are inefficient and poorly robust, and relying on a single image data to detect abnormalities, such as temperature abnormalities and internal defects.
High-resolution cameras and multimodal sensors are used to collect data in real time, extract multi-scale features through deep learning models, perform data fusion and adaptive optimization, and use neural network models to detect and classify abnormalities in real time, and provide optimization suggestions.
Improve the accuracy and stability of detection, reduce false detection and missed detection phenomena, provide comprehensive abnormal detection information, enhance the robustness of the system, and improve the efficiency and product quality of the production line through real-time feedback and self-optimization. .
Smart Images

Figure CN119067936A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of flexible circuit board patch detection technology, and in particular to a method and system for detecting anomalies in flexible circuit board patches, which are specifically applied in complex and changeable production environments, and realize real-time detection and classification of anomalies in flexible circuit board patches through high robustness algorithms and multimodal data fusion technology. Background Art
[0002] In the modern electronic manufacturing industry, Flexible Printed Circuit (FPC) is widely used in the production of high-tech products such as smartphones, tablets, and wearable devices due to its advantages such as lightness, high flexibility, and adaptability to complex structures. However, the production environment of FPC is complex and changeable, with a cluttered background and various noises, which puts extremely high demands on abnormality detection during the patch process.
[0003] Traditional FPC anomaly detection methods mainly rely on artificial visual detection and simple image processing technology. These methods usually have the following problems:
[0004] (1) Low efficiency of manual inspection: Manual inspection is not only time-consuming and labor-intensive, but also easily affected by operator fatigue and subjective judgment, and the accuracy of inspection is difficult to guarantee.
[0005] (2) Poor robustness of image processing methods: Traditional image processing methods are prone to false detection and missed detection when faced with complex backgrounds and noisy environments, and the detection results are unstable.
[0006] (3) Single data and lack of multimodal information: Single image data cannot provide comprehensive abnormality information, such as temperature anomalies, internal defects, etc. Summary of the invention
[0007] The present invention provides a method and system for detecting abnormalities in a flexible circuit board patch, so as to solve the problem of how to achieve efficient abnormality detection and classification by using multimodal data fusion technology and high robustness algorithm in a complex and changeable FPC production environment.
[0008] In order to solve the above problems, the present invention provides a method and system for detecting abnormality of a patch on a flexible circuit board, comprising:
[0009] A method and system for detecting abnormality of a patch on a flexible circuit board, comprising:
[0010] S100, using high-resolution cameras and multimodal sensors to collect images and sensor data of flexible circuit boards in real time and perform preprocessing;
[0011] S200, using deep learning models to extract multi-scale features from images and multimodal data, and perform data fusion and adaptive optimization;
[0012] S300, uses a neural network model to detect and classify anomalies in real time, feeds the detection results and classification information back to the production line, and provides optimization suggestions;
[0013] S400 first stores and manages feedback data, then conducts detailed analysis, and uses machine learning and artificial intelligence technologies to optimize the system itself and provide long-term optimization and improvement suggestions.
[0014] As a preferred solution, step S100 includes:
[0015] S110, using a high-resolution camera to capture an image of the flexible circuit board in real time, wherein the camera has a resolution greater than 50 million pixels, a frame rate of at least 30 frames per second, and a signal-to-noise ratio of at least 50 dB;
[0016] S120, using an infrared sensor, a laser radar, and an ultrasonic sensor to respectively collect temperature, three-dimensional shape, and internal defect information of the flexible circuit board;
[0017] S130: Perform noise reduction, image enhancement and geometric correction processing on the collected images and sensor data.
[0018] As a preferred solution, step S200 includes:
[0019] S210, extracting low-level, mid-level, and high-level features from preprocessed images and multimodal data using a deep learning model;
[0020] S220, standardizing the extracted features and performing weighted fusion to form fusion features;
[0021] S230, dynamically adjust feature extraction and fusion strategies according to changes in the production environment, and use online learning and reinforcement learning methods for self-optimization.
[0022] As a preferred solution, step S300 includes:
[0023] S310, using a neural network model to perform real-time analysis on the fusion features, and detect and classify anomalies;
[0024] S320: Detailed classification and analysis of detected anomalies, and provision of detailed information on anomaly types and causes.
[0025] As a preferred solution, the step S400 further includes:
[0026] The test results are fed back to the production line in real time, optimization suggestions are provided, and the system is self-optimized based on the feedback data.
[0027] As a preferred solution, it includes:
[0028] A data acquisition module, used to execute step S100;
[0029] A feature extraction module, used to execute step S200;
[0030] An anomaly detection module, used to execute step S300;
[0031] The self-optimization module is used to execute step S400.
[0032] As a preferred solution, the data acquisition module includes:
[0033] High-resolution cameras and multimodal sensors for real-time acquisition of images and sensor data from flexible circuit boards;
[0034] Intelligent pre-processing unit for performing noise reduction, image enhancement and geometric correction processing of data.
[0035] As a preferred solution, the feature extraction module includes:
[0036] Deep learning models to extract multi-scale features from preprocessed data;
[0037] The feature fusion unit is used to perform feature normalization and weighted fusion.
[0038] As a preferred solution, the anomaly detection module includes:
[0039] Neural network models to detect and classify anomalies in real time;
[0040] Classification analysis unit, which provides detailed information on the type and cause of anomalies.
[0041] As a preferred solution, the self-optimization module includes:
[0042] Storage and management system, used to store and manage feedback data;
[0043] A detailed analysis unit is used to use machine learning and artificial intelligence technologies to optimize the system itself and provide long-term optimization and improvement suggestions. BRIEF DESCRIPTION OF THE DRAWINGS
[0044] Figure 1 The present invention provides a method and system for detecting abnormalities of a flexible circuit board patch according to an embodiment of the present invention. DETAILED DESCRIPTION
[0045] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0046] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.
[0047] Reference Figure 1 , is a flow chart of a method and system for detecting abnormality of a patch on a flexible circuit board according to an embodiment of the present invention. The method and system for detecting abnormality of a patch on a flexible circuit board may at least include steps S100-S400:
[0048] S100 uses high-resolution cameras and multimodal sensors to collect images and sensor data of flexible circuit boards in real time and perform preprocessing.
[0049] S200, uses deep learning models to extract multi-scale features from images and multimodal data, and performs data fusion and adaptive optimization.
[0050] S300 uses a neural network model to detect and classify anomalies in real time, feeds the detection results and classification information back to the production line, and provides optimization suggestions.
[0051] S400 first stores and manages feedback data, then conducts detailed analysis, and uses machine learning and artificial intelligence technologies to optimize the system itself and provide long-term optimization and improvement suggestions.
[0052] The embodiment of the present invention provides a supply chain monitoring and management method and system based on a distributed architecture, which are mainly used to solve the technical problem solved by the present invention, namely, "how to detect and classify anomalies in flexible circuit board patches in real time through high-robustness algorithms and multimodal data fusion technology in a complex and changeable FPC production environment."
[0053] The present invention discloses a method and system for abnormal detection of flexible circuit board (FPC) patches, which involves high-resolution image acquisition and multi-modal sensor data fusion. The system includes real-time data acquisition, preprocessing, feature extraction, abnormal detection and classification, and feedback optimization modules. First, a high-resolution camera is used to capture FPC images, and infrared, laser radar and ultrasonic sensors are combined to obtain temperature, three-dimensional morphology and internal defect information; secondly, the data is denoised, enhanced and corrected; thirdly, a deep learning model is used to extract multi-scale features, and features are optimized through data fusion; then, a neural network is used to detect and classify abnormalities in real time, and feedback is given to the production line and suggestions are provided; finally, data is stored, analyzed and self-optimized to improve FPC production quality control. The present invention solves the problem of real-time detection and classification of FPC abnormalities in complex production environments, and improves the accuracy and stability of detection.
[0054] S100, using a high-resolution camera and a multi-modal sensor to collect images and sensor data of the flexible circuit board in real time and perform pre-processing. Step S100 includes steps S110-S130:
[0055] S110, using a high-resolution camera to capture an image of the flexible circuit board in real time.
[0056] (1) Real-time image data collection on a flexible printed circuit (FPC) production line, including:
[0057] Resolution: higher than 50 megapixels to ensure that the captured images have sufficient details and clarity.
[0058] Frame rate: at least 30 frames per second, ensuring real-time capture of dynamic changes of FPC on the production line.
[0059] Signal-to-noise ratio (SNR): at least 50dB to ensure high image quality and reduce noise interference.
[0060] (2) Select a suitable high-resolution camera to capture the image, specifically:
[0061] Camera selection: Use a high-resolution camera of 50 megapixels or more with optical zoom and autofocus capabilities.
[0062] Installation location: Install cameras at key locations on the production line to ensure full coverage of the FPC production area.
[0063] Lighting configuration: Configure appropriate lighting equipment to ensure that the camera can capture high-quality images under various lighting conditions.
[0064] Real-time image capture: The camera captures images of the flexible circuit board in real time, at least 30 frames per second, to ensure continuous and clear recording of the flexible circuit board production process.
[0065] Automatic adjustment: The camera automatically adjusts the focus and exposure according to changes in light and the position of the flexible circuit board C to ensure image clarity and quality.
[0066] S120, use infrared, laser radar and ultrasonic sensors to collect temperature, three-dimensional shape and internal defect information of flexible circuit boards. Specifically:
[0067] (1) Use infrared sensors to collect temperature data, specifically:
[0068] Sensor selection: Use high-precision infrared sensor with a resolution of 0.1°C.
[0069] Installation location: In the key position of the production line, the infrared sensor is installed at a certain angle to ensure full coverage of the FPC.
[0070] Real-time monitoring: Infrared sensors collect temperature data of flexible circuit boards in real time, paying special attention to the temperature distribution of solder joints and other key areas.
[0071] (2) Use laser radar to collect three-dimensional morphological data of the flexible circuit board, specifically:
[0072] Sensor selection: Use high-resolution lidar with a resolution of 0.01 mm.
[0073] Installation location: The LiDAR is fixed on the top or side of the production line to obtain the best 3D scanning angle.
[0074] Real-time scanning: The LiDAR scans the surface of the FPC in real time to generate high-resolution three-dimensional morphological data.
[0075] (3) Use ultrasonic sensors to detect internal defects of flexible circuit boards, specifically:
[0076] Sensor selection: Use a high-sensitivity ultrasonic sensor with a sensitivity of 0.05 mm.
[0077] Installation location: The ultrasonic sensor is installed below or on the side of the production line to accommodate the movement and detection of the FPC.
[0078] Real-time detection: Ultrasonic sensors detect internal defects of FPC, such as bubbles, cracks, etc., in real time.
[0079] S130, performing noise reduction, image enhancement and geometric correction processing on the image and sensor data collected in S110 and S120. Step S130 includes steps S131-S133:
[0080] S131, performing data noise reduction processing on the image and sensor data collected in S110 and S120, specifically:
[0081] (1) Image denoising: Use an adaptive median filter algorithm to remove random noise in the image and improve image quality. Perform wavelet transform on the image to decompose noise components of different scales, then remove high-frequency noise and retain image details. After denoising, the signal-to-noise ratio (SNR) of the image is increased to more than 60dB.
[0082] (2) De-noising of temperature data: Apply Kalman filter algorithm to smooth the temperature data and remove measurement noise. The accuracy of the denoised temperature data is maintained at 0.1°C, and the influence of noise is significantly reduced.
[0083] (3) De-noising of 3D morphological data: Use statistical filtering and conditional filtering methods to remove noise and abnormal points in 3D point cloud data. The resolution of the 3D morphological data after denoising is maintained at 0.01 mm, and the noise points are reduced by more than 80%.
[0084] (4) Denoising defect data: Adaptive filtering method is used to denoise the data of ultrasonic sensors to improve detection sensitivity. The sensitivity of the denoised defect data is increased to 0.05 mm, and noise interference is reduced by more than 50%.
[0085] S132, performing image enhancement processing on the images and sensor data collected in S110 and S120, specifically:
[0086] (1) Contrast enhancement: Perform histogram equalization on the image to improve the contrast of the image and make the details clearer. Enhance the contrast of the local part of the image to prevent artifacts caused by over-enhancement. The image contrast is increased to 1.5 times that of the original image, and the edge clarity is increased by more than 30%.
[0087] (2) Edge enhancement: Apply the Laplace operator to sharpen the image edges and highlight the details, which improves the image edge clarity by more than 30% and enriches the detail information.
[0088] S133, performing geometric correction processing on the image and sensor data collected in S110 and S120, specifically:
[0089] (1) Perform perspective correction: Use the perspective transformation algorithm to correct the perspective distortion introduced during the camera shooting process so that the image is consistent with the actual shape. This ensures that the image deviation after perspective correction is controlled within 0.5 pixels.
[0090] (2) Deformation correction: geometric transformation of the image is performed to correct the deformation caused by the change of camera angle or FPC position. The accuracy of the geometrically corrected image reaches 0.01 mm, which is highly consistent with the actual shape.
[0091] The quality and robustness of the collected data can be significantly improved through the noise reduction, image enhancement and geometric correction processing of the S110-S130 intelligent pre-processing module. The high-quality data after pre-processing provides a reliable foundation for subsequent feature extraction and anomaly detection, ensuring the accuracy and stability of the detection system in complex production environments.
[0092] Here are some examples:
[0093] Example 1: High-resolution image acquisition and preprocessing
[0094] Step S110: Use a high-resolution camera with more than 50 million pixels to capture images of the flexible circuit board in real time on the production line. The camera is installed at key locations on the production line and equipped with appropriate lighting equipment to ensure that high-quality images are captured under various lighting conditions. The camera captures images in real time, at least 30 frames per second, and automatically adjusts the focus and exposure according to changes in lighting and the position of the flexible circuit board.
[0095] Step S130: Use adaptive median filtering algorithm and wavelet transform to perform denoising on the image. After denoising, the signal-to-noise ratio of the image is increased to above 60dB;
[0096] Perform histogram equalization and local contrast enhancement on the image to improve the image contrast to 1.5 times of the original image and improve the edge clarity by more than 30%;
[0097] Use perspective transformation and geometric transformation algorithms to perform perspective correction and deformation correction on the image, so that the image accuracy reaches 0.01 mm, which is highly consistent with the actual shape.
[0098] Through the above preprocessing steps, the quality and robustness of the collected high-resolution image data are significantly improved, providing a reliable foundation for subsequent feature extraction and anomaly detection.
[0099] Example 2: Multimodal data acquisition and preprocessing
[0100] Step S120: Use high-precision infrared sensors to collect temperature data of the flexible circuit board at key locations of the production line, paying special attention to the temperature distribution of solder joints and other key areas. The infrared sensor monitors the temperature data in real time with a resolution of 0.1°C.
[0101] Use high-resolution LiDAR to scan the surface of the flexible circuit board in real time to generate high-resolution 3D morphological data. The LiDAR has a resolution of 0.01 mm and is fixed on the top or side of the production line to obtain the best 3D scanning angle.
[0102] Use highly sensitive ultrasonic sensors to detect internal defects of flexible circuit boards in real time, such as bubbles, cracks, etc. The ultrasonic sensor has a sensitivity of 0.05 mm and is installed below or on the side of the production line to accommodate the movement and detection of the FPC.
[0103] Step S130: Apply the Kalman filter algorithm to smooth the temperature data and remove the measurement noise, so that the accuracy of the temperature data after denoising is maintained at 0.1°C.
[0104] Statistical filtering and conditional filtering methods are used to remove noise and abnormal points in the three-dimensional point cloud data, so that the resolution of the denoised three-dimensional morphological data is maintained at 0.01 mm.
[0105] An adaptive filtering method is used to denoise the data from the ultrasonic sensor, so that the sensitivity of the defect data after denoising is increased to 0.05 mm.
[0106] After the above preprocessing steps, the quality and robustness of multimodal data are significantly improved, providing comprehensive basic data for subsequent feature extraction and anomaly detection.
[0107] S200, using a deep learning model to extract multi-scale features from images and multimodal data, and perform data fusion and adaptive optimization. Step S200 includes steps S210-S230:
[0108] S210, using a deep learning model to extract features at different levels from the image and multimodal data of S130, step S210 includes steps S211-S212, specifically:
[0109] S211, selecting a deep learning model for the data to be processed, specifically:
[0110] Convolutional Neural Network (CNN): Choose advanced CNN architectures such as ResNet and EfficientNet to extract multi-scale features of images.
[0111] Multimodal Fusion Network: Use deep learning models such as multimodal fusion networks to combine image and sensor data to extract multi-scale features.
[0112] S212, extracting features from the image data processed in S130, specifically:
[0113] (1) Low-level feature extraction method:
[0114] F low,i =σ(W low,i *X i +b low,i )
[0115] Among them, Xi represents the image data after preprocessing in S130, W low,i represents the low-level convolution kernel, b low,i represents the bias, σ represents the activation function, * represents the convolution operation, F low,i Represents the extracted low-level features.
[0116] (2) Methods for extracting intermediate features:
[0117] F mid,i =σ(W mid,i *F low,i +b mid,i )
[0118] Among them, W mid,i represents the intermediate convolution kernel, b mid,i Indicates bias, F mid,i Represents the extracted mid-level features.
[0119] (3) Advanced feature extraction method:
[0120] F high,i =σ(W high,i *F mid,i +b high,i )
[0121] Where: W high,i represents the advanced convolution kernel, b high,i Indicates bias, F high,i Represents the extracted high-level features.
[0122] S220, fusing the image and sensor feature data after intelligent preprocessing and multi-scale feature extraction in S210. Step S220 includes steps S221-S224, specifically:
[0123] S221, performing feature standardization processing on the data processed in S210, including temperature data standardization processing, three-dimensional shape data standardization processing, and internal defect data standardization processing, specifically:
[0124] (1) Temperature data standardization:
[0125]
[0126] Among them, T i represents the temperature data in S212, μT and σT represent the mean and standard deviation of the temperature data respectively, and T'i represents the standardized temperature data.
[0127] (2) Standardization of three-dimensional morphological data:
[0128]
[0129] Among them, S i represents the three-dimensional morphological data in S212, μS and σS represent the mean and standard deviation of the three-dimensional morphological data respectively, and S'i represents the standardized three-dimensional morphological data.
[0130] (3) Internal defect data standardization:
[0131]
[0132] Among them, U i represents the internal defect data in S212, μU and σU represent the mean and standard deviation of the internal defect data respectively, and U'i represents the standardized internal defect data.
[0133] S222, perform feature weighted fusion processing on the data processed in S221, and the specific fusion formula is:
[0134] F fusion,i =φ(αF high,i +βT′ i +γS′ i +6U′ i )
[0135] Among them, F fusion,i is the fusion feature, F high,i Represents the high-level features of the image, T' i Represents the temperature data in S221, S' i Represents the three-dimensional morphological data in S221, U' i represents the internal defect data in S221, α, β, γ, δ represent the fusion weights, which need to be optimized through training, and φ represents the activation function of the fusion network.
[0136] S223, performing network training of feature fusion on the fusion weights in S222, specifically:
[0137]
[0138] Among them, y i represents the true label, represents the predicted value, λ is the regularization parameter, N represents the number of samples, and α, β, γ, and δ represent the fusion weights.
[0139] The specific training process is: use the gradient descent method or the Adam optimization algorithm to minimize the loss function and optimize the fusion weight. The weight is updated each iteration so that the fusion feature can better represent the comprehensive information of multimodal data.
[0140] S224: Output the fusion features in S221-S223, so that the feature data processed by the fusion network contains the comprehensive information of the image and multimodal sensor data, and enhances the representation ability and relevance of the features. fusion,i , providing comprehensive and highly robust basic data for subsequent anomaly detection.
[0141] S230: dynamically adjust the feature extraction and fusion strategies of the data outputted by S220 according to the changes in the production environment, and use online learning and reinforcement learning methods for self-optimization. Step S230 includes steps S231-S234, specifically:
[0142] S231. The system continuously learns and updates the model during operation, so that it can adapt to new data and environmental changes, receive new input data in real time, and use the new data to incrementally update the existing model. It can also adjust the feature extraction and fusion strategy according to the new data to ensure that the model always maintains the best state. Specifically:
[0143]
[0144] Where: θ t represents the current model parameters, η represents the learning rate, L represents the loss function, X t and Y t Represents new input data and corresponding labels.
[0145] S232. Through interaction with the environment, the system learns the optimal strategy through a trial-and-error mechanism, thereby performing reinforcement learning.
[0146]
[0147] Among them, s t Indicates the current state, a t Represents an action, such as adjusting feature extraction or fusion strategies. t Represents the reward function, according to the reward R t Update strategies to maximize long-term rewards.
[0148] S233. Dynamically adjust feature extraction and fusion strategies according to changes in the production environment so that the system can adapt to environmental changes.
[0149]
[0150] Among them, α, β, γ, δ represent feature fusion weights, L represents the gradient of the loss function.
[0151] By real-time monitoring of changes in the production environment, such as noise levels and temperature changes, strategy adjustments can be made. For example, feature extraction and fusion strategies can be adjusted according to environmental changes to ensure high detection rate and robustness of the system in different environments.
[0152] S234. Through online learning and reinforcement learning, the system continuously optimizes itself and dynamically adjusts feature extraction and fusion strategies to ensure continuous and efficient detection. It obtains adaptively optimized feature data and provides high-quality basic data for subsequent anomaly detection.
[0153] Here are some examples:
[0154] Example 1: Image-based multi-scale feature extraction and fusion
[0155] Assume that we have completed steps S100 to S130, which means we have pre-processed high-resolution image data with a signal-to-noise ratio of more than 60dB, a contrast improvement of 1.5 times, and an edge definition improvement of more than 30%. Now, we will use a deep learning model to extract multi-scale features from these images.
[0156] Feature extraction: Using the initial layers of a convolutional neural network (CNN), such as ResNet or EfficientNet, basic features such as texture and edges are extracted from the image. These features will reflect the shape of the solder joint, the width of the line, and the texture of the material. Further through the deep layers of the network, more abstract features are extracted, such as the recognition of specific patterns and preliminary judgments on the quality of solder joints. Finally, the top layer of the network extracts features for global understanding, which may include the layout structure of the entire FPC and potential abnormal patterns.
[0157] After feature extraction, we use a multimodal fusion network to fuse features from different levels together to create a comprehensive representation for detecting and classifying anomalies. For example, low-level features can help locate the location of suspected defects, while high-level features can confirm whether an anomaly really exists and its type.
[0158] Example 2: Fusion and adaptive optimization of multimodal data
[0159] In steps S120 and S130, we collect and pre-process the data from the infrared sensor, lidar and ultrasonic sensor, which provide information on temperature, 3D morphology and internal defects respectively. These data are processed by Kalman filtering, statistical filtering and conditional filtering, and are standardized and ready.
[0160] Normalized temperature data provides detailed information about the heat distribution of the solder joint, helping to identify areas of overheating or insufficient cooling. Normalized 3D morphology data reveals subtle deformations on the FPC surface, indicating possible physical damage or manufacturing deviations. De-noised ultrasonic data reveals the integrity of the internal structure, indicating the presence of bubbles or cracks.
[0161] Using a deep learning model, we perform feature weighted fusion on these multimodal data, where temperature, morphology, and defect data are assigned different weights (α, β, γ) to reflect their relative importance to overall anomaly detection. Through training, these weights are optimized so that the model can detect and classify anomalies more effectively. For example, if the temperature of the production environment fluctuates greatly, the model may automatically increase the weight of the temperature data to better adapt to this change.
[0162] In these instances, the deep learning model not only extracted multi-scale features from the image and sensor data, but also performed data fusion, enabling the system to more fully understand and respond to abnormal situations in the FPC production process. Through adaptive optimization, the system can adjust its detection strategy as production conditions change, maintaining high performance and robustness.
[0163] S300, using the neural network model to detect and classify anomalies in real time, feeding back the detection results and classification information to the production line, and providing optimization suggestions. Step S300 includes steps S310-S320:
[0164] S310: Use a multi-layer neural network model to perform real-time analysis on feature data to detect and classify anomalies. Specifically:
[0165] (1) Select a Multi-Layer Perceptron (MLP) or Convolutional Neural Network (CNN) model to process complex feature data, and design a network layer with sufficient depth, including an input layer, multiple hidden layers, and an output layer, to ensure that the model has sufficient expression and classification capabilities.
[0166] (2) Standardize the input feature data to ensure that the feature data is on a uniform scale, and input the standardized feature data into the input layer of the network.
[0167] (3) Perform forward propagation of hidden layer calculation and output layer calculation on the standardized data:
[0168] Hidden layer calculation:
[0169] h (l) =σ(W (l) h (l-1) +b(l) )
[0170] Among them, h( l ) represents the output of the lth layer, W( l ) represents the weight matrix of the lth layer, b( l ) represents the bias vector of the lth layer, and σ represents the activation function.
[0171] Output layer calculation:
[0172] y=softmax(w (L) h (L-1) +b (L) )
[0173] Among them, y represents the output of the network, that is, the probability distribution of each category, W (L) represents the weight matrix of the output layer, b (L) represents the bias vector of the output layer, and softmax represents the softmax activation function, which converts the output into a probability distribution.
[0174] (4) Set the classification threshold τ. When the probability of a certain category exceeds the threshold, it is judged as an anomaly of that category, thereby performing anomaly detection and classification.
[0175] Classification results:
[0176]
[0177] in, Indicates the predicted category, that is, the anomaly type.
[0178] (5) The detection and classification results are output to the production line control system in real time for reference by the production line operator. Based on the detection results, the abnormal position on the FPC is marked in real time to facilitate rapid positioning and processing.
[0179] S320: Classify and analyze the abnormality detected in S310 in detail, and provide detailed information on the abnormality type and cause.
[0180] (1) Select a support vector machine (SVM), random forest, or a more complex deep learning model (such as LSTM) to perform anomaly classification. Use historical data and known anomaly types to train the classification model so that it can accurately classify different types of anomalies.
[0181] (2) For the input feature vector F anomaly,i Standardization is performed to ensure that the feature data is on a uniform scale.
[0182] Classification prediction:
[0183]
[0184] in, represents the abnormal type after detailed classification, f represents the classification model, and θ represents the model parameters.
[0185] (3) Use feature importance evaluation methods (such as SHAP value) to analyze which features contribute most to anomaly classification.
[0186] Cause analysis formula:
[0187]
[0188] Among them, F j represents the jth feature in the feature vector, Importance(F j ) represents the influence of this feature on the classification result.
[0189] Anomaly type and cause: Provides anomaly type and possible cause based on feature importance and classification results.
[0190] (4) Detailed classification of abnormal types The results of the cause analysis are output to the production line control system. A graphical report of anomaly detection and classification is generated to show the location, type and cause of the anomaly, helping operators to quickly understand and deal with the problem.
[0191] Here are some examples:
[0192] Example 1: Anomaly detection and classification based on image data
[0193] The FPC collected from the production line has high image clarity, a signal-to-noise ratio of more than 60dB, a contrast ratio of 1.5 times, and an edge clarity of more than 30%. Noise reduction, contrast enhancement, and geometric correction have been completed. Convolutional neural networks (CNNs), such as ResNet or EfficientNet, are used to extract multi-scale features from images. The standardized feature data is input into a multi-layer neural network model, such as MLP, for forward propagation to calculate the probability distribution of anomalies. A classification threshold τ is set. When the probability of a certain category exceeds τ, it is judged as an anomaly of that category, such as a bad solder joint or a broken line.
[0194] The detected abnormality location and type are marked on the image in real time and fed back to the production line control system, where the operator can view and handle it immediately; analyze the abnormality frequency and distribution, generate statistical reports, and identify bottlenecks or problem areas in the production process; and make optimization suggestions, such as adjusting welding parameters or improving line design, to reduce the occurrence of specific types of abnormalities.
[0195] Example 2: Anomaly Detection and Classification Based on Multimodal Data
[0196] Infrared sensor temperature data (accuracy 0.1°C), lidar 3D shape data (resolution 0.01 mm) and ultrasonic sensor defect data (sensitivity 0.05 mm).
[0197] Use deep learning models to fuse features of multimodal data to obtain comprehensive feature representation; apply support vector machine (SVM) or random forest (Random Forest) to classify fused features and identify different types of anomalies, such as excessive temperature, surface deformation or internal cracks; analyze feature importance to determine which features contribute most to anomaly classification, such as the impact of temperature data on solder joint quality.
[0198] Output detailed classification of abnormality types and possible causes to the production line control system to help operators quickly locate problems; generate graphical reports of abnormality detection and classification to intuitively display the location, type and cause of abnormalities for easy analysis and discussion; based on feature importance and classification results, make long-term optimization suggestions, such as improving temperature control of the production line or strengthening material inspection to prevent specific types of abnormalities.
[0199] S400, first stores and manages feedback data, then conducts detailed analysis, and uses machine learning and artificial intelligence technology to optimize the system itself and provide long-term optimization and improvement suggestions. Specifically:
[0200] S410: Feedback the test results to the production line in real time, provide optimization suggestions, and perform system self-optimization based on the feedback data, specifically:
[0201] (1) The system collects feedback data and optimization effect data from the production line in real time, including execution results, production parameter changes, and production efficiency data; uses a distributed database or cloud storage platform to store the collected feedback data to ensure data security and scalability; designs the database structure to include fields such as data timestamp, exception type, location, cause, optimization suggestions, and execution results to facilitate subsequent query and analysis; and backs up data regularly to ensure data integrity and security, implements data permission management, and ensures that only authorized personnel can access and modify data.
[0202] (2) Conduct preliminary statistical analysis on the feedback data, calculate the frequency of occurrence of various types of anomalies, cause distribution, and optimization effects; generate statistical reports to display the key performance indicators (KPIs) of the production line, such as detection accuracy, classification accuracy, production efficiency, etc.; use machine learning and data mining technology to conduct in-depth analysis of feedback data to discover potential patterns and optimization opportunities. For example, use clustering algorithms to identify common abnormal patterns and use regression analysis to predict the trend of production efficiency changes.
[0203] (3) Based on the data analysis results, conduct long-term trend analysis to predict possible problems and challenges in the future. For example, based on historical data, predict the probability of a certain type of abnormality in the future and take preventive measures in advance; based on the analysis results, provide long-term optimization and improvement suggestions, including process optimization, equipment upgrades, personnel training, etc. Generate an optimization plan, and describe in detail the specific steps, expected results and implementation schedule of each measure.
[0204] (4) Regularly review and evaluate the effectiveness of optimization measures, adjust and update optimization strategies, and ensure that the system always maintains the best state; through feedback loops, continuously accumulate and utilize data to improve the system's self-optimization capabilities and the overall production level of the production line.
[0205] Based on the existing technology, the present invention introduces a highly robust deep learning algorithm and multimodal data fusion technology to achieve real-time detection and classification of abnormalities in patches in a complex and changeable FPC production environment. The specific beneficial effects are as follows:
[0206] (1) Improve detection accuracy: By using multimodal data collected by high-resolution cameras and multimodal sensors (such as infrared sensors, lidar, and ultrasonic sensors), multi-scale features are extracted through deep learning models, and data fusion is performed, which significantly improves the accuracy of the detection system and reduces false detections and missed detections. By extracting multi-level features of images through convolutional neural networks (such as ResNet and EfficientNet), combined with multimodal data, the representation ability and relevance of features are enhanced.
[0207] (2) Enhance system robustness: Traditional image processing methods are prone to failure in complex backgrounds and noisy environments. The present invention effectively improves the robustness of the system through multimodal data fusion and deep learning algorithms. By standardizing and weighted fusion of image data, temperature data, three-dimensional morphology data, and defect data, the high detection rate and stability of the system in different production environments are ensured.
[0208] (3) Real-time feedback and adaptive optimization: This invention combines online learning and reinforcement learning methods to dynamically adjust feature extraction and fusion strategies according to changes in the production environment, and provide real-time feedback of detection results and optimization suggestions to the production line. The system can self-optimize, continuously improve detection efficiency and accuracy, and ensure efficient operation of the production line.
[0209] (4) Comprehensive anomaly detection information: Compared with traditional methods that rely only on single image data, the present invention uses multimodal data (such as temperature, three-dimensional morphology, and internal defect information) to provide more comprehensive anomaly detection information. Through methods such as Kalman filtering, statistical filtering, and adaptive filtering, multimodal data is denoised to ensure high quality and high precision of the data, providing a solid foundation for anomaly detection.
[0210] (5) Efficient data processing: The present invention uses complex mathematical formulas, such as convolution operations, multivariate regression equations, and reinforcement learning formulas, to perform in-depth processing and analysis of data. For example, the convolutional neural network extracts image features through multi-layer convolution, fuses multimodal data, and uses a feature weighted fusion formula to generate a comprehensive feature representation. The reinforcement learning method maximizes long-term rewards and improves the system's adaptive capabilities by dynamically adjusting fusion weights and feature extraction strategies.
[0211] (6) Long-term optimization and improvement: The system can not only detect and provide feedback in real time, but also store and manage feedback data, conduct detailed analysis, and provide long-term optimization and improvement suggestions. Through machine learning and data mining technology, the system can discover potential patterns and optimization opportunities, generate detailed optimization plans, and improve the overall production level and product quality of the production line.
[0212] The present invention significantly improves the accuracy and robustness of anomaly detection of flexible circuit board patches through the combination of high-resolution cameras, multimodal sensors, deep learning models and adaptive optimization technology, provides real-time feedback and optimization suggestions, improves the efficiency of the production line and product quality, and realizes an efficient and stable anomaly detection system.
[0213] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the described specific features, structures, materials or characteristics may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.
[0214] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.
[0215] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.
[0216] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.
[0217] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.
[0218] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.
[0219] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A method for detecting abnormality of a flexible circuit board patch, characterized in that: include: S100, using high-resolution cameras and multimodal sensors to collect images and sensor data of flexible circuit boards in real time and perform preprocessing; S200, using deep learning models to extract multi-scale features from images and multimodal data, and perform data fusion and adaptive optimization; S300, uses a neural network model to detect and classify anomalies in real time, feeds the detection results and classification information back to the production line, and provides optimization suggestions; S400 first stores and manages feedback data, then conducts detailed analysis, and uses machine learning and artificial intelligence technologies to optimize the system itself and provide long-term optimization and improvement suggestions.
2. The method according to claim 1, characterized in that: The step S100 includes: S110, using a high-resolution camera to capture an image of the flexible circuit board in real time, wherein the camera has a resolution greater than 50 million pixels, a frame rate of at least 30 frames per second, and a signal-to-noise ratio of at least 50 dB; S120, using an infrared sensor, a laser radar, and an ultrasonic sensor to respectively collect temperature, three-dimensional shape, and internal defect information of the flexible circuit board; S130: Perform noise reduction, image enhancement and geometric correction processing on the collected images and sensor data.
3. The method according to claim 1, characterized in that The step S200 includes: S210, extracting low-level, mid-level, and high-level features from preprocessed images and multimodal data using a deep learning model; S220, standardizing the extracted features and performing weighted fusion to form fusion features; S230, dynamically adjust feature extraction and fusion strategies according to changes in the production environment, and use online learning and reinforcement learning methods for self-optimization.
4. The method according to claim 1, characterized in that: The step S300 includes: S310, using a neural network model to perform real-time analysis on the fusion features, and detect and classify anomalies; S320: Detailed classification and analysis of detected anomalies, and provision of detailed information on anomaly types and causes.
5. The method according to claim 1, characterized in that The step S400 further includes: The test results are fed back to the production line in real time, optimization suggestions are provided, and the system is self-optimized based on the feedback data.
6. A flexible circuit board patch abnormality detection system, the system implements the method according to claim 1, characterized in that: include: A data acquisition module is used to collect images and sensor data of the flexible circuit board in real time using a high-resolution camera and a multimodal sensor, and perform preprocessing; Feature extraction module, which is used to extract multi-scale features from images and multimodal data using deep learning models, and perform data fusion and adaptive optimization; Anomaly detection module, which uses a neural network model to detect and classify anomalies in real time, feeds the detection results and classification information back to the production line, and provides optimization suggestions; The self-optimization module is used to first store and manage feedback data, then conduct detailed analysis, and use machine learning and artificial intelligence technologies to perform system self-optimization and provide long-term optimization and improvement suggestions.
7. The system according to claim 6, characterized in that The data acquisition module comprises: High-resolution cameras and multimodal sensors for real-time acquisition of images and sensor data from flexible circuit boards; Intelligent pre-processing unit for performing noise reduction, image enhancement and geometric correction processing of data.
8. The system according to claim 6, characterized in that The feature extraction module comprises: Deep learning models to extract multi-scale features from preprocessed data; The feature fusion unit is used to perform feature normalization and weighted fusion.
9. The system according to claim 6, characterized in that The anomaly detection module comprises: Neural network models to detect and classify anomalies in real time; Classification analysis unit, which provides detailed information on the type and cause of anomalies.
10. The system according to claim 6, characterized in that The self-optimization module comprises: Storage and management system, used to store and manage feedback data; A detailed analysis unit is used to use machine learning and artificial intelligence technologies to optimize the system itself and provide long-term optimization and improvement suggestions.
Citation Information
Patent Citations
Multi-modal sentiment analysis method based on three-level image features and graph data text
CN115512377A
Computer visual identification system and method
CN116958771A
High-precision circuit board tracing system and method
CN116976757A
Unmanned aerial vehicle electric power inspection report production method
CN117371985A
Weld defect detecting and marking method based on multilayer CNN (Convolutional Neural Network) and image processing
CN117557497A
Cited By
Circuit board production quality evaluation system based on artificial intelligence
CN119691582A
Automobile fastener state detection method and system based on artificial intelligence
CN119762444A
A training method and system for 5G flexible circuit board defect detection model
CN119784741A
Slope crack detection method and device based on super-resolution processing
CN120031788A
Multi-mode temperature control method and system of three-in-one laminating plastic packaging machine
CN120161888A
