Intelligent evaluation system for warping degree of PCB (Printed Circuit Board) by fusing visual positioning and multi-mode sensing

Abstract

The invention relates to the technical field of intelligent detection in the electronic manufacturing industry, in particular to a PCB warping degree intelligent evaluation system integrating visual positioning and multi-modal sensing, which comprises a multi-modal sensing unit, a visual positioning unit, an intelligent evaluation engine and a closed-loop control unit, the multi-modal sensing unit integrates laser displacement, infrared thermal imaging and strain sensors to acquire three-dimensional deformation, temperature and stress data; the visual positioning unit realizes sub-pixel-level positioning by using a high-resolution industrial camera and a feature point matching algorithm, and compensates vibration errors; the intelligent evaluation engine fuses data based on a time-space synchronization protocol, predicts a thermal deformation trend through an improved multi-modal convolutional neural network, and dynamically adjusts a qualified threshold value; and the closed-loop control unit executes sorting and rechecking according to an evaluation result, and optimizes warping and leveling parameters. According to the system, multi-dimensional accurate detection and intelligent control are realized, the PCB warping degree detection accuracy is effectively improved, the process can be dynamically optimized according to the production working condition, and the equipment fault risk is reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent detection technology in the electronics manufacturing industry, and specifically to an intelligent evaluation system for the warpage of PCB boards that integrates visual positioning and multi-modal sensing. Background Art

[0002] In the modern electronics manufacturing field, the printed circuit board (PCB), as a key carrier of electronic components, its quality directly affects the performance and reliability of electronic products. With the development of electronic products towards miniaturization, high density, and multi-function, higher requirements are put forward for the manufacturing precision of PCB boards. Among them, the warpage is one of the important indicators to measure the quality of PCB boards.

[0003] Traditional methods for detecting the warpage of PCB boards mainly rely on single sensors, such as laser displacement sensors or vision inspection equipment. These methods have obvious limitations: on the one hand, a single sensor can only obtain information in a single dimension and cannot comprehensively reflect the comprehensive state of the PCB board in a complex production environment. For example, it cannot simultaneously consider the influence of temperature changes and stress distribution on the warpage, resulting in insufficient accuracy of the detection results; on the other hand, in the face of interference factors such as conveyor belt vibration and fluctuations in temperature and humidity in the production environment, it is difficult to ensure the detection accuracy, and false judgments or missed judgments are likely to occur.

[0004] In terms of production process control, existing technologies mostly use fixed standards and empirical parameters for the evaluation and leveling of the warpage of PCB boards. This method cannot dynamically adjust process parameters according to the actual material properties of the PCB board (such as the difference in Young's modulus of different substrates) and real-time production conditions (such as temperature changes), resulting in some PCB boards not achieving the ideal leveling effect, which not only affects product quality but also causes waste of raw materials and energy.

[0005] In addition, traditional detection equipment lacks an effective monitoring and maintenance mechanism for its own operating state. During the long-term operation of the equipment, problems such as mechanical structure fatigue and sensor accuracy decline are difficult to detect in advance, which easily leads to unplanned shutdowns, reduces production efficiency, and increases maintenance costs.

[0006] With the development of intelligent manufacturing technology, electronics manufacturing enterprises urgently need a warpage evaluation system for PCB boards that can achieve multi-dimensional information fusion, precise detection, intelligent decision-making, and adaptive control to meet the growing high-quality production requirements and enhance the core competitiveness of enterprises. Therefore, in view of the above problems, an intelligent evaluation system for the warpage of PCB boards that integrates visual positioning and multi-modal sensing is proposed. Summary of the Invention

[0007] The purpose of the present invention is to provide an intelligent evaluation system for the warpage of PCB boards that integrates visual positioning and multi-modal sensing to solve the problems raised in the above background art.

[0008] To achieve the above object, the present invention provides the following technical solutions:

[0009] The intelligent PCB warpage assessment system that integrates visual positioning and multi-modal sensing includes:

[0010] The multi-modal sensing unit integrates a laser displacement sensor, an infrared thermal imager, and a strain sensor to respectively collect the three-dimensional deformation data, temperature distribution data, and stress distribution data of the printed circuit board in real time;

[0011] The visual positioning unit, which includes a high-resolution industrial camera and a ring light source, performs sub-pixel positioning of printed circuit boards through a feature point matching algorithm to compensate for measurement errors caused by conveyor belt vibration;

[0012] The intelligent evaluation engine, deployed on edge computing devices, performs the following operations:

[0013] Based on the spatiotemporal synchronization protocol, the multimodal sensing data and visual positioning data are integrated to generate a dynamic model of board warpage;

[0014] An improved multimodal convolutional neural network is used to analyze the temperature-stress coupling field and predict the thermal deformation trend;

[0015] Dynamically adjust the board warpage qualification threshold according to the IPC-6012 standard and output the grading evaluation results;

[0016] Closed-loop control unit, receives the evaluation results and links the production line actuators:

[0017] Triggering the sorting mechanism to isolate unqualified printed circuit boards;

[0018] Initiate re-inspection process for warning printed circuit boards;

[0019] Generate warping and leveling parameters based on the dynamic model of plate warpage, and optimize the process library of warping and leveling equipment;

[0020] Predictive maintenance module monitors the fatigue status of the equipment's mechanical structure in real time.

[0021] As a preferred solution, the visual positioning unit realizes posture correction through the following steps:

[0022] Extract the scale-invariant feature transformation descriptor of the current frame and the previous frame image;

[0023] Calculate the minimum sum of squares of feature point coordinate differences and introduce a regularization constraint on the displacement gradient tensor;

[0024] Among them, the vibration suppression coefficient ranges from 0.05 to 0.2, and the output posture correction accuracy is ≤0.02 mm.

[0025] As a preferred solution, the loss function of the improved multi-modal convolutional neural network consists of three parts:

[0026] The mean square error term between the measured value and the predicted value of the laser deformation;

[0027] The KL divergence term between the infrared thermal image and the strain temperature distribution;

[0028] The Frobenius norm term between the measured covariance matrix of stress and the predicted matrix;

[0029] The sum of the three weight coefficients is 1, and the weight of the laser deformation term is greater than the weight of the thermal term, and the weight of the thermal term is greater than the weight of the stress term.

[0030] As a preferred solution, the dynamic adjustment rule of the qualified threshold of the board warpage is:

[0031] Based on the IPC-6012 standard basic threshold;

[0032] Perform linear scaling according to the deviation ratio of the Young's modulus and compressive strength of the printed circuit board material relative to the FR4 substrate;

[0033] Introduce an exponential decay compensation for the temperature deviation, and the temperature correction coefficient is fixed at 50 degrees Celsius;

[0034] Among them, the scaling coefficient of the ceramic substrate is 0.8, and that of the metal substrate is 1.2.

[0035] As a preferred solution, the process optimization algorithm of the closed-loop control unit performs the following operations:

[0036] In the spatial domain of the printed circuit board, minimize the integral of the sum of the squares of the second-order partial derivatives of the warping function;

[0037] Generate the parameters for pressing and flattening: the target pressure value , the flattening temperature , the holding pressure duration ;

[0038] Restrict the pressure and temperature of the pressing and flattening equipment within the process allowable range.

[0039] As a preferred solution, the fatigue warning rule of the predictive maintenance module is:

[0040] Calculate the stress attenuation based on the power-law relationship between the cumulative operating cycle number of the equipment and the designed fatigue life cycle;

[0041] Superimpose the linear compensation terms of the temperature fluctuation amplitude and the pressure fluctuation amplitude;

[0042] When the measured stress exceeds the attenuated threshold, trigger a warning.

[0043] As a preferred solution, the scanning density control logic of the laser displacement sensor is as follows:

[0044] Taking the reference scanning density as the initial value;

[0045] Attenuating according to the negative exponential function of the ratio of the conveyor belt speed to the maximum designed speed;

[0046] The maximum designed speed Adjusted to 5 - 10 m / min according to the rhythm of the press and flattening equipment;

[0047] Superimposing the minimum safety scanning density protection value to prevent too low density.

[0048] As a preferred solution, the adaptive adjustment rule of the three weight coefficients is as follows:

[0049] The weight of the laser deformation term is equal to the ratio of its signal - to - noise ratio to the total signal - to - noise ratio of the multi - modality;

[0050] The weight of the thermal term is equal to the ratio of the infrared signal - to - noise ratio to the total signal - to - noise ratio;

[0051] The weight of the stress term is determined by the remaining ratio of the total signal - to - noise ratio.

[0052] As a preferred solution, the spatio - temporal synchronization protocol satisfies:

[0053] The timestamp of the visual data acquisition plus the optical path transmission delay;

[0054] Is equal to the timestamp of the multi - modality sensor data acquisition superimposed with the hardware delay calibration amount and the action delay amount of the press and flattening equipment;

[0055] The speed of light is fixed at 3×10 8 m / s.

[0056] As a preferred solution, the method for optimizing the press and flattening parameters includes:

[0057] Adopting a hierarchical pressure control strategy: applying 1.5 times the reference pressure to the area of the PCB board with a warpage gradient ≥ 0.1 mm / cm²;

[0058] Real - time monitoring of the stress relaxation rate after flattening. If the relaxation rate is < 90% within 10 seconds, trigger secondary flattening.

[0059] It can be seen from the technical solutions provided by the present invention above that the intelligent evaluation system for PCB board warpage degree integrating visual positioning and multi - modality sensing provided by the present invention has the following beneficial effects:

[0060] High-precision detection ensures quality: The multi-modal sensing unit collaborates with laser, infrared, and strain sensors to collect data from multiple physical field dimensions. Combining with the high-precision positioning technology of the vision positioning unit, it realizes the accurate measurement of the warpage of the PCB board. The intelligent evaluation engine uses advanced neural network algorithms to deeply analyze the data and dynamically adjusts the evaluation criteria according to material characteristics and environmental factors, effectively avoiding misjudgments, significantly improving the quality of the PCB board, and meeting the strict requirements of the high-end manufacturing field;

[0061] Efficient operation improves production capacity: The closed-loop control unit quickly responds and executes sorting, re-inspection, and leveling operations based on the intelligent evaluation results. The dynamic scanning strategy of the laser displacement sensor and the process optimization algorithm significantly shorten the detection and processing time while ensuring detection accuracy. The fully automated operation of the system reduces manual intervention, significantly improves the detection efficiency and processing speed of the production line, and speeds up the production rhythm;

[0062] Intelligent maintenance reduces costs: The predictive maintenance module monitors the equipment status in real time, anticipates potential faults in advance through scientific algorithm models, and reduces unplanned downtime. Precise process optimization avoids waste of raw materials and excessive wear of equipment, reduces equipment maintenance and production operation costs, and improves the economic benefits of the enterprise;

[0063] Flexible adaptation enhances flexibility: The system has strong adaptive capabilities and can automatically adjust detection and processing parameters according to different materials, specifications of PCB boards, and changes in the production environment. Without complex manual debugging, it can quickly adapt to diverse production requirements, improve the flexibility level of the production line, and enhance the enterprise's response ability to market changes;

[0064] Data-driven promotes innovation: A large amount of data generated during the operation of the system is deeply mined and analyzed, which is not only used to optimize core algorithms and technologies, but also provides a decision-making basis for the improvement of PCB manufacturing processes and equipment upgrades, helping enterprises achieve intelligent and digital transformation and maintain technological leading advantages. Brief Description of the Drawings

[0065] Figure 1 It is a schematic diagram of the overall structure of the intelligent evaluation system for the warpage of PCB boards integrating vision positioning and multi-modal sensing of the present invention. Detailed Embodiments

[0066] In order to make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0067] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the drawings of the specification and specific embodiments.

[0068] As Figure 1 shown, the embodiment of the present invention provides an intelligent evaluation system for PCB board warpage degree integrating visual positioning and multi-modal sensing, including:

[0069] A multi-modal sensing unit, integrating a laser displacement sensor, an infrared thermal imager, and a strain sensor, respectively collecting three-dimensional deformation data, temperature distribution data, and stress distribution data of the printed circuit board in real time;

[0070] A visual positioning unit, including a high-resolution industrial camera and an annular light source, performing sub-pixel level positioning on the printed circuit board through a feature point matching algorithm to compensate for measurement errors caused by conveyor belt vibration;

[0071] An intelligent evaluation engine, deployed on an edge computing device, performing the following operations:

[0072] Based on a spatio-temporal synchronization protocol, fusing multi-modal sensing data and visual positioning data to generate a dynamic warpage degree model of the board;

[0073] Using an improved multi-modal convolutional neural network to analyze the temperature-stress coupling field and predict the thermal deformation trend;

[0074] Dynamically adjusting the qualified threshold of the board warpage degree according to the IPC-6012 standard and outputting a hierarchical evaluation result;

[0075] A closed-loop control unit, receiving the evaluation result and linking with the production line actuator:

[0076] Triggering the sorting mechanism to isolate unqualified printed circuit boards;

[0077] Starting a re-inspection process for warning printed circuit boards;

[0078] Generating press-flattening parameters based on the dynamic warpage degree model of the board and optimizing the process library of the press-flattening equipment;

[0079] A predictive maintenance module, real-time monitoring the fatigue state of the mechanical structure of the equipment.

[0080] In this embodiment, the multi-modal sensing unit, as the "perception core" of the intelligent evaluation system for PCB board warpage degree integrating visual positioning and multi-modal sensing, realizes the accurate acquisition of multi-dimensional physical parameters of the PCB board through integrating multiple sensors; the following will elaborate in detail from aspects such as overall function, sub-module composition, key technical principles, working process, and application value:

[0081] I. Overview of overall function:

[0082] The multimodal sensing unit realizes real-time data collection of 3D deformation, temperature distribution and stress distribution of printed circuit boards by coordinating laser displacement sensors, infrared thermal imagers and strain sensors. Its core function is to convert physical quantities into digital information and perform preliminary data preprocessing to provide highly reliable raw data for subsequent analysis by the intelligent evaluation engine, which is the basis for building a dynamic model of PCB warpage.

[0083] 2. Sub-module composition and functions:

[0084] 1. Laser displacement sensor module:

[0085] 3D data acquisition: Based on the principle of triangulation, the laser displacement sensor emits a laser beam to the surface of the PCB board, and calculates the 3D coordinates of the measured point by receiving the angle and time difference of the reflected light; during the scanning process, the scanning density is dynamically adjusted according to the conveyor belt speed. The calculation formula is: (in, is the adaptive laser scanning point density, in points / square centimeter; is the base scanning density, which is 25 dots / cm2; is the conveyor belt speed attenuation coefficient, which takes a value of 1.5; The current conveyor belt running speed, in meters per minute; is the maximum design speed of the equipment, which is 5m / min; The minimum safe scanning density is 8 points / cm2, ensuring that high-precision 3D deformation data can be obtained even at high speed.

[0086] Data preprocessing: De-noising and filtering the collected point cloud data to remove abnormal points caused by environmental interference or sensor errors, and interpolation algorithms are used to supplement data in sparse areas to improve data integrity and continuity;

[0087] (II) Infrared thermal imager module:

[0088] Temperature field capture: The infrared thermal imager detects the infrared radiation energy on the surface of the PCB board, converts it into a temperature distribution image, and outputs the temperature data in matrix form. The equipment has high resolution and high sensitivity, which can identify tiny temperature differences and capture the temperature changes of PCB boards caused by current distribution, component heating and other factors during the production process;

[0089] Data calibration and normalization: Perform non-uniformity correction on the original temperature data to eliminate the measurement error caused by the performance difference of the detector itself; at the same time, normalize the temperature data to a unified range to facilitate fusion analysis with other modal data;

[0090] (3) Strain Sensor Module:

[0091] Stress Distribution Monitoring: An array of foil strain gauges is used. The strain gauges are pasted on key parts of the PCB board. When the PCB board deforms, the resistance value of the strain gauges changes. The resistance change is converted into a voltage signal through a Wheatstone bridge circuit, and then the stress values of each measurement point are calculated to construct an actual measurement matrix of stress covariance. ;

[0092] Signal Amplification and Filtering: Since the voltage signal generated by the strain gauge is weak, it needs to be amplified by a signal amplifier. At the same time, band-pass filtering technology is adopted to filter out high-frequency noise and low-frequency drift, improving the signal-to-noise ratio and accuracy of stress data.

[0093] III. Key Technical Principles:

[0094] (1) Principle of Multi-Sensor Cooperative Acquisition:

[0095] The multi-modal sensing unit realizes precise alignment of data of each sensor based on the spatio-temporal synchronization protocol. The formula is: (where is the timestamp of the vision system, is the distance from the camera to the measured point, m / s is the speed of light, is the sensor timestamp, is the system calibration delay, is the action delay of the press and flattening equipment); Through this protocol, the consistency of laser displacement, infrared temperature, and stress data in time and space is ensured, laying a foundation for subsequent multi-modal data fusion;

[0096] (2) Principle of Physical Quantity Conversion and Data Processing:

[0097] The laser displacement sensor uses the optical triangulation method to convert the spatial distance into an electrical signal. The infrared thermal imager converts the infrared radiation energy into temperature data based on Planck's law. The strain sensor converts mechanical strain into an electrical signal through the piezoresistive effect; After the analog signals output by each sensor are converted into digital signals through A / D conversion, targeted data processing algorithms (such as point cloud filtering, temperature correction, stress signal amplification, etc.) are respectively used to convert the original physical signals into effective data available for analysis;

[0098] IV. Working Process of the Module:

[0099] (1) Initialization Phase:

[0100] After the multi-modal sensing unit is started, hardware self-checks are performed on the laser displacement sensor, infrared thermal imager, and strain sensor to check whether the sensor power supply is normal, whether the communication interface is stably connected, and whether there is physical damage to the measurement components.

[0101] Load the calibration parameters and configuration information of each sensor, establish a data transmission channel with the intelligent evaluation engine, and prepare for data acquisition work;

[0102] (2) Data acquisition stage:

[0103] The laser displacement sensor collects 3D point cloud data on the surface of the PCB board according to the scanning density control logic; the infrared thermal imager captures the temperature field distribution image of the PCB board in real time; the strain sensor continuously monitors the stress changes at each measurement point and synchronously outputs voltage signals;

[0104] Each sensor transmits the collected raw data to the data preprocessing unit for operations such as denoising, filtering, calibration, and normalization;

[0105] (3) Data output stage:

[0106] Timestamp the preprocessed 3D deformation data, temperature distribution data, and stress distribution data according to the spatio-temporal synchronization protocol and align the spatial coordinates, and then package and send them to the intelligent evaluation engine for subsequent model construction and analysis;

[0107] V. Application value of the module:

[0108] (1) Improve detection accuracy:

[0109] Through the collaborative acquisition of multi-modal data, break through the detection limitations of a single sensor, and achieve multi-dimensional accurate measurement of the warpage of the PCB board; for example, by combining laser displacement data and stress distribution data, the mechanical causes of warpage can be analyzed more accurately, improving the reliability of detection results;

[0110] (2) Enhance fault diagnosis ability:

[0111] The fusion of infrared thermal imaging data and stress data can be used to analyze the potential fault risks of the PCB board caused by thermal stress, discover warpage or component damage problems caused by abnormal temperature in advance, and provide rich information dimensions for fault diagnosis;

[0112] (3) Optimize the production process:

[0113] The multi-modal data collected in real time can provide a basis for optimizing the production process of the PCB board; for example, adjust the process parameters of the warpage flattening equipment according to the temperature distribution and stress distribution to achieve precise control of the warpage and improve product quality and production efficiency.

[0114] In this embodiment, the visual positioning unit realizes pose correction through the following steps:

[0115] Extract the scale-invariant feature transform descriptors of the current frame and the previous frame of images;

[0116] Calculate the least squares sum of the coordinate differences of the feature points, and introduce a regularization constraint term for the displacement gradient tensor;

[0117] Among them, the vibration suppression coefficient ranges from 0.05 to 0.2, and the accuracy of the output pose correction amount ≤ 0.02 mm;

[0118] Furthermore, as the core positioning module of the "Intelligent Evaluation System for PCB Warpage Degree Integrating Visual Positioning and Multimodal Sensing", the visual positioning unit is like the "visual nerve center" of the system. With the help of advanced hardware and intelligent algorithms, it realizes high-precision spatial positioning of the PCB board, laying a solid foundation for the system to accurately evaluate the warpage degree. The following is a comprehensive analysis from aspects such as functional architecture, core components, algorithm principle, operation process, and application advantages:

[0119] I. Overall functional architecture:

[0120] The visual positioning unit relies on a high-resolution industrial camera, a ring light source, and a supporting processing unit to build a complete visual positioning system. Its core task is to obtain accurate spatial coordinate information through feature point extraction, matching, and pose correction during the high-speed movement of the PCB board along the conveyor belt, provide a precise spatial reference for multimodal sensing data, and at the same time eliminate positioning errors caused by factors such as conveyor belt vibration, ensuring the measurement accuracy and stability of the entire evaluation system;

[0121] II. Detailed explanation of core components and functions:

[0122] (I) High-resolution industrial camera:

[0123] High-speed high-definition imaging: The industrial camera equipped with a high-pixel and high-frame-rate image sensor can capture clear images with a resolution of more than 2000×2000 pixels at a rate of dozens of frames per second under the working conditions where the conveyor belt speed is up to . Even for extremely small feature points on the PCB board, such as fine solder pads and component pins, it can accurately image, providing rich image details for subsequent sub-pixel positioning;

[0124] Intelligent parameter adjustment: It has a powerful ability to adaptively adjust parameters, and can automatically optimize the exposure time, focal length, and aperture according to the illumination intensity of the production environment, the surface material of the PCB board, and the reflection characteristics. For example, when processing a metal substrate with strong surface reflection, the camera can quickly shorten the exposure time to avoid overexposure of the image and ensure that the feature points are clearly distinguishable;

[0125] II. Ring light source:

[0126] Omnidirectional uniform illumination: The LED light source array distributed in a ring around the industrial camera can provide 360-degree dead-angle-free uniform illumination. Through the carefully designed light source angle and brightness distribution, it effectively eliminates the shadows generated on the PCB board surface due to unevenness or component occlusion, while suppressing the reflection interference, significantly enhancing the contrast between the feature points and the background in the image, making the key features such as the PCB board edge contour and identification pattern more prominent, facilitating the accurate extraction by the algorithm.

[0127] Intelligent brightness adaptation: Supports dynamic adjustment of brightness and can automatically adjust the light source intensity according to the PCB board material (such as ceramic substrate, ordinary FR-4 substrate) and surface color differences. For PCB boards with darker colors and strong light absorption, the light source will automatically increase the brightness. For light-colored or highly reflective boards, the brightness will be reduced to ensure that ideal imaging effects can be obtained for various PCB boards.

[0128] (III) Image acquisition and processing system:

[0129] High-speed data transmission: The image acquisition card uses high-speed data interfaces such as GigE Vision and Camera Link, which can transmit a large amount of image data collected by the industrial camera to the processing unit at an extremely fast speed. The transmission delay is almost negligible, ensuring the real-time nature of the data and avoiding affecting the timeliness and accuracy of positioning due to slow transmission.

[0130] Efficient algorithm processing: The processing unit is equipped with a high-performance processor and integrates core algorithms such as feature point matching and pose correction. Using parallel computing technology, it can simultaneously process multiple groups of image data, greatly improving the operation efficiency of the algorithm. During the rapid movement of the PCB board, it can also quickly complete image analysis and positioning calculations and output accurate positioning results in real time.

[0131] III. Analysis of the key algorithm principle:

[0132] (I) SIFT feature point extraction and matching algorithm:

[0133] Robust feature point extraction: Using the Scale-Invariant Feature Transform (SIFT) algorithm, feature points are extracted from two consecutive frames of images and . This algorithm constructs a multi-scale space, detects the extreme points in the image at different scales, and precisely locates and assigns directions to them, thereby obtaining a large number of stable and unique feature points. These feature points have extremely strong robustness to the scale change, rotation, and illumination intensity change of the image and can accurately capture the key geometric features on the PCB board.

[0134] Precise matching and screening: Calculate the similarity between feature descriptors based on Euclidean distance, and screen out potential matching point pairs with relatively high similarity; further adopt the ratio method of the nearest neighbor and the second nearest neighbor distance (usually set the ratio to be less than 0.7) to eliminate mis-matching points with relatively low similarity; at the same time, combine the RANSAC (Random Sample Consensus) algorithm, and through random sampling and model verification, remove incorrect matches again to ensure the accuracy and reliability of the final matching point pairs;

[0135] (2) Pose correction algorithm:

[0136] Calculate the pose correction amount by minimizing the objective function : (wherein, is the vibration suppression coefficient, and its value range is between 0.05 - 0.2, which is used to balance the feature matching error and the smoothness of the displacement field, and effectively suppress the irregular displacement caused by the vibration of the conveyor belt; is the number of successfully matched feature points; represents the -th coordinate of the feature point in the image; is the displacement vector to be solved; is the displacement field gradient, which ensures the continuity and smoothness of the displacement change; represents the Scale - Invariant Feature Transform descriptor, which is used to extract feature points with scale and rotation invariance in the image), this algorithm accurately calculates the best pose correction amount of the current frame image relative to the previous frame through continuous iterative optimization, controls the positioning error of the PCB board to be ≤ 0.02 mm, and realizes high - precision pose calibration;

[0137] IV. Full analysis of the working process:

[0138] (1) Initialization and preparation stage:

[0139] Comprehensive hardware self - inspection: After the unit starts, immediately conduct a system inspection on hardware devices such as industrial cameras, ring lights, and image acquisition cards, check whether the device power supply is stable, the communication connection is normal, and the functions of each component are intact, to ensure that the hardware is in the best working state;

[0140] Parameter configuration and connection establishment: Load the SIFT algorithm parameters and the initial parameters of the pose correction model, and configure the acquisition parameters such as the camera exposure time and the light source brightness according to the production process requirements; at the same time, establish a data communication link with the intelligent evaluation engine to prepare for data interaction;

[0141] (2) Image acquisition and pre - processing stage:

[0142] Real-time image acquisition: The industrial camera continuously acquires images of the PCB board on the conveyor belt at a preset frame rate. The annular light source synchronously provides stable and uniform illumination to ensure that each captured image is clear and complete.

[0143] Image optimization processing: After the acquired original image is transmitted to the processing unit, it is first subjected to grayscale processing to convert the color image into a grayscale image, simplifying the complexity of data processing. Denoising algorithms such as median filtering are used to remove noise interference in the image, enhance the image quality, and create good conditions for subsequent feature extraction.

[0144] (III) Positioning and correction execution stage:

[0145] Feature extraction and matching: For the preprocessed image, the SIFT algorithm is used to extract feature points and match them with the feature points of the previous frame image. Through screening and verification, reliable matching point pairs are determined.

[0146] Precise pose correction: Based on the matching point pairs, the pose correction algorithm is used to calculate the pose correction amount of the current frame image , and the spatial coordinates of the PCB board are corrected in real time to obtain accurate position and attitude information.

[0147] Data fusion and transmission: The corrected positioning data, including information such as the precise position and attitude of the PCB board, is sent to the intelligent evaluation engine in a timely manner and fused with multi-modal sensing data to provide accurate spatial coordinate basis for warpage evaluation.

[0148] (IV) Dynamic adaptive optimization stage:

[0149] According to actual production conditions such as changes in conveyor belt speed and switching of PCB board types, the vision positioning unit automatically adjusts the camera acquisition parameters (such as frame rate, exposure time) and algorithm parameters (such as vibration suppression coefficient ); for example, when the conveyor belt speed increases, the camera frame rate is increased to capture more image frames; in the face of PCB boards of different sizes and materials, the algorithm parameters are optimized to ensure stable and high-precision vision positioning in various complex production scenarios.

[0150] V. Application value and advantage manifestation:

[0151] (I) Accurately eliminate motion errors:

[0152] With the real-time pose correction technology, the vision positioning unit can effectively offset the measurement errors caused by factors such as conveyor belt vibration and speed fluctuations, making the warpage evaluation result of the PCB board more stable and reliable; in actual production tests, when the conveyor belt speed is 3m / min, the error of the traditional vision positioning method is about 0.1mm, while after using this unit, the error is significantly reduced to within 0.02mm, significantly improving the detection accuracy.

[0153] (2) Strengthen the efficiency of multi-modal data fusion:

[0154] Provide an accurate spatial coordinate reference for multi-modal sensing data such as laser displacement sensors and infrared thermal imagers, ensure the accurate alignment and seamless fusion of data from different types of sensors in the same coordinate system, avoid evaluation errors caused by positioning deviations, and greatly enhance the accuracy and reliability of the dynamic model of the PCB board warpage;

[0155] (3) Widely adapt to diverse production requirements:

[0156] Through the parameter adaptive adjustment and algorithm optimization mechanism, the vision positioning unit can flexibly accommodate the positioning requirements of PCB boards with different sizes, materials, and production processes; whether it is the fine detection of high-density interconnect (HDI) boards, the rapid scanning of large multi-layer PCB boards, or the positioning of special material boards such as ceramic substrates and metal substrates, it can efficiently and accurately complete the tasks, greatly improving the versatility and adaptability of the system, and providing strong support for the intelligent upgrade of PCB board production and manufacturing.

[0157] In this embodiment, the loss function of the improved multi-modal convolutional neural network consists of three parts:

[0158] The mean square error term between the measured value and the predicted value of the laser deformation;

[0159] The KL divergence term between the infrared thermal map and the strain temperature distribution;

[0160] The Frobenius norm term between the measured covariance matrix and the predicted matrix of stress;

[0161] The sum of the three weight coefficients is 1, and the weight of the laser deformation term is greater than the weight of the thermal term, and the weight of the thermal term is greater than the weight of the stress term;

[0162] The dynamic adjustment rule of the qualified threshold of the board warpage is:

[0163] Based on the basic threshold of the IPC-6012 standard;

[0164] Perform linear scaling according to the deviation ratio of the Young's modulus and compressive strength of the printed circuit board material relative to the FR4 substrate;

[0165] Introduce an exponential decay compensation for the temperature deviation, and the temperature correction coefficient is fixed at 50 degrees Celsius;

[0166] Among them, the scaling coefficient of the ceramic substrate is 0.8, and that of the metal substrate is 1.2;

[0167] Furthermore, as the "decision-making brain" of the "Intelligent Warpage Degree Evaluation System for PCB Boards Integrating Visual Positioning and Multimodal Sensing", the intelligent evaluation engine realizes the accurate evaluation and trend prediction of the warpage degree of PCB boards through multimodal data fusion, intelligent algorithm analysis, and dynamic threshold decision-making. The following will elaborate in detail from aspects such as functional architecture, core modules, algorithm principles, working processes, and application values:

[0168] I. Overall Functional Architecture:

[0169] The intelligent evaluation engine is deployed on edge computing devices. Taking the three-dimensional deformation, temperature distribution, and stress distribution data collected by the multimodal sensing unit, as well as the accurate pose information provided by the visual positioning unit as inputs, through data fusion, model construction, intelligent analysis, and dynamic decision-making, it realizes the real-time evaluation of the warpage degree of PCB boards, the prediction of thermal deformation trends, and quality grading. Its core functions cover data processing, model calculation, threshold optimization, and result output, providing decision-making basis for the closed-loop control unit and driving the intelligent adjustment and optimization of the production line;

[0170] II. Detailed Explanation of Core Modules and Functions:

[0171] (I) Multimodal Data Fusion Module:

[0172] Spatio-temporal Synchronization Processing: Based on the spatio-temporal synchronization protocol formula (where is the timestamp of the visual system, is the distance from the camera to the measured point, m / s is the speed of light, is the sensor timestamp, is the system calibration delay, is the action delay of the press and flattening equipment) to align the timestamps and map the spatial coordinates of the laser displacement, infrared thermal imaging, strain sensor data, and visual positioning data, ensuring the consistency of different modal data in the spatio-temporal dimension and eliminating the deviations caused by transmission delays and physical distances;

[0173] Feature-Level Fusion: Adopt the principal component analysis (PCA) and mutual information fusion algorithms to extract the key features of each modal data (such as the curvature features of three-dimensional point clouds, the gradient features of temperature fields, and the extreme value features of stress distributions), and fuse them into a high-dimensional feature vector to construct a dynamic model of the warpage degree of PCB boards containing multi-physical field information;

[0174] (II) Intelligent Analysis and Prediction Module:

[0175] Improved Multimodal Convolutional Neural Network (CNN): Deploy a customized multimodal CNN model, and its loss function is (where is the measured value of laser deformation, is the predicted value, is the infrared temperature distribution matrix, is the strain inversion temperature field, is the measured matrix of stress covariance, is the prediction matrix, and the weight coefficient , , satisfies and , and through , , realizes adaptive adjustment, are the signal-to-noise ratios of each sensor respectively);

[0176] This network automatically extracts the non-linear characteristics of the temperature-stress coupling field through multiple convolutional layers, pooling layers and fully connected layers, predicts the thermal deformation trend of the PCB board under different working conditions, and identifies potential warping risks in advance;

[0177] Dynamic threshold adjustment algorithm: According to the IPC-6012 standard, combined with real-time production data, through the formula (where is the IPC-6012 standard threshold, is the current Young's modulus of the substrate, GPa is the standard Young's modulus of FR-4 material, is the material sensitivity factor, taking 0.8 for ceramic substrates and 1.2 for metal substrates, is the temperature deviation, °C) dynamically adjusts the qualified threshold of the board warping degree to achieve differential evaluation of PCB boards under different material and temperature conditions;

[0178] (3) Result output and decision-making module:

[0179] Hierarchical evaluation output: According to the warping degree evaluation result, referring to the dynamically adjusted threshold, the PCB board is divided into three grades of "qualified", "warning" and "unqualified", and a visualization report including 3D deformation cloud map, temperature-stress distribution heat map and evaluation conclusion is generated;

[0180] Control instruction generation: Send control instructions to the closed-loop control unit, such as triggering the sorting mechanism to isolate unqualified PCB boards, starting the re-inspection process for warning PCB boards, and at the same time outputting the pressure warping and leveling parameters (pressure , temperature , time ) optimized based on the board warping degree dynamic model to guide the process adjustment of the production line;

[0181] III. Analysis of Key Technical Principles:

[0182] 1. Multimodal Data Fusion Technology:

[0183] Based on information theory and machine learning theory, eliminate data asynchrony through spatio-temporal alignment, and use feature-level fusion algorithms to mine the correlation between multimodal data. For example, by combining the geometric features of laser displacement data with the temperature features of infrared thermal imaging, the impact of temperature changes on the deformation of PCB boards can be analyzed, improving the accuracy of the evaluation model.

[0184] 2. Adaptive Threshold Optimization Algorithm:

[0185] Based on the principles of material mechanics and thermodynamics, establish a dynamic threshold adjustment model by quantifying the influence of the Young's modulus of the substrate and temperature deviation on the warping degree. When the substrate material is ceramic, because , its threshold is less sensitive to temperature changes than that of metal substrates ( ), thus achieving precise evaluation of material specificity.

[0186] 3. Deep Learning Prediction Model:

[0187] The improved multimodal CNN minimizes the loss function through the backpropagation algorithm , and optimizes the network parameters. Among them, the mean square error (MSE) ensures the accuracy of deformation prediction, the KL divergence measures the consistency of the temperature field distribution, and the Frobenius norm constrains the prediction accuracy of the stress matrix. The three work together to achieve high-precision prediction of the thermal deformation trend.

[0188] IV. Full Analysis of the Workflow:

[0189] 1. Data Reception and Preprocessing Stage:

[0190] Multi-source data access: Receive 3D deformation, temperature, and stress data from multimodal sensing units, as well as pose correction information from visual positioning units, and perform format unification and integrity verification on the data.

[0191] Data cleaning and standardization: Remove outliers and normalize numerical data (such as normalizing temperature data to the [0,1] interval) to prepare for subsequent fusion analysis.

[0192] 2. Data Fusion and Model Construction Stage:

[0193] Spatio-temporal synchronous fusion: According to the spatio-temporal synchronization protocol, perform timestamp alignment and spatial coordinate mapping on multimodal data to generate a fusion dataset.

[0194] Dynamic model construction: Based on the fusion dataset, construct a dynamic model of the warping degree of PCB boards containing multi-physical field information to provide a basis for intelligent analysis.

[0195] (3) Intelligent Analysis and Decision-making Stage:

[0196] Prediction of Thermal Deformation Trend: Input the fused data into the improved multi-modal CNN to predict the thermal deformation trend of the PCB board and output the warpage change curve within a certain period in the future;

[0197] Calculation of Dynamic Threshold: Calculate the qualified threshold of warpage after dynamic adjustment according to the current substrate material and temperature conditions;

[0198] Hierarchical Evaluation and Decision-making: Compare the measured warpage with the dynamic threshold, conduct hierarchical evaluation of the PCB board quality, and generate corresponding control instructions and optimization parameters;

[0199] (4) Result Output and Feedback Stage:

[0200] Output of Evaluation Results: Send the hierarchical evaluation results and control instructions to the closed-loop control unit, and at the same time upload the visualization report to the production management system;

[0201] Model Iterative Optimization: Collect the feedback data from the production line (such as the effect of pressing and leveling warpage and the re-inspection results) to improve the parameters of the multi-modal CNN model and the dynamic threshold adjustment algorithm, and continuously improve the evaluation accuracy;

[0202] V. Application Value and Advantage Embodiment:

[0203] (1) Improvement of Evaluation Accuracy and Timeliness:

[0204] Through multi-modal data fusion and intelligent algorithm analysis, high-precision real-time evaluation of the warpage of the PCB board is realized, and the prediction error rate of the thermal deformation trend is less than 5%. Compared with the traditional single detection method, the evaluation efficiency is increased by more than 60%;

[0205] (2) Enhancement of System Adaptability:

[0206] The dynamic threshold adjustment algorithm enables the system to automatically adapt to the detection requirements of PCB boards under different materials and temperature conditions, without the need for manual frequent parameter adjustment, reducing the operation complexity and improving the flexibility level of the production line;

[0207] (3) Driving Intelligent Production:

[0208] Provide accurate decision-making basis for the closed-loop control unit. By optimizing the pressing and leveling parameters, the defective rate of the warpage of the PCB board can be reduced by 30%-40%. At the same time, combined with the predictive maintenance module, the equipment failure downtime is reduced, and the production efficiency and product quality are significantly improved.

[0209] In this embodiment, the process optimization algorithm of the closed-loop control unit performs the following operations:

[0210] Integrate and minimize the sum of squares of the second-order partial derivatives of the warping function in the spatial domain of the printed circuit board;

[0211] Generate press-warping and flattening parameters: target pressure value , flattening temperature , pressure holding duration ;

[0212] Restrict the pressure and temperature of the press-warping and flattening equipment within the process allowable range;

[0213] Furthermore, as the "execution center" of the "Intelligent Warpage Evaluation System for PCB Boards Integrating Visual Positioning and Multimodal Sensing", the closed-loop control unit plays a key role in connecting the evaluation results with production operations. By responding in real time to the instructions of the intelligent evaluation engine, it realizes the dynamic regulation and optimization of the production line. The following elaborates in detail from aspects such as functional architecture, core modules, algorithm principles, working processes, and application values:

[0214] I. Overall functional architecture:

[0215] The closed-loop control unit receives the hierarchical evaluation results and optimization parameters of the PCB board warpage degree output by the intelligent evaluation engine. According to preset rules and algorithms, it links the production line actuators to realize the sorting of unqualified products, the re-inspection of warning products, and the automatic optimization of the production process. Its core functions include result response, action execution, parameter optimization, and feedback adjustment. By forming a closed-loop link of "evaluation - decision - execution - feedback", it ensures the efficient and stable operation of the production line and improves the consistency of product quality;

[0216] II. Detailed explanation of core modules and functions:

[0217] (I) Evaluation result analysis module:

[0218] Hierarchical instruction recognition: Receive the hierarchical results of "qualified", "warning", and "unqualified" of the PCB board output by the intelligent evaluation engine, analyze the attached detailed evaluation data (such as warpage degree value, prediction of thermal deformation trend, dynamic threshold, etc.), and clarify the quality status and potential risks of each PCB board;

[0219] Instruction priority division: According to the hierarchical results and production process requirements, sort the control instructions by priority. For example, the sorting instruction for "unqualified" products has the highest priority and needs to be executed immediately; the re-inspection instruction for "warning" products is next, ensuring that the inspection is completed without affecting the production rhythm;

[0220] (II) Actuator linkage module:

[0221] Sorting mechanism control: For "unqualified" PCB boards, send a trigger signal to the sorting mechanism to control the execution components such as the robotic arm and pneumatic push rod, remove the unqualified products from the production line and transfer them to the designated area to prevent them from flowing into the next process;

[0222] Re-inspection process start: When receiving the "warning" instruction, start the re-inspection process, control the conveyor belt to transport the PCB board to an independent re-inspection station, and link auxiliary detection equipment (such as high-precision microscopes, X-ray detectors) for secondary detection to verify the accuracy of the evaluation results;

[0223] Warping and flattening equipment regulation: According to the optimization parameters (pressure , temperature , time ) provided by the intelligent evaluation engine, adjust the process parameters of the warping and flattening equipment; through the PID (Proportional-Integral-Derivative) control algorithm, precisely control the pressure output and temperature regulation of the equipment to ensure the best effect during the flattening process of the PCB board;

[0224] (III) Process optimization algorithm module (method for optimizing warping and flattening parameters):

[0225] Warping function optimization: Execute the process optimization algorithm (where is the warping function, describing the warping degree of the PCB board in the two-dimensional space ; is the pressure constraint range, is the temperature constraint range, is the flattening time constraint range); by minimizing the curvature integral of the warping function, determine the optimal combination of pressure and temperature parameters to achieve precise control of the warping degree of the PCB board;

[0226] Layered pressure control: For high-curvature regions that satisfy (where is the gradient of the warping function representing the warping rate change), apply layered pressure( is the reference pressure) to ensure effective correction of severely warped local areas;

[0227] Stress relaxation verification: After flattening, use a stress sensor to continuously monitor the residual stress of the PCB board and calculate the stress relaxation rate (where is the initial stress, is the residual stress after holding pressure for 10 seconds); if , then trigger the secondary flattening process to ensure that the product quality meets the standards;

[0228] (IV) Feedback adjustment module:

[0229] Execution effect monitoring: Real-time collect the action status of the actuator (such as the grasping success rate of the sorting mechanism, the actual output value of the parameters of the press and flattening equipment) and the quality data of the PCB board (such as the warpage degree and stress distribution after flattening), and monitor the execution effect of the control instruction;

[0230] Dynamic parameter correction: Feed back the monitoring data to the intelligent evaluation engine and the process optimization algorithm module. By comparing the difference between the target value and the actual value, automatically adjust the control parameters; for example, if the warpage degree of the PCB board after pressing and flattening does not meet the expectation, the algorithm module will recalculate the optimized parameters and send them to the actuator for readjustment, forming a closed-loop feedback mechanism;

[0231] III. Analysis of key technical principles:

[0232] (I) PID control technology:

[0233] In the parameter adjustment of the press and flattening equipment, the PID controller dynamically adjusts the control quantity according to the deviation between the set value (the optimized parameter output by the intelligent evaluation engine) and the actual value (the parameter feedback by the equipment sensor). The proportional (P), integral (I), and derivative (D) links work together. The proportional link quickly responds to the deviation, the integral link eliminates the steady-state error, and the derivative link predicts the change trend of the deviation to ensure the stability and accuracy of pressure and temperature control;

[0234] (II) Hierarchical pressure optimization algorithm:

[0235] Based on the principle of finite element analysis (FEA), by performing mechanical modeling on the warping deformation of different regions of the PCB board, determine the stress concentration in the high-curvature region; the hierarchical pressure control algorithm applies differential pressure to specific regions according to the calculation results of the model, while avoiding excessive overall pressure from damaging the substrate, and efficiently correcting local warping problems;

[0236] (III) Stress relaxation verification mechanism:

[0237] Based on the stress relaxation theory in material mechanics, by monitoring the stress decay of the PCB board during the pressure holding process, judge whether the residual stress inside the material reaches an acceptable level; when the stress relaxation rate does not meet the standard, trigger secondary flattening to ensure the reliability of the product during long-term use;

[0238] IV. Full analysis of the working process:

[0239] (I) Instruction reception and parsing stage:

[0240] Data acquisition: The closed-loop control unit continuously listens to the output port of the intelligent evaluation engine and receives the grading evaluation results and optimized parameter data of the PCB board;

[0241] Instruction parsing: Decode and analyze the received data, extract key information (such as product quality grade, press and flatten parameters, re-inspection requirements, etc.), and divide the instruction priorities;

[0242] (II) Actuator action stage:

[0243] Sorting of unqualified products: If it is an "unqualified" product, immediately send a trigger signal to the sorting mechanism to control it to complete the product rejection action within the specified time, and record the sorting result;

[0244] Re-inspection of warning products: For "warning" products, start the re-inspection process, control the conveyor belt to transport the products to the re-inspection station, link the detection equipment to complete the secondary detection, and wait for the feedback of the detection results;

[0245] Execution of the press and flatten process: According to the optimized parameters, adjust the pressure, temperature and time parameters of the press and flatten equipment through the PID control algorithm to flatten the PCB board, and monitor the equipment operation status and parameter output in real time;

[0246] (III) Process optimization and feedback stage:

[0247] Effect evaluation: After flattening, collect data such as the warpage and stress distribution of the PCB board, calculate the stress relaxation rate, and evaluate the process execution effect;

[0248] Parameter correction: If the detection result does not meet the expectation, feedback the deviation data to the process optimization algorithm module, recalculate the optimized parameters, and send them to the actuator for readjustment; if the result meets the standard, record the current optimized parameters in the equipment process library for the production of subsequent similar products;

[0249] (IV) Status recording and log update stage:

[0250] Process data recording: Record the processing process data of each PCB board (such as control instruction content, actuator action time, actual process parameter values, detection results, etc.), and store it in the local database;

[0251] Log generation: Regularly generate the operation logs of the closed-loop control unit, including information such as equipment working status, instruction execution success rate, and process optimization times, providing a basis for system maintenance and performance analysis;

[0252] V. Application value and advantage manifestation:

[0253] (I) Improvement of product quality:

[0254] Through precise process optimization and strict quality control, the closed-loop control unit can reduce the defective rate of PCB board warpage by 30%-50%, effectively reducing product scrapping and rework caused by warpage problems, and improving the overall qualified rate and reliability of products;

[0255] (2) Improving production efficiency:

[0256] The automated execution and feedback mechanism reduces the manual intervention links and shortens the time for handling unqualified products and process adjustment. At the same time, by optimizing the process parameters of pressing, warping and leveling, the leveling time of a single PCB board can be shortened by 10%-20%, improving the overall production rhythm of the production line;

[0257] (3) Reducing production costs:

[0258] Prevent unqualified products from flowing into subsequent processes, reducing waste of raw materials and repeated processing costs; through process optimization and automatic adjustment of equipment parameters, extend the service life of equipment and reduce maintenance costs; in addition, precise quality control can also reduce after-sales costs caused by product quality problems;

[0259] (4) Enhancing production flexibility:

[0260] It can quickly respond to the production requirements of PCB boards with different materials and specifications. Through the dynamic optimization parameters provided by the intelligent evaluation engine, automatically adjust the control strategy, and achieve flexible switching between multiple products on the production line to meet diverse production tasks.

[0261] In this embodiment, the fatigue warning rule of the predictive maintenance module is:

[0262] Calculate the stress attenuation based on the power-law relationship between the cumulative operation cycle number of the equipment and the designed fatigue life cycle;

[0263] Superimpose the linear compensation term of the temperature fluctuation amplitude and the pressure fluctuation amplitude;

[0264] When the measured stress exceeds the attenuated threshold, trigger a warning;

[0265] Furthermore, as the "health steward" of the "Intelligent Evaluation System for PCB Board Warpage Degree Integrating Visual Positioning and Multimodal Sensing", the predictive maintenance module can predict potential failures in advance by real-time monitoring the fatigue state of the equipment mechanical structure, avoiding production interruption and detection errors caused by equipment failure. The following will elaborate in detail from aspects such as functional architecture, core modules, algorithm principles, working processes and application values:

[0266] I. Overall functional architecture:

[0267] The predictive maintenance module collects the operating parameters of the key components of the equipment in real time through integrated sensors, constructs an evaluation model by combining material mechanics and fatigue damage theory, and realizes dynamic monitoring of the fatigue state of the equipment's mechanical structure, fault risk prediction, and maintenance decision support. Its core functions include data acquisition, status evaluation, warning trigger, and maintenance recommendation generation, forming a closed-loop management of "monitoring - analysis - warning - maintenance" to ensure the long-term stable operation of the system and reduce the risk of unplanned downtime.

[0268] II. Detailed Explanation of Core Modules and Functions:

[0269] (1) Operating Parameter Acquisition Module:

[0270] Multi-source data acquisition: Deploy devices such as vibration sensors, strain gauges, and temperature sensors to collect parameters such as vibration acceleration, stress and strain, and temperature of the equipment's mechanical structure (such as the conveyor drive shaft, hydraulic rod of the pressing and leveling equipment, camera bracket, etc.) in real time. Among them, the vibration sensor uses a three-axis accelerometer to capture the vibration signal during equipment operation at a sampling frequency of 1000 Hz; the strain gauge is attached to the surface of the key stressed components to monitor the stress change.

[0271] Data preprocessing: Filter the collected original signals, and use the wavelet transform algorithm to remove noise interference; unify the data to the standard range through normalization operations, and at the same time mark and repair abnormal data to ensure the accuracy and reliability of the input data.

[0272] (2) Fatigue State Evaluation Module:

[0273] Construction of fatigue life model: Based on Miner's linear cumulative damage theory and combined with the material characteristic parameters of the equipment, establish a fatigue life evaluation model; the fatigue warning rule formula is: (where is the fatigue threshold stress, is the material yield strength, is the cumulative number of operating cycles, is the designed fatigue life, is the strain hardening index, is the amplitude of temperature cycle fluctuation, is the amplitude of pressure fluctuation); this model comprehensively considers the influence of factors such as operating cycles, temperature fluctuations, and pressure changes on the fatigue of the equipment.

[0274] Real-time status analysis: Substitute the collected operating parameters into the fatigue life model to calculate the fatigue damage degree and remaining life of the current mechanical structure; by comparing the measured stress with the fatigue threshold stress , judge whether there is a fatigue risk for the equipment; when , trigger the warning mechanism.

[0275] (3) Early warning and maintenance decision-making module:

[0276] Multi-level early warning mechanism: Set different levels of early warning signals according to the fatigue risk level; The first-level early warning (slight risk) is prompted by a pop-up window on the system interface, and it is recommended to strengthen monitoring; The second-level early warning (moderate risk) activates the sound and light alarm and sends a text message notification to the maintenance personnel, requiring an inspection to be arranged as soon as possible; The third-level early warning (severe risk) immediately suspends the operation of the equipment to prevent the expansion of the fault;

[0277] Generation of maintenance strategies: When the early warning is triggered, combined with the historical maintenance data and the current operating status of the equipment, generate targeted maintenance suggestions; For example, for the hydraulic rod of the press and leveling equipment that is fatigued due to pressure fluctuations, it is recommended to adjust the pressure control parameters and replace the worn seals; For the conveyor drive shaft with abnormal vibration, dynamic balance calibration and bearing lubrication are recommended; At the same time, provide a guide to maintenance operation steps and a list of required spare parts to assist the maintenance personnel to complete the repair work efficiently;

[0278] (4) Data management and analysis module:

[0279] Storage of historical data: Store data such as equipment operation parameters, fatigue assessment results, early warning records, and maintenance information in the local database to establish a full life cycle file of the equipment; The data is stored in a time series database, supporting efficient time series data query and analysis;

[0280] Trend analysis and optimization: Use machine learning algorithms (such as LSTM long short-term memory network) to analyze historical data, mine the potential relationship between equipment operation parameters and fatigue damage, and predict the trend of equipment performance degradation; According to the analysis results, dynamically optimize the parameters of the fatigue life model and the early warning threshold to improve the prediction accuracy; At the same time, provide data support for equipment design improvement and process optimization;

[0281] III. Analysis of key technical principles:

[0282] (1) Miner's linear cumulative damage theory:

[0283] This theory assumes that the fatigue damage of materials at different stress levels can be linearly superimposed, that is, when the equipment is under the action of multiple stress cycles, its total damage degree is the sum of the damage degrees of each stress cycle ( ; When the total damage degree reaches 1, the material undergoes fatigue failure; Based on this theory, the predictive maintenance module quantifies the fatigue damage process in combination with the actual operating stress of the equipment;

[0284] (2) Machine learning prediction algorithm:

[0285] The LSTM network can effectively handle long-term dependencies in time series data through memory units and gating mechanisms. In equipment fatigue prediction, the LSTM model learns the mapping relationship between historical operating parameters and fatigue states, predicts the fatigue trend of the equipment in the future for a period of time, and discovers potential fault hazards in advance.

[0286] (III) Multi-parameter fusion analysis:

[0287] Fuse and analyze multi-source parameters such as vibration, stress, and temperature to reflect the operating state of the equipment from different dimensions. For example, abnormal vibration signals may indicate loose or unbalanced components, sudden stress changes may indicate cracks in the structure, and rising temperatures may be accompanied by poor lubrication or overload. Comprehensive evaluation of multi-parameters can improve the accuracy and reliability of fault diagnosis.

[0288] IV. Full analysis of the work process:

[0289] (I) Initialization and data collection stage:

[0290] System startup configuration: After the predictive maintenance module is started, complete the self-check of sensors to ensure normal equipment connection, sampling frequency, and accuracy; load configuration information such as equipment material parameters and initial parameters of the fatigue life model.

[0291] Real-time data collection: Vibration sensors, strain gauges, temperature sensors, etc. collect equipment operating parameters at the set frequency, and the data is transmitted to the fatigue state evaluation module after preprocessing.

[0292] (II) State evaluation and warning stage:

[0293] Fatigue calculation and analysis: The fatigue state evaluation module substitutes the collected data into the fatigue life model, calculates the current fatigue damage degree, remaining life, and fatigue threshold stress of the equipment, and compares the measured stress with the threshold stress to judge the risk level.

[0294] Warning trigger processing: If the measured stress exceeds the fatigue threshold, trigger corresponding warning signals according to the risk level, and push the warning information to the system interface and the maintenance personnel's terminal; if there is no risk, continue real-time monitoring.

[0295] (III) Maintenance decision-making and execution stage:

[0296] Maintenance strategy generation: After the warning is triggered, the warning and maintenance decision-making module generates maintenance suggestions according to the equipment status, including maintenance content, operation steps, required spare parts, and estimated maintenance time, etc.

[0297] Maintenance task execution: Maintenance personnel receive maintenance instructions and perform maintenance operations according to the suggestions, replace worn parts, adjust parameters, or calibrate the equipment; after completion, feedback the maintenance results to the predictive maintenance module to update the equipment status information.

[0298] 4. Data Management and Optimization Phase:

[0299] Data Recording and Archiving: Store device operation data, warning records, maintenance information, etc. in the database to form a complete device health file;

[0300] Model Optimization and Iteration: Regularly analyze historical data, and use machine learning algorithms to optimize the parameters of the fatigue life model and warning thresholds, continuously improving the accuracy and effectiveness of predictive maintenance;

[0301] V. Application Value and Advantage Demonstration:

[0302] (1) Reduction of Unplanned Downtime:

[0303] By predicting equipment failures in advance, convert passive maintenance to proactive maintenance, and avoid production line interruptions caused by sudden equipment failures; According to statistics, after adopting predictive maintenance, the unplanned downtime can be reduced by more than 50%, significantly improving production continuity;

[0304] (2) Reduction of Maintenance Costs:

[0305] Accurate maintenance decisions avoid over-maintenance and unnecessary spare part replacements, extending the service life of equipment; At the same time, reduce production losses caused by downtime and emergency procurement costs for maintenance, and the comprehensive maintenance cost can be reduced by 30% - 40%;

[0306] (3) Guarantee of Detection Accuracy:

[0307] Timely detect and repair fatigue problems in the mechanical structure of the equipment, prevent detection errors in the warpage of PCB boards caused by equipment performance degradation, ensure the long-term high-precision operation of the evaluation system, and improve product quality consistency;

[0308] (4) Optimization of Production Management:

[0309] Through digital management of the equipment health status, provide a decision-making basis for production scheduling and resource allocation; For example, reasonably arrange production tasks according to the remaining life of the equipment, avoid producing key products during the high-risk period of the equipment, and improve the scientificity and flexibility of the production plan.

[0310] Although the embodiments of the present invention have been shown and described, for those of ordinary skill in the art, it can be understood that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. An intelligent evaluation system for PCB board warpage degree that integrates visual positioning and multi-modal sensing, characterized in that: include: The multi-modal sensing unit integrates a laser displacement sensor, an infrared thermal imager, and a strain sensor to respectively collect the three-dimensional deformation data, temperature distribution data, and stress distribution data of the printed circuit board in real time; The visual positioning unit, which includes a high-resolution industrial camera and a ring light source, performs sub-pixel positioning of printed circuit boards through a feature point matching algorithm to compensate for measurement errors caused by conveyor belt vibration; The intelligent evaluation engine, deployed on edge computing devices, performs the following operations: Based on the spatiotemporal synchronization protocol, the multimodal sensing data and visual positioning data are integrated to generate a dynamic model of board warpage; An improved multimodal convolutional neural network is used to analyze the temperature-stress coupling field and predict the thermal deformation trend; Dynamically adjust the board warpage qualification threshold according to the IPC-6012 standard and output the grading evaluation results; Closed-loop control unit, receives the evaluation results and links the production line actuators: Triggering the sorting mechanism to isolate unqualified printed circuit boards; Initiate re-inspection process for warning printed circuit boards; Generate warping and leveling parameters based on the dynamic model of plate warpage, and optimize the process library of warping and leveling equipment; Predictive maintenance module monitors the fatigue status of the equipment's mechanical structure in real time.

2. The intelligent evaluation system for PCB board warpage degree integrating vision positioning and multi-modal sensing according to claim 1, characterized in that: The visual positioning unit realizes posture correction through the following steps: Extract the scale-invariant feature transformation descriptor of the current frame and the previous frame image; Calculate the minimum sum of squares of feature point coordinate differences and introduce a regularization constraint on the displacement gradient tensor; Among them, the vibration suppression coefficient ranges from 0.05 to 0.2, and the output posture correction accuracy is ≤0.02 mm.

3. The intelligent evaluation system for PCB board warpage degree integrating visual positioning and multi-modal sensing according to claim 1, characterized in that: The loss function of the improved multimodal convolutional neural network consists of three parts: The mean square error term between the measured and predicted values of laser deformation; KL divergence term of infrared thermogram and strain temperature distribution; Frobenius norm terms of the stress covariance measured matrix and the predicted matrix; The sum of the three weight coefficients is 1, and the weight of the laser deformation term is greater than the weight of the thermal term, and the weight of the thermal term is greater than the weight of the stress term.

4. The intelligent evaluation system for PCB board warpage degree integrating visual positioning and multi-modal sensing according to claim 1, wherein: The dynamic adjustment rule of the board warpage qualified threshold is: Based on the basic threshold of IPC-6012 standard; Linear scaling is performed based on the deviation ratio of Young's modulus and compressive strength of printed circuit board materials relative to FR4 substrate; Exponential decay compensation for temperature deviation is introduced, and the temperature correction coefficient is fixed at 50 degrees Celsius; Among them, the scaling factor of the ceramic substrate is 0.8, and that of the metal substrate is 1.

2.

5. The intelligent evaluation system for PCB board warpage degree integrating vision positioning and multi-modal sensing according to claim 1, wherein: The process optimization algorithm of the closed-loop control unit performs the following operations: In the PCB space domain, the integral of the sum of squares of the second-order partial derivatives of the warpage function is minimized; Generate press-bending and leveling parameters: target pressure value , leveling temperature , pressure holding duration ; Constrain the pressure and temperature of the warping and leveling equipment to be within the process allowable range.

6. The intelligent evaluation system for PCB board warpage degree integrating vision positioning and multi-modal sensing according to claim 1, wherein: The fatigue warning rules of the predictive maintenance module are: Calculate stress decay based on the power law relationship between the cumulative number of equipment operating cycles and the designed fatigue life cycle; Superimpose linear compensation items of temperature fluctuation amplitude and pressure fluctuation amplitude; When the measured stress exceeds the post-attenuation threshold, an early warning is triggered.

7. The intelligent evaluation system for PCB board warpage degree integrating vision positioning and multi-modal sensing according to claim 1, characterized in that: The scanning density control logic of the laser displacement sensor is: The reference scanning density is taken as the initial value; The decay is carried out according to the negative exponential function of the ratio of the conveyor belt speed to the maximum design speed; Maximum design speed Adjusted to 5 - 10 m / min according to the rhythm of the press and flattening equipment; Overlay the minimum safe scanning density protection value to prevent the density from being too low.

8. The intelligent evaluation system for PCB board warpage degree integrating visual positioning and multi-modal sensing according to claim 3, characterized in that: The adaptive adjustment rules for the three weight coefficients are as follows: The weight of the laser deformation term is equal to the ratio of its signal-to-noise ratio to the total signal-to-noise ratio of the multi-modal; The weight of the thermal term is equal to the ratio of the infrared signal-to-noise ratio to the total signal-to-noise ratio; The weight of the stress term is determined by the remaining ratio of the total signal-to-noise ratio.

9. The intelligent evaluation system for PCB board warpage degree integrating visual positioning and multi-modal sensing according to claim 1, wherein: The spatio-temporal synchronization protocol satisfies: The timestamp of visual data acquisition plus the optical path transmission delay; Is equal to the timestamp of multi-modal sensor data acquisition plus the hardware delay calibration amount and the action delay amount of the press and flatten equipment; The speed of light is fixed at 3×10 8 m / s.

10. The intelligent evaluation system for PCB board warpage degree integrating vision positioning and multi-modal sensing according to claim 5, characterized in that: The method for optimizing the press and flatten parameters includes: Adopting a hierarchical pressure control strategy: applying 1.5 times the reference pressure to the area where the warpage gradient of the PCB board is ≥ 0.1 mm / cm²; Real-time monitoring of the stress relaxation rate after flattening. If the relaxation rate is < 90% within 10 seconds, trigger secondary flattening.

Images