Chip packaging welding alignment detection method and system based on image recognition

Abstract

The invention discloses a chip packaging welding alignment detection method and system based on image recognition, and relates to the technical field of image recognition, in the welding process of a chip packaging part, a plurality of welding sub-parts are determined according to image recognition of a real-time image of the chip packaging part, and a plurality of welding sub-parts are determined according to the image recognition of the real-time image of the chip packaging part; the welding alignment map is determined according to the welding alignment requirements of the multiple welding sub-parts and the chip package, image recognition of the real-time image of the chip package is achieved, the multiple welding sub-parts are determined in the image recognition process to construct the welding alignment map, and the accuracy of the real-time alignment level of the chip package is improved. Determining a closed-loop alignment detection system according to the plurality of welding alignment factors, the material attribute of the chip package and the welding temperature; the alignment adjustment amount of the chip package is marked in sequence along the welding process, and a plurality of corresponding alignment adjustment orientations are matched according to the alignment adjustment amount, so that the accuracy of the alignment measure of the chip package is improved.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, and in particular to a chip packaging welding alignment detection method and system based on image recognition. Background Technology

[0002] As semiconductor packaging technology advances towards higher density and miniaturization, advanced packaging methods such as flip chips have become mainstream. During the soldering process of chip packages, the soldering alignment accuracy directly determines the electrical connection performance and long-term reliability of the chip.

[0003] Existing visual inspection technologies mostly involve static or single-frame image analysis. They typically treat the chip package as a whole for image acquisition and alignment judgment, ignoring the differences in physical state of different areas on the chip package surface (such as pads, edge spacing lines, and non-welded areas) during the welding process. This affects the accuracy of the real-time alignment level of the chip package, resulting in low accuracy of the alignment measures for the chip package. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a chip packaging welding alignment detection method and system based on image recognition.

[0005] This invention provides a chip packaging solder alignment detection method based on image recognition, comprising: During the soldering process of the chip package, real-time images of the chip package are acquired. Multiple sub-soldering parts are determined based on image recognition of the real-time images of the chip package, and a soldering alignment diagram is determined along the soldering alignment requirements of the multiple sub-soldering parts and the chip package. In the solder alignment diagram, the soldered positions are marked, the primary alignment event is determined based on the soldered positions and the unsoldered positions of the chip package, and the real-time alignment level of the chip package is marked. The primary alignment event changes with the dynamic welding of the chip package, and the corresponding alignment change content is marked in each change. The alignment change content is iterated in multiple levels to output multiple welding alignment factors. Based on the multiple welding alignment factors, the material properties of the chip package and the welding temperature, a closed-loop alignment detection system is determined. In this closed-loop alignment detection system, the welding process of the chip package is marked, the alignment adjustment amount of the chip package is marked sequentially along the welding process, and multiple corresponding alignment adjustment positions are matched according to the alignment adjustment amount. The alignment measures of the chip package are determined based on the multiple alignment adjustment positions and the current posture of the chip package.

[0006] This invention provides a chip packaging soldering alignment detection system based on image recognition, which is applied to the aforementioned chip packaging soldering alignment detection method based on image recognition; the image recognition-based chip packaging soldering alignment detection system includes: The image recognition module is used to acquire real-time images of the chip package during the soldering process, determine multiple sub-soldering parts based on the image recognition of the real-time images of the chip package, and determine a soldering alignment diagram along the soldering alignment requirements of the multiple sub-soldering parts and the chip package. The primary alignment module is used to mark the welded positions in the welding alignment diagram, determine the primary alignment event based on the welded positions and the unwelded positions of the chip package, and mark the real-time alignment level of the chip package. The closed-loop alignment detection system module is used to change the primary alignment event as the chip package is dynamically welded, and to mark the corresponding alignment change content in each change. The alignment change content is iterated in multiple levels to output multiple welding alignment factors. The closed-loop alignment detection system is determined based on multiple welding alignment factors, the material properties of the chip package and the welding temperature. The alignment measure module is used in the closed-loop alignment detection system to mark the welding process of the chip package, mark the alignment adjustment amount of the chip package sequentially along the welding process, match multiple corresponding alignment adjustment positions according to the alignment adjustment amount, and determine the alignment measure of the chip package based on the multiple alignment adjustment positions and the current posture of the chip package.

[0007] Compared with the prior art, the beneficial effects of the present invention are: (1) During the soldering process of the chip package, real-time images of the chip package are acquired. Multiple sub-soldering parts are determined based on the image recognition of the real-time images of the chip package. A soldering alignment map is determined along the soldering alignment requirements of the multiple sub-soldering parts and the chip package. In the soldering alignment map, the soldered positions are marked. The primary alignment event is determined based on the soldered positions and the unsoldered positions of the chip package. The real-time alignment level of the chip package is marked. This realizes the image recognition of the real-time images of the chip package and determines multiple sub-soldering parts during the image recognition process to construct the soldering alignment map, thereby improving the accuracy of the real-time alignment level of the chip package.

[0008] (2) The primary alignment event changes with the dynamic welding of the chip package and marks the corresponding alignment change content in each change. The alignment change content is iterated in multiple levels to output multiple welding alignment factors. A closed-loop alignment detection system is determined based on multiple welding alignment factors, chip package material properties and welding temperature. In this closed-loop alignment detection system, the welding process of the chip package is marked, the alignment adjustment amount of the chip package is marked along the welding process, and multiple alignment adjustment positions are matched according to the alignment adjustment amount. The alignment measures of the chip package are determined based on the multiple alignment adjustment positions and the current posture of the chip package. The system fully considers multiple welding alignment factors, chip package material properties and welding temperature, improves the accuracy of the closed-loop alignment detection system, and controls the multiple alignment adjustment positions and the current posture of the chip package, improving the accuracy of the chip package alignment measures. Attached Figure Description

[0009] Figure 1 This is a flowchart illustrating the chip packaging welding alignment detection method based on image recognition in an embodiment of the present invention. Figure 2 This is a flowchart illustrating step S11 in the chip packaging welding alignment detection method based on image recognition in an embodiment of the present invention. Figure 3 This is a flowchart illustrating step S12 in the chip packaging welding alignment detection method based on image recognition in an embodiment of the present invention. Figure 4 This is a flowchart illustrating step S13 in the chip packaging welding alignment detection method based on image recognition in an embodiment of the present invention. Figure 5 This is a flowchart illustrating step S14 of the chip packaging welding alignment detection method based on image recognition in an embodiment of the present invention. Figure 6 This is a schematic diagram of the structural composition of a chip packaging welding alignment detection system based on image recognition in an embodiment of the present invention. Detailed Implementation

[0010] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0011] Please see Figures 1 to 6 A chip packaging solder alignment detection method based on image recognition is applied to image recognition scenarios. The image recognition-based chip packaging solder alignment detection method includes: Step S11: During the soldering process of the chip package, real-time images of the chip package are acquired, multiple sub-soldering parts are determined based on image recognition of the real-time images of the chip package, and a soldering alignment diagram is determined along the soldering alignment requirements of the multiple sub-soldering parts and the chip package. Step S12: In the soldering alignment diagram, mark the soldered positions of the soldering alignment diagram, determine the primary alignment event based on the soldered positions and the unsoldered positions of the chip package, and mark the real-time alignment level of the chip package. Step S13: The primary alignment event changes with the dynamic welding of the chip package, and the corresponding alignment change content is marked in each change. The alignment change content is iterated in multiple levels to output multiple welding alignment factors. Based on the multiple welding alignment factors, the material properties of the chip package and the welding temperature, a closed-loop alignment detection system is determined. Step S14: In this closed-loop alignment detection system, the welding process of the chip package is marked, the alignment adjustment amount of the chip package is marked sequentially along the welding process, and multiple corresponding alignment adjustment positions are matched according to the alignment adjustment amount. The alignment measures of the chip package are determined according to the multiple alignment adjustment positions and the current posture of the chip package.

[0012] refer to Figure 2 In step S11, the specific steps are as follows: S111: Real-time monitoring of chip package soldering. An external camera captures images of the chip package and obtains real-time images of it. Simultaneously, an infrared thermal imaging unit is triggered to detect the chip package, integrating temperature field distribution information into the real-time image. Image recognition is performed on the real-time image, and multiple sub-soldering parts are identified during the recognition process. These multiple sub-soldering parts are located at different positions on the chip package, and adjacent sub-soldering parts have edge spacing lines. S112: Collect the model number of the chip package, determine the corresponding welding alignment requirements based on the traceability of the chip package model, perform semantic segmentation on the welding alignment requirements, determine multiple sub-alignment requirements during the segmentation process, construct a framework alignment map based on each alignment requirement and the corresponding sub-welding part, and determine the welding alignment map based on the multi-factor fusion of the framework alignment map and the previous alignment deviation information of the chip package.

[0013] In the embodiments of this application, the soldering of the chip package is monitored in real time. An external camera takes pictures of the chip package and obtains real-time images of the chip package. At the same time, an infrared thermal imaging unit is triggered to detect the chip package so that temperature field distribution information is integrated into the real-time image. Image recognition is performed on the real-time image, and multiple sub-soldering parts are identified during the recognition process. The multiple sub-soldering parts are located at different positions on the chip package, and two adjacent sub-soldering parts have edge spacing lines.

[0014] At this time, the welding of the chip package is monitored in real time. An external camera (such as a linear array or high frame rate area array camera) is responsible for capturing the geometric features and texture details of the chip surface and obtaining real-time images in the two-dimensional spatial domain. At the same time, the system triggers the infrared thermal imaging unit to capture the infrared radiation energy during the welding process and convert it into a temperature field distribution matrix. Meanwhile, since the optical path and sensor resolution of visible light and infrared thermal imaging are different, the low-resolution temperature field data of infrared thermal imaging needs to be mapped and superimposed onto the high-resolution visible light image coordinate system through a pre-calibrated mapping matrix to form a four-dimensional data tensor (x,y,rgb,t) in which each pixel contains both grayscale / color information and temperature features.

[0015] Image recognition is performed on the integrated multidimensional image data, using a hybrid segmentation method that combines edge detection and region growing. The system utilizes temperature field gradient information to assist in geometric feature extraction: the welding area usually exhibits high thermal radiation characteristics (high temperature zone), while the substrate or non-welding area exhibits low thermal radiation characteristics. It also identifies high-brightness welding pools or solder wetting areas, and expands outward based on this to determine the boundary of the "sub-welding part". This process utilizes the physical properties of heat conduction to effectively distinguish areas with similar optical properties but different thermal properties.

[0016] Between the segmented sub-welding sections, the system focuses on the topology of the "edge spacing lines," extracting the metal lines or insulation gaps connecting adjacent sub-welding sections. The width, direction, and relative position of these spacing lines to the edges of the sub-welding sections are crucial. These lines are not only physical connection channels but also heat conduction paths, and their geometric accuracy directly affects the stress distribution of the welding alignment. The system calculates the geometric center lines of these spacing lines and uses them as a reference line for the relative positional deviation of adjacent sub-welding sections to construct a local geometric constraint model.

[0017] Specifically, the chip package is a high-density flip chip with four symmetrically distributed rectangular solder pads (i.e., target sub-soldering parts) on its packaging substrate. Adjacent solder pads are connected by fine serpentine lines. The plating on the surface of the solder pads is prone to specular reflection at high temperatures, which interferes with conventional visual inspection.

[0018] When the chip package enters the soldering station, the system activates a high-speed area array camera to capture images of the chip surface with microsecond-level exposure time, freezing the moment of soldering motion; the synchronously triggered infrared thermal imaging unit detects the thermal radiation of the soldering laser or reflow soldering heating area; the system uses pre-stored calibration parameters to interpolate the 320×240 resolution temperature matrix obtained by the infrared sensor and map it to the 2048×2048 coordinate system of the visible light image. At this time, in the real-time image generated by the system, the pixels in the pad area not only record its surface texture (such as the solder wetting state), but also superimpose the real-time temperature value (for example, the central area reaches 250℃ and the edge area reaches 180℃).

[0019] Based on temperature field information, connected regions in the image with temperatures higher than the solder melting point (e.g., 217℃) were initially identified as valid solder pools, and four independent high-temperature regions were successfully segmented and labeled as sub-welding parts P1, P2, P3, and P4, respectively. During this process, due to the strong reflection on the surface of the chip package pads, the purely visual method misjudged the reflective spots as solder overflow. However, combined with the temperature field data, the temperature of the reflective spot area was displayed as room temperature (25℃), thus it was accurately removed, ensuring the accuracy of the boundary identification of sub-welding parts P1 to P4.

[0020] The system further analyzes the two adjacent sub-welded sections, P1 and P2. In the non-high-temperature region between them, the system identifies a 50μm wide serpentine metal line—the "edge spacing line." The system extracts the skeleton centerline of this line and measures its perpendicular distance from the edges of P1 and P2. The system detects that the spacing line has an offset of 5μm relative to the right edge of P1, indicating that P1 underwent a slight positional drift during the previous welding process. Based on this, the system locks the relative geometric constraint relationship between P1 and P2, providing precise geometric and physical boundary basis for marking the welded positions and determining the primary alignment event in the subsequent S12 step.

[0021] Furthermore, the model number of the chip package is collected, and the corresponding welding alignment requirements are determined based on the traceability of the chip package model. The welding alignment requirements are semantically segmented, and multiple sub-alignment requirements are determined during the segmentation process. A framework alignment map is constructed based on each alignment requirement and the corresponding sub-welding part. The welding alignment map is determined based on the multi-factor fusion of the framework alignment map and the chip package's previous alignment deviation information. This comprehensive consideration of multi-factor fusion of the framework alignment map and the chip package's previous alignment deviation information ensures the accuracy of the welding alignment map.

[0022] At this point, the system collects the model code of the chip package through optical character recognition (OCR) or barcode scanning module, and uses it as the primary key for deep indexing in the process knowledge graph. The system not only extracts the basic design parameters of the model (such as nominal coordinates and dimensions of the pads), but also retrieves the matching soldering process specifications. For high-density packaging, the system needs to automatically load specific tolerance level standards (such as IPC-610 Class III standard) and convert unstructured process documents into structured constraint parameter vectors to provide a data benchmark for subsequent semantic segmentation.

[0023] The system analyzes key entities (such as "pads", "pins", and "alignment marks") and their relationships (such as "alignment", "concentricity", and "offset tolerance") in the process documentation. The system deconstructs the overall alignment requirements into "sub-alignment requirements" that correspond one-to-one with the physical sub-welding parts. Each sub-requirement includes specific indicators such as position tolerance, solder coverage threshold, and edge misalignment limit. For sub-welding parts at different locations, the sub-requirements differ (for example, the thermal expansion tolerances of the center pad and the edge pad are different), and the system needs to construct differentiated constraint logic.

[0024] The system maps the aforementioned multiple sub-alignment requirements onto the various "sub-welding parts" identified in step S111 based on coordinate transformation. At this point, the system constructs a frame alignment map in the image coordinate system. This is not a simple static image, but a data model containing geometric topology and attribute labels. The nodes in the map represent each sub-welding part, and the node attributes are loaded with their respective sub-alignment requirements (such as theoretical center coordinates and allowable deviation range). The edges represent the relative positional constraints between the solder pads. This frame map establishes the theoretical benchmark for detection.

[0025] The system calls the "past alignment deviation information" database of the chip model and uses Bayesian estimation or neural network models to analyze historical trends. The system takes into account factors such as thermal deformation trends, inherent systematic errors of the equipment, and batch differences of materials. For example, if historical data shows that the chip model has a systematic shift to the upper left at a specific temperature range, the system will pre-compensate or weight the theoretical benchmark in the "frame alignment diagram" based on this prior knowledge. The fused result is the final welding alignment diagram, which is a dynamic detection benchmark that combines theoretical standards and historical experience.

[0026] Specifically, the chip package is a high-density flip chip with four symmetrically distributed rectangular solder pads on its substrate, labeled P1 (top left), P2 (top right), P3 (bottom left), and P4 (bottom right). The system camera captures the laser-engraved "Type-A-2026" on the corner of the chip package substrate, and the OCR module recognizes this string. Using this as the primary key, the system instantly retrieves the chip package design drawings and soldering specifications from the cloud-based process database. The search results include: substrate material is FR-4, pad size is 1.5mm × 1.5mm, and overall alignment accuracy requirement is ±25μm.

[0027] The system parses the soldering specification document: "The four solder pads must be precisely aligned with the corresponding solder pads on the PCB, and the edge misalignment shall not exceed 10% of the solder pad width." Through semantic segmentation, the system deconstructs this overall requirement into four specific sub-alignment requirements: Sub-requirement R1 (corresponding to P1): X-axis offset ≤ 15μm, Y-axis offset ≤ 15μm (considering that the heat dissipation is faster in the upper left corner, the thermal shrinkage compensation coefficient is set to 1.05); Sub-requirements R2 to R4 are generated in the same way, but their compensation coefficients are slightly different according to the thermodynamic characteristics of their respective positions; The system successfully transforms unstructured text into specific numerical constraint vectors.

[0028] The system uses the four sub-welding parts (P1, P2, P3, P4) identified in step S111 as four nodes and establishes a topological relationship in the image coordinate system. The sub-requirements R1 to R4 deconstructed above are loaded into the attribute list of the corresponding nodes. The generated "frame alignment map" is a standardized geometric model that marks the theoretical center coordinates of the four pads and their allowable tolerance range.

[0029] The system retrieved production data for the chip packages from the past six months and discovered that due to minor wear on the guide rails of the welding equipment, chips in batch B2025 exhibited a systematic rightward deviation of an average of +8μm in the X-axis direction. Based on this historical deviation information, the system implemented a multi-factor fusion approach, dynamically correcting the theoretical reference coordinates in the "frame alignment map": pre-compensating the X-axis detection reference of all pads by 8μm to the left; simultaneously, considering the difference in dielectric constant of the substrate material in the current batch, the system fine-tuned the alignment weights of each sub-region; the final output "welding alignment map" not only included the geometric positions of the four pads but also embedded the corrected detection reference lines and dynamic tolerance bands. This enabled the system to automatically filter out known systematic errors and accurately pinpoint accidental welding offsets during subsequent inspections, significantly improving the signal-to-noise ratio and accuracy of the inspection.

[0030] refer to Figure 3 In step S12, the specific steps are as follows: S121: The soldering process of the chip package is regarded as a continuous time stream. The real-time image is compared with the soldering alignment map along the time stream, and the corresponding soldered position and the corresponding position deviation information are extracted. The corresponding unsoldered position is marked. Based on the comparison of the soldered position, the corresponding position deviation information and the unsoldered position, multiple sub-alignment error information is determined, and the corresponding primary alignment event is determined in combination with the current posture of the chip package. S122: In the primary alignment event, multiple sub-alignment items are determined based on the identification of the primary alignment event, and the alignment accuracy of each sub-alignment item is marked. The real-time alignment level of the chip package is determined according to the item content of each sub-alignment item, the corresponding alignment accuracy, and the welding constraints of the chip package.

[0031] In the embodiments of this application, the soldering process of the chip package is regarded as a continuous time stream. The real-time image is compared with the soldering alignment map along the time stream, and the corresponding soldered positions and corresponding position deviation information are extracted. The corresponding unsoldered positions are marked. Based on the comparison of the soldered positions, the corresponding position deviation information and the unsoldered positions, multiple sub-alignment error information is determined. The corresponding primary alignment event is determined in combination with the current posture of the chip package. This approach is compatible with the overall consideration of comparing the soldered positions, the corresponding position deviation information and the unsoldered positions, ensuring the accuracy of multiple sub-alignment error information.

[0032] At this point, the system establishes a time axis coordinate system based on the moving trajectory of the welding heat source (such as a laser or welding head); the system uses the "welding alignment diagram" generated by S112 as an ideal template and performs frame-by-frame differential calculations along the time flow and the real-time acquired image sequence; using image registration methods (such as elastic registration based on mutual information), the system eliminates global rigid displacement caused by conveyor belt vibration or thermal expansion, ensuring the consistency of the reference for local welding area comparison. This process realizes the accurate mapping of the physical welding process to the digital time flow.

[0033] Based on grayscale abrupt changes or temperature field thresholds (inherited from S111 thermal imaging data), the system identifies the "welded positions" where welding operations have been completed. The system calculates the Euclidean distance and rotation angle between the actual geometric center of the welded position and the corresponding theoretical center in the "welding alignment diagram," and outputs a position deviation vector. At the same time, the system analyzes the edge gradient of the welded area to determine whether there are microscopic defects such as solder overflow or insufficient wetting, forming a multi-dimensional deviation feature descriptor.

[0034] The system subtracts the welded areas from the "welding alignment diagram," and the remaining part is the "unwelded position." The system marks the unwelded position with characteristic status, which is not only marking coordinates, but also evaluating its surface condition. For example, it detects whether there is oxidation, contamination, or heat-affected zone (such as discoloration) caused by previous welding processes on the surface of the unwelded pads. These status marks will serve as important input variables for dynamically adjusting welding parameters in the subsequent S13 step, reflecting the forward-looking nature of the detection.

[0035] The system integrates the "position deviation information" of the soldered positions, the "status markers" of the unsoldered positions, and the real-time "current orientation of the chip package" (determined by the relative positional relationship of the sub-soldering parts identified by S111). The system analyzes the error distribution pattern of multiple sub-soldering parts to determine whether it is a translation error, rotation error, or nonlinear thermal warping error. If the deviation of the soldered parts shows a certain trend (such as gradual offset), the system uses this trend to predict the potential risks of the unsoldered parts. Finally, the system defines this set of complex error states as a specific "primary alignment event", which includes the error type, error magnitude, and confidence level, triggering the subsequent closed-loop adjustment mechanism.

[0036] Specifically, at time T1, the welding head completes welding at P1 and moves above P2 to prepare for welding. The system compares the real-time image acquired at time T1 with the "welding alignment diagram". Through continuous tracking over time, the system confirms that the welding process has covered the P1 area and the current focus is on the P2 area.

[0037] The system focuses on the soldered P1 (top left pad); using image processing, it calculates that the actual soldering center of P1 is offset by +12μm in the positive X-axis and -5μm in the negative Y-axis relative to the theoretical center; combined with thermal imaging data, it is found that there is slight uneven solder spreading at the edge of P1, forming a positional deviation vector Verr1=(+12,-5) and an edge feature descriptor.

[0038] The system marks the remaining P2, P3, and P4 as unwelded positions in the welding alignment diagram. For P2 (upper right pad) which is about to be welded, the system detects that its surface cleanliness is good, but due to the heat conduction during the welding of P1, the temperature of the substrate around P2 has risen by 15°C. This temperature field information is marked as "preheating state". At the same time, the system predicts that the theoretical coordinates of P2 need to take into account the current coefficient of thermal expansion.

[0039] The system comprehensively analyzes the current posture: P1 has a significant rightward offset, and P2 is under heat. The system couples and analyzes the error information of all sub-welding parts, and determines that the offset is not a random error, but a slight rotation caused by mechanical lag of the welding head in the X-axis direction or uneven heating of the substrate. Therefore, the system generates a "primary alignment event": the event type is defined as "systematic linear drift", the magnitude is "X-axis +12μm, confidence level 95%", this event immediately triggers the closed-loop iterative logic in step S13, prompting the system to perform pre-compensation in the negative X-axis direction when welding P2, in order to offset the discovered systematic deviation and prevent the accumulation of errors from causing the subsequent pads P3 and P4 to be scrapped.

[0040] Furthermore, in this primary alignment event, multiple sub-alignment items are determined based on the identification of the primary alignment event, and the alignment accuracy of each sub-alignment item is marked. The real-time alignment level of the chip package is determined according to the item content, corresponding alignment accuracy, and welding constraints of each sub-alignment item. This approach is compatible with the overall considerations of the primary alignment event identification, ensuring the accuracy of multiple sub-alignment items. At the same time, image recognition of the chip package in real time is realized, and multiple sub-welding parts are identified during the image recognition process to construct a welding alignment map, thereby improving the accuracy of the real-time alignment level of the chip package.

[0041] At this point, the system decomposes the alignment event into specific "sub-alignment items" using topological mapping based on the type of alignment event (such as translation, rotation, thermal deformation). Each sub-alignment item not only corresponds to a physical pad area but is also associated with specific geometric feature dimensions (such as edge alignment, center coincidence, and angle parallelism). The system allocates the error information extracted in S121 to different sub-items according to the feature dimensions, constructing a multi-dimensional detection task queue to realize the logical transformation from "state description" to "task execution".

[0042] The system calculates the actual detection accuracy of each sub-item based on the output of image recognition methods (such as edge gradient operators or template matching correlation coefficients). The system not only calculates the deviation value, but also evaluates the confidence interval of the deviation value. By analyzing the signal-to-noise ratio (SNR) of the image, edge sharpness, and radiometric resolution of thermal imaging, the system marks each sub-item with "alignment accuracy". This accuracy value is not a fixed threshold, but a measurement uncertainty that dynamically fluctuates according to the current environment, reflecting the reliability of the current detection result and avoiding misjudgments caused by image quality fluctuations.

[0043] The system introduces "welding constraints" as an evaluation benchmark. Welding constraints include rigid indicators such as upper limits of design tolerances, lower limits of process windows, and electrical connectivity requirements. The system constructs a three-dimensional evaluation space consisting of "project content (deviation magnitude)," "alignment accuracy (confidence level)," and "welding constraints (tolerance threshold)." Through weighted fusion or fuzzy logic reasoning, the system calculates a comprehensive score and maps it to a preset "real-time alignment level" scale. This level is usually divided into discrete states such as "excellent (no intervention required)," "good (monitoring required)," "medium (fine-tuning required)," and "poor (shutdown alarm required)," which directly guide subsequent control strategies.

[0044] Specifically, the system has identified a rightward offset in P1 and triggered a "primary alignment event." At the current moment, the system is preparing to evaluate the status of subsequent pads. The system deconstructs the "systematic linear drift" primary alignment event into three specific sub-alignment items: Item I1 (P2 pre-alignment detection): for the upper right pad P2 that is about to be soldered, the content is to detect the offset of its current position relative to the theoretical coordinates; Item I2 (P1 edge integrity): for the soldered upper left pad P1, the content is to detect the parallelism between the solder edge and the pad boundary and the overflow situation; Item I3 (substrate thermal deformation): for the substrate area between P1 and P2, the content is to detect the relative position drift caused by thermal expansion. The system has constructed a detection task queue that includes three dimensions: position, morphology, and deformation.

[0045] The system assigns accuracy labels to the above items: For item I1, since P2 has not yet been soldered, there is no solder reflection interference on the surface, and the image edge gradient is steep, the system labels its alignment accuracy as "subpixel level (±2μm)", with high confidence; For item I2, since P1 has been soldered, there is specular reflection and some high-temperature noise on the solder surface, which increases the uncertainty of edge recognition, the system labels its alignment accuracy as "normal level (±8μm)", with medium confidence. These accuracy labels directly reflect the reliability of the current data and provide a weighting basis for subsequent grade determination.

[0046] The system makes a comprehensive judgment based on the welding constraints of the chip package (design tolerance is ±25μm): For project I1: the detection found that the pre-offset of P2 is +14μm (within the tolerance range, but close to the boundary). Combined with its high-precision marking, the system calculates its risk value as 0.6 (out of 1.0); For project I2: the detection found that the edge deviation of P1 is +12μm. Combined with its medium-precision marking, the system calculates its compliance as 0.75.

[0047] The system uses weighted logic. Given that P1 is already finalized and P2 is about to be soldered, P2 is given a higher weight. Ultimately, the system determines the current real-time alignment level of the chip package to be "Level 2 (requires dynamic fine-tuning)". This level indicates that although the current soldering status is not out of tolerance, a significant drift trend has been detected. An active compensation mechanism must be activated when soldering P2 to prevent the risk of out-of-tolerance issues in subsequent P3 and P4.

[0048] refer to Figure 4 In step S13, the specific steps are as follows: S131: As the chip package is dynamically soldered, the soldered position of the chip package continuously changes, triggering a dynamic change in the primary alignment event. The primary alignment event is compared before and after the change to output the corresponding alignment change content, which covers the position deviation trajectory of the unsoldered position. S132: Input the alignment change content into the preset iterative model and trigger multi-level iteration of the alignment change content. Output the iteration results of each level in sequence according to the multi-level iteration of the alignment change content. Determine the corresponding welding alignment factors according to the identification of the iteration results of each level to obtain multiple welding alignment factors. Combine the material properties of the chip package and the welding temperature to perform multi-factor fusion to construct the corresponding closed-loop alignment detection system. The closed-loop alignment detection system outputs the next welding node and the current alignment level of the chip package in real time.

[0049] In the embodiments of this application, as the chip package is dynamically soldered, the soldered position of the chip package continuously changes, triggering a dynamic change in the primary alignment event. The primary alignment event is compared before and after the change to output the corresponding alignment change content. This alignment change content covers the position deviation trajectory of the unsoldered position, thus introducing the alignment change content to cover the position deviation trajectory of the unsoldered position.

[0050] At this time, as the welding heat source (such as a laser welding head or reflow nozzle) moves along the preset trajectory, the system tracks the changes in physical state in real time; the system regards the "welded position" as a discrete set of points in the state space, and this set of points expands continuously as the time axis advances; the formation of each new weld point not only means a change in the physical properties of the image features (such as a jump in gray value or the appearance of a temperature field peak), but also triggers the system to re-evaluate the overall welding topology; the system needs to capture the critical state transition from "not welded" to "welding" and then to "welded" with a time resolution of milliseconds.

[0051] The system introduces an event-driven update mechanism: when the set of "welded positions" undergoes an incremental change, the update calculation of the "primary alignment event" is immediately triggered; the system locks the time ti-1 before the change and the time ti after the change, and performs a difference operation on the primary alignment events at these two times. This is not a simple numerical subtraction, but a comparative analysis of event attributes (such as deviation vector direction, error magnitude, and confidence level). Through this comparison, the system can identify whether the welding error shows a convergence trend (process under control) or a divergence trend (process out of control), thereby judging the stability of the welding process.

[0052] The system outputs "alignment change content" containing both explicit and implicit information. Explicitly, the system records the actual deviation increment at the welded positions. Implicitly, the system uses a heat conduction model and a materials mechanics model to calculate the overall stress redistribution caused by localized welding. The system maps the deviation data of the welded positions to the geometric coordinate system of the unwelded areas, constructing a "position deviation trajectory." This is a predictive trajectory based on a physical model, describing the direction and magnitude of positional drift at the unwelded positions in the future, providing crucial dynamic input for subsequent multi-level iterations.

[0053] Specifically, the chip package contains four symmetrically distributed rectangular solder pads (P1, P2, P3, P4); the soldering sequence is P1→P2→P3→P4; at the current moment, the system has just completed the soldering of P2 and is preparing to move to P3; at time T2, the soldering head completes the soldering operation on the upper right solder pad P2; real-time image acquisition shows that the solder in the P2 area has been wetted and solidified, and infrared thermal imaging shows that the temperature in this area has reached its peak and then started to decrease; the system recognizes that the set of "soldered positions" has been incrementally updated from {P1} to {P1,P2}, and this change in physical state immediately triggers the dynamic update process of the primary alignment event.

[0054] The system retrieves the primary alignment event data at time T1 (when only P1 welding is completed) and compares it with the data at the current time T2: Event at time T1: The deviation type is "single-point translation", and the overall deviation vector points to the upper right, mainly dominated by the deviation of P1; Event at time T2: The deviation type evolves into "composite rotation"; Due to the thermal stress applied to the upper right corner of the substrate during P2 welding, the substrate undergoes a slight clockwise warping. Through timing difference calculation, the system detects that the addition of P2 has changed the overall error distribution pattern, and the nature of the alignment event changes from a simple linear drift to a complex deformation with a rotation component.

[0055] Based on the above comparison, the system outputs the "alignment change content": Change in the welded area: The actual center of P2 has an X-axis deviation of +8μm and a Y-axis deviation of -3μm relative to the theoretical center; Trajectory prediction of the unwelded area: Based on the combined deviation of P1 and P2 and the thermal stress distribution, the system calculates that the substrate has a slight warping to the lower right; As a result, the system generates position deviation trajectories for the unwelded positions P3 and P4: It is predicted that P3 (lower left) will be subjected to tensile stress, resulting in an additional displacement trend of about 5μm to the lower left; It is predicted that P4 (lower right) will be most affected by compressive stress, resulting in a displacement trend of about 10μm to the upper right; This predicted trajectory is encapsulated in the "alignment change content" and transmitted to step S132 in real time, prompting the system to move the welding head to the upper right in advance for reverse compensation when welding P3.

[0056] Furthermore, the alignment change content is input into a preset iterative model, triggering multi-level iterations of the alignment change content. Based on the multi-level iterations of the alignment change content, the iteration results of each level are output sequentially. Based on the identification of the iteration results of each level, the corresponding welding alignment factors are determined to obtain multiple welding alignment factors. These factors are then combined with the material properties of the chip package and the welding temperature to perform multi-factor fusion, thereby constructing a corresponding closed-loop alignment detection system. This closed-loop alignment detection system outputs the next welding node and the current alignment level of the chip package in real time, taking into account the overall consideration of identifying the iteration results of each level, and ensuring the accuracy of the corresponding welding alignment factors.

[0057] At this point, the system will perform feature vectorization on the positional changes (including positional deviation trajectory and error evolution trend) and input it into the preset iterative model. A cascaded structure or deep neural network architecture is often used, with each iteration corresponding to feature extraction at a different granularity. The first level focuses on smoothing instantaneous deviations, the second level focuses on fitting local trends, and the third level focuses on the evolution of global stress distribution. The system automatically adjusts the iteration depth according to the confidence and completeness of the input data to ensure a balance between computational efficiency and accuracy.

[0058] The system outputs the iteration results at each level sequentially, with each level representing a specific dimension of welding alignment factors. The system identifies key factors affecting alignment from the iteration results, such as: geometric factors: simple mechanical coordinate offset; thermodynamic factors: warping caused by thermal expansion coefficient (CTE) mismatch due to temperature gradient; and dynamic factors: vibrations or hydrodynamic disturbances during the welding process. These factors are no longer single data points but parameter vectors with physical meaning. The system extracts these factors from the iterative data stream to form a "welding alignment factor set".

[0059] The system calls upon the chip package's "material properties" library (such as Young's modulus, Poisson's ratio, and coefficient of thermal expansion of the substrate and chip) and real-time "welding temperature" field data. The system constructs a multi-factor fusion matrix to couple the above welding alignment factors with physical property parameters. For example, the temperature field data is converted into physical deformation using the coefficient of thermal expansion and then weighted and fused with the geometric deviations identified by the image. This process uses a simplified inverse model of finite element analysis (FEA) to correct the limitations of pure visual inspection and generate a closed-loop alignment detection system that conforms to physical laws and fits real-time data.

[0060] The closed-loop alignment detection system calculates the optimal subsequent operation path based on the fused state space. The system outputs two key indicators: first, the recommended coordinates and orientation of the next welding node (including pre-compensation), and second, the corrected current alignment level. This level reflects the true quality status of the current welding process after considering all physical interference factors, and directly determines the aggressiveness of the adjustment measures in the subsequent S14 step.

[0061] Specifically, at the current moment, the system has completed the welding of P1 and P2 and detected the clockwise rotation trend of the substrate due to heat accumulation; the next step is to weld P3 (lower left pad); the system encapsulates the "alignment change content" output by S131, in which key data include: the X-axis deviation increment of +8μm after welding P2, and the predicted clockwise rotation trend vector of the substrate. This data stream is input into the preset three-level iterative model, and the model then starts the calculation process.

[0062] The model performs multi-level iterations and outputs results: First-level iteration output: The system performs Kalman filtering on the original deviation data to remove high-frequency jitter noise and determines the "geometric translation factor" as positive X-axis +10μm; Second-level iteration output: The system analyzes the rate of change of deviation over time and finds that the deviation has a slight rebound trend as the temperature decreases, determining the "thermoelastic recovery factor" as the estimated -2μm; Third-level iteration output: The system combines the positional relationship between P1 and P2 to calculate the overall torsion vector of the substrate and determines the "global warping factor" as a nonlinear distribution along the diagonal direction; The system finally obtains the "welding alignment factor set" containing the above three dimensions.

[0063] The system retrieves the material properties of the chip package: the coefficient of thermal expansion (CTE) of the substrate (FR-4) is 14 ppm / ℃, and that of the chip (silicon) is 2.6 ppm / ℃, indicating a significant CTE mismatch. Combined with the current welding temperature of 180℃ (average temperature) fed back by the infrared thermal imaging unit, the system performs a fusion calculation: using material properties and temperature difference, the theoretical thermal expansion difference is calculated to be approximately 5 μm. This physical deformation is then vector-superimposed with the "geometric translation factor" and the "global warpage factor." The fusion result shows that 40% of the current visual deviation is due to reversible deformation caused by thermal expansion, and this error needs to be corrected after cooling. The remaining 60% is due to mechanical offset, which needs immediate compensation. Based on this fusion logic, the system constructs the current closed-loop alignment detection system, which incorporates a temperature-deformation correction curve.

[0064] The closed-loop system calculates the current state and outputs the following instructions: Next welding node: For the P3 pad, the system suggests that the target coordinates of the welding head be offset to the left by 6μm from the theoretical position (to compensate for the combined result of thermal expansion and mechanical drift), and the Z-axis height be reduced by 2μm to accommodate substrate warping; Current alignment level: The system determines that the current state is within a controllable range, but the accuracy margin is reduced, and updates the real-time alignment level of the chip package to "Level 3 (requires precision compensation)", prompting that a high-precision multi-level adjustment strategy needs to be started in the subsequent S14 step.

[0065] refer to Figure 5 In step S14, the specific steps are as follows: S141: Monitor the closed-loop alignment detection system in real time and collect the welding process of the chip package. At the same time, based on the phase transition characteristics of the chip package, divide the welding process into multiple welding stages and construct corresponding control boundaries for each welding stage. Combine the control boundaries of the welding stages and the corresponding welding alignment factors to perform multi-step prediction and predict the alignment adjustment amount of the chip package. S142: Perform matrix analysis on the alignment adjustment amount of the chip package in multiple directions, and output sub-matrix information in multiple directions. Determine the corresponding alignment adjustment orientation by recognizing the sub-matrix information in each direction to obtain multiple alignment adjustment orientations. Combine the multiple alignment adjustment orientations with the current posture of the chip package to trigger local alignment of the chip package and output multiple local alignment results. Determine the alignment measures of the chip package based on the combination of multiple local alignments and the mapping relationship of alignment measures.

[0066] In the embodiments of this application, the closed-loop alignment detection system is monitored in real time, and the welding process of the chip package is collected. At the same time, based on the phase transition characteristics of the chip package, the welding process is divided into multiple welding stages, and corresponding control boundaries are constructed for each welding stage. Multi-step prediction is performed by combining the control boundaries of the welding stages and the corresponding welding alignment factors, and the alignment adjustment amount of the chip package is predicted. The prediction of the alignment adjustment amount of the chip package is introduced.

[0067] At this point, the system continuously monitors the closed-loop alignment detection system constructed by S132 at a high sampling frequency, and reads the system's output state vector (including the current alignment level, prediction deviation, etc.) in real time. Simultaneously, it collects "welding process" data synchronously through the encoder and visual timestamp. The system maps the physical coordinate trajectory of the welding head, the output power curve of the heat source, and the deviation curve of visual detection onto the same time axis, forming a continuous process data stream. This not only records "where the welding has reached," but also "to what extent the welding has progressed," providing a dynamic data foundation for subsequent stage division.

[0068] Based on the "phase transition characteristics"—that is, the abrupt change points of the material's physical state during the welding process (such as solder melting point, wetting angle change rate, and temperature plateau period)—the system divides the welding process into several distinct stages. The system uses differential operators to process temperature and image gradient data to identify the critical points of phase transition. Typical stage divisions include: preheating stage (temperature rise, solder solidification), wetting stage (solder liquefaction, surface tension dominance), reflow stage (interconnect formation, temperature peak), and cooling and solidification stage (liquid to solid, stress stabilization). Each stage corresponds to a different physical mechanism, thus requiring different detection and control strategies.

[0069] Based on the process knowledge base and real-time status, the system constructs dynamic "control boundaries" for each stage. These boundaries are not static geometric lines, but multi-dimensional constraint spaces. The control boundaries cover: geometric boundaries: the maximum allowable positional deviation (to prevent short circuits between solder pads); thermal boundaries: the maximum allowable instantaneous temperature and duration (to prevent thermal damage to devices); and kinetic boundaries: the maximum allowable adjustment speed (to prevent solder splashing). These boundaries adaptively narrow or widen as the soldering stage progresses. For example, the geometric boundaries are relatively loose during the wetting stage, while they become extremely tight during the cooling and solidification stage, where any large adjustment will damage the solder joint structure.

[0070] The system inputs the current "welding alignment factors" (from S132) into the prediction model (such as a Long Short-Term Memory network LSTM or a state-space model), and performs multi-step prediction deduction in combination with the current stage's "control boundary". The system simulates the evolution of the welding state within a certain number of time steps in the future and predicts whether the error will reach the control boundary if no intervention is made. Based on this prediction, the system reverse-engineers the optimal adjustment scheme to keep the trajectory within the boundary and outputs the "alignment adjustment amount". This adjustment amount is a comprehensive instruction set that includes translation amount, rotation angle and execution time, realizing the leap from "post-correction" to "pre-judgment".

[0071] Specifically, at the current moment, the system is welding P3 (lower left pad); the closed-loop system shows that there is residual stress caused by the welding of P1 and P2, and predicts a tendency to warp to the lower left; the system monitors the closed-loop detection system in real time, and collects data that the welding head is currently above the P3 pad, the moving speed is 5mm / s, the infrared temperature display shows that the current area temperature is 210℃, and the visual deviation reading is +3μm (X-axis). These data constitute the current welding process snapshot.

[0072] The system analyzes the temperature curve and image features and detects that the solder in the P3 region reaches a peak in the temperature curve at time T0 and then experiences a brief plateau. Visually, the solder changes from a solid to a liquid state in terms of gloss (phase transition characteristics). The system determines that the soldering process has officially crossed the critical point from the "preheating stage" and entered the "wetting stage". This is a key physical turning point, which means that the solder begins to flow and the relative position between the substrate and the pad is adjustable, but it is also easily affected by surface tension.

[0073] For the newly entered "wetting stage", the system dynamically constructs control boundaries: Geometric boundary: Considering the fluidity of the solder, the maximum allowable deviation of P3 is set to ±20μm (which is more lenient than ±5μm in the cooling stage), but it must be ensured that the solder does not overflow into adjacent lines; Dynamic boundary: The adjustment acceleration is limited to be less than the preset acceleration to prevent liquid solder from splashing and causing short circuits. This boundary provides a constraint space for subsequent predictions and ensures that the adjustment behavior will not cause secondary defects.

[0074] The system calls S132 to obtain welding alignment factors (including "residual thermal stress factor" and "material warpage factor"); within the control boundary of the wetting stage, the system performs multi-step prediction: Model prediction: If the current posture is maintained, P3 will shift to the lower left by about 12μm as it cools and shrinks, which will touch the geometric boundary, resulting in the edge spacing line spacing between P3 and P1 being too small; Solution adjustment amount: In order to offset this prediction deviation, the system calculates the alignment adjustment amount that needs to be executed immediately: The welding head needs to move to the upper right by 8μm within 50ms and make a slight clockwise adjustment of 0.05°. This adjustment amount is quickly output to the actuator, so that a reverse pre-offset is artificially applied before the solder solidifies, thereby achieving perfect alignment after final cooling.

[0075] Furthermore, the alignment adjustment amount of the chip package is analyzed by matrix analysis in multiple directions, and sub-matrix information in multiple directions is output. The corresponding alignment adjustment orientation is determined by identifying the sub-matrix information in each direction to obtain multiple alignment adjustment orientations. Combining multiple alignment adjustment orientations with the current posture of the chip package triggers local alignment of the chip package and outputs multiple local alignment results. The alignment measures of the chip package are determined based on the mapping relationship between multiple local alignment combinations and alignment measures. This overall consideration of multiple local alignment combinations and alignment measures ensures the accuracy of the chip package alignment measures. At the same time, multiple welding alignment factors, chip package material properties and welding temperature are fully considered, improving the accuracy of the closed-loop alignment detection system. Controlling multiple alignment adjustment orientations and the current posture of the chip package further improves the accuracy of the chip package alignment measures.

[0076] At this point, the system uses Jacobi matrix transformation or tensor decomposition to map the global adjustment amount to a specific motion control dimension. The system resolves the total adjustment amount into translational component matrices (X, Y, Z axes) and rotational component matrices (θX, θY, θZ axes) along the Cartesian coordinate system. Each matrix element corresponds to an independent drive axis or micro stage and includes parameters such as displacement, velocity curve, and acceleration limit. This process transforms the macroscopic "position correction" into a microscopic "set of drive commands".

[0077] The system performs feature recognition on the sub-matrix information in each direction, and determines the specific "alignment adjustment orientation" through threshold judgment and symbolic logic judgment. Not all matrix components need to be executed. The system filters out tiny components below mechanical precision (such as nanometer-level jitter) and retains only the orientations with substantial correction effects. For example, the main adjustment orientations are identified as "positive translation on the X-axis" and "rotation on the Z-axis". These orientations constitute the action topology of the subsequent actuators.

[0078] The system reads the "current posture" of the chip package (real-time recognition from S121) and uses it as the initial state for action execution. Subsequently, the system triggers the "local alignment" mechanism. Unlike global handling, local alignment specifically refers to precise fine-tuning of a specific pad or sub-region. The system controls the soldering head, stage, or micro-ejector to perform physical movement according to the determined alignment adjustment orientation. This process involves multi-axis linkage control to ensure that no additional Y-axis deviation or angular deviation is introduced when adjusting the X-axis deviation. The system collects real-time feedback from the grating ruler or vision and outputs the "local alignment result," which is the immediate state after adjustment.

[0079] The system collects multiple local alignment results and constructs a "local alignment combination" state set. Using a pre-set expert knowledge base or fuzzy logic rules, it queries the mapping relationship between "local alignment combination" and "alignment measures". Alignment measures not only include a single "position compensation", but also involve "welding parameter adjustment" (such as modifying laser power or welding time to adapt to small deviations), "path replanning" or "manual intervention alarm", etc. Based on the residual error distribution after local adjustment, the system finally locks the optimal system-level response measures, completing a complete closed loop from detection to execution.

[0080] Specifically, at the current moment, the system is processing the soldering of the P3 pad. Step S141 predicts that an adjustment of "moving 8μm to the upper right and making a slight clockwise adjustment" is required. The system receives the total adjustment vector Vadj = [ΔX = +8μm, ΔY = +2μm, Δθ = +0.05°] output by S141. The system starts the matrix analysis method and decomposes the vector into the control matrix of the actuator (six-axis micro-motion platform and soldering head): translation sub-matrix Mtrans: resolved as X-axis motor pulse increment +800 pulses (corresponding to 8μm), Y-axis motor pulse increment +200 pulses (corresponding to 2μm); rotation sub-matrix Mrot: resolved as R-axis (rotation axis) angle correction command +0.05°; vertical sub-matrix Mvert: considering the warping of the substrate, it is determined that the Z-axis needs to be raised by 1μm to maintain a constant focal length.

[0081] The system performs feature recognition on the above sub-matrices: it identifies the main feature of the translation sub-matrix and determines the main alignment adjustment orientation as "horizontal rightward offset"; it identifies the non-zero feature of the rotation sub-matrix and determines the secondary alignment adjustment orientation as "clockwise angle correction"; it identifies the vertical sub-matrix and determines the auxiliary alignment adjustment orientation as "vertical following"; the system thus locks in the three main execution orientations and eliminates other inactive redundant degrees of freedom.

[0082] The system reads the current orientation of the P3 pad: position (x3, y3), current tilt angle 0.02°; the system triggers the local alignment mechanism: the stage micro-motion moves according to the command, driving the chip substrate to translate 8μm to the right, while the rotation axis rotates 0.05° clockwise; during the execution, the vision system monitors the relative position of P3 with the surrounding substrates in real time; due to the use of local flexible support technology, this adjustment mainly affects P3 and its adjacent areas, without causing stress compression on the already soldered P1 and P2; after the execution is completed, the system outputs the local alignment result: the current position deviation of P3 has been reduced to within ±1μm, meeting the high precision requirements.

[0083] The system integrates multiple local results: P3 is aligned, but the rotation adjustment caused a 3μm offset in the theoretical position of P4 (the unsoldered pad in the lower right corner). Based on this "local alignment combination" state, the system queries the expert rule base: rule trigger: if the pad accuracy has been adjusted to meet the standard, but the offset of the unsoldered pad is less than 5μm, then "pre-compensation welding measures" are executed. The alignment measures finally determined by the system are: confirm the current position of P3 and immediately perform welding and solidification, while updating the welding target coordinates of P4 and correcting its theoretical position to the left by 3μm. This decision ensures that while correcting the current error, potential deviations in subsequent processes are prevented, achieving global optimal control.

[0084] Please see Figure 6 , Figure 6 This is a schematic diagram of the structural composition of a chip packaging welding alignment detection system based on image recognition according to an embodiment of the present invention; the chip packaging welding alignment detection system based on image recognition is applied to the above-mentioned chip packaging welding alignment detection method based on image recognition; the chip packaging welding alignment detection system based on image recognition includes: The image recognition module 21 is used to acquire real-time images of the chip package during the welding process, determine multiple sub-welding parts based on the image recognition of the real-time images of the chip package, and determine a welding alignment map along the welding alignment requirements of the multiple sub-welding parts and the chip package. The primary alignment module 22 is used to mark the welded positions in the welding alignment diagram, determine the primary alignment event based on the welded positions and the unwelded positions of the chip package, and mark the real-time alignment level of the chip package. The closed-loop alignment detection system module 23 is used to change the primary alignment event as the chip package is dynamically welded, and to mark the corresponding alignment change content in each change, and to perform multi-level iteration on the alignment change content to output multiple welding alignment factors. The closed-loop alignment detection system is determined based on the multiple welding alignment factors, the material properties of the chip package and the welding temperature. The alignment measure module 24 is used in the closed-loop alignment detection system to mark the welding process of the chip package, mark the alignment adjustment amount of the chip package sequentially along the welding process, match multiple corresponding alignment adjustment positions according to the alignment adjustment amount, and determine the alignment measure of the chip package according to the multiple alignment adjustment positions and the current posture of the chip package.

[0085] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A chip packaging soldering alignment detection method based on image recognition, characterized in that, include: During the soldering process of the chip package, real-time images of the chip package are acquired. Based on image recognition of the real-time images of the chip package, multiple sub-soldering parts are identified, and a soldering alignment map is determined along the soldering alignment requirements of the multiple sub-soldering parts and the chip package. This includes: real-time monitoring of the soldering of the chip package; an external camera taking pictures of the chip package and acquiring real-time images of the chip package; simultaneously triggering an infrared thermal imaging unit to detect the chip package, so that temperature field distribution information is integrated into the real-time images; image recognition is performed on the real-time images, and multiple sub-soldering parts are identified during the recognition process; the multiple sub-soldering parts are located at different positions on the chip package, and adjacent sub-soldering parts have edge spacing lines. In the solder alignment diagram, the soldered positions are marked, the primary alignment event is determined based on the soldered positions and the unsoldered positions of the chip package, and the real-time alignment level of the chip package is marked. The primary alignment event changes with the dynamic welding of the chip package, and the corresponding alignment change content is marked in each change. The alignment change content is iterated in multiple levels to output multiple welding alignment factors. Based on the multiple welding alignment factors, the material properties of the chip package and the welding temperature, a closed-loop alignment detection system is determined. In this closed-loop alignment detection system, the welding process of the chip package is marked, the alignment adjustment amount of the chip package is marked sequentially along the welding process, and multiple corresponding alignment adjustment positions are matched according to the alignment adjustment amount. The alignment measures of the chip package are determined based on the multiple alignment adjustment positions and the current posture of the chip package.

2. The chip packaging welding alignment detection method based on image recognition according to claim 1, characterized in that, The process of soldering the chip package involves acquiring real-time images of the chip package, determining multiple sub-soldering sections based on image recognition of these real-time images, and determining a soldering alignment map along the soldering alignment requirements of the multiple sub-soldering sections and the chip package. The process also includes: The model of the chip package is collected, and the corresponding welding alignment requirements are determined by tracing the model of the chip package. The welding alignment requirements are semantically segmented, and multiple sub-alignment requirements are determined during the segmentation process. A framework alignment map is constructed based on each alignment requirement and the corresponding sub-welding part. The welding alignment map is determined by multi-factor fusion based on the framework alignment map and the previous alignment deviation information of the chip package.

3. The chip packaging welding alignment detection method based on image recognition according to claim 1, characterized in that, In the solder alignment diagram, the soldered positions are marked, and a primary alignment event is determined based on the soldered positions and the unsoldered positions of the chip package. The real-time alignment level of the chip package is also marked, including: The chip package soldering process is regarded as a continuous time stream. The real-time image is compared with the soldering alignment map along the time stream, and the corresponding soldered positions and corresponding position deviation information are extracted. The corresponding unsoldered positions are also marked. Based on the comparison of the soldered positions, the corresponding position deviation information and the unsoldered positions, multiple sub-alignment error information is determined. The corresponding primary alignment event is determined by combining the current posture of the chip package.

4. The chip packaging welding alignment detection method based on image recognition according to claim 3, characterized in that, The process of marking the soldered positions in the solder alignment diagram, determining primary alignment events based on the soldered positions and the unsoldered positions of the chip package, and marking the real-time alignment level of the chip package further includes: In this primary alignment event, multiple sub-alignment items are identified based on the identification of the primary alignment event, and the alignment accuracy of each sub-alignment item is marked. The real-time alignment level of the chip package is determined according to the item content of each sub-alignment item, the corresponding alignment accuracy, and the welding constraints of the chip package.

5. The chip packaging welding alignment detection method based on image recognition according to claim 1, characterized in that, The primary alignment event changes with the dynamic soldering of the chip package, and the corresponding alignment change content is marked in each change. This alignment change content is iterated at multiple levels to output multiple soldering alignment factors. Based on these multiple soldering alignment factors, the material properties of the chip package, and the soldering temperature, a closed-loop alignment detection system is determined, including: As the chip package is dynamically soldered, the soldered positions of the chip package continuously change, triggering dynamic changes in the primary alignment event. The primary alignment event is compared before and after the change to output the corresponding alignment change content, which covers the position deviation trajectory of the unsoldered position.

6. The chip packaging welding alignment detection method based on image recognition according to claim 5, characterized in that, The primary alignment event changes with the dynamic soldering of the chip package, and the corresponding alignment change content is marked in each change. This alignment change content is iterated at multiple levels to output multiple soldering alignment factors. A closed-loop alignment detection system is determined based on these multiple soldering alignment factors, the material properties of the chip package, and the soldering temperature. The system also includes: The alignment change content is input into a preset iterative model, and multi-level iteration of the alignment change content is triggered. The iteration results of each level are output sequentially according to the multi-level iteration of the alignment change content. The corresponding welding alignment factors are determined based on the identification of the iteration results of each level to obtain multiple welding alignment factors. These factors are then combined with the material properties of the chip package and the welding temperature to perform multi-factor fusion, so as to construct a corresponding closed-loop alignment detection system. This closed-loop alignment detection system outputs the next welding node and the current alignment level of the chip package in real time.

7. The chip packaging welding alignment detection method based on image recognition according to claim 1, characterized in that, In this closed-loop alignment detection system, the welding process of the chip package is marked, the alignment adjustment amount of the chip package is marked sequentially along the welding process, and multiple corresponding alignment adjustment positions are matched according to the alignment adjustment amount. The alignment measures of the chip package are determined based on the multiple alignment adjustment positions and the current posture of the chip package, including: The closed-loop alignment detection system is monitored in real time, and the welding process of the chip package is collected. At the same time, based on the phase transition characteristics of the chip package, the welding process is divided into multiple welding stages, and corresponding control boundaries are constructed for each welding stage. Multi-step prediction is performed by combining the control boundaries of the welding stages and the corresponding welding alignment factors, and the alignment adjustment amount of the chip package is predicted.

8. The chip packaging welding alignment detection method based on image recognition according to claim 7, characterized in that, In this closed-loop alignment detection system, the welding process of the chip package is marked, the alignment adjustment amount of the chip package is marked sequentially along the welding process, and multiple corresponding alignment adjustment positions are matched according to the alignment adjustment amount. The alignment measures of the chip package are determined according to the multiple alignment adjustment positions and the current posture of the chip package. The system also includes: The alignment adjustment amount of the chip package is analyzed by matrix in multiple directions, and sub-matrix information in multiple directions is output. The corresponding alignment adjustment orientation is determined by identifying the sub-matrix information in each direction to obtain multiple alignment adjustment orientations. The local alignment of the chip package is triggered by combining the multiple alignment adjustment orientations and the current posture of the chip package, and multiple local alignment results are output. The alignment measures of the chip package are determined according to the mapping relationship between the combination of multiple local alignments and the alignment measures.

9. A chip packaging welding alignment detection system based on image recognition, characterized in that, The image recognition-based chip packaging welding alignment detection system is applied to the image recognition-based chip packaging welding alignment detection method as described in any one of claims 1-8; The image recognition-based chip packaging welding alignment detection system includes: The image recognition module is used to acquire real-time images of the chip package during the soldering process, determine multiple sub-soldering parts based on the image recognition of the real-time images of the chip package, and determine a soldering alignment diagram along the soldering alignment requirements of the multiple sub-soldering parts and the chip package. The primary alignment module is used to mark the welded positions in the welding alignment diagram, determine the primary alignment event based on the welded positions and the unwelded positions of the chip package, and mark the real-time alignment level of the chip package. The closed-loop alignment detection system module is used to change the primary alignment event as the chip package is dynamically welded, and to mark the corresponding alignment change content in each change. The alignment change content is iterated in multiple levels to output multiple welding alignment factors. The closed-loop alignment detection system is determined based on multiple welding alignment factors, the material properties of the chip package and the welding temperature. The alignment measure module is used in the closed-loop alignment detection system to mark the welding process of the chip package, mark the alignment adjustment amount of the chip package sequentially along the welding process, match multiple corresponding alignment adjustment positions according to the alignment adjustment amount, and determine the alignment measure of the chip package based on the multiple alignment adjustment positions and the current posture of the chip package.

Images