Appearance inspection apparatus, multi-stage inspection method and system for circuit board

Abstract

The application discloses a multi-stage detection method and system of a circuit board by appearance detection equipment, and relates to the technical field of visual detection.In each visual detection area, the corresponding electronic circuit is determined based on image recognition of the visual detection area, the circuit detection part and the appearance detection part are determined according to the track of the electronic circuit, and the first defect combination of the circuit detection part and the second defect combination of the appearance detection part are marked, thereby improving the accuracy of the first defect combination and the second defect combination.In the detection space of the appearance detection equipment, the corresponding multi-stage visual detection system is constructed based on the review path and the multi-stage defect content, the corresponding key defect content is output based on the multi-stage visual detection system, and the corresponding appearance-circuit defect is marked, the multi-stage visual detection system is considered from multiple factors, the accuracy of the appearance-circuit defect is improved, and the appearance cleaning measure of the circuit board is determined.

Description

Technical Field

[0001] This invention relates to the field of visual inspection technology, and more particularly to a method and system for multi-level inspection of circuit boards using an appearance inspection device. Background Technology

[0002] In the manufacturing process of printed circuit boards (PCBs), visual inspection is a crucial step in ensuring product quality. As electronic products become increasingly integrated, the electronic circuitry on PCBs becomes denser and their layout more complex. Existing PCB visual inspection technologies typically employ a single vision inspection system, using an industrial camera to acquire images of the entire board or a fixed area, identifying defects such as scratches, short circuits, open circuits, and foreign objects. However, current visual inspection equipment largely relies on static image data for inspection, failing to incorporate the processing information from the PCB's manufacturing process. This affects the accuracy of identifying the first and second defect combinations, resulting in lower accuracy for multi-level vision inspection systems. Summary of the Invention

[0003] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a method and system for multi-level inspection of circuit boards using an appearance inspection device.

[0004] This invention provides a multi-level inspection method for circuit boards using an appearance inspection device, comprising:

[0005] The camera of the appearance inspection equipment captures the circuit board and obtains the current image of the circuit board. Based on the current image and the processing information of the circuit board, multiple visual inspection areas are determined, and the multiple visual inspection areas cover different electronic circuits.

[0006] In each visual inspection area, the corresponding electronic circuit is determined based on the image recognition of the visual inspection area. The circuit inspection part and the appearance inspection part are determined according to the trajectory of the electronic circuit, and the first defect combination of the circuit inspection part and the second defect combination of the appearance inspection part are marked.

[0007] Multi-level defect content is determined based on the matching of the first defect combination and the second defect combination, and multiple defect control nodes are determined based on the identification of the multi-level defect content. The re-inspection path of the appearance inspection equipment on the circuit board is determined according to the node position and corresponding node content of each defect control node.

[0008] Within the inspection space of the appearance inspection equipment, a corresponding multi-level visual inspection system is constructed for the review path and multi-level defect content. Based on this multi-level visual inspection system, the corresponding key defect content is output, and the corresponding appearance-circuit defects are marked to determine the appearance cleaning measures for the circuit board.

[0009] This invention provides a multi-level inspection system for circuit boards using an appearance inspection device. This system is applied to the aforementioned multi-level inspection method for circuit boards using an appearance inspection device. The multi-level inspection system for circuit boards includes:

[0010] The image detection module is used by the camera of the appearance inspection equipment to capture images of the circuit board and obtain the current image of the circuit board. Based on the current image and the processing information history of the circuit board, multiple visual inspection areas are determined, and the multiple visual inspection areas cover different electronic circuits.

[0011] The defect combination module is used to determine the corresponding electronic circuit in each visual inspection area based on image recognition of the visual inspection area, determine the circuit inspection part and the appearance inspection part according to the trajectory of the electronic circuit, and mark the first defect combination of the circuit inspection part and the second defect combination of the appearance inspection part.

[0012] The review path module is used to determine multi-level defect content based on the matching of the first defect combination and the second defect combination, and to determine multiple defect control nodes based on the identification of the multi-level defect content. The review path of the appearance inspection equipment on the circuit board is determined according to the node position and corresponding node content of each defect control node.

[0013] The multi-level inspection module is used to construct a corresponding multi-level visual inspection system for the inspection path and multi-level defect content within the inspection space of the appearance inspection equipment. Based on the multi-level visual inspection system, it outputs the corresponding key defect content and marks the corresponding appearance-circuit defects to determine the appearance cleaning measures for the circuit board.

[0014] Compared with the prior art, the beneficial effects of the present invention are:

[0015] (1) The camera of the appearance inspection equipment takes pictures of the circuit board and obtains the current image of the circuit board. Based on the current image and the processing information history of the circuit board, multiple visual inspection areas are determined, and the multiple visual inspection areas cover different electronic circuits. In each visual inspection area, the corresponding electronic circuit is determined based on the image recognition of the visual inspection area. The circuit inspection part and the appearance inspection part are determined based on the trajectory of the electronic circuit. The first defect combination of the circuit inspection part and the second defect combination of the appearance inspection part are marked. Each visual inspection area is introduced, and the circuit inspection part and the appearance inspection part are controlled, which improves the accuracy of the first defect combination and the second defect combination.

[0016] (2) Multi-level defect content is determined based on the matching of the first defect combination and the second defect combination, and multiple defect control nodes are determined based on the identification of the multi-level defect content. The inspection path of the appearance inspection equipment on the circuit board is determined according to the node position and corresponding node content of each defect control node. In the inspection space of the appearance inspection equipment, a corresponding multi-level visual inspection system is constructed for the inspection path and the multi-level defect content. The corresponding key defect content is output based on the multi-level visual inspection system, and the corresponding appearance-line defects are marked. The inspection path of the appearance inspection equipment on the circuit board is introduced, and the multi-level visual inspection system is considered from multiple factors, which improves the accuracy of appearance-line defects and determines the appearance cleaning measures of the circuit board. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the multi-level inspection method for circuit boards using an appearance inspection device in an embodiment of the present invention.

[0018] Figure 2 This is a flowchart illustrating step S11 of the multi-level inspection method for circuit boards using an appearance inspection device in an embodiment of the present invention.

[0019] Figure 3 This is a flowchart illustrating step S12 of the multi-level inspection method for circuit boards using an appearance inspection device in an embodiment of the present invention.

[0020] Figure 4 This is a flowchart illustrating step S13 of the multi-level inspection method for circuit boards using an appearance inspection device in this embodiment of the invention.

[0021] Figure 5 This is a flowchart illustrating step S14 of the multi-level inspection method for circuit boards using an appearance inspection device in an embodiment of the present invention.

[0022] Figure 6 This is a schematic diagram of the structure of the multi-level inspection system for circuit boards by the appearance inspection equipment in this embodiment of the invention;

[0023] Figure 7 This is a schematic diagram of the structural composition of the appearance inspection device in an embodiment of the present invention. Detailed Implementation

[0024] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0025] Please see Figures 1 to 7 A multi-level inspection method for circuit boards using an appearance inspection device, applied to visual inspection scenarios; the multi-level inspection method for circuit boards using an appearance inspection device includes:

[0026] Step S11: The camera of the appearance inspection equipment takes a picture of the circuit board and obtains the current image of the circuit board. Based on the current image and the processing information history of the circuit board, multiple visual inspection areas are determined, and the multiple visual inspection areas cover different electronic circuits.

[0027] Step S12: In each visual inspection area, the corresponding electronic circuit is determined based on the image recognition of the visual inspection area, the circuit inspection part and the appearance inspection part are determined according to the trajectory of the electronic circuit, and the first defect combination of the circuit inspection part and the second defect combination of the appearance inspection part are marked.

[0028] Step S13: Determine the multi-level defect content based on the matching of the first defect combination and the second defect combination, and determine multiple defect control nodes based on the identification of the multi-level defect content. Determine the re-inspection path of the appearance inspection equipment on the circuit board according to the node position and corresponding node content of each defect control node.

[0029] Step S14: Within the inspection space of the appearance inspection equipment, construct a corresponding multi-level visual inspection system for the review path and multi-level defect content, and output the corresponding key defect content based on the multi-level visual inspection system, and mark the corresponding appearance-circuit defects to determine the appearance cleaning measures for the circuit board.

[0030] refer to Figure 2 In step S11, the specific steps are as follows:

[0031] S111: When the circuit board appearance inspection equipment is in the inspection station, the camera of the appearance inspection equipment is triggered to perform multi-dimensional visual inspection of the circuit board and output multiple sub-images of the circuit board at different angles. The current image of the circuit board is constructed based on the multiple sub-images and the visual inspection parameters of the camera.

[0032] S112: Based on the traceability of the circuit board, determine multiple processing steps of the circuit board and mark the processing information history of the circuit board. At this time, the processing information history of the circuit board presents the processing content of the circuit board in each processing step.

[0033] S113: Register the processing information history of the circuit board with the current image of the circuit board, and dynamically generate irregularly shaped visual inspection areas. In each visual inspection area, the visual inspection area covers different electronic circuits. Each electronic circuit presents the corresponding line inflection point and the corresponding pin connection point, and shields the non-conductive substrate area.

[0034] In the embodiments of this application, when the circuit board appearance inspection equipment is in the inspection station, the camera of the appearance inspection equipment is triggered to perform multi-dimensional visual inspection of the circuit board and output multiple sub-images of the circuit board at different angles. The current image of the circuit board is constructed based on the multiple sub-images and the visual inspection parameters of the camera, which takes into account the overall consideration of multiple sub-images and the visual inspection parameters of the camera, and ensures the accuracy of the current image of the circuit board.

[0035] At this time, when the photoelectric sensor or encoder confirms that the circuit board has reached the detection station and is in a stationary (or uniform speed of dynamic tracking) state, the system sends a hardware trigger signal to the multi-camera controller; the detection system adopts a multi-camera array or a single-camera multi-angle imaging scheme; including but not limited to: front view (0°): used to obtain the layout of components on the board, silkscreen characters and macroscopic circuit routing; side view (30°~45°): using side lighting to focus on capturing the geometric height of the component body, the solder ramping of the pins and whether there is tombstoning or offset; oblique view / multi-angle synthesis: illuminating from different directions to eliminate the reflective blind spots on the metal surface; after receiving the trigger signal, each camera performs photoelectric conversion and analog-to-digital conversion (ADC) to output the original image data from its own perspective, i.e., "multiple sub-images".

[0036] The key to giving physical meaning to image data is mapping pixel coordinates to the world coordinate system. Visual detection parameters include the camera's intrinsic parameters (focal length, principal point coordinates, distortion coefficients, etc.) and extrinsic parameters (rotation matrix, translation vector, i.e., the camera's spatial position and orientation relative to the detection coordinate system). When reading the metadata of a sub-image, the corresponding visual detection parameters are loaded synchronously. This means that each sub-image not only contains pixel grayscale values, but also carries the corresponding calibration matrix, aperture value, exposure time, and imaging depth parameters, providing a mathematical basis for subsequent 3D reconstruction or geometric measurement.

[0037] Distortion correction is performed on each sub-image to eliminate barrel or pincushion distortion and restore the true straight-line features. Based on the extrinsic relationship between each camera, the overlapping area of ​​adjacent viewpoint sub-images is found and aligned using feature point matching algorithms (such as SIFT, ORB) or phase correlation methods. The final constructed "current image" is a high-fidelity digital panoramic image that has undergone coordinate normalization, geometric correction and information complementarity, which truly reflects the physical state of the circuit board at the current moment.

[0038] Specifically, the circuit board (power management board) has an onboard QFN packaged main control chip (located in the center), an array of electrolytic capacitors distributed around it, and connectors on the edge of the board. The main testing difficulty lies in the fact that the QFN chip solder pads are located at the bottom and are not visible, as well as the solder saturation of the capacitor pins. The appearance inspection equipment is equipped with a five-camera imaging system with "top view + four-way oblique view" and has an automatic calibration function.

[0039] As the circuit board enters the inspection station of the appearance inspection equipment via the conveyor belt, the obstruction-type photoelectric sensor is blocked, and the system locks the position of the circuit board. The top-view camera (vertically downward) is triggered to capture the entire board and outputs the sub-image Img-Top, which clearly shows the laser marking on the surface of the main control chip and the surrounding wiring. At the same time, four oblique-view cameras (from the northwest, northeast, southwest, and southeast at 45° angles respectively) are triggered simultaneously and output sub-images Img-45-NW, Img-45-NE, etc. These images capture the side contours of the electrolytic capacitor pins on the circuit board and the chamfer details of the connector metal terminals, which cannot be observed in the front view.

[0040] While reading the Img-Top and four oblique sub-images, the software system of the appearance inspection equipment retrieves the visual inspection parameters calibrated before leaving the factory. For example, for the Img-45-NW image, the system loads the distortion mapping table and spatial projection matrix of the camera. This means that for each pixel of the electrolytic capacitor in the upper left corner of the image, its X, Y, and Z coordinates in the physical world coordinate system can be calculated through the parameters, and the system knows the physical height of the image center point from the surface of the circuit board.

[0041] The system initiates image processing to synthesize the image of the circuit board; it performs distortion correction on the Img-Top image to eliminate the PCB edge stretching caused by the wide-angle lens, ensuring that straight lines on the board edges appear straight; the system performs pixel-level feature alignment between the Img-Top image and the oblique view images from four directions; using the solder reflection points (highlight areas) captured in the oblique view images and the pad positions in the top view images, the system calculates the three-dimensional height information of each solder joint; the appearance inspection equipment successfully constructs the current image of the circuit board, which is not only a clear 2D photograph but also a 2.5D image that incorporates height data; in the image, although the QFN solder joints around the main control chip on the circuit board are located at the bottom, clear solder joint features are synthesized in the current image through the reflection of oblique light; and the pin ramp height of the electrolytic capacitors is also quantified in the image data, providing an accurate data source for subsequent multi-level inspections.

[0042] Furthermore, based on the traceability of the circuit board, multiple processing steps of the circuit board are determined, and the processing information history of the circuit board is marked. At this time, the processing information history of the circuit board presents the processing content of the circuit board in each processing step, which is compatible with the overall consideration of circuit board traceability and ensures the accuracy of multiple processing steps of the circuit board.

[0043] At this point, the unique serial number (SN) or batch number on the circuit board is obtained through OCR (Optical Character Recognition) or barcode / QR code reading technology. This serial number is used as an index key to retrieve the information in the Manufacturing Execution System (MES) database. Based on this index, the system traverses the production plan, work order records, and equipment logs to extract the sequence of all production stages that the circuit board has gone through, forming an ordered processing chain. This chain includes not only the process name but also the specific equipment number, operator ID, and operation timestamp.

[0044] For each processing step (such as SMT placement, reflow soldering, wave soldering, selective soldering, conformal coating, etc.), the system extracts specific process parameters and design data for that step. For example, for the SMT step, the BOM (Bill of Materials) and placement coordinate data are extracted; for the reflow soldering step, the oven temperature profile settings are extracted; for the AOI testing step, the historical defect distribution map is extracted. These unstructured or semi-structured data are converted into standardized data objects to form a unique digital feature set for the circuit board.

[0045] These unstructured or semi-structured data are converted into standardized data objects, forming a unique set of digital features for the circuit board. On a virtual circuit board model or image, the processing content is layered and marked. For example, "mount area", "soldering area" and "inspection area" are marked with different logical labels. This process map not only tells the system "what this board has gone through", but more importantly, "what operation was performed at what location", thus providing a benchmark for defect judgment in subsequent visual inspection (such as distinguishing between process defects and raw material defects).

[0046] Specifically, the barcode reader on top of the appearance inspection equipment scans the QR code SN:PM-202X-V2.0-24001 on the circuit board paper tape; the system immediately sends a query request to the MES server; the returned data shows that the circuit board has undergone the following processing steps: PCB photolithography > inner layer lamination > SMT placement (Line-3) > leaded reflow soldering (Zone-A) > irregular component insertion (Manual-Sta-2) > selective wave soldering (SelWave-05) > in-circuit testing (ICT).

[0047] The software module of the appearance inspection equipment extracted the processing content that is highly relevant to visual inspection from the above process logs: from the SMT placement process, it extracted the package model of the U1 main control chip as QFN-48 and its precise placement center coordinates (X:150.5, Y:80.2); from the selective wave soldering process, it extracted the solder spraying parameters of the J1-J12 connector pins on the bottom of the board, recorded that SAC305 lead-free solder was used in this area, and the nozzle width used in this process was 5mm; the system also noted that the board was set with high TGFR-4 substrate parameters in the inner layer lamination process, which has specific requirements for the backlight compensation threshold of the image.

[0048] The visual inspection equipment constructs the circuit board's processing information history in memory and marks it in the logical layer of the software interface: a timeline is displayed on the screen, with selective wave soldering nodes highlighted on the axis; the system divides the circuit board image into two logical regions: Region Alpha (SMT area): marked as "high-density surface mount area", the main processing content is precision soldering, and the inspection focus is set on pin offset and solder bridging; Region Beta (wave soldering area): marked as "through-hole insertion area", the main processing content is solder spraying, and the inspection focus is set on solder voids, solder spikes, and through-hole solder penetration rate; at this time, the processing information history clearly tells the visual inspection equipment: in Region Beta of the circuit board, there is a risk of wave soldering-specific "shadow effect" or "solder bridge".

[0049] Therefore, the processing information history of the circuit board is registered with the current image of the circuit board, and irregularly shaped visual inspection areas are dynamically generated. In each visual inspection area, different electronic circuits are covered. Each electronic circuit presents the corresponding line inflection point and the corresponding pin connection point, and the non-conductive substrate area is shielded, thus introducing a shielded non-conductive substrate area.

[0050] At this point, the system searches for preset optical positioning reference points (usually circular or cross-shaped copper marks) in the current image of the circuit board. It obtains the pixel coordinates of the reference points in the current image through image processing algorithms (such as gray-scale centroid method or edge fitting method). These measured pixel coordinates are compared with the standard design coordinates in the manufacturing information history (such as Gerber file). The rotation matrix and translation vector between the two are calculated to establish an affine transformation model from the "pixel coordinate system" to the "physical PCB coordinate system". This ensures that the subsequently generated detection area can accurately cover the actual physical circuit, even if the circuit board has slight rotation or translation.

[0051] Based on the registered processing information history, the vector direction of the electronic circuit is extracted. The system does not simply draw a frame, but calculates a geometric expansion vector along the actual copper foil trajectory of the circuit. The circuit trajectory is expanded in the normal direction to both sides to generate a closed polygon that is close to the edge of the circuit. This irregular shape (such as a strip or serpentine shape) visual detection area can contain the effective features of the circuit to the maximum extent, while eliminating irrelevant background interference on both sides.

[0052] Based on the curvature analysis algorithm, the system scans along the center line of the irregular detection area to identify the locations where the curvature changes abruptly (such as 90-degree corners or the starting point of an arc). These line inflection points are areas of stress concentration and key points for adjusting the layout of inductors and capacitors. The system marks them as high-priority detection features.

[0053] By combining pad data from the manufacturing process, the system identifies the metallized connection area of ​​the circuit terminal. The system marks the location where the line width suddenly widens (from line to pad) as the "pin connection point". This often involves SMT pads or through-hole pads and is the core area for detecting cold solder joints and solder bridges.

[0054] By using color space conversion (such as RGB to HSI) or texture analysis, electronic circuits with a metallic luster are distinguished from non-conductive substrates with uniform color and rough texture (usually green, black, or blue solder mask / FR-4 material). Within the generated visual inspection area, the grayscale values ​​of pixels identified as substrate material are forcibly set to zero or marked as "ignore state". This means that subsequent defect detection algorithms (such as scratch detection and short circuit detection) will only run in conductive electronic circuits and solder joint areas, thereby avoiding misjudging scratches and stains on the substrate surface as circuit defects.

[0055] Specifically, the software of the appearance inspection equipment accurately identified two 1mm circular optical mark points in the lower left and upper right corners of the current image of the circuit board. The system found that the line connecting these two mark points in the current image was rotated 0.15 degrees counterclockwise relative to the standard design drawing and shifted 3 pixels to the right. Based on this deviation, the system immediately updated the transformation matrix, so that the Gerber design drawing called by the software perfectly overlapped with the actual photographed circuit board, and the error was controlled at the sub-pixel level.

[0056] For the serpentine sampling lines on the circuit board, the system did not draw a huge rectangle. Instead, it extracted the contours based on the registered Gerber data and along the actual direction of the copper foil. The system generated a polygonal visual inspection area V-Region-Coil that closely followed the serpentine lines. This area was meandering in shape and only 0.1mm wider than the line. It completely covered the copper foil body but perfectly avoided the large area of ​​green ink substrate next to the line.

[0057] Within the V-Region-Coil area, the visual inspection equipment initiates a geometric feature extraction algorithm. For line bends: the system identifies the bends at the 5th, 6th, and 7th turns of the serpentine line and marks them as feature points P-Corner-01 to P-Corner-03. Copper foil at these bends is prone to over-etching, so the system will increase the sensitivity of line width detection at these points. For pin mating points: at the end of the serpentine line, the system detects that the line widens and connects to the MOSFET pad, accurately marking the pin mating point P-Pad-Mosfet. The system will focus on detecting defects such as "pad peeling" or "solder mask vias" at this point.

[0058] Within the aforementioned V-Region-Coil, although the region has a meandering shape, it is not entirely composed of copper foil (there are gaps). The visual inspection equipment utilizes the high reflectivity of copper foil to identify portions of the image with brightness below the copper foil threshold as non-conductive substrates or solder resist ink. The system generates a dynamic mask to logically shield these substrate areas. Even if there is a noticeable scratch on the surface of the circuit board substrate (but outside the copper foil), the system will not falsely report it as a circuit break because the substrate is shielded, thus ensuring extremely high detection accuracy.

[0059] refer to Figure 3 In step S12, the specific steps are as follows:

[0060] S121: Real-time monitoring of each visual detection area, determining multiple corresponding electronic lines based on image recognition of each visual detection area, constructing corresponding electronic circuits along multiple electronic lines, and at this time, triggering the extraction of electronic circuits based on the image skeletonization mechanism and determining the trajectory of electronic circuits.

[0061] S122: Based on the identification of the trajectory of the electronic circuit, a circuit detection area and an appearance inspection area are determined. In the circuit detection area, multiple circuit features are determined based on the identification of the circuit detection area. The circuit detection part is determined according to the multiple circuit features and the regional content of the circuit detection area. Multiple circuit defects are determined based on the tracing of the circuit detection part. The multiple circuit defects are constructed into a first defect combination.

[0062] S123: In the appearance inspection area, multiple appearance inspection features are determined based on the identification of the appearance inspection area, and an appearance inspection part is determined based on the multiple appearance inspection features and the regional content of the appearance inspection area. Multiple appearance defects are determined based on the tracing of the appearance inspection part, and the multiple appearance defects are constructed into a second defect combination.

[0063] In the embodiments of this application, each visual detection area is monitored in real time, and multiple electronic segments are determined based on the image recognition of each visual detection area. The corresponding electronic circuit is constructed along the multiple electronic segments. At this time, the extraction of the electronic circuit is triggered based on the image skeletonization mechanism, and the trajectory of the electronic circuit is determined. This takes into account the overall consideration of image recognition of each visual detection area and ensures the accuracy of the corresponding multiple electronic segments.

[0064] At this point, the system performs histogram analysis for each visual detection area and uses a local adaptive thresholding algorithm (such as the Otsu method or the Niblack method) to binarize the grayscale image. This step can distinguish between the high-brightness "electronic circuits (copper foil / gold fingers)" and the low-brightness "background (solder resist / substrate)".

[0065] Within the binarized irregular region, edge detection and connected component analysis algorithms are applied; the system identifies independent, continuous pixel connected blocks and defines them as "electronic segments"; this step filters out noise (such as tiny dust particles) and breaks up adhering artifacts, accurately locking each independent conductive entity.

[0066] The system calculates the endpoint distance, direction angle, and overlap between each electronic line segment. When the endpoint distance between two electronic line segments is less than a preset tolerance threshold (e.g., half a line width) and their directions are consistent (collinear or smooth transition), the system logically connects them. This connection is not only a connection of physical pixels, but also an aggregation based on the attributes of the electronic line segments (e.g., line width level, network label). A continuous structure spliced ​​from multiple line segments is identified as a complete "electronic circuit" and assigned a unique circuit ID.

[0067] A morphological thinning algorithm (such as the Zhang-Suen algorithm or distance-transform-based morphological skeletonization) is used to iteratively erode the constructed electronic circuit. Edge pixels of the circuit are eroded away until the remaining line width is only one pixel (i.e., the central axis of the connected region). This single-pixel-width line is the "skeleton" of the circuit, which perfectly represents the geometric center of the electronic circuit. Based on the pixel coordinate chain of the skeleton, the system generates an ordered coordinate sequence. Combined with tangent direction calculation, the system can determine the precise trajectory of the electronic circuit in space, including its direction trend, degree of curvature, and start and end coordinates, providing a geometric reference for subsequent defect detection (such as notches and burrs).

[0068] Specifically, in step S113, the system has generated an irregular visual inspection area for the "main power circuit" on the board. This circuit includes a relatively thick main power line and two relatively thin sampling branches, with the line forming a "T-shaped" branch structure. The image processing module of the appearance inspection equipment focuses on the visual inspection area of ​​the "main power circuit". Due to the lighting angle, the edge of the line produces a certain gradient. The system initiates local dynamic threshold processing and successfully separates the copper foil line from the green solder resist ink. Within this area, the system identifies three independent electronic line segments: Seg-1: the vertical main part of the T-shape (line width approximately 2.0 mm); Seg-2: the branch part in the upper left (line width approximately 0.8 mm); Seg-3: the branch part in the upper right (line width approximately 0.8 mm).

[0069] The system analyzes the spatial relationship between Seg-1, Seg-2, and Seg-3; it finds that the starting endpoints of Seg-2 and Seg-3 fall on both sides of Seg-1 and are very close to each other; based on the connectivity rules, the system logically connects these three electronic segments, confirms that they all belong to an electronic circuit named Power_Trail_01, and records that the circuit contains one main node and two branch nodes.

[0070] To detect whether there are "gap" or "overwidth" issues in the unevenly wide circuit, the system performs an image skeletonization mechanism on Power_Trail_01. It begins by "peeling" the circuit from the edge towards the center until the 2.0mm wide main trunk and 0.8mm wide branch lines are reduced to single-pixel widths. The system tracks a continuous single-pixel path in the skeleton diagram, representing the trajectory of the electronic circuit. Trajectory data shows that it starts from the lower end of the main trunk (X100, Y100), extends vertically upwards to (X100, Y50), and forks at (X100, Y50). One path smoothly extends to the left to (X50, Y20), while the other extends to the right to (X150, Y20). Based on this determined trajectory, the visual inspection equipment calculates the distance from the skeleton point to the actual circuit edge (i.e., the skeleton radius), thereby accurately reconstructing the linewidth contour model of the power circuit on the circuit board. This provides a precise geometric reference for subsequent determination of "copper foil gaps" (abrupt changes in skeleton radius).

[0071] Furthermore, based on the identification of the electronic circuit's trajectory, a circuit detection area and an appearance inspection area are determined. Within the circuit detection area, multiple circuit features are identified based on the identification of the circuit detection area. The circuit detection part is determined based on the multiple circuit features and the regional content of the circuit detection area. Multiple circuit defects are identified based on the tracing of the circuit detection part. The multiple circuit defects are constructed into a first defect combination, which is compatible with the overall consideration of tracing the circuit detection part and ensures the accuracy of the multiple circuit defects.

[0072] At this point, normal expansion is performed on both sides of the skeleton trajectory of the electronic circuit as the center line; Circuit detection area: The expansion distance is strictly controlled within the range of 1.1 to 1.2 times the standard line width. This area only includes the circuit body and the pixels at its very edge, focusing on the analysis of conductive geometry; Appearance inspection area: The expansion distance is expanded to 2 to 3 times the standard line width, or extended to the safe distance between adjacent components. This area covers the circuit body, the edge of the solder mask layer and the surrounding environment, focusing on the analysis of surface cleanliness and physical damage.

[0073] Calculate the distance along the normal direction of each point along the skeleton trajectory to obtain the linewidth profile; Key features extracted: Linewidth features: average linewidth, maximum linewidth (short circuit risk), minimum linewidth (gap risk), linewidth variation coefficient; Edge features: edge roughness, jagged protrusions (burrs), edge smoothness; Topological features: skeleton connectivity (open circuit), skeleton bifurcation (short circuit).

[0074] The extracted line features are mapped back to the pixel coordinates of the line detection area; the area content includes not only pixel grayscale, but also the spatial distribution of these features (such as "line width abruptly shrinks at the 50th pixel"); the "line detection area with attribute labels" is defined as the "line detection part". This part is no longer a simple image block, but a digital model with line width threshold table, impedance specification and electrical attribute parameters. It is the direct carrier for subsequent defect judgment.

[0075] The measured characteristics of the circuit inspection section are compared with the standard values ​​(from the manufacturing information history). For example, if the line width is less than the standard line width - 20%, it is judged as a "gap"; if the line width is greater than the standard line width + 20% and extends to the adjacent line, it is judged as a "short circuit". The location coordinates, type, severity and corresponding line ID (traceability) of each judged defect are recorded. All electrical geometric defects found in the circuit inspection section (such as short circuit at Pin 1, gap at Zone 3, open circuit at Track 5) are summarized and packaged to generate the first defect combination. This combination represents the quality status of the PCB's electrical connection integrity.

[0076] Specifically, there is a main power supply line called TRK-PWR on the circuit board (power management board), with a designed line width of 1.5mm; its skeleton trajectory has been extracted in S121; the appearance inspection equipment executes a region expansion algorithm based on the skeleton trajectory of TRK-PWR; the line inspection area: the system expands 0.9mm to each side of the skeleton, generating a compact strip area with a width of about 1.8mm, which precisely wraps around the copper foil line and is used to detect the geometric integrity of the line itself; the appearance inspection area: the system expands 2.5mm to each side of the skeleton, generating a wide strip area with a width of about 5.0mm, which covers the copper foil and the green solder mask layer on both sides, and is used to detect dirt or scratches around the line.

[0077] The system identified several line features in the area: Feature F1: the line width remained around 1.50mm in most areas; Feature F2: at the coordinate point, an inward indentation of the edge was detected; Feature F3: at the end of the line, an unexpected interruption of the skeleton line was detected at the pixel point; combining the above features with the actual pixel content of the line detection area, the line detection part L-Part-01 was determined. This part of the model clearly indicated that: at the location of Feature F2, the area content showed missing copper foil; at the location of Feature F3, the area content showed exposed substrate without copper foil connection; the system locked these two abnormal locations as the core basis for subsequent judgment.

[0078] The visual inspection equipment uses the standard line width (1.5mm ± 10%) and connectivity requirements for judgment. For position F2, the measured line width is only 0.8mm (40% over the standard), and is traced back to the defect type "line gap". For position F3, the distance between the skeleton breakpoint exceeds the tolerance, and is traced back to the defect type "line open circuit". The system packages these two defects to construct the first defect combination of the TRK-PWR circuit on the circuit board:

[0079] {Defect_1:Type=Nick,Location=(x1,y1);Defect_2:Type=Open,Location=(x2,y2)}, this first defect combination will be sent to the subsequent steps for comprehensive matching analysis with the defects in the appearance inspection section.

[0080] Therefore, in the appearance inspection area, multiple appearance inspection features are determined based on the identification of the appearance inspection area. The appearance inspection part is determined according to the multiple appearance inspection features and the regional content of the appearance inspection area. Multiple appearance defects are determined based on the traceability of the appearance inspection part. Multiple appearance defects are constructed into a second defect combination, which is compatible with the overall consideration of the traceability of the appearance inspection part and ensures the accuracy of multiple appearance defects. At the same time, various visual inspection areas are introduced and the line inspection part and appearance inspection part are controlled, which improves the accuracy of the first defect combination and the second defect combination.

[0081] At this point, for the wide "appearance inspection area" divided in S122, various image analysis algorithms are used to extract non-geometric features of the surface state; texture and color feature extraction: unlike circuit detection which focuses on geometric dimensions, this step focuses on the grayscale distribution of pixels, color gradient (RGB / HSV space) and texture roughness; the algorithm identifies foreign spots, oil stain diffusion, character blurring and color difference changes of solder mask layer in the area.

[0082] Surface morphology feature extraction: Based on 3D reconstruction data or photometric stereo vision at a specific angle, extract the physical undulation features of the surface, such as scratch depth, bubble height, solder mask peeling, or green oil blistering; Contrast feature extraction: Calculate the contrast between the local area and the background substrate to identify discoloration caused by oxidation or gloss reduction caused by dirt.

[0083] The extracted texture, color, and morphological features are mapped back to the pixel space of the appearance inspection area; the system analyzes the distribution patterns of these features (such as clustered particles and linear scratches); the "appearance inspection area containing surface state attributes" is defined as the "appearance inspection part". This part not only contains pixel coordinates, but also includes material attribute labels (such as "solder resist", "character", "substrate") and surface quality benchmark models. It is the direct object for evaluating non-conductive defects.

[0084] Combining the cleaning process standards and coating standards in the processing information history, the physical properties of abnormal features are analyzed; for example, distinguishing between "dust adhering to the surface" (removable) and "hard damage embedded in the substrate" (irreparable); defect tracing: foreign matter: identified as white spots, black residue; damage: identified as scratches, exposed copper, collisions; process: identified as solder pads on solder resist, character offset, oxidation discoloration; the system records the category, location, area, and confidence level of each defect; all independent appearance defects (non-electrical connectivity defects) traced in the appearance inspection section are summarized; a second defect combination is generated, which contains a list of defects and their corresponding coordinate set, used for spatial matching with the first defect combination (circuit defects) in subsequent step S13 to determine whether the appearance defect affects the circuit function (e.g., whether scratches damage copper foil).

[0085] Specifically, in step S122, the system generates an appearance inspection area of ​​approximately 5.0 mm in width for the main power supply line TRK-PWR. This area includes not only the copper foil line but also large areas of green solder resist ink on both sides of the line. The appearance inspection equipment focuses on the appearance inspection area on both sides of the TRK-PWR line and performs surface analysis. The system identifies several appearance features of this area: Feature A1: A thin, high-contrast bright line is detected on the green solder resist layer on the right side of the copper foil line; Feature A2: An irregular dark spot with jagged edges is detected in the upper left of the copper foil line; Feature A3: A dark brown color is detected in the pad area at the beginning of the line, which is significantly different from the surrounding bright copper or green.

[0086] By combining the above features with the area content, the system identified the appearance inspection part A-Part-01. This part of the model clearly indicates that: feature A1 presents as physical cutting marks (scratches); feature A2 presents as adhering particles (dirt / foreign matter); and feature A3 presents as chemical discoloration (oxidation). The system accurately locks the positions of these features within the visual inspection area and confirms their correlation with the material properties of the substrate.

[0087] The appearance inspection equipment calls upon the appearance quality standards of the circuit board (IPC-A-600 standard or enterprise internal control standard) for traceability judgment. For feature A1 (scratches): the system measures its length and depth, and traces it as "appearance scratches". For feature A2 (dark spots): the system analyzes its spectral characteristics and confirms it as flux residue, and traces it as "non-conductive foreign matter". For feature A3 (dark brown): the system compares it with the oxide color board and traces it as "pad oxidation". The system packages all the above-identified appearance problems to construct the second defect combination of the TRK-PWR area of ​​the circuit board: {Defect_A:Type=Scratch,Location=(x3,y3);Defect_B:Type=ForeignMaterial,Location=(x4,y4);Defect_C:Type=Oxidation,Location=(x5,y5)}. At this point, the appearance inspection equipment has a complete first defect combination (circuit break, gap) and second defect combination (scratches, foreign matter, oxidation).

[0088] refer to Figure 4 In step S13, the specific steps are as follows:

[0089] S131: The first defect combination contains multiple line defects, and the second defect combination contains multiple appearance defects. At the same time, a spatial correlation matrix is ​​constructed between the first defect combination and the second defect combination. In this spatial correlation matrix, each line defect and each appearance defect is analyzed in depth, and the corresponding multi-level defect content is determined according to the overlap of spatial positions.

[0090] S132: Cluster the various multi-level defect contents and identify multiple neighboring defect points during the clustering process. Merge the neighboring defect points into corresponding defect control nodes to determine multiple defect control nodes. Mark the node position and corresponding node content of each defect control node. At the same time, take all defect control nodes as target points and combine the kinematic characteristics of the appearance inspection equipment to determine multiple staged review ranges. Based on the multiple staged review ranges and the distribution pattern of each defect control node, determine the review path of the appearance inspection equipment on the circuit board. Introduce a priority weight mechanism in the review path and prioritize the planning of node ranges that have a serious impact on electrical function.

[0091] In the embodiments of this application, the first defect combination includes multiple line defects, and the second defect combination includes multiple appearance defects. At the same time, a spatial correlation matrix between the first defect combination and the second defect combination is constructed, and each line defect and each appearance defect is analyzed in depth in the spatial correlation matrix. The corresponding multi-level defect content is determined according to the overlap of spatial positions, which takes into account the overall consideration of the overlap of spatial positions and ensures the accuracy of the corresponding multi-level defect content.

[0092] At this point, the discrete defect data output by S122 and S123 is transformed into a set of computable spatial objects; Line defect set (SetL): Extracts all elements of the first defect combination; each element contains defect type (e.g., Open, Short, Nick), center coordinates (xL, yL), circumscribed polygon contour PL, and the geometric dimensions of the defect; Appearance defect set (SetK): Extracts all elements of the second defect combination; each element contains defect type (e.g., Scratch, Stain, Oxidation), center coordinates (xK, yK), circumscribed polygon contour PK, and the coverage area; Ensure that the coordinate systems of both sets have been unified to the same physical coordinate system (PCB coordinate system) through step S113, eliminating the deviation caused by pixel resolution and imaging perspective.

[0093] Construct an M×N two-dimensional matrix MatrixSpatial, where M is the number of defects in SetL and N is the number of defects in SetK; each element Mij in the matrix represents the spatial correlation S(Li,Kj) between the i-th line defect Li and the j-th appearance defect Kj; at the same time, calculate the following indices: Euclidean distance: the physical distance between the center points of two defects; IoU (Intersection over Union): the intersection-union ratio of the contour regions of two defects; containment relationship: determining whether a defect is completely located within the region of another defect (e.g., whether the line contour is within the appearance scratch contour).

[0094] Set a spatial overlap threshold (e.g., distance less than Dthresh or IoU > 0.10.1); if Li and Kj overlap spatially (e.g., an appearance scratch Kj exactly covers a line gap Li), then the two are determined to be "related defects"; at this time, multi-level defect content is defined as a high-level defect with a causal relationship (e.g., "physical scratches that cause line damage"); if the distance between Li and all Kj exceeds the threshold, then Li is determined to be a pure process defect (e.g., incomplete etching), unrelated to appearance; similarly, isolated appearance defects are only surface dirt; logically bind the successfully matched "line defect + appearance defect" pairs to generate multi-level defect content, which not only indicates "what the defect is", but also "the result of the interaction between the internal circuit and the external surface", providing a dual basis for subsequent cleaning measures and maintenance strategies.

[0095] Specifically, the appearance inspection equipment has already completed the inspection in the previous steps and obtained the defect data of the main power supply line TRK-PWR; the first defect combination (line): includes defect L1 ("line gap" located at coordinate P1); the second defect combination (appearance): includes defect K1 ("appearance scratch" located at coordinate P2) and defect K2 ("foreign object residue" located far away from the line area).

[0096] The visual inspection equipment retrieves the defect data of the circuit board: Line defect L1: coordinates (100.5, 50.2), type "line gap", width reduced to 0.8mm, outline is long and thin; Visual defect K1: coordinates (100.6, 50.3), type "visual scratch", length 3mm, direction is consistent with the line direction; Visual defect K2: coordinates (200.0, 200.0), type "foreign object residue", located on the edge of the board.

[0097] The system constructs a spatial correlation matrix and calculates the relationship between L1 and K1, and L1 and K2. For L1 and K1, the Euclidean distance d(L1,A1)≈0.14mm (very close, almost overlapping). IoU calculation shows that the area of ​​the notch L1 is completely covered by the area of ​​the scratch K1; correlation score: high. For L1 and K2, the Euclidean distance d(L1,K2)>150mm (extremely far); correlation score: zero.

[0098] Because L1 (notch) and K1 (scratch) have a very high degree of spatial overlap, the system determines that these are not two independent defects, but a multi-level defect with a causal relationship. The system outputs the final judgment result: "Risk of circuit breakage caused by mechanical scratches". This means that the problem with the circuit board is not only that the surface is dirty (appearance), but also that the core circuit has been cut off (circuit). Cleaning (decontamination) alone cannot repair it. Circuit jumper repair is necessary. Meanwhile, the other defect K2 is marked as an independent "non-related dirt" that only needs to be cleaned.

[0099] Furthermore, the various multi-level defect contents are clustered, and multiple neighboring defect points are identified during the clustering process. These neighboring defect points are then merged into corresponding defect control nodes to determine multiple defect control nodes. The node position and corresponding node content of each defect control node are marked. Simultaneously, using all defect control nodes as target points and combining the kinematic characteristics of the appearance inspection equipment, multiple phased review ranges are determined. Based on the multiple phased review ranges and the distribution pattern of each defect control node, the review path of the appearance inspection equipment on the circuit board is determined. A priority weight mechanism is introduced into this review path, and priority is given to planning the node range that has a serious impact on electrical function. This approach takes into account the overall consideration of multiple phased review ranges and the distribution pattern of each defect control node, ensuring the accuracy of the review path of the appearance inspection equipment on the circuit board.

[0100] At this point, density-based clustering algorithms (such as DBSCAN) or hierarchical clustering algorithms are used to analyze the coordinates of all multi-level defect content; the Euclidean distance between defect points is calculated; if the distance between two or more defect points (such as "circuit break" and the "scratch" above it) is less than the preset clustering threshold (e.g., 5mm), these points are determined to be in a "nearby state".

[0101] All adjacent defect points are grouped into a single geometric entity, namely the defect control node. The location of this node is usually determined by the centroid of the cluster center, and its coverage is determined by the minimum bounding rectangle or convex hull of the defect point distribution. This avoids the equipment frequently traveling back and forth to dense small defects and improves the efficiency of re-inspection.

[0102] Record the center coordinates (Xc, Yc) of each defect control node in the PCB coordinate system and the dimensions (length, width, height) of its geometric envelope; logically associate all original defect information merged into the node; the content includes: defect type set: such as {Open, Scratch, Stain}; severity level: the criticality of the defect size and location (such as whether it is located on the power main line); multi-level attributes: mark the node as a "pure appearance defect" or a "line-appearance coupling defect".

[0103] Read the motion parameters of the appearance inspection equipment, including maximum acceleration, maximum speed, Z-axis focusing time, and optical zoom switching time; divide the scope into stages: global coarse inspection stage: for low-density or low-priority node ranges, use a low-magnification large-field-of-view camera for rapid scanning; local fine inspection stage: for high-density or high-priority node ranges, plan small movement steps and switch to a high-magnification lens or 3D laser module; define a "re-inspection range box" for each stage, which must include the node itself and reserve a certain mechanical positioning margin.

[0104] Assign a weight Wii to each defect control node; weight calculation factors: whether the node content contains "short circuit / open circuit" (high weight), whether it is located in "functional line" (high weight), and whether it is only "surface dirt" (low weight); total path time T = ∑(MoveTimei + InspectTimei) - λ∑(PriorityWeighti); tend to prioritize accessing nodes with higher weights, even if node B is physically closer. However, if node A has "power line open circuit" (fatal defect) and node B has "silk screen blurry" (minor defect), the system will plan a path to A first and then B, or if the physical path is unreachable, A will be placed first in the scan queue.

[0105] Specifically, after S131 analysis, three multi-level defects were identified: Location A: a combination of "scratches + open circuits" on the main power supply line (high risk); Location B: "flux residue" 2mm next to Location A (adjacent to A); Location C: "minor scratches" on the corner of the board edge (low risk, no damage to the circuit).

[0106] The path planning module of the appearance inspection equipment analyzes the coordinates of the three defects; it finds that the distance between location A (open circuit) and location B (residue) is only 2mm, which is less than the system's set proximity threshold of 5mm; the system merges A and B into a composite defect control node N-01; location C is farther away and is identified as a separate defect control node N-02; at this point, what originally required running 3 points is now optimized to 2 control nodes.

[0107] N-01: Position coordinates are locked at the geometric center of A and B; node content is marked as: {Type:Composite;Defects:[Open_Circuit,Solder_Residue];Severity:Critical}; N-02: Position coordinates are locked at the corner; node content is marked as: {Type:Cosmetic;Defects:[Surface_Scratch];Severity:Minor}.

[0108] Based on the kinematic characteristics of the appearance inspection equipment (maximum speed of 1000mm / s on the X / Y axis, zooming requires 0.5s), the system determines the phased review range: For N-01: due to the inclusion of "open circuit", extremely high precision is required; the review range is set to a small 5mm×5mm interval centered on N-01, and the instruction requires the Z-axis to decrease to a high-magnification imaging depth of field; For N-02: only appearance needs to be checked; the review range is set to a larger 20mm×20mm interval, using a low-magnification wide-angle lens, and keeping the Z-axis at a high position.

[0109] The system introduces a priority weighting mechanism: N-01 is weighted at 1.0 (fatal defect), and N-02 is weighted at 0.2 (minor defect). Calculations show that although N-02 is physically closer to the device origin, N-01 is the node most severely affected by electrical functions. The device camera starts from the origin and moves at full speed to N-01 (high weight is prioritized) to perform detailed 3D imaging to confirm the depth of the break. Only then does it move to N-02 for rapid visual confirmation. This path ensures that if the circuit board needs to be judged as scrapped immediately (due to N-01), the device can give a conclusion in the shortest time without wasting time returning to the main circuit area after confirming the edge scratches.

[0110] refer to Figure 5 In step S14, the specific steps are as follows:

[0111] S141: The current position of the circuit board is input to the appearance inspection equipment. The appearance inspection equipment matches the corresponding inspection space based on the current position of the circuit board, monitors the inspection space in real time, and performs multi-dimensional feature extraction on each defect control node on the inspection path. At this time, for the circuit inspection part, the texture fracture feature is extracted using the phase consistency method; for the appearance inspection part, the two-dimensional geometric feature is extracted using the surface gradient direction histogram; the texture fracture feature, the two-dimensional geometric feature, and the distribution map of the circuit board are input into the multi-modal feature fusion network, and the corresponding multi-level visual inspection system is constructed.

[0112] S142: In this multi-level visual inspection system, the defect parts corresponding to multiple defect control nodes are classified and marked with corresponding functional defect level, potential risk level and pure appearance defect level; the corresponding key defect content is determined according to the defect part, the corresponding defect level and the current image of the circuit board. The defect content presents the corresponding appearance-circuit defect. At the same time, the cleaning measures-defect feature knowledge base is collected, and cleaning process parameters are dynamically generated according to the physical properties of appearance-circuit defects. The appearance cleaning measures of the circuit board are determined along the cleaning process parameters and the corresponding defect level.

[0113] In the embodiments of this application, the current position of the circuit board is input to the appearance inspection device. The appearance inspection device matches the corresponding inspection space based on the current position of the circuit board, monitors the inspection space in real time, and performs multi-dimensional feature extraction on each defect control node on the inspection path. At this time, for the circuit inspection part, the texture fracture feature is extracted using the phase consistency method; for the appearance inspection part, the two-dimensional geometric feature is extracted using the surface gradient direction histogram; the texture fracture feature, the two-dimensional geometric feature, and the distribution map of the circuit board are input into the multi-modal feature fusion network, and a corresponding multi-level visual inspection system is constructed, thus introducing the corresponding multi-level visual inspection system.

[0114] At this point, the current physical coordinates (X,Y,θ) of the circuit board fed back by the motion control system are mapped to the logical coordinate system of the vision system; based on this position, the system calls the predefined detection space parameters (including the optimal focal plane, light source combination mode, and ROI mask) of the region from the database.

[0115] Unlike analyzing only grayscale amplitude, phase consistency uses Fourier transform to analyze the phase consistency of image frequency components. When there are significant edges or feature points in the image, the phase of each frequency component tends to be consistent. The system extracts the shape, length, and direction of these phase collapse regions and defines them as texture fracture features. These features can effectively filter out shadow interference caused by uneven lighting and accurately restore the real physical fracture of the line.

[0116] For surface morphology and stains, statistical shape descriptors are used; the gradient magnitude and direction of pixels in the appearance detection area in the X and Y directions are calculated; the detection window is divided into small cell units, the histogram of gradient direction in each unit is calculated, and adjacent units are combined into blocks for normalization; the generated surface gradient direction histogram describes the contour statistical characteristics of appearance defects; for example, scratches will show a strong and directional gradient distribution, while stains will show a random or gentle gradient distribution. These two-dimensional geometric features are used to distinguish different types of surface defects.

[0117] It receives three heterogeneous data streams: texture breakage features (one-dimensional vectors or feature maps representing electrical continuity); two-dimensional geometric features (HOG vectors or their feature maps representing surface morphology); and circuit board distribution maps (containing prior topological information about component layout and circuit networks). A multi-branch neural network is used; the feature extraction branches process the above features separately, and the features are spliced ​​or attention-weighted in the fusion layer; the network learns the correlation between different feature modalities (e.g., "texture breakage" is often accompanied by "scratch gradient") to build a comprehensive representation vector.

[0118] Based on the output of the fusion network, a dynamic inspection pipeline for the current circuit board is constructed. This system not only outputs the final defect label, but also the confidence score. The system has multiple levels of thresholds. For example, the first level determines whether there is a geometric anomaly (HOG feature), and the second level determines whether there is an electrical break (phase feature). The results of these two levels of determination are combined to form the final decision on the defect control node.

[0119] Specifically, the appearance inspection equipment has been moved above defect control node N-01 (this node contains suspected "circuit breakage" and "scratches"); current status: the equipment camera is focused, and the light source has been switched to mixed light (coaxial light + multi-angle ring light); the motion controller of the appearance inspection equipment reports the current position coordinates as (X=150.0, Y=80.0); the system immediately matches the detection space parameters corresponding to these coordinates, selects the high-resolution macro lens mode, and locks the Z-axis height at 10mm (optimal focal plane); the camera monitors node N-01 in the field of view in real time to ensure that the image is jitter-free.

[0120] For the line section in the field of view (the line detection section defined by S122), the system runs the phase consistency algorithm; in the middle of the copper foil line, the originally high-frequency and consistent phase response suddenly shows a trough of width, and the frequency components at the trough are lost; the system marks this trough as a significant texture break feature F_Break, and calculates its length as 0.5mm, confirming it as a discontinuous interruption of the line texture.

[0121] For the green solder mask layer area around the line (the appearance inspection part defined by S123), the system performs HOG feature extraction; the system calculates the gradient of this area and finds a strong linear gradient distribution with its direction at a 45-degree angle to the line direction; the system generates the corresponding two-dimensional geometric feature F_Geom, which perfectly matches the geometric model of "physical scratch", indicating that there are mechanical damage traces on the surface along the 45-degree direction.

[0122] The visual inspection equipment inputs F_Break (texture breakage), F_Geom (scratch gradient), and the circuit board wiring diagram (showing the main power supply line) into a multimodal feature fusion network. The network analyzes the data at the fusion layer and finds that the spatial location of F_Break and the gradient center of F_Geom completely coincide on the distribution diagram. Combining prior knowledge (main line + breakage + scratch), the network outputs a judgment result with extremely high confidence—this is not two independent defects, but a "physical scratch that causes the main power supply line to be open-circuited." Based on this reasoning result, the visual inspection equipment constructs detection logic for this type of defect at the current node. The final report from the multi-level visual inspection system is: "Node N-01 confirms the existence of a critical defect, type: [fatal - line open circuit], cause: [caused by surface scratch], confidence level: 99.8%." This output directly triggers subsequent cleaning and maintenance decisions.

[0123] Furthermore, in this multi-level visual inspection system, the defect portions corresponding to multiple defect control nodes are classified and labeled with corresponding functional defect levels, potential risk levels, and purely aesthetic defect levels. Based on the defect portion, the corresponding defect level, and the current image of the circuit board, the corresponding key defect content is determined. This defect content presents the corresponding aesthetic-circuit defect. Simultaneously, a cleaning measures-defect feature knowledge base is collected, and cleaning process parameters are dynamically generated based on the physical properties of the aesthetic-circuit defect. The aesthetic cleaning measures for the circuit board are determined along the cleaning process parameters and the corresponding defect level. This system considers the defect portion, the corresponding defect level, and the current image of the circuit board holistically, ensuring the accuracy of the corresponding key defect content. Additionally, a re-inspection path for the circuit board by the aesthetic inspection equipment is introduced, and multiple factors are considered in the multi-level visual inspection system, improving the accuracy of aesthetic-circuit defects and determining the aesthetic cleaning measures for the circuit board.

[0124] At this point, within the multi-level visual inspection system, the detected defects are qualitatively and quantitatively classified according to their severity. Functional defect level: assesses the direct impact of the defect on the electrical performance of the circuit board; for example, open circuits and short circuits belong to the highest level (Level 1), while changes in resistance and capacitance belong to the intermediate level (Level 2). This level determines whether the product should be scrapped directly. Potential risk level: assesses the probability that the defect will cause failure in the future; for example, microcracks (currently conductive but fragile) and excessively thin solder mask (prone to future corrosion) belong to high potential risk; this level determines whether preventative maintenance is required. Pure appearance defect level: assesses defects that only affect aesthetics or ease of assembly; for example, blurred silkscreen printing and dirt in non-functional areas of the surface belong to the low level (Level 3); this level determines whether only simple cleaning is required. The system assigns the above three levels of labels to each defect control node, forming the attribute vector of that node.

[0125] The system logically binds the circuit inspection part determined in S121, the appearance inspection part determined in S123, and the currently determined defect level; the system no longer views the problem in isolation, but generates a composite description; for example, it merges "circuit break" (circuit part) with "physical scratch" (appearance part) and presents it as the key defect content "circuit break caused by surface scratch". This presentation method clearly reveals the causal chain between appearance damage and electrical fault.

[0126] The system retrieves rules of thumb for handling defects from a pre-set expert database. The knowledge base structure includes a mapping table Map(Cleaning, Feature). The Key is the physical properties of the defect (e.g., material type, contaminant composition, adhesion, area, depth). The Value is the recommended cleaning process template (e.g., laser power, plasma time, chemical reagent ratio, wiping intensity). Based on the currently identified physical properties of the defect (e.g., flux residue, oxide layer, metal debris), the system calls the corresponding data records in the knowledge base in real time.

[0127] The system analyzes the physical properties of "appearance-circuit defects," including: contaminant type: organic (glue), inorganic (solder dross), or oxide layer; sensitivity: whether the defect is surrounded by heat-sensitive components or fragile circuits; if it is organic and the surrounding area is not heat-resistant, the generation parameters are: {Type:Plasma;Power:Low;Time:10s}; if it is metal debris and the substrate is robust, the generation parameters are: {Type:Laser_Ablation;Wavelength:1064nm;Energy:30%}; if it is an oxide layer, the generation parameters are: {Type:Chemical_Wipe;Agent:Flux_Remover}.

[0128] Based on the severity of the defect, the final cleaning strategy is determined: High-level (functional defect): If the defect level is "functional defect" (such as open circuit) and the defect cannot be repaired by cleaning (such as copper foil breakage), the cleaning measure is determined to be "no cleaning required, directly transfer to repair / scrap"; Medium-level (potential risk): such as cracks, partial open circuit, the cleaning measure tends to be "non-contact cleaning" (such as laser, airflow) to avoid secondary mechanical damage; Low-level (pure appearance defect): such as dust, fingerprints, the cleaning measure tends to be "contact rapid cleaning" (such as roller wiping, brush cleaning); The system outputs a specific appearance cleaning measure instruction sheet to guide the executing agency to carry out precise operations.

[0129] Specifically, the visual inspection equipment found defect control node N-01 (composite defect on the main power line) and defect control node N-02 (minor dirt on the edge) at the starting point of the inspection path; the current image clearly shows the physical scratches and broken copper wire at N-01, as well as the white spots at N-02.

[0130] The inspection system of the appearance inspection equipment rates two nodes; Node N-01: Functional defect level: Level 1 (fatal) - Main power line is broken, causing the board to be unable to work; Potential risk level: Low - It is already a fact, with no potential risk; Pure appearance defect level: High - Accompanied by severe physical scratches; Node N-02: Functional defect level: None - Does not affect electrical function; Potential risk level: Medium - Contaminants absorb moisture and cause future corrosion; Pure appearance defect level: Level 3 (minor) - Only surface foreign matter.

[0131] The system combines the current image and grading results of N-01 to generate key defect content: description: "A composite appearance-circuit defect was detected at the main power circuit location; the physical morphology is a sharp object scratch, which caused the bottom copper foil circuit to be completely broken." This directly clarifies that N-01 is a structural damage, rather than surface contamination.

[0132] For N-02 (white spots) and N-01 (accompanied metal debris), the system collects cleaning measures - defect feature knowledge base; query Feature: White_Residue, the database returns recommended template: Template_Alcohol_Wipe; query Feature: Metal_Debris_near_crack, the database returns recommended template: Template_Air_Knife.

[0133] The system analyzes defect attributes and generates specific parameters; for N-01: due to physical breakage (copper foil defect), cleaning cannot repair the circuit; the system generates the parameter: {Repair:False;Action:Stop}; for N-02: identified as "non-conductive flux residue"; based on the knowledge base, cleaning process parameters are dynamically generated:

[0134] {Method:Precision_Spray;Pressure:0.2MPa;Solvent:Isopropyl_Alcohol;Nozzle_Distance:50mm}, this ensures that spots can be removed without damaging the surrounding solder mask.

[0135] The visual inspection equipment, based on the defect level, ultimately outputs the following cleaning measures for the circuit board: For N-01: Measures = "Skip cleaning, mark as scrap"; because cleaning cannot solve the open circuit problem, and wiping damages the original state of the break point, affecting subsequent repair analysis; For N-02: Measures = "Perform precision chemical cleaning"; because the level is a pure visual defect and the risk is medium, the equipment instructs the nozzle to move to coordinates (200, 200) and perform targeted spraying according to the generated parameters; through this graded decision-making, the visual inspection equipment avoids wasting cleaning resources on irreparable open circuit boards, while accurately handling dirt that poses potential hazards.

[0136] Please see Figure 6 , Figure 6 This is a schematic diagram of the structure of a multi-level inspection system for circuit boards using an appearance inspection device in an embodiment of the present invention; the multi-level inspection system for circuit boards using the appearance inspection device is applied to the aforementioned multi-level inspection method for circuit boards using an appearance inspection device; the multi-level inspection system is applied to the appearance inspection device; the multi-level inspection system for circuit boards using the appearance inspection device includes:

[0137] The image detection module 21 is used by the camera of the appearance inspection equipment to capture images of the circuit board and obtain the current image of the circuit board. Based on the current image and the processing information history of the circuit board, multiple visual inspection areas are determined, and the multiple visual inspection areas cover different electronic circuits.

[0138] The defect combination module 22 is used to determine the corresponding electronic circuit in each visual inspection area based on image recognition of the visual inspection area, determine the circuit inspection part and the appearance inspection part according to the trajectory of the electronic circuit, and mark the first defect combination of the circuit inspection part and the second defect combination of the appearance inspection part.

[0139] The review path module 23 is used to determine multi-level defect content based on the matching of the first defect combination and the second defect combination, and to determine multiple defect control nodes based on the identification of the multi-level defect content. The review path of the appearance inspection equipment on the circuit board is determined according to the node position and corresponding node content of each defect control node.

[0140] The multi-level inspection module 24 is used to construct a corresponding multi-level visual inspection system for the review path and multi-level defect content within the inspection space of the appearance inspection equipment, and output the corresponding key defect content based on the multi-level visual inspection system, and mark the corresponding appearance-circuit defects to determine the appearance cleaning measures for the circuit board.

[0141] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

Claims

1. A multi-level inspection method for circuit boards using an appearance inspection device, characterized in that, include: The camera of the appearance inspection equipment captures the circuit board and obtains the current image of the circuit board. Based on the current image and the processing information of the circuit board, multiple visual inspection areas are determined, and the multiple visual inspection areas cover different electronic circuits. In each visual inspection area, the corresponding electronic circuit is determined based on the image recognition of the visual inspection area. The circuit inspection part and the appearance inspection part are determined according to the trajectory of the electronic circuit, and the first defect combination of the circuit inspection part and the second defect combination of the appearance inspection part are marked. Specifically, this includes: real-time monitoring of each visual inspection area; determining multiple corresponding electronic line segments based on image recognition of each visual inspection area; constructing corresponding electronic circuits along these electronic line segments; triggering electronic circuit extraction based on an image skeletonization mechanism and determining the trajectory of the electronic circuit; determining circuit detection areas and appearance inspection areas based on the recognition of the electronic circuit trajectory; determining multiple circuit features based on the recognition of the circuit detection area; determining the circuit detection part based on the multiple circuit features and the regional content of the circuit detection area; determining multiple circuit defects based on the tracing of the circuit detection part; and constructing a first defect combination based on the multiple circuit defects. In the appearance inspection area, determining multiple appearance inspection features based on the recognition of the appearance inspection area. The appearance inspection area defines the appearance inspection portion, and based on the traceability of this appearance inspection portion, multiple appearance defects are identified, and multiple appearance defects are constructed into a second defect combination; the circuit inspection area is centered on the skeleton trajectory of the electronic circuit, and expands normally to both sides, with the expansion distance controlled within the range of 1.1-1.2 times the standard line width. This circuit inspection area only includes the circuit body and extremely edge pixels, focusing on the analysis of conductive geometry; the appearance inspection area is centered on the skeleton trajectory of the electronic circuit, and expands normally to both sides, with the expansion distance increased to 2-3 times the standard line width, or extended to the safety distance between adjacent components. This appearance inspection area covers the circuit body, the edge of the solder mask layer, and the surrounding environment, focusing on the analysis of surface cleanliness and physical damage; Multi-level defect content is determined based on the matching of the first and second defect combinations, and multiple defect control nodes are identified based on the identification of this multi-level defect content. The re-inspection path of the appearance inspection equipment on the circuit board is determined according to the node position and corresponding node content of each defect control node. Specifically, this includes: the first defect combination contains multiple circuit defects, and the second defect combination contains multiple appearance defects. Simultaneously, a spatial correlation matrix is ​​constructed between the first and second defect combinations, and each circuit defect and each appearance defect is analyzed in depth within this spatial correlation matrix. The corresponding multi-level defect content is determined based on the overlap of spatial positions. A spatial overlap threshold is set; if the i-th circuit defect L... i With the j-th appearance defect K j If they overlap spatially, they are determined to be related defects; in this case, the multi-level defect content is defined as a high-level defect with a causal relationship; if L i If the distance to all appearance defects exceeds the threshold, then L is determined to be defective. i For purely process defects unrelated to appearance, similarly, isolated appearance defects are merely surface contamination. Successfully matched "circuit defects + appearance defects" pairs are logically bound to generate multi-level defect content. This multi-level defect content not only indicates what the defect is but also that it is the result of the interaction between internal circuitry and the external surface, providing a dual basis for subsequent cleaning measures and maintenance strategies. Each multi-level defect content is clustered, and multiple neighboring defect points are identified during the clustering process. These neighboring defect points are merged into corresponding defect control nodes to determine multiple defect control nodes. The node positions and corresponding node content of each defect control node are marked. Simultaneously, using all defect control nodes as target points and combining the kinematic characteristics of the appearance inspection equipment, multiple phased review ranges are determined. Based on these phased review ranges and the distribution of each defect control node, the review path for the circuit board by the appearance inspection equipment is determined. This review path introduces a priority weighting mechanism, prioritizing the planning of node ranges that severely impact electrical functionality. The position of the defect control node is determined by the centroid of the cluster center, and its coverage is determined by the minimum bounding rectangle or convex hull of the defect point distribution. Within the inspection space of the appearance inspection equipment, a corresponding multi-level visual inspection system is constructed for the review path and multi-level defect content. Based on the multi-level visual inspection system, the corresponding key defect content is output, and the corresponding appearance-line defects are marked to determine the appearance cleaning measures for the circuit board.

2. The multi-level inspection method for circuit boards using the appearance inspection equipment according to claim 1, characterized in that, The camera of the appearance inspection device captures an image of the circuit board and obtains a current image of the circuit board. Based on this current image and the manufacturing process information of the circuit board, multiple visual inspection areas are determined. These multiple visual inspection areas cover different electronic circuits, including: When the circuit board is inspected at the inspection station of the appearance inspection equipment, the camera of the appearance inspection equipment is triggered to perform multi-dimensional visual inspection of the circuit board and output multiple sub-images of the circuit board from different angles. The current image of the circuit board is constructed based on the multiple sub-images and the visual inspection parameters of the camera. Based on the traceability of the circuit board, multiple processing steps of the circuit board are determined, and the processing information history of the circuit board is marked. At this time, the processing information history of the circuit board presents the processing content of the circuit board in each processing step.

3. The multi-level inspection method for circuit boards using the appearance inspection equipment according to claim 2, characterized in that, The appearance inspection device uses a camera to capture images of the circuit board and obtains a current image of the board. Based on this current image and the manufacturing process information of the circuit board, multiple visual inspection areas are determined. These multiple visual inspection areas cover different electronic circuits and also include: The processing information history of the circuit board is registered with the current image of the circuit board, and irregularly shaped visual inspection areas are dynamically generated. In each visual inspection area, different electronic circuits are covered. Each electronic circuit presents the corresponding line inflection point and the corresponding pin connection point, and the non-conductive substrate area is shielded.

4. The multi-level inspection method for circuit boards using the appearance inspection equipment according to claim 1, characterized in that, Within the inspection space of the appearance inspection equipment, a multi-level visual inspection system is constructed corresponding to the review path and multi-level defect content. Based on this multi-level visual inspection system, the corresponding key defect content is output, and the corresponding appearance-line defects are marked to determine the appearance cleaning measures for the circuit board, including: The current position of the circuit board is input into the appearance inspection equipment. The appearance inspection equipment matches the corresponding inspection space based on the current position of the circuit board, monitors the inspection space in real time, and performs multi-dimensional feature extraction on each defect control node on the inspection path. At this time, for the circuit inspection part, the texture fracture feature is extracted using the phase consistency method; for the appearance inspection part, the two-dimensional geometric feature is extracted using the surface gradient direction histogram. The texture fracture feature, two-dimensional geometric feature and the distribution map of the circuit board are input into the multi-modal feature fusion network, and the corresponding multi-level visual inspection system is constructed.

5. The multi-level inspection method for circuit boards using the appearance inspection equipment according to claim 4, characterized in that, Within the inspection space of the appearance inspection equipment, a multi-level visual inspection system is constructed corresponding to the review path and multi-level defect content. Based on this multi-level visual inspection system, the corresponding key defect content is output, and the corresponding appearance-line defects are marked to determine the appearance cleaning measures for the circuit board. The method also includes: In this multi-level visual inspection system, the defect parts corresponding to multiple defect control nodes are classified and marked with corresponding functional defect level, potential risk level and pure appearance defect level. Based on the defect part, the corresponding defect level and the current image of the circuit board, the corresponding key defect content is determined. The defect content presents the corresponding appearance-line defect. At the same time, the cleaning measures-defect feature knowledge base is collected, and cleaning process parameters are dynamically generated based on the physical properties of appearance-line defects. The appearance cleaning measures of the circuit board are determined along the cleaning process parameters and the corresponding defect level.

6. A multi-level inspection system for circuit boards using an appearance inspection device, characterized in that, The multi-level inspection system for circuit boards described in the appearance inspection equipment is applied to the multi-level inspection method for circuit boards using the appearance inspection equipment as described in any one of claims 1-5; The appearance inspection equipment includes a multi-level inspection system for circuit boards, comprising: The image detection module is used by the camera of the appearance inspection equipment to capture images of the circuit board and obtain the current image of the circuit board. Based on the current image and the processing information history of the circuit board, multiple visual inspection areas are determined, and the multiple visual inspection areas cover different electronic circuits. The defect combination module is used to determine the corresponding electronic circuit in each visual inspection area based on image recognition of the visual inspection area, determine the circuit inspection part and the appearance inspection part according to the trajectory of the electronic circuit, and mark the first defect combination of the circuit inspection part and the second defect combination of the appearance inspection part. The review path module is used to determine multi-level defect content based on the matching of the first defect combination and the second defect combination, and to determine multiple defect control nodes based on the identification of the multi-level defect content. The review path of the appearance inspection equipment on the circuit board is determined according to the node position and corresponding node content of each defect control node. The multi-level inspection module is used to construct a corresponding multi-level visual inspection system for the inspection path and multi-level defect content within the inspection space of the appearance inspection equipment. Based on the multi-level visual inspection system, it outputs the corresponding key defect content and marks the corresponding appearance-line defects to determine the appearance cleaning measures for the circuit board.

Images