IC carrier plate defect detection method, system and equipment based on distributed computing and medium

Abstract

The invention discloses an IC carrier plate defect detection method, system and device based on distributed computing and a medium, mainly relates to the technical field of defect detection, and aims to solve the problems that low-contrast defects and process texture noise are difficult to distinguish in a single visual mode, image transmission and processing delay is high, and image quality is poor in the prior art. The real-time response requirement of a high-speed production line cannot be met, and physical property detection and visual information are mutually independent. Comprising the following steps: fusing an RGB image and a depth image into a preset aggregation image; obtaining a property detection result of the current preset core process through a preset physical property detection device on the preset core process; generating an IC carrier plate defect conclusion according to the property detection result and a preset aggregation image; when the IC carrier plate defect conclusion is that the IC carrier plate defect exists, the closest IC carrier plate screening mechanical arm is determined according to the actual position of the collected IC carrier plate; and the mechanical arm is used for screening out the corresponding IC carrier plates.

Description

Technical Field

[0001] This application relates to the field of IC substrate defect detection technology, and in particular to an IC substrate defect detection method, system, device and medium based on distributed computing. Background Technology

[0002] Currently, IC substrate defect detection mainly relies on automated optical inspection systems (AOIs), which use high-resolution industrial cameras to acquire RGB images of the surface. Visible defects such as solder joint misalignment, solder bridges, and missing components are identified through preset template matching or edge detection algorithms. Some production lines supplement this with X-ray inspection to observe interlayer voids or microcracks, or use laser scanning to obtain surface morphology changes. The inspection process typically centers on a central server processing image data, while manual visual inspection remains a crucial review step to correct false alarms and missed detections in the AOI system, forming a hybrid "machine inspection + human judgment" model.

[0003] Existing technologies have limitations: a single visual modality is difficult to distinguish between low-contrast defects and process texture noise, resulting in a high false spot rate, a heavy workload for manual re-inspection, and a high risk of missed detection due to fatigue; the detection system has a centralized architecture, resulting in high latency in image transmission and processing, which cannot meet the real-time response requirements of high-speed production lines; physical property detection (such as temperature and conductivity) and visual information are independent of each other, lacking a multimodal collaborative analysis mechanism, making it difficult to accurately locate hidden defects. Summary of the Invention

[0004] This application provides a method, system, device, and medium for IC substrate defect detection based on distributed computing, in order to solve the problems of existing solutions, such as the inability of a single visual modality to distinguish between low-contrast defects and process texture noise, the centralized architecture of the detection system with high image transmission and processing latency, which cannot meet the real-time response requirements of high-speed production lines, and the independence of physical property detection (such as temperature and conductivity) from visual information, lacking a multimodal collaborative analysis mechanism.

[0005] In a first aspect, this application provides a method for detecting defects in IC substrates based on distributed computing, the method comprising: Deploy distributed edge nodes on the pre-defined core processes of the IC carrier board; Based on the collectors installed on the distributed edge nodes, RGB and depth images of the IC carrier board on the current preset core process are collected; Distributed edge nodes use an attention weight algorithm to fuse RGB images and depth images into a preset aggregated image; The property test results of the current core process are obtained by using the physical property testing equipment pre-set in the core process. Based on the property detection results and the preset aggregated image, a conclusion on the IC substrate defect is generated; When the IC carrier board defect conclusion is that there is an IC carrier board defect, the nearest IC carrier board screening robot arm is determined based on the actual location of the collected IC carrier board; the robot arm is used to screen out the corresponding IC carrier board.

[0006] In one implementation of this application, distributed edge nodes are deployed on a preset core process of the IC carrier board, specifically including: At the preset core process nodes of the IC carrier board, distributed edge computing nodes are deployed in sequence according to the process flow. Each node corresponds to an independent IC carrier board inspection station. Each edge node is fixedly mounted with a binocular industrial camera and an infrared structured light depth sensor by a rigid bracket. The camera and sensor are triggered synchronously to align the RGB image and depth image on the time axis. The acquisition frequency of the RGB image and depth image is set to 10 frames per second. The trigger signal for the synchronous triggering of the camera and sensor is driven by the pulse of the pipeline encoder so that each IC carrier board is only acquired once by the edge node when passing through the inspection area.

[0007] In one implementation of this application, distributed edge nodes utilize an attention weight algorithm to fuse RGB images and depth images into a preset aggregated image, specifically including: The query target Q1 value is generated using the RGB image and a preset first weight; The index anchor point K1 value is generated using the depth image and a preset second weight; Using depth images and preset third weights, extractable content V1 values ​​are generated; Through the formula: A1 = Softmax[(Q1·K1)] ) / d k ]·V1, calculate the first weighted depth feature A1; where, d k Indicates the preset factor; The Q2 value of the query target is generated using the depth image and a preset fourth weight; The index anchor point K2 value is generated using the RGB image and the preset fifth weight; Using RGB images and a preset sixth weight, extractable content V2 values ​​are generated; Through the formula: A2 = Softmax[(Q2·K2)] ) / d k V2, calculate the second weighted depth feature A2; Channel concatenation is performed on the first and second weighted depth features, which are then fed into a 1×1 convolution to compress the number of channels to 128. After upsampling, the images are restored to the preset image resolution, and a pixel-level preset aggregated image is output.

[0008] In one implementation of this application, the property test results of the current preset core process are obtained through a preset physical property testing device, specifically including: The testing equipment communicates with edge nodes in real time via an industrial bus; The testing equipment automatically starts when the IC carrier board passes by and collects physical characteristics, including conductivity, thickness, resistance, surface flatness, and whether it is an abnormal area. The sampling point density is no less than 5 per square centimeter. The collection results of the sampling points are bound to the timestamp and spatial coordinates. The initial value for whether it is an abnormal area is empty. The acquisition results are transmitted to the corresponding edge nodes in a structured binary format. The property detection results of sampling points whose acquisition results exceed the preset range are marked as abnormal areas.

[0009] In one implementation of this application, a defect conclusion for the IC substrate is generated based on the property detection results and a preset aggregated image, specifically including: Identify visual defects in pre-defined aggregated images; The property detection results and visual defects are converted into vector data to construct multidimensional feature vectors; The multidimensional feature vector is input into the preset decision rule engine. The decision rule engine uses a logical judgment tree built based on the historical defect sample library to cross-validate the visual defects in the preset aggregated image with the physical features in the property detection results. When the visual defects and physical features trigger the preset defect combination conditions at the same time, it is determined that there is an IC carrier board defect.

[0010] In one implementation of this application, when the IC carrier board defect conclusion is that an IC carrier board defect exists, the nearest IC carrier board screening robot arm is determined based on the actual location of the collected IC carrier board; the robot arm is used to screen out the corresponding IC carrier boards, specifically including: When the IC carrier board defect conclusion is that there is an IC carrier board defect, the edge node calculates the spatial distance between the real-time coordinates of the IC carrier board and the positions of all robotic arms in the global coordinate system of the pipeline, and selects the nearest robotic arm for screening. After receiving the coordinate command, the robotic arm control unit starts the servo motor to drive the end effector to move to the target position and screen out the corresponding IC carrier board.

[0011] Secondly, this application provides an IC substrate defect detection system based on distributed computing, the system comprising: The deployment module is used to deploy distributed edge nodes on the preset core processes of the IC carrier board; The acquisition module is used to acquire RGB and depth images of the IC carrier board in the current preset core process based on the acquisition devices installed on the distributed edge nodes. The aggregation module is used by distributed edge nodes to fuse RGB images and depth images into a preset aggregated image using an attention weight algorithm; The acquisition module is used to obtain the property test results of the current preset core process through the preset physical property testing equipment in the preset core process; The generation module is used to generate a conclusion on IC substrate defects based on the property detection results and the preset aggregated image; The screening module is used to determine the nearest IC carrier screening robot arm based on the actual location of the collected IC carrier when the IC carrier defect conclusion is that there is an IC carrier defect; the robot arm is then used to screen out the corresponding IC carrier.

[0012] In one implementation of this application, the aggregation module includes an aggregation unit. Used to generate the query target Q1 value using an RGB image and a preset first weight; The index anchor point K1 value is generated using the depth image and a preset second weight; Using depth images and preset third weights, extractable content V1 values ​​are generated; Through the formula: A1 = Softmax[(Q1·K1)] ) / d k ]·V1, calculate the first weighted depth feature A1; where, d k Indicates the preset factor; The Q2 value of the query target is generated using the depth image and a preset fourth weight; The index anchor point K2 value is generated using the RGB image and the preset fifth weight; Using RGB images and a preset sixth weight, extractable content V2 values ​​are generated; Through the formula: A2 = Softmax[(Q2·K2)] ) / d k V2, calculate the second weighted depth feature A2; Channel concatenation is performed on the first and second weighted depth features, which are then fed into a 1×1 convolution to compress the number of channels to 128. After upsampling, the images are restored to the preset image resolution, and a pixel-level preset aggregated image is output.

[0013] Thirdly, this application provides an IC substrate defect detection device based on distributed computing, the device comprising: processor; And a memory that stores executable code, which, when executed, causes the processor to perform a distributed computing-based IC substrate defect detection method as described above.

[0014] Fourthly, this application provides a non-volatile computer storage medium storing computer instructions thereon, which, when executed, implement a distributed computing-based IC substrate defect detection method as described above.

[0015] As can be seen from the above technical solutions, this application has the following advantages: Benefit 1: Reduced false positive rate, improved defect identification accuracy: By simultaneously acquiring RGB and depth images of the IC substrate on distributed edge nodes and employing an attention weighting algorithm for multimodal fusion, an aggregated image with enhanced spatial structure is generated, effectively distinguishing between low-contrast defects and process texture noise. The depth image provides three-dimensional topography information, compensating for the shortcomings of RGB images in recognizing minute surface elevation differences, while the attention mechanism dynamically weights key regions, suppressing interference responses from non-defect textures. This mechanism directly eliminates the root cause of misjudgments due to grayscale similarity in a single visual modality, upgrading defect judgment from single-color comparison to geometric-light intensity joint feature analysis, thereby reducing the false positive detection rate, alleviating the burden of manual review, and improving overall detection reliability.

[0016] Benefit 2: Enables rapid response, meeting the real-time detection needs of high-speed production lines: By offloading image acquisition, fusion computation, and preliminary defect assessment functions to distributed edge nodes deployed near core processes, the communication latency and computational queuing delays introduced by long-distance image data transmission and centralized processing on a central server in traditional centralized architectures are completely avoided. Each edge node independently completes image processing and decision-making for its local area, without needing to transmit data back to the central platform. The processing loop is completed locally, reducing system response time from seconds to milliseconds. This architecture design directly adapts to the high-speed continuous production cycle of IC carrier boards, ensuring that each carrier board is judged as it passes through the inspection station, guaranteeing smooth production line operation and throughput efficiency.

[0017] Benefit 3: Constructing a physical-visual collaborative diagnostic mechanism to accurately locate latent defects: For the first time, real-time output data from physical property detection devices (such as temperature sensors and conductivity probes) is jointly analyzed with visual aggregated images to establish cross-modal correlation logic. When the aggregated image shows a suspected defect area, the corresponding temperature anomaly or conductivity deviation trend is simultaneously compared, achieving dual verification of "morphological anomaly + physical property anomaly". This mechanism transforms previously isolated visual cues and physical parameters into a complementary chain of evidence, improving the accuracy of locating latent defects without obvious surface features, such as internal cracks, delamination, and micro-short circuits. Defect conclusions no longer rely on fuzzy judgments of a single modality but are based on cross-verification of multi-source heterogeneous data, significantly improving the reliability and interpretability of defect classification. Attached Figure Description

[0018] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart of an IC substrate defect detection method based on distributed computing provided in an embodiment of this application.

[0020] Figure 2 This is a schematic diagram of the internal structure of an IC substrate defect detection system based on distributed computing, provided in an embodiment of this application.

[0021] Figure 3 This is a schematic diagram of the internal structure of an IC substrate defect detection device based on distributed computing, provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Those skilled in the art should understand that the embodiments described below are merely preferred embodiments of this disclosure and do not imply that this disclosure can only be implemented through these preferred embodiments. These preferred embodiments are merely used to explain the technical principles of this disclosure and are not intended to limit the scope of protection of this disclosure. Based on the preferred embodiments provided by this disclosure, all other embodiments obtained by those skilled in the art without creative effort should still fall within the scope of protection of this disclosure.

[0024] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0025] The technical solutions proposed in the embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0026] The embodiment provides a method for detecting defects in IC carrier boards based on distributed computing, such as... Figure 1 As shown in the embodiments of this application, the method mainly includes the following steps: Step 110: Deploy distributed edge nodes on the preset core processes of the IC carrier board.

[0027] In some embodiments, distributed edge nodes are deployed on a preset core process of the IC carrier board, specifically including: At the preset core process nodes of the IC carrier board, distributed edge computing nodes are deployed in sequence according to the process flow. Each node corresponds to an independent IC carrier board inspection station. Each edge node is fixedly mounted with a binocular industrial camera and an infrared structured light depth sensor by a rigid bracket. The camera and sensor are triggered synchronously to align the RGB image and depth image on the time axis. The acquisition frequency of the RGB image and depth image is set to 10 frames per second. The trigger signal for the synchronous triggering of the camera and sensor is driven by the pulse of the pipeline encoder so that each IC carrier board is only acquired once by the edge node when passing through the inspection area.

[0028] It should be noted that distributed edge nodes are deployed at the core process nodes of the IC carrier board. Each node is equipped with a fixedly installed binocular industrial camera and an infrared structured light depth sensor, and the acquisition is triggered synchronously by pulses from the production line encoder, ensuring that each carrier board is acquired only once when it passes through the inspection station, with a stable image frame rate of 10 frames per second. This deployment method directly achieves temporal alignment and spatial positioning consistency of image data, avoiding image registration errors caused by motion blur or asynchronous sampling. At the same time, because the acquisition behavior is strictly bound to the production line cycle, the possibility of duplicate acquisition or missed acquisition is eliminated, ensuring the uniqueness and traceability of the inspection samples. The edge nodes independently complete local image acquisition without relying on the central system scheduling, so that the temporal determinism and spatial accuracy of the image acquisition process are directly bound to the physical workstation, providing a stable and non-redundant raw data input foundation for subsequent multimodal fusion and defect judgment.

[0029] Step 120: Collect RGB and depth images of the IC carrier board on the current preset core process using the collectors installed on the distributed edge nodes.

[0030] It should be further clarified that: "Acquirer" refers to an industrial image acquisition device adapted for installation on distributed edge nodes, capable of simultaneously acquiring RGB and depth images; "preset core processes" refers to key processes in the IC substrate production process that require focused inspection (such as substrate etching and pin forming); "IC substrate" refers to the substrate used to support and fix integrated circuit chips; "RGB image" refers to a two-dimensional image captured by the acquirer, containing red (R), green (G), and blue (B) color information; "depth image" refers to a two-dimensional image captured by the acquirer, containing distance information between each pixel in the image and the acquirer. During acquisition, the acquirer establishes a stable data connection with the distributed edge nodes, synchronously triggering the acquisition of both images to ensure that the acquired RGB and depth images correspond to the same IC substrate and the same acquisition time, guaranteeing the accuracy of subsequent feature calculations.

[0031] Step 130: Distributed edge nodes use an attention weight algorithm to fuse RGB images and depth images into a preset aggregated image.

[0032] It should be noted that this step can be specifically described as follows: The query target Q1 value is generated using the RGB image and a preset first weight; The index anchor point K1 value is generated using the depth image and a preset second weight; Using depth images and preset third weights, extractable content V1 values ​​are generated; Through the formula: A1 = Softmax[(Q1·K1)] ) / d k ]·V1, calculate the first weighted depth feature A1; where, d k Indicates the preset factor; The Q2 value of the query target is generated using the depth image and a preset fourth weight; The index anchor point K2 value is generated using the RGB image and the preset fifth weight; Using RGB images and a preset sixth weight, extractable content V2 values ​​are generated; Through the formula: A2 = Softmax[(Q2·K2)] ) / d k V2, calculate the second weighted depth feature A2; Channel concatenation is performed on the first and second weighted depth features, which are then fed into a 1×1 convolution to compress the number of channels to 128. After upsampling, the images are restored to the preset image resolution, and a pixel-level preset aggregated image is output.

[0033] It should be further explained that the preset weights from the first to the sixth are all coefficients pre-calibrated through experiments to adjust the influence of the corresponding image data in feature generation. Each weight ranges from 0 to 1 and can be flexibly adjusted according to the IC carrier board detection accuracy requirements. The query target values ​​Q1 and Q2 refer to the feature vectors calculated based on the corresponding image data and weights, used for feature matching queries. The index anchor values ​​K1 and K2 refer to the anchor vectors calculated based on the corresponding image data and weights, used for feature matching and positioning. The extractable content values ​​V1 and V2 refer to the extractable vectors containing key image features, calculated based on the corresponding image data and weights. The preset factor d... k The term "preset constant" refers to a preset constant used to adjust the scale of the inner product of feature vectors to avoid excessively large calculation results, and its value is adapted to the dimension of the feature vectors. The Softmax function is an activation function used to normalize the calculation results of feature vectors to the 0-1 range to facilitate weight allocation. Channel splicing refers to merging the feature channels of the first weighted depth feature A1 and the second weighted depth feature A2 in a preset order. 1×1 convolution is used to compress the number of feature channels after merging. Upsampling refers to restoring the compressed feature map to a preset image resolution through interpolation operations (the preset image resolution refers to a standard resolution that is pre-set and consistent with the resolution of the acquired image). The pixel-level preset aggregated image refers to an aggregated feature image that contains the comprehensive features of each pixel on the IC carrier board and can be used for subsequent detection and analysis.

[0034] Step 140: Obtain the property test results of the current core process by using the physical property testing equipment preset in the preset core process.

[0035] In some embodiments, the property test results of the current preset core process are obtained through a preset physical property testing device in the preset core process, specifically including: The testing equipment communicates with edge nodes in real time via an industrial bus; The testing equipment automatically starts when the IC carrier board passes by and collects physical characteristics, including conductivity, thickness, resistance, surface flatness, and whether it is an abnormal area. The sampling point density is no less than 5 per square centimeter. The collection results of the sampling points are bound to the timestamp and spatial coordinates. The initial value for whether it is an abnormal area is empty. The acquisition results are transmitted to the corresponding edge nodes in a structured binary format. The property detection results of sampling points whose acquisition results exceed the preset range are marked as abnormal areas.

[0036] It should be further clarified that the preset core processes refer to the process steps in the IC substrate manufacturing process that have a critical impact on electrical performance and structural integrity, such as electroplating, etching, or lamination. Their locations are pre-set according to the process flow chart, and each process corresponds to only one set of physical testing equipment. The physical property testing equipment consists of non-contact or micro-contact industrial sensors, including eddy current conductivity meters, laser thickness gauges, four-probe resistance testing modules, and white light interferometric surface topography instruments. These are used to acquire conductivity, thickness, resistance, and surface flatness data in real time, with a response time of less than 20ms and a sampling frequency of no less than 10Hz. The timestamp is the acquisition time recorded by the device's built-in high-precision clock, with an accuracy of 1ms, synchronized with the production line encoder pulse. The spatial coordinates are determined by the encoder position feedback of the substrate conveying mechanism, with the device's optical reference point as the origin, in millimeters, ensuring that each sampling point can be traced back to its specific physical location on the substrate. The structured binary format is a custom data packet format, containing a 4-byte timestamp, 8-byte coordinates (X / Y), 4-byte conductivity, 4-byte thickness, 4-byte resistance, and 4-byte surface flatness RMS value, with a total length of 32 bytes per sampling point. It is uncompressed and unencrypted, and edge nodes can be directly parsed. The preset range is a static threshold range derived from historical good product data statistics, such as conductivity ±5%, thickness ±2μm, resistance ±10%, and surface flatness ≤0.8μm RMS. Sampling points outside this range are marked as abnormal areas for subsequent use by the multimodal fusion module.

[0037] Step 150: Based on the property detection results and the preset aggregated image, generate a conclusion on the defects of the IC carrier board.

[0038] This step can be specifically described as follows: Identify visual defects in a preset aggregated image (which can be achieved by existing image recognition algorithms, and this application does not limit this); The property detection results and visual defects are converted into vector data to construct multidimensional feature vectors; The multidimensional feature vector is input into the preset decision rule engine. The decision rule engine uses a logical judgment tree built based on the historical defect sample library to cross-validate the visual defects in the preset aggregated image with the physical features in the property detection results. When the visual defects and physical features trigger the preset defect combination conditions at the same time, it is determined that there is an IC carrier board defect.

[0039] It should be further explained that the preset decision rule engine is a static logical judgment tree manually constructed based on a historical defect sample library. Its node conditions are defined by process engineers according to Failure Mode and Effects Analysis (FMEA). For example, "A visual pattern of edge cracks exists at coordinates (X,Y), and the conductivity at that point is below the lower threshold of -5%" constitutes a leaf node rule. The logical judgment tree consists of several conditional paths combining "AND" logic. Each path corresponds to a known defect type. The tree structure is fixed before system deployment and is not dynamically updated with new data. The preset defect combination conditions are specific paths defined in the judgment tree. The engine outputs the conclusion "An IC carrier board defect exists" only when the visual and physical features simultaneously satisfy all conditions in the path at the same spatial coordinates; otherwise, it outputs "No defect." This engine has no probability output, no confidence score, and the judgment result is a binary deterministic output.

[0040] Step 160: When the IC carrier board defect conclusion is that there is an IC carrier board defect, determine the nearest IC carrier board screening robot arm based on the actual location of the collected IC carrier board; use the robot arm to screen out the corresponding IC carrier board.

[0041] This step can be specifically described as follows: When the IC carrier board defect conclusion is that there is an IC carrier board defect, the edge node calculates the spatial distance between the real-time coordinates of the IC carrier board and the positions of all robotic arms in the global coordinate system of the pipeline, and selects the nearest robotic arm for screening. After receiving the coordinate command, the robotic arm control unit starts the servo motor to drive the end effector to move to the target position and screen out the corresponding IC carrier board.

[0042] It should be further explained that the global coordinate system of the production line is a unified spatial coordinate system established based on a fixed reference datum of the production line. Its origin is set at the mechanical reference point at the entrance of the first workstation. The X-axis is along the direction of the carrier board conveying, the Y-axis is perpendicular to the conveying surface and points to the side, and the Z-axis is vertically upward. The unit is millimeters. The coordinate update frequency is synchronized with the encoder pulse. The center position of each IC carrier board is calculated in real time by the encoder of the conveying mechanism and injected into the edge nodes to ensure that all detection and execution actions are based on the same spatial reference system. The screening robotic arms are multiple industrial six-axis robotic arms fixedly installed along the side of the production line. Each arm covers the lateral range of three adjacent detection stations. The position of the arm base is pre-calibrated in the coordinate system. The end effector is a vacuum suction clamp with a maximum response delay of no more than 50ms. Each robotic arm has an independent control channel and no shared motion path.

[0043] In addition, this application Figure 2 This application provides an IC substrate defect detection system based on distributed computing, as an embodiment of the present application. Figure 2 As shown in the embodiments of this application, the system mainly includes: Deployment module 210 is used to deploy distributed edge nodes on a preset core process of the IC carrier board.

[0044] The acquisition module 220 is used to acquire RGB and depth images of the IC carrier board in the current preset core process based on the acquisition devices installed on the distributed edge nodes.

[0045] The aggregation module 230 is used by distributed edge nodes to fuse RGB images and depth images into a preset aggregated image using an attention weight algorithm.

[0046] Aggregation module 230 includes aggregation units, Used to generate the query target Q1 value using an RGB image and a preset first weight; The index anchor point K1 value is generated using the depth image and a preset second weight; Using depth images and preset third weights, extractable content V1 values ​​are generated; Through the formula: A1 = Softmax[(Q1·K1)] ) / d k ]·V1, calculate the first weighted depth feature A1; where, d k Indicates the preset factor; The Q2 value of the query target is generated using the depth image and a preset fourth weight; The index anchor point K2 value is generated using the RGB image and the preset fifth weight; Using RGB images and a preset sixth weight, extractable content V2 values ​​are generated; Through the formula: A2 = Softmax[(Q2·K2)] ) / d k V2, calculate the second weighted depth feature A2; Channel concatenation is performed on the first and second weighted depth features, which are then fed into a 1×1 convolution to compress the number of channels to 128. After upsampling, the images are restored to the preset image resolution, and a pixel-level preset aggregated image is output.

[0047] The module 240 is used to obtain the property test results of the current preset core process through the preset physical property testing equipment on the preset core process.

[0048] The generation module 250 is used to generate a conclusion on IC substrate defects based on the property detection results and the preset aggregated image.

[0049] The screening module 260 is used to determine the nearest IC carrier screening robot arm based on the actual location of the collected IC carrier when the IC carrier defect conclusion is that there is an IC carrier defect; and to use the robot arm to screen out the corresponding IC carrier.

[0050] The above are method embodiments of this application. Based on the same inventive concept, this application also provides an IC substrate defect detection device based on distributed computing. Figure 3 As shown, the device includes: a processor; and a memory storing executable code thereon, which, when executed, causes the processor to perform an IC carrier board defect detection method based on distributed computing as described in the above embodiments.

[0051] Specifically, the server deploys distributed edge nodes on the preset core processes of the IC carrier board; based on the collectors installed on the distributed edge nodes, it collects RGB and depth images of the IC carrier board at the current preset core process; the distributed edge nodes use an attention weight algorithm to fuse the RGB and depth images into a preset aggregated image; through the preset physical property detection equipment on the preset core process, it obtains the property detection results of the current preset core process; based on the property detection results and the preset aggregated image, it generates an IC carrier board defect conclusion; when the IC carrier board defect conclusion is that an IC carrier board defect exists, it determines the nearest IC carrier board screening robot arm based on the actual position of the collected IC carrier board; the robot arm is used to screen out the corresponding IC carrier board.

[0052] In addition, this application embodiment also provides a non-volatile computer storage medium storing executable instructions, which, when executed, implement the IC carrier board defect detection method based on distributed computing as described above.

[0053] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting defects in IC substrates based on distributed computing, characterized in that, The method includes: Deploy distributed edge nodes on the pre-defined core processes of the IC carrier board; Based on the collectors installed on the distributed edge nodes, RGB and depth images of the IC carrier board in the current preset core process are collected; Distributed edge nodes use an attention weight algorithm to fuse RGB images and depth images into a preset aggregated image; The property test results of the current core process are obtained by using the physical property testing equipment pre-set in the core process. Based on the property detection results and the preset aggregated image, a conclusion on the IC substrate defect is generated; When the IC carrier board defect conclusion is that there is an IC carrier board defect, the nearest IC carrier board screening robot arm is determined based on the actual location of the collected IC carrier board; the robot arm is used to screen out the corresponding IC carrier board.

2. The IC carrier board defect detection method based on distributed computing according to claim 1, characterized in that, Deploying distributed edge nodes on the pre-defined core processes of the IC carrier board specifically includes: At the preset core process nodes of the IC carrier board, distributed edge computing nodes are deployed in sequence according to the process flow. Each node corresponds to an independent IC carrier board inspection station. Each edge node is fixedly mounted with a binocular industrial camera and an infrared structured light depth sensor by a rigid bracket. The camera and sensor are triggered synchronously to align the RGB image and depth image on the time axis. The acquisition frequency of the RGB image and depth image is set to 10 frames per second. The trigger signal for the synchronous triggering of the camera and sensor is driven by the pulse of the pipeline encoder so that each IC carrier board is only acquired once by the edge node when passing through the inspection area.

3. The IC carrier board defect detection method based on distributed computing according to claim 1, characterized in that, Distributed edge nodes utilize an attention weight algorithm to fuse RGB and depth images into a pre-defined aggregated image, specifically including: The query target Q1 value is generated using the RGB image and a preset first weight; The index anchor point K1 value is generated using the depth image and a preset second weight; Using depth images and preset third weights, extractable content V1 values ​​are generated; Through the formula: A1 = Softmax[(Q1·K1)] ) / d k ]·V1, calculate the first weighted depth feature A1; where, d k Indicates the preset factor; The Q2 value of the query target is generated using the depth image and a preset fourth weight; The index anchor point K2 value is generated using the RGB image and the preset fifth weight; Using RGB images and a preset sixth weight, extractable content V2 values ​​are generated; Through the formula: A2 = Softmax[(Q2·K2)] ) / d k V2, calculate the second weighted depth feature A2; Channel concatenation is performed on the first and second weighted depth features, which are then fed into a 1×1 convolution to compress the number of channels to 128. After upsampling, the images are restored to the preset image resolution, and a pixel-level preset aggregated image is output.

4. The IC carrier board defect detection method based on distributed computing according to claim 1, characterized in that, By using pre-set physical property testing equipment for the pre-set core process, the property testing results of the current pre-set core process are obtained, specifically including: The testing equipment communicates with edge nodes in real time via an industrial bus; The testing equipment automatically starts when the IC carrier board passes by and collects physical characteristics, including conductivity, thickness, resistance, surface flatness, and whether it is an abnormal area. The sampling point density is no less than 5 per square centimeter. The collection results of the sampling points are bound to the timestamp and spatial coordinates. The initial value for whether it is an abnormal area is empty. The acquisition results are transmitted to the corresponding edge nodes in a structured binary format. The property detection results of sampling points whose acquisition results exceed the preset range are marked as abnormal areas.

5. The IC carrier board defect detection method based on distributed computing according to claim 1, characterized in that, Based on the property detection results and the preset aggregated image, a defect conclusion for the IC carrier board is generated, specifically including: Identify visual defects in pre-defined aggregated images; The property detection results and visual defects are converted into vector data to construct multidimensional feature vectors; The multidimensional feature vector is input into the preset decision rule engine. The decision rule engine performs cross-validation on the visual defects in the preset aggregated image and the physical features in the property detection results based on the logical judgment tree established by the historical defect sample library. When the visual defects and physical features trigger the preset defect combination conditions at the same time, it is determined that there is an IC carrier board defect.

6. The IC carrier board defect detection method based on distributed computing according to claim 1, characterized in that, When the IC carrier board defect conclusion is that an IC carrier board defect exists, the nearest IC carrier board screening robot arm is determined based on the actual location of the collected IC carrier board; the robot arm is used to screen out the corresponding IC carrier boards, specifically including: When the IC carrier board defect conclusion is that there is an IC carrier board defect, the edge node calculates the spatial distance between the real-time coordinates of the IC carrier board and the positions of all robotic arms in the global coordinate system of the pipeline, and selects the nearest robotic arm for screening. After receiving the coordinate command, the robotic arm control unit starts the servo motor to drive the end effector to move to the target position and screen out the corresponding IC carrier board.

7. A defect detection system for IC substrates based on distributed computing, characterized in that, The system includes: The deployment module is used to deploy distributed edge nodes on the preset core processes of the IC carrier board; The acquisition module is used to acquire RGB and depth images of the IC carrier board in the current preset core process based on the acquisition devices installed on the distributed edge nodes. The aggregation module is used by distributed edge nodes to fuse RGB images and depth images into a preset aggregated image using an attention weight algorithm; The acquisition module is used to obtain the property test results of the current preset core process through the preset physical property testing equipment in the preset core process; The generation module is used to generate a conclusion on IC substrate defects based on the property detection results and the preset aggregated image; The screening module is used to determine the nearest IC carrier screening robot arm based on the actual location of the collected IC carrier when the IC carrier defect conclusion is that there is an IC carrier defect; the robot arm is then used to screen out the corresponding IC carrier.

8. The IC carrier board defect detection system based on distributed computing according to claim 7, characterized in that, The aggregation module includes aggregation units. Used to generate the query target Q1 value using an RGB image and a preset first weight; The index anchor point K1 value is generated using the depth image and a preset second weight; Using depth images and preset third weights, extractable content V1 values ​​are generated; Through the formula: A1 = Softmax[(Q1·K1)] ) / d k ]·V1, calculate the first weighted depth feature A1; where, d k Indicates the preset factor; The Q2 value of the query target is generated using the depth image and a preset fourth weight; The index anchor point K2 value is generated using the RGB image and the preset fifth weight; Using RGB images and a preset sixth weight, extractable content V2 values ​​are generated; Through the formula: A2 = Softmax[(Q2·K2)] ) / d k V2, calculate the second weighted depth feature A2; Channel concatenation is performed on the first and second weighted depth features, which are then fed into a 1×1 convolution to compress the number of channels to 128. After upsampling, the images are restored to the preset image resolution, and a pixel-level preset aggregated image is output.

9. An IC substrate defect detection device based on distributed computing, characterized in that, The device includes: processor; And a memory having executable code stored thereon, which, when executed, causes the processor to perform an IC carrier board defect detection method based on distributed computing as described in any one of claims 1-6.

10. A non-volatile computer storage medium, characterized in that, It stores computer instructions, which, when executed, implement an IC carrier board defect detection method based on distributed computing as described in any one of claims 1-6.

Images