AOI equipment defect detection method and system

Abstract

The embodiment of the invention provides an AOI equipment defect detection method and system, and relates to the technical field of AOI equipment defect detection.The method comprises the steps that image collection is conducted on a circuit board in a first illumination mode, a first image is obtained, and a first type of imaging problem area is recognized according to the first image; performing image acquisition on the circuit board by adopting a second illumination mode to obtain a second image, and identifying a second type of imaging problem area according to the second image; according to the first type of imaging problem areas and the second type of imaging problem areas, an illumination adjustment map is generated, and the illumination adjustment map indicates spatial distribution of different imaging problem areas; according to the illumination adjustment map, customized illumination is applied to different imaging problem areas, and a composite illumination field is formed; and performing single image acquisition in the composite illumination field to obtain a complete image of the circuit board. According to the embodiment of the invention, the integrity and accuracy of image information can be improved, and the risk of missing detection and false alarm can be reduced.

Description

Technical Field

[0001] This application relates to the field of AOI equipment defect detection technology, and in particular to an AOI equipment defect detection method and system. Background Technology

[0002] In related technologies, with the continuous improvement of the integration of electronic products, the design of circuit boards has become increasingly complex and dense. Components with highly reflective surfaces are increasingly used on circuit boards, such as shields with polished metal casings, certain types of surface-mount capacitors, or gold-plated connector pins. When AOI equipment illuminates these highly reflective surfaces using traditional vertical or fixed-angle light sources, these highly reflective surfaces produce strong specular reflections, or "glare." In the photographs taken by the camera, these glare areas appear as pure white or overexposed bright spots, completely obscuring the actual details of the component surface, such as the silkscreen characters on the component body and the solder joint patterns on the component edges. When analyzing these areas covered by bright spots, the image processing system cannot extract effective feature information, thus potentially missing defects that should have been detected, resulting in serious missed detections. Simultaneously, the system may also mistakenly identify these intense bright spots themselves as an anomaly, generating false alarms and increasing the burden of manual re-inspection. Summary of the Invention

[0003] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes an AOI equipment defect detection method and system, aiming to improve the integrity and accuracy of image information and reduce the risk of missed detections and false alarms.

[0004] In a first aspect, embodiments of this application provide a method for detecting defects in AOI equipment, including: The circuit board is image acquired using a first illumination method to obtain a first image, and a first type of imaging problem area is identified based on the first image; The circuit board is imaged using a second illumination method to obtain a second image, and the second type of imaging problem area is identified based on the second image. An illumination adjustment map is generated based on the first type of imaging problem area and the second type of imaging problem area, and the illumination adjustment map indicates the spatial distribution of different imaging problem areas; Based on the illumination adjustment map, customized illumination is applied to the different imaging problem areas to form a composite illumination field; Under the composite illumination field, a single image acquisition is performed to obtain a complete image of the circuit board.

[0005] According to some embodiments of this application, the step after forming the composite illumination field further includes: After the composite illumination field is formed, the circuit board is pre-imaged to obtain a pre-image of the circuit board. Analyze the pre-image to identify calibration points or component feature points on the circuit board; Calculate the positional deviation of the circuit board based on the calibration points or component feature points; Based on the positional deviation, the projection area, angle, or brightness of each independently controllable light source in the composite illumination field is adjusted.

[0006] According to some embodiments of this application, the step of analyzing the pre-imaging image and identifying calibration points or component feature points of the circuit board includes: Multi-scale feature extraction is performed on the pre-image to obtain image information at different scales; Based on the image information at different scales, an enhanced feature representation is formed; Based on the enhanced feature representation, the calibration points or component feature points of the circuit board are identified.

[0007] According to some embodiments of this application, the step of identifying calibration points or component feature points of the circuit board based on the enhanced feature representation further includes: Obtain camera intrinsic parameters and distortion parameters, as well as camera attitude parameters relative to the circuit board; The image coordinates of the identified calibration points or component feature points are used to perform distortion correction using the camera intrinsic parameters and distortion parameters to obtain the corrected image coordinates. Based on the camera attitude parameters, the corrected image coordinates are converted into physical coordinates on the circuit board plane through projection transformation; Under a preset physical coordinate system, the calibration point or component feature point is located at the sub-pixel level.

[0008] According to some embodiments of this application, the step of performing multi-scale feature extraction on the pre-imaging image to obtain image information at different scales includes: Edge detection is performed on the pre-image to identify the outlines of components and the boundaries of imaging problem areas in the image; The feature extraction range is determined based on the identified component outlines and the boundaries of the imaging problem area; Within the extraction range, image representations of different resolutions are generated; Feature extraction is performed on the image representations at different resolutions to obtain image information at different scales.

[0009] According to some embodiments of this application, the step of forming enhanced feature representations based on the image information at different scales includes: The image information at different scales is analyzed to obtain the confidence information of the image information at each scale. Based on the confidence information of each scale image information, assign a corresponding weight to each scale image information; The image information at each scale is weighted and summed with the corresponding weights to obtain the fused feature representation. The fused feature representation is subjected to a nonlinear transformation to form an enhanced feature representation.

[0010] According to some embodiments of this application, the step of performing a nonlinear transformation on the fused feature representation to form an enhanced feature representation includes: The fused feature representation is analyzed to obtain information on the type and intensity of residual image interference; Based on the type and intensity information of the residual image interference, a nonlinear transformation function is selected from a preset nonlinear transformation function library; Based on the intensity information, adjust the parameters of the selected nonlinear transformation function; The fused feature representation is transformed using an adjusted nonlinear transformation function to form an enhanced feature representation.

[0011] According to some embodiments of this application, the step of analyzing the fused feature representation to obtain information on the type and intensity of residual image interference includes: The fused feature representation is then divided into local regions; Statistical characteristic analysis is performed on the feature representation of each local region to obtain the statistical parameters of the local region, including brightness distribution, contrast, and texture roughness. Based on the statistical parameters, the type and intensity information of residual image interference are obtained.

[0012] According to some embodiments of this application, after obtaining the type and intensity information of residual image interference based on the statistical parameters, the method further includes: The spatial distribution of residual image interference is obtained based on the type and intensity information of the residual image interference; Obtain the deformation information of the circuit board; Based on the deformation information, the spatial distribution of the residual image interference is geometrically corrected to obtain the corrected spatial distribution, which is then aligned with the deformed circuit board area. The corrected spatial distribution is fused with the real-time position of the circuit board to generate a dynamic interference map; Based on the dynamic disturbance mapping, the effective range and intensity of the nonlinear transformation function are adjusted.

[0013] Secondly, embodiments of this application provide an AOI equipment defect detection system, the system comprising: The image acquisition and problem area identification module is used to acquire images of the circuit board using a first illumination method to obtain a first image, and identify a first type of imaging problem area based on the first image; to acquire images of the circuit board using a second illumination method to obtain a second image, and identify a second type of imaging problem area based on the second image; the illumination adjustment map generation module is used to generate an illumination adjustment map based on the first type of imaging problem area and the second type of imaging problem area, wherein the illumination adjustment map indicates the spatial distribution of different imaging problem areas; The composite illumination field generation module is used to adjust the map according to the illumination and apply customized illumination to the different imaging problem areas to form a composite illumination field.

[0014] The image acquisition module is used to perform a single image acquisition under the composite illumination field to obtain a complete image of the circuit board.

[0015] The technical solution according to the embodiments of this application has at least the following beneficial effects: The embodiments of this invention first use a first illumination method to acquire an image of the circuit board, obtaining a first image, and identify a first type of imaging problem area based on the first image; then use a second illumination method to acquire an image of the circuit board, obtaining a second image, and identify a second type of imaging problem area based on the second image; based on the first type of imaging problem area and the second type of imaging problem area, generate an illumination adjustment map, the illumination adjustment map indicating the spatial distribution of different imaging problem areas; based on the illumination adjustment map, apply customized illumination to the different imaging problem areas to form a composite illumination field; under the composite illumination field, perform a single image acquisition to obtain a complete image of the circuit board. The embodiments of this application can improve the integrity and accuracy of image information and reduce the risk of missed detections and false alarms.

[0016] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0017] The accompanying drawings are used to provide a further understanding of the technical solutions of this application and constitute a part of the specification. They are used together with the embodiments of this application to explain the technical solutions of this application and do not constitute a limitation on the technical solutions of this application.

[0018] Figure 1 This is a flowchart illustrating an AOI equipment defect detection method provided in one embodiment of this application; Figure 2 This is a schematic diagram of an AOI equipment defect detection system provided in one embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical methods, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0020] It should be noted that the meaning of "multiple" (or "more than") in the description of the embodiments of this application refers to two or more, and "greater than," "less than," "exceeding," etc. are understood to exclude the number itself, while "above," "below," "within," etc. are understood to include the number itself. If "first," "second," etc. are used in the description, they are only for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0021] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of singular or plural items. For example, at least one of a, b, and c can represent: the existence of a alone, the existence of b alone, the existence of c alone, the simultaneous existence of a and b, the simultaneous existence of a and c, the simultaneous existence of b and c, or the simultaneous existence of a, b, and c, where a, b, and c can be single or multiple.

[0022] In the description of this application, unless otherwise expressly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0023] Based on the above, this application proposes an AOI equipment defect detection method and system, aiming to improve the integrity and accuracy of image information and reduce the risk of missed detections and false alarms.

[0024] The AOI device defect detection method provided in this application can be applied to a terminal, a server, or software running on either a terminal or a server. In some embodiments, the terminal can be a smartphone, tablet, laptop, desktop computer, etc.; the server can be configured as an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms; the software can be an application that implements the AOI device defect detection method, but is not limited to the above forms.

[0025] This application can be applied to numerous general-purpose or special-purpose computer system environments or configurations. Examples include: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, and distributed computing environments including any of the above systems or devices. This application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific tasks or implement specific abstract data types. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via communication networks. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices. It should be noted that in various specific embodiments of this invention, when processing is required based on data related to the characteristics of an object (e.g., user attributes or sets of attribute information), permission or consent from the corresponding object is obtained first, and the collection, use, and processing of this data comply with relevant laws and standards. Furthermore, when the embodiments of the present invention need to obtain the attribute information of an object, they will obtain the separate permission or separate consent of the corresponding object through pop-up windows or redirection to a confirmation page. After obtaining the separate permission or separate consent of the corresponding object, they will then obtain the relevant data of the object necessary for the embodiments of the present invention to operate normally.

[0026] See Figure 1 , Figure 1This is a flowchart illustrating an AOI equipment defect detection method according to an embodiment of this application. The AOI equipment defect detection method provided in this embodiment includes, but is not limited to, steps S110 to S170, which will be described in detail below.

[0027] Step S110: Use the first illumination method to acquire an image of the circuit board, obtain a first image, and identify the first type of imaging problem area based on the first image; Step S120: Use the second illumination method to acquire images of the circuit board, obtain a second image, and identify the second type of imaging problem area based on the second image; Step S130: Generate an illumination adjustment map based on the first type of imaging problem area and the second type of imaging problem area. The illumination adjustment map indicates the spatial distribution of different imaging problem areas. Step S140: Adjust the map according to the illumination, apply customized illumination to different imaging problem areas to form a composite illumination field; Step S150: Under a composite illumination field, perform a single image acquisition to obtain a complete image of the circuit board.

[0028] It should be noted that the first and second lighting methods are two or more different lighting strategies. They can employ different light source types, projection angles, light intensities, or combinations to address different types of imaging problems on the circuit board. The first lighting method can be a high-angle ring light designed to highlight the surface details of components, but may produce glare in highly reflective areas; the second lighting method can be a low-angle side light designed to reduce glare, but may produce shadows at the edges of components. The first and second types of imaging problem areas refer to areas on the circuit board with poor image quality under different lighting conditions. The first type of imaging problem area may mainly manifest as overexposed glare areas caused by highly reflective components, while the second type of imaging problem area may mainly manifest as shadow areas formed by tall components. Identifying these areas is the basis for subsequently generating a lighting adjustment map. The lighting adjustment map is a spatial mapping that records in detail the specific location and type of different imaging problem areas on the circuit board. This map can be represented in the form of a two-dimensional matrix, where each pixel or region cell stores the corresponding imaging problem type (such as glare, shadow) and its severity information. Customized lighting involves precisely controlling illumination of specific areas on a circuit board based on a lighting adjustment map. This can be achieved through independently controllable light source arrays. For glare areas, local light intensity can be reduced or the projection angle changed; for shadowed areas, local light intensity can be increased or supplemental lighting can be provided from different directions. A composite lighting field is a comprehensive lighting environment optimized for different areas within the entire circuit board detection area, formed through customized lighting. This lighting field is no longer a single, uniform illumination, but a dynamically adjusted combination of illuminations based on the characteristics of each area of ​​the circuit board.

[0029] In one embodiment, the first illumination method can be a high-angle ring illumination, characterized by uniform light that clearly reveals the surface details of most components on the circuit board. A ring light source composed of multiple LED beads can be used, placed around the camera lens and illuminating the circuit board at a near-vertical angle. After acquiring the first image, image processing algorithms can be used to identify first-type imaging problem areas based on brightness thresholds or gradient analysis. When the pixel brightness of a certain area in the image exceeds a preset threshold, that area may be identified as a glare area, i.e., a first-type imaging problem area. A set of strip light sources located to the side or slightly above the circuit board can be used to illuminate the circuit board at a small incident angle. After acquiring the second image, image processing algorithms are similarly used to identify second-type imaging problem areas based on brightness below a threshold or texture loss. When the pixel brightness of a certain area in the image is below a preset threshold, that area may be identified as a shadow area, i.e., a second-type imaging problem area. Subsequently, an illumination adjustment map is generated based on the first and second-type imaging problem areas. The illumination adjustment map indicates the spatial distribution of different imaging problem areas. The first and second images can be registered, and then the identified glare and shadow areas can be superimposed onto the two-dimensional coordinate system of the circuit board to form a spatial distribution map containing different types of problem areas. This map can be a two-dimensional array, with each element recording the imaging problem type (e.g., glare, shadow, normal) and its severity at the corresponding location. Then, if the illumination adjustment map shows glare in a certain area, the corresponding independent controllable light source can be instructed to reduce its brightness or change its projection angle to eliminate the glare; if a shadow exists in a certain area, the corresponding independent controllable light source can be instructed to increase its brightness or provide supplementary lighting from different directions to eliminate the shadow. This can be achieved using an array of multiple independent controllable light sources, each of which can precisely adjust its projection area, angle, and brightness according to the instructions from the illumination adjustment map. Finally, since the composite illumination field has been optimized for imaging problems in different areas of the circuit board, a complete image of the circuit board with clear details and no glare or shadows can be obtained in a single acquisition. This image can be directly used for subsequent defect detection and analysis without complex image fusion or multiple acquisitions.

[0030] It should be noted that pre-imaging is a rapid, typically low-resolution image acquisition of the circuit board using the camera system of the AOI device before the actual single image acquisition, to obtain preliminary image information of the circuit board. Calibration points are pre-designed or known markers on the circuit board with clear geometric features, such as fiducial marks on a PCB. Component feature points are geometric feature points of specific components (such as chips, resistors, capacitors, etc.) on the circuit board, such as corner points, center points, or specific pin positions. Calculating the positional deviation of the circuit board involves comparing the actual positions of the calibration points or component feature points identified in the pre-imaged image with the preset ideal positions to determine the translational deviation of the circuit board in the X and Y directions, the height deviation in the Z-axis direction, and the rotational deviation around the X, Y, and Z axes. This can be achieved through methods such as least squares, iterative nearest point (ICP) algorithms, or geometric transformations based on feature matching. Adjusting the projection area, angle, or brightness of each independent, controllable light source in the composite illumination field involves dynamically adjusting the parameters of each independent light source based on the calculated circuit board position deviation. If the composite illumination field consists of a programmable LED array, the activation area, beam projection angle, or luminous intensity of specific LED units can be adjusted to ensure that customized illumination accurately covers the corresponding imaging problem area on the circuit board, even if the circuit board has slight displacement or rotation.

[0031] In one embodiment, it is assumed that a circuit board is placed on an inspection platform in an AOI device. First, the system generates and initially forms a composite illumination field based on a preset imaging problem area. Then, to ensure illumination accuracy, the system performs a rapid pre-imaging. In the pre-imaging image, the image processing algorithm identifies two reference points A and B on the circuit board. By comparing the pixel coordinates of these two reference points in the pre-imaging image with their expected coordinates in the ideal model, the system calculates that the circuit board currently has a translational deviation of 2 mm to the right and 1 mm upward, as well as a rotational deviation of 0.5 degrees clockwise. Based on these calculated positional deviations, the illumination control unit of the AOI device immediately adjusts the individual controllable LED light source arrays in the composite illumination field. If an LED light source was originally designed to illuminate a specific area in the upper left corner of the circuit board, its projection area will be fine-tuned due to the circuit board's rightward and upward movement. The center of its light spot will be moved 2 mm to the right and 1 mm upward, and its projection angle will also be fine-tuned to compensate for the 0.5-degree rotation, ensuring that the illumination still accurately covers the corresponding upper left corner area of ​​the circuit board. In this way, even if there is a slight placement deviation in the circuit board, the customized lighting can always be applied precisely to the target area, thereby ensuring the quality of the final single image acquisition and providing high-quality image data for subsequent defect detection.

[0032] It should be noted that techniques such as image pyramids, wavelet transforms, or multi-layer convolutional neural networks can be used to generate a series of image representations with different spatial resolutions, thereby acquiring image information at different scales. This image information at different scales can include features such as edges, corners, textures, and shapes, aiming to comprehensively capture the various visual characteristics of calibration points or component feature points on the circuit board, to address potential variations in size, orientation, or illumination. Features at different scales can be weighted and summed, feature stitched, or selectively fused using attention mechanisms to highlight the most discriminative features for the target and suppress noise and redundant information. Image information at different scales can be integrated through weighted fusion, feature stitching, or the use of feature fusion modules in deep learning. Identifying calibration points or component feature points on the circuit board based on enhanced feature representations can be achieved using methods such as template matching, feature point descriptor-based matching (e.g., SIFT, SURF, ORB), or deep learning models (e.g., object detection networks). Calibration points are typically specific geometric markings pre-designed on the circuit board, while component feature points can be component pins, pads, specific corners, or center locations. By leveraging enhanced feature representations, these key points can be located more accurately, maintaining a high recognition rate even in complex backgrounds or under uneven lighting conditions.

[0033] In one embodiment, obtaining camera intrinsic parameters and distortion parameters involves acquiring a set of parameters describing the camera's internal optical characteristics and geometric distortions through a camera calibration process. Camera intrinsic parameters typically include focal length, principal point coordinates, and pixel size, used to project points in three-dimensional space onto a two-dimensional image plane. Distortion parameters describe the radial and tangential distortions produced by the lens, which cause straight lines in the image to appear curved. Simultaneously, obtaining the camera's attitude parameters relative to the circuit board refers to determining the camera's rotational and translational relationship relative to the circuit board in three-dimensional space, typically obtained through external calibration or a preset installation position. Furthermore, by utilizing the pre-acquired camera intrinsic parameters and distortion parameters, the pixel coordinates affected by distortion in the original image can be mapped to ideal image coordinates without distortion, thereby ensuring the accuracy of subsequent coordinate transformations. Thus, based on the camera attitude parameters, the corrected image coordinates are converted into physical coordinates on the circuit board plane through projection transformation. Through perspective projection or homography matrix transformation, the image coordinates are converted into actual physical size coordinates at the millimeter or micrometer level, providing a basis for subsequent precise measurement and positioning. Finally, in the physical coordinate system, sub-pixel-level positioning is performed on calibration points or component feature points to further improve positioning accuracy beyond the limitations of a single pixel. Sub-pixel-level positioning technology, by analyzing pixel grayscale distribution, edge gradient information, or using interpolation algorithms, can improve the positioning accuracy of feature points to below the pixel level, which can be 0.1 pixels or even higher.

[0034] In one embodiment, it is assumed that the AOI device uses an industrial camera for image acquisition. Before the device is put into use, the intrinsic parameter matrix and distortion coefficients of the camera are first obtained using a standard camera calibration board and calibration algorithm (e.g., based on the Zhang Zhengyou calibration method). These parameters are stored in the system's configuration database. When the circuit board is placed in the detection area and pre-imaged, the system identifies calibration points (e.g., specific marker points on the circuit board) or component feature points (e.g., the center of a BGA solder ball) in the image. For the image coordinates of these identified points, the system calls the distortion correction function in the image processing library to correct them using the pre-stored camera intrinsic parameters and distortion parameters, eliminating image distortion caused by lens distortion. Subsequently, in order to convert these corrected two-dimensional image coordinates into actual physical coordinates on the circuit board plane, the system uses pre-calibrated attitude parameters between the camera and the circuit board (e.g., a transformation matrix obtained through hand-eye calibration). By applying a projection transformation (such as homography transformation), the corrected image coordinates are mapped to the physical coordinate system of the circuit board. For example, a point located at (x, y) pixels on an image might correspond to a physical location of (X_mm, Y_mm) millimeters on a circuit board after transformation. Finally, to achieve higher positioning accuracy, the system performs sub-pixel-level positioning on these physical coordinates. For an identified feature point, grayscale interpolation or Gaussian fitting can be performed on its surrounding pixel area to find its true sub-pixel center position. When a feature point is initially located at physical coordinates (10.0mm, 20.0mm), its precise position might be corrected to (10.015mm, 20.023mm) through sub-pixel-level positioning. This high-precision physical coordinate information is then used to accurately calculate the positional deviation of the circuit board and guide the fine adjustment of the projection area, angle, and brightness of each light source in the composite illumination field, ensuring that the illumination accurately covers the target area, thereby optimizing the defect detection effect.

[0035] It should be noted that edge detection on the pre-image aims to accurately identify key structures in the image. Component outlines refer to the geometric boundaries of various electronic components on the circuit board, such as the external shapes of resistors, capacitors, and integrated circuits. The boundaries of the imaging problem area refer to the edges of image anomalies that may occur during the pre-imaging process due to factors such as uneven lighting, reflection, and shadows. Edge detection algorithms, such as the Canny, Sobel, or Laplacian operators, can effectively extract this boundary information, providing accurate localization for subsequent feature extraction. Furthermore, based on the identified component outlines and the boundaries of the imaging problem area, the feature extraction range can be determined. This range is determined to focus the subsequent feature extraction process on the areas most relevant to the identification of calibration points or component feature points, avoiding calculations on irrelevant areas, thereby improving processing efficiency and accuracy. These ranges can be defined based on the envelope rectangle of the component outline or the minimum bounding rectangle of the imaging problem area. Within this extraction range, image representations of different resolutions can be generated. The original pre-image, within the defined feature extraction range, will be processed into multiple image copies with different levels of detail. For example, a series of low-resolution images can be generated from the original high-resolution image through downsampling (such as bilinear interpolation, nearest neighbor interpolation, or average pooling), thereby capturing image features at different scales. Subsequently, feature extraction is performed on the image representations at different resolutions to obtain image information at different scales. This can include, but is not limited to, extracting texture features (such as Gabor features, LBP features), shape features (such as Hu invariant moments, Fourier descriptors), or local descriptors (such as SIFT, SURF). By performing feature extraction on images at different resolutions, a wide range of information, from fine details to macroscopic structures, can be captured, thus providing a more comprehensive and robust data foundation for subsequent feature representation and recognition.

[0036] In one embodiment, analyzing image information at different scales to obtain confidence information for each scale refers to evaluating image features obtained from different resolutions or processing paths to quantify their reliability or effectiveness. Confidence can be calculated based on indicators such as image sharpness, contrast, signal-to-noise ratio, feature point density, or uniformity of distribution. A high confidence level generally indicates that the image information at that scale is clearer, has less noise, or more prominent features.

[0037] In one embodiment, assigning a corresponding weight to each scale of image information based on the confidence level of that image information can be understood as assigning an importance coefficient to each scale of image information based on the evaluated confidence level. Generally, image information with higher confidence levels receives a larger weight, and vice versa. This ensures that high-quality information dominates the subsequent fusion process.

[0038] In one embodiment, if image information at different scales is in the form of feature vectors, these vectors can be linearly combined according to their weights to form a comprehensive feature vector. The aim is to effectively aggregate multi-scale information while taking into account the quality differences at each scale. Nonlinear transformations can be used to enhance the discriminative power of features, suppress residual noise, correct nonlinear distortion, or map features to a more discriminative space. Transformations can be performed using the Sigmoid function, ReLU function, Tanh function, or more complex activation functions to highlight key features and weaken irrelevant information.

[0039] In one embodiment, it is assumed that after multi-scale feature extraction of the pre-image, image information at three different scales is obtained: scale A (high-resolution detail information), scale B (medium-resolution structural information), and scale C (low-resolution global information). First, the image information at these three scales is analyzed by calculating the local contrast, edge sharpness, and response intensity of the feature point detection algorithm for each scale to obtain its confidence information. Assume the calculated confidence scores for scale A are 0.9, scale B is 0.7, and scale C is 0.5. Based on these confidence scores, corresponding weights can be assigned to each scale. Normalization can be used to ensure the sum of the weights is 1, or the confidence scores can be directly used as weights. Assume the assigned weights are WA = 0.45, WB = 0.35, and WC = 0.20. Subsequently, the image information at these three scales (e.g., their feature vectors) is weighted and summed with the corresponding weights to obtain a fused feature representation. Finally, a nonlinear transformation is performed on this fused feature representation. A sigmoid function can be used to map the fused feature values ​​to the range of 0 to 1, while enhancing the non-linear expressive power of the features. If there are some weak but important features in the fused feature representation, the sigmoid function can amplify them, while suppressing some noisy or unimportant features. The resulting enhanced feature representation can more effectively highlight the calibration points or component features of the circuit board and suppress background interference, thus providing high-quality input for subsequent accurate identification.

[0040] It should be noted that residual image interference refers to factors such as noise, artifacts, uneven lighting, shadows, and reflections that are not completely eliminated during image feature fusion, negatively impacting image quality and feature extraction. These interferences may stem from the complexity of the original image acquisition environment, the surface characteristics of the circuit board, or limitations of preprocessing steps. The pre-defined nonlinear transformation function library contains a set of nonlinear mapping functions tailored to different types of image interference and feature enhancement needs. These may include gamma correction functions, logarithmic transformation functions, sigmoid functions, tanh functions, and various custom nonlinear mapping curves. Each function may offer superior processing results for specific types of interference. If a severe low-contrast problem is detected in the image, a logarithmic transformation or sigmoid function might be selected; if there is local overexposure or underexposure, a gamma correction function might be chosen. Furthermore, the parameters of the selected nonlinear transformation function are dynamically adjusted based on the intensity of the residual image interference. For the gamma correction function, the gamma value can be adjusted according to the overall brightness distribution and contrast intensity of the image; for the sigmoid function, its slope and offset can be optimized according to the intensity of the interference to ensure that the transformation can effectively suppress the interference and enhance the target features, while avoiding the introduction of new distortions.

[0041] In one embodiment, when analyzing the fused feature representation, the system detects problems such as excessively high brightness in local areas of the image (e.g., due to strong reflections from component surfaces) and low overall contrast. At this point, the type of residual image interference is identified as localized overexposure and overall low contrast, with intensity information quantifying the severity of these problems. Based on this information, the system can select a combined transformation function from a pre-defined library of nonlinear transformation functions. For example, it can first apply a local gamma correction function (gamma value less than 1) for the bright areas to suppress overexposure, and then apply a global sigmoid function to enhance overall contrast. Specifically, the gamma value of the gamma correction function can be dynamically adjusted based on the average brightness intensity of the bright areas; for example, the higher the brightness, the smaller the gamma value. The slope and center point of the sigmoid function can be optimized based on the overall image's grayscale histogram distribution and contrast intensity to ensure effective stretching of the grayscale range without loss of detail.

[0042] It's important to note that local region partitioning of the fused feature representation involves dividing the entire fused feature representation image or data into multiple smaller, independent regions. These regions can be regular grid-like partitions, uniformly dividing the image into M rows and N columns; or they can be adaptive partitions, clustering or segmenting based on image content or feature distribution to ensure relatively uniform image interference characteristics within each local region. This decomposes the complex global image interference problem into multiple manageable local problems, enabling refined analysis of specific interference in different regions. Furthermore, statistical characteristic analysis is performed on the feature representation of each local region to obtain statistical parameters, including brightness distribution, contrast, and texture roughness. Brightness distribution can be characterized by calculating the pixel gray-level histogram, average brightness, and median brightness of the local region; contrast can be measured by calculating the standard deviation of pixel gray-level values ​​and the difference between the maximum and minimum values ​​within the local region; texture roughness can be obtained through features such as the energy, entropy, and contrast of the gray-level co-occurrence matrix (GLCM), or through frequency domain analysis methods such as wavelet transform and Fourier transform. These statistical parameters can quantitatively describe the image quality and interference characteristics within a local region.

[0043] In one embodiment, when the brightness distribution of a local area exhibits a significant peak concentrated at the high end, coupled with low contrast, it may indicate overexposure or highlight clipping interference. The intensity can be measured by the peak brightness or the size of the clipped area. When the contrast of a local area is extremely low and the texture is blurred, it may indicate haze or low-light interference, the intensity of which can be measured by the degree of contrast reduction. When the texture roughness is abnormally high and irregular, it may indicate random noise interference, the intensity of which can be measured by the variance or signal-to-noise ratio of the noise. By establishing a mapping relationship between statistical parameters and known interference types and their intensities, accurate identification and quantification of residual image interference can be achieved.

[0044] In one embodiment, after obtaining information on the type and intensity of residual image interference, this information can be further analyzed to determine the specific distribution of the residual image interference in the image space, i.e., to obtain the spatial distribution of the residual image interference. This is achieved by constructing an interference heatmap or interference density map, the purpose of which is to accurately depict the intensity and type of interference at different locations.

[0045] In one embodiment, the circuit board may undergo minor physical deformations during production, transportation, or inspection, such as bending, twisting, or warping. These deformations alter the actual positions of feature points on the circuit board, thus affecting the correspondence between the interference area in the image and the actual physical area. Deformation information can be obtained by using a laser scanner to perform a 3D scan of the circuit board to acquire its precise geometry, or by inferring the deformation through the positional deviation of preset calibration points under different lighting conditions. Specifically, geometrically correcting the spatial distribution of interference in the residual image based on the deformation information to align it with the deformed circuit board area means adjusting the previously acquired spatial distribution of interference based on the assumption of an ideal plane according to the actual deformation of the circuit board. If the circuit board has local protrusions, the image of that area will be locally magnified or compressed, and the corresponding spatial distribution of interference also needs to undergo corresponding geometric transformations to ensure that the interference information accurately matches the true physical position of the circuit board. This can be achieved by applying image registration algorithms or geometric transformation matrices, with the aim of eliminating the interference positional deviation caused by circuit board deformation. Furthermore, the spatial distribution of residual image interference after deformation correction is combined with the real-time position information of the circuit board acquired by the AOI device during inspection. The real-time position of the circuit board may vary slightly due to placement deviations or mechanical movements. Through fusion, a dynamically updated mapping map that accurately reflects the interference distribution in the current state of the circuit board can be generated. This mapping map can guide subsequent image processing in real time. Therefore, the application area and intensity of the nonlinear transformation function are adjusted according to the dynamic interference map. The nonlinear transformation function selected and adjusted in the above embodiment can now be adjusted more finely based on the precise spatial distribution and intensity information indicated by the dynamic interference map. For areas with high interference intensity, a stronger nonlinear transformation can be applied; for areas with weak or no interference, the transformation can be weakened or not applied at all, thereby achieving local, adaptive compensation for image interference and avoiding over-processing or under-processing.

[0046] See Figure 2 , Figure 2 This is a schematic diagram of an AOI equipment defect detection system provided in one embodiment of this application. The AOI equipment defect detection system 200 includes: The image acquisition and problem area identification module 210 is used to acquire images of the circuit board using a first illumination method, obtain a first image, and identify a first type of imaging problem area based on the first image; to acquire images of the circuit board using a second illumination method, obtain a second image, and identify a second type of imaging problem area based on the second image; the illumination adjustment map generation module is used to generate an illumination adjustment map based on the first type of imaging problem area and the second type of imaging problem area, and the illumination adjustment map indicates the spatial distribution of different imaging problem areas; The composite illumination field generation module 220 is used to adjust the map according to the illumination and apply customized illumination to different imaging problem areas to form a composite illumination field.

[0047] The image acquisition module 230 is used to perform a single image acquisition under a composite illumination field to obtain a complete image of the circuit board.

[0048] It should be noted that the information interaction and execution process between the above modules are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0049] Those skilled in the art will understand that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0050] The foregoing has provided a detailed description of the preferred embodiments of this application. However, this application is not limited to the above-described embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of this application. All such equivalent modifications or substitutions are included within the scope defined in this application.

Claims

1. An AOI apparatus defect detection method, characterized by, The method comprises the following steps: adopting a first light mode to collect images of the circuit board to obtain a first image, and identifying a first type of imaging problem area according to the first image; adopting a second light mode to collect images of the circuit board to obtain a second image, and identifying a second type of imaging problem area according to the second image; generating a light adjustment map according to the first type of imaging problem area and the second type of imaging problem area, the light adjustment map indicating the spatial distribution of different imaging problem areas; applying customized light to the different imaging problem areas according to the light adjustment map to form a composite light field; under the composite light field, performing single image collection to obtain a complete image of the circuit board.

2. The AOI apparatus defect detection method of claim 1, wherein, The step of forming the composite light field further comprises: after the composite light field is formed, pre-imaging the circuit board to obtain a pre-imaging image of the circuit board; analyzing the pre-imaging image to identify calibration points or component feature points of the circuit board; calculating the position deviation of the circuit board according to the calibration points or component feature points; adjusting the projection area, angle or brightness of each independently controllable light source in the composite light field according to the position deviation.

3. The method of claim 2, wherein the method further comprises: The step of analyzing the pre-imaging image to identify calibration points or component feature points of the circuit board comprises: performing multi-scale feature extraction on the pre-imaging image to obtain image information of different scales; forming an enhanced feature representation according to the image information of different scales; identifying calibration points or component feature points of the circuit board according to the enhanced feature representation.

4. The AOI apparatus defect detection method of claim 3, wherein, The step of identifying calibration points or component feature points of the circuit board according to the enhanced feature representation further comprises: obtaining camera intrinsic and distortion parameters, and camera pose parameters relative to the circuit board; performing distortion correction on the image coordinates of the identified calibration points or component feature points using the camera intrinsic and distortion parameters to obtain corrected image coordinates; converting the corrected image coordinates to physical coordinates on the circuit board plane through projection transformation according to the camera pose parameters; performing sub-pixel level positioning of the calibration points or component feature points in a pre-set physical coordinate system.

5. The AOI apparatus defect detection method of claim 3, wherein, The step of performing multi-scale feature extraction on the pre-imaging image to obtain image information of different scales comprises: performing edge detection on the pre-imaging image to identify the boundaries of component outlines and imaging problem areas in the image; determining a feature extraction range according to the identified boundaries of component outlines and imaging problem areas; generating image representations of different resolutions within the extraction range; performing feature extraction on the image representations of different resolutions to obtain image information of different scales.

6. The AOI apparatus defect detection method of claim 3, wherein, The step of forming an enhanced feature representation according to the image information of different scales comprises: analyzing the image information of different scales to obtain confidence information of each scale of image information; assigning a corresponding weight to each scale of image information according to the confidence information of each scale of image information; performing weighted summation of each scale of image information and the corresponding weight to obtain a fused feature representation; performing a non-linear transformation on the fused feature representation to form an enhanced feature representation.

7. The AOI apparatus defect detection method of claim 6, wherein, The step of performing a non-linear transformation on the fused feature representation to form an enhanced feature representation comprises: performing an analysis on the fused feature representation to obtain type and intensity information of residual image interference; selecting a non-linear transformation function from a pre-set non-linear transformation function library according to the type of residual image interference and the intensity information; adjusting parameters of the selected non-linear transformation function according to the intensity information; transforming the fused feature representation using the adjusted non-linear transformation function to form an enhanced feature representation.

8. The AOI apparatus defect detection method of claim 7, wherein, The step of performing an analysis on the fused feature representation to obtain type and intensity information of residual image interference comprises: dividing the fused feature representation into local regions; performing a statistical property analysis on the feature representation of each local region to obtain statistical parameters of the local region, the statistical parameters including brightness distribution, contrast, and texture roughness; obtaining type and intensity information of residual image interference according to the statistical parameters.

9. The AOI apparatus defect detection method of claim 8, wherein, After the step of obtaining type and intensity information of residual image interference according to the statistical parameters, the method further comprises: obtaining a spatial distribution of residual image interference according to the type and intensity information of residual image interference; obtaining deformation information of the circuit board; performing geometric correction on the spatial distribution of residual image interference according to the deformation information to obtain a corrected spatial distribution, and aligning the corrected spatial distribution with the deformed circuit board region; fusing the corrected spatial distribution with a real-time position of the circuit board to generate a dynamic interference map; adjusting an action area and intensity of the non-linear transformation function according to the dynamic interference map.

10. An AOI apparatus defect detection system, comprising: The system comprises: an image acquisition and problem region identification module, configured to acquire images of a circuit board using a first light mode to obtain a first image, and identify a first type of imaging problem region according to the first image; acquire images of the circuit board using a second light mode to obtain a second image, and identify a second type of imaging problem region according to the second image; a light adjustment map generation module, configured to generate a light adjustment map according to the first type of imaging problem region and the second type of imaging problem region, the light adjustment map indicating a spatial distribution of different imaging problem regions; a composite light field generation module, configured to apply customized light to the different imaging problem regions according to the light adjustment map to form a composite light field; an image acquisition module, configured to acquire a complete image of the circuit board through single image acquisition under the composite light field.

Images