PCB board detection method and device based on artificial intelligence

Abstract

The present application discloses an artificial intelligence-based PCB board detection method and device. The solution obtains a multi-view original image set of a PCB board; uses an adaptive region generation network to process the target layer PCB image to generate a candidate region set containing potential defects; performs multi-scale context enhancement processing on the candidate region based on the impedance change characteristics inside and outside the candidate region; performs feature extraction on the enhanced candidate region to extract defect features with spatial perception capabilities, and performs multi-dimensional embedded coding processing on the position information of the enhanced candidate region and the board layer identification to generate position features with process relevance; performs cross-modal fusion processing on the defect features and the position features to generate fused features with process context perception; performs defect detection processing on the PCB board based on the fused features to obtain the defect type of the PCB board and the associated process link information, which can improve the accuracy and reliability of PCB board detection.

Description

Technical Field

[0001] The present application relates to the fields of PCB (printed circuit board) detection technology and artificial intelligence technology, and in particular to an artificial intelligence-based PCB board detection method and device. Background Art

[0002] As an essential and fundamental component of modern electronic devices, the performance and quality of printed circuit boards (PCBs) are crucial to the proper functioning of the entire electronic device. The PCB manufacturing process involves numerous complex steps, including but not limited to circuit printing, etching, drilling, and component installation. Due to the numerous steps and extremely high technical requirements, and the interference of various factors, such as slight variations in raw materials, fluctuations in the precision of production equipment, and environmental factors, PCBs are prone to various defects. Common defects include short circuits, opens, incomplete etching, offset drilling, and poor component soldering. Once these defects exist on PCBs, they can lead to serious problems such as functional failure, performance degradation, reduced stability, or even complete inoperability in electronic devices. Therefore, effective PCB testing is crucial to ensuring the quality of electronic devices.

[0003] Traditional PCB inspection methods have numerous shortcomings. While manual inspection can reveal some obvious defects based on the inspector's experience, this method is extremely inefficient and susceptible to subjective factors such as fatigue and lack of concentration, making it difficult to guarantee the reliability and accuracy of inspection results. Furthermore, traditional single-method inspection methods based on machine vision or electrical testing often have limitations when dealing with the complex structures and diverse defect types of PCBs, making it equally difficult to guarantee the reliability and accuracy of inspection results. Summary of the Invention

[0004] The main purpose of this application is to provide an artificial intelligence-based PCB board detection method and device, which can improve the accuracy and reliability of PCB board detection.

[0005] To achieve the above objectives, an embodiment of the present invention provides a PCB board detection method based on artificial intelligence, the method comprising:

[0006] Acquire a set of multi-view original images of a PCB board, each original image being associated with a view angle identifier and a board surface layer identifier, wherein the view angle identifier represents an observation angle of the imaging device relative to the PCB board, and the board surface layer identifier indicates a currently inspected PCB board layer;

[0007] Using an adaptive region generation network to process the target layer PCB image to generate a candidate region set containing potential defects, wherein the candidate region set includes at least one candidate region;

[0008] Performing multi-scale context enhancement processing on the candidate region according to impedance change characteristics inside and outside the candidate region to obtain an enhanced candidate region;

[0009] Performing feature extraction on the enhanced candidate area to extract defect features with spatial perception capabilities, and performing multi-dimensional embedding coding processing on the position information of the enhanced candidate area and the panel surface layer identifier to generate position features with process relevance;

[0010] Performing cross-modal fusion processing on the defect feature and the position feature to generate a fusion feature with process context awareness;

[0011] Defect detection processing is performed on the PCB board according to the fusion feature to obtain the defect type of the PCB board and related process link information.

[0012] Accordingly, an embodiment of the present application further provides a PCB board detection device based on artificial intelligence, the device comprising:

[0013] An image acquisition module is configured to acquire a set of original images of a PCB from multiple perspectives, each original image being associated with a perspective identifier and a board layer identifier. The perspective identifier represents the observation angle of the imaging device relative to the PCB, and the board layer identifier indicates the PCB layer currently being inspected.

[0014] An image processing module is configured to process a target-level PCB image using an adaptive region generation network to generate a candidate region set containing potential defects, wherein the candidate region set includes at least one candidate region;

[0015] an enhancement module, configured to perform multi-scale context enhancement processing on the candidate region according to impedance variation characteristics inside and outside the candidate region to obtain an enhanced candidate region;

[0016] a feature encoding module for performing feature extraction on the enhanced candidate area to extract defect features with spatial perception capability, and performing multi-dimensional embedding encoding processing on the position information of the enhanced candidate area and the panel surface layer identifier to generate position features with process relevance;

[0017] A fusion module, configured to perform cross-modal fusion processing on the defect feature and the position feature to generate a fusion feature with process context awareness;

[0018] The detection module is used to perform defect detection processing on the PCB board according to the fusion feature to obtain the defect type of the PCB board and related process link information.

[0019] The artificial intelligence-based PCB board detection method provided in the embodiment of the present application can obtain a multi-view original image set of the PCB board, which can fully cover the information of the PCB board from multiple angles and different board layers, overcoming the limitations of single-view or single-level detection, thereby greatly reducing the possibility of missing defects and improving the accuracy and reliability of PCB board detection. The adaptive region generation network is used to process the target layer PCB image, which can focus on the area where defects may exist in a targeted manner, accurately locate potential defects, effectively reduce the amount of data for subsequent processing, and improve detection efficiency. Multi-scale context enhancement processing is performed based on the impedance change characteristics inside and outside the candidate area, which fully considers the electrical characteristics of the PCB board. Since the electrical performance of the PCB board is closely related to the presence of defects, this can more accurately highlight the characteristics of the potential defect area and improve the accuracy of defect judgment. When extracting features from the enhanced candidate area, defect features with spatial perception capabilities are extracted, and the position information and the board layer identification are multi-dimensionally embedded and encoded to generate position features with process relevance. The defect features can be combined with the process structure and layer information of the PCB board, so that defects can be accurately associated with specific process links in subsequent detection. Finally, the defect features and position features are cross-modally fused to generate fused features with process context awareness. Based on this, defect detection of PCB boards can more accurately determine the defect type of PCB boards and their association with manufacturing process links, thereby improving the overall accuracy and reliability of PCB board detection and ensuring the performance and reliability of electronic devices using these PCB boards. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0021] Figure 1 Schematic diagram of a scenario of a PCB board detection method based on artificial intelligence in an embodiment of the present application;

[0022] Figure 2 A flowchart of an artificial intelligence-based PCB board detection method provided in an embodiment of the present application;

[0023] Figure 3 A schematic diagram of the multi-dimensional embedded coding process provided in an embodiment of the present application;

[0024] Figure 4 A schematic diagram of the regional enhancement process provided in an embodiment of the present application;

[0025] Figure 5A schematic diagram of the structure of a PCB board detection device based on artificial intelligence provided in an embodiment of the present application;

[0026] Figure 6 Another structural diagram of the artificial intelligence-based PCB board detection device provided in an embodiment of the present application;

[0027] Figure 7 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without making creative efforts are within the scope of protection of this application.

[0029] The embodiments of the present application provide a PCB board detection method and device based on artificial intelligence, which will be described in detail below.

[0030] PCB board testing refers to the process of using a series of specific technical means and methods to detect and identify various defects that may occur in PCB boards during manufacturing, assembly and use.

[0031] During the manufacturing process, PCB boards can experience a variety of defects. For example, circuit defects, such as an open circuit, mean that a circuit that should be connected is broken, preventing current from flowing normally. A short circuit occurs when a circuit that should not be connected is connected, which can cause electrical failures. Etching can also involve over-etching or under-etching. Over-etching can make the circuit thinner or even disconnected, while under-etching can reduce the spacing between circuits, increasing the risk of short circuits. Drilling-related defects include drill offset, which can affect the subsequent component installation accuracy and may result in improper component installation or poor electrical connections. There are also defects related to component installation, such as poor soldering, including cold solder joints (where solder joints appear to be connected but are actually poorly connected), and short-circuit solder joints (where connections between adjacent solder joints are not expected).

[0032] During the assembly process, component installation errors may occur, such as installing the wrong component in a specific location or installing a component with incorrect polarity.

[0033] During use, PCBs may also malfunction due to environmental factors such as temperature, humidity, and electromagnetic interference. For example, prolonged exposure to high temperatures can cause the performance of certain materials on the PCB to degrade, leading to circuit aging and component degradation.

[0034] The purpose of PCB board testing in this application is to ensure the quality and reliability of PCB boards, ensure the normal operation of electronic devices using these PCB boards, improve the overall quality of products, and reduce the repair costs and product recall risks of electronic equipment failures caused by PCB board defects.

[0035] like Figure 1 As shown, a PCB board detection system is provided, which includes a PCB board to be detected, an imaging device, an electrical testing device and a detection and analysis computer device, wherein the imaging device, the electrical testing device and the detection and analysis computer device are connected via a wired or wireless network.

[0036] An imaging device is used to capture multi-view raw image data of the PCB to be inspected. This imaging device can be a high-precision industrial camera capable of capturing images of the PCB from various angles, ensuring that each raw image accurately reflects the PCB's surface condition. After capturing the images, the imaging device transmits the raw image data to a computer for inspection and analysis. Each raw image is associated with a viewpoint identifier (indicating the imaging device's observation angle relative to the PCB) and a board layer identifier (indicating the PCB layer currently being inspected).

[0037] Electrical testing equipment is used to obtain impedance variation characteristics within and outside the candidate area of the PCB board. This electrical testing equipment can use a professional impedance tester to obtain relevant data through contact or non-contact testing methods, and then send this data to the detection and analysis computer equipment.

[0038] The detection and analysis computer equipment processes the received data. First, an adaptive region generation network is used to process the target layer PCB image (the target layer image is selected from the multi-view original images obtained by the imaging device) to generate a set of candidate regions containing potential defects. Next, based on the impedance variation characteristics within and outside the candidate regions obtained from the electrical testing equipment, the candidate regions are subjected to multi-scale context enhancement to obtain enhanced candidate regions. Feature extraction is then performed on the enhanced candidate regions. A specific algorithm is used to extract spatially aware defect features. Furthermore, the location information of the enhanced candidate regions is combined with the board layer identifiers through multi-dimensional embedding coding to generate process-relevant location features. The defect features and location features are then cross-modally fused to generate fused features that are aware of the process context. Finally, defect detection is performed on the PCB based on the fused features to determine the defect type and associated process steps. The detection and analysis computer equipment can also output and display the defect detection results so that the inspectors can obtain the results. The results contain detailed information such as the defect location and type of the PCB board and its association with the manufacturing process. This helps manufacturers accurately determine the problem and take targeted improvement measures to improve the production quality of PCB boards.

[0039] In the embodiment of the present invention, the detection and analysis computer device can be a smart phone, a camera, a desktop computer, a tablet computer, a server, or other devices.

[0040] refer to Figure 2 , Figure 2 This is a flow chart of an artificial intelligence-based PCB board detection method provided in an embodiment of the present application. The execution subject of the method can be a computer device, which can be a single computer device or a cluster of multiple computer devices. The computer device can be a terminal device or a server. The artificial intelligence-based PCB board detection method provided in an embodiment of the present application specifically includes:

[0041] S101: Acquire a set of original images of a PCB from multiple perspectives, each original image being associated with a perspective identifier and a board layer identifier. The perspective identifier represents the observation angle of the imaging device relative to the PCB, and the board layer identifier indicates the PCB layer currently being inspected.

[0042] In one embodiment, a multi-view original image set of a PCB board can be acquired by an imaging device, which includes various types of imaging devices, such as industrial-grade charge-coupled device (CCD) cameras and complementary metal oxide semiconductor (CMOS) cameras.

[0043] The multi-view original image set refers to a set of images obtained by photographing the PCB board from multiple different angles, wherein each original image is associated with a viewpoint identifier and a board surface layer identifier.

[0044] The viewing angle indicator represents the imaging device's viewing angle relative to the PCB. PCBs are multi-layered electronic components, and different viewing angles can capture different information. For example, an image of a six-layer PCB taken from directly above, perpendicular to the board surface, clearly reveals the layout of the components on the surface. Meanwhile, an image taken from the side at a 45-degree angle reveals information such as the connections between different layers and the routing of side traces.

[0045] The board layer identifier indicates the PCB layer currently being inspected. The hierarchical structure of a PCB is crucial for its manufacturing and functional implementation, and different layers may have different circuit patterns, copper foil thicknesses, and component layouts. For example, in a multi-layer PCB, the power layer primarily provides power transmission, typically featuring simpler circuit patterns and thicker copper foil to carry higher currents. The signal layer, primarily used for signal transmission, features more complex circuitry and requires higher signal integrity. The board layer identifier clearly identifies the layer corresponding to the captured image.

[0046] The embodiment of the present application adopts a multi-perspective, labeled image acquisition method to comprehensively and meticulously obtain various information of the PCB board, providing a rich raw data basis for subsequent detection and avoiding inaccurate detection results due to missing information.

[0047] S102: Using an adaptive region generation network to process the target layer PCB image to generate a set of candidate regions containing potential defects.

[0048] Among them, the adaptive region generation network is a network architecture built based on a deep learning algorithm, which can contain multiple network layers, such as convolutional layers, pooling layers, and fully connected layers. Among them, the convolutional layer is a network layer that extracts image features by sliding a convolution kernel on the image. For example, when a 3x3 convolution kernel slides on the image, it will perform a convolution operation with the local area of the image to extract the features of the area. The pooling layer is a network layer that downsamples the feature map output by the convolution layer to reduce the amount of data. Common pooling methods include maximum pooling and average pooling. For example, maximum pooling selects the point with the largest feature value in a small area as the representative of the area. The fully connected layer is a network layer that integrates and classifies the features extracted by the previous network layer, in which each neuron is connected to all neurons in the previous layer.

[0049] In this embodiment, the target-level PCB image is an image of a specific layer selected from the acquired multi-view original image set. The selection rules can be determined based on the specific inspection requirements or specific process focus. For example, if the focus is on defects in the power supply layer, the PCB image corresponding to the power supply layer will be used as the target-level PCB image.

[0050] The embodiment of the present application adopts an adaptive region generation network that can automatically adjust various parameters in the network according to the characteristics of the input PCB image. For example, when processing PCB board images of different complexities and types, the network can adaptively adjust the weights of the convolution kernel, the sampling strategy of the pooling layer, and the connection weights of the fully connected layer. Generating a set of candidate regions containing potential defects is the main goal of this step. These candidate regions are areas where defects may exist that the network identifies based on the learned normal and abnormal image feature patterns of the PCB board. For another example, for short circuit defects on PCB boards, the network can mark areas where short circuits may exist as candidate regions by learning the connectivity of normal lines and different image features of short circuit areas, such as color changes, line interruptions or abnormal connections. The embodiment of the present application can efficiently and accurately locate areas where defects may exist through adaptive processing, reduce the amount of data for subsequent processing, and improve the efficiency of the entire detection process and the accuracy of locating potential defect areas.

[0051] S103: Perform multi-scale context enhancement processing on the candidate area according to impedance change characteristics inside and outside the candidate area.

[0052] In the circuit structure of a PCB board, impedance is a physical quantity that reflects the circuit's resistance to alternating current and is affected by a variety of factors. For example, the circuit material (such as copper purity), circuit geometry (length, width, thickness), spacing between circuits, and the electrical characteristics of surrounding components all affect impedance. Impedance variation characteristics within and outside the candidate region can be obtained using specialized electrical testing equipment, such as a high-precision impedance analyzer. In one embodiment, the electrical testing equipment can accurately measure the impedance value and its variation within and outside the candidate region using contact or non-contact measurement methods.

[0053] Among them, multi-scale context enhancement processing is an enhancement method that comprehensively considers information at different scales. The scale in the embodiment of the present application can be understood as the range or granularity of observation, which can include size. In one embodiment, at a small scale, such as a single circuit or a tiny circuit unit as the observation scale, very subtle feature changes in the candidate area can be captured. For example, slight corrosion on the surface of the circuit may cause slight changes in local impedance, which can be detected at a small scale.

[0054] In one embodiment, at a large scale, with the observation scale including multiple circuit units or the entire candidate area and its surrounding environment, it is possible to pay attention to the mutual influence relationship between the candidate area and the surrounding lines and components. For example, there may be multiple line branches and components in a larger candidate area, and the failure of a certain component may affect the impedance distribution of the entire area. This relationship at a large scale can be identified. Through this multi-scale processing method, the electrical state of the candidate area can be analyzed more comprehensively and in-depth, thereby more effectively enhancing the features related to potential defects. The embodiment of the present application can improve the sensitivity to the potential defect features in the candidate area by combining the electrical characteristics of the PCB board and adopting multi-scale analysis, which helps to identify defects more accurately and provide a more accurate information basis for subsequent detection.

[0055] S104: Feature extraction is performed on the enhanced candidate area to extract defect features with spatial perception capabilities, and multi-dimensional embedding coding is performed on the position information of the enhanced candidate area and the board surface layer identification to generate position features with process relevance.

[0056] In one embodiment, a convolutional neural network (CNN) can be used to perform preliminary feature extraction on the enhanced candidate regions. CNN is a deep learning model widely used in image processing. It consists of multiple convolutional layers, pooling layers, and fully connected layers. A convolutional layer extracts image features by sliding a convolution kernel across the image. For example, when a 5x5 convolution kernel is slid across the image, it convolves with a local region of the image, extracting features from that region. A pooling layer downsamples the feature map output by the convolutional layer to reduce the amount of data. Common pooling methods include max pooling and average pooling. For example, average pooling calculates the average of all feature values within a small region as a representative representation of that region. A fully connected layer integrates and classifies features extracted by previous network layers. Each neuron is connected to all neurons in the previous layer. This structure enables CNN to effectively extract features from images. After preliminary feature extraction of the enhanced candidate regions, a basic feature map is generated. Spatial pooling is then performed on this basic feature map. Spatial pooling mainly includes max pooling and average pooling. For example, in max pooling, the point with the largest eigenvalue within a small area is selected as the representative of that area. This reduces the amount of data while retaining the most significant spatial information, resulting in a spatially aggregated feature. Finally, the spatially aggregated feature is input into a fully connected layer for mapping. Each neuron in the fully connected layer is connected to all neurons in the previous layer. By adjusting the connection weights, the spatially aggregated feature is mapped into a spatially aware defect feature. This defect feature can reflect the spatial distribution of defects, such as whether the defects are concentrated in a local area or dispersed throughout the candidate area, as well as spatial information such as the defect's shape and size.

[0057] In the embodiment of the present application, the position information refers to information such as the specific coordinate position of the enhanced candidate area on the PCB board. For example, in a rectangular coordinate system established with a corner of the PCB board as the origin, the position information can be expressed as the value range of the horizontal coordinate and the vertical coordinate, and may also include height information in three-dimensional space (if the three-dimensional structure of the PCB board is considered). Among them, the board surface layer identifier clearly defines the PCB board level where the candidate area is located, and different levels may be different in terms of the manufacturing and functional implementation of the PCB board. For example, in a multi-layer PCB board, the inner layer is mainly used for signal transmission and power distribution, and the outer layer may be more used for the connection and physical support of components.

[0058] In one embodiment, multidimensional embedded coding can be performed in a variety of ways. For example, the position information and board layer identifiers can be quantized and converted. For example, the position information can be normalized according to the overall dimensions of the PCB board, and the board layer identifiers can be converted to a specific encoding format, such as integer encoding based on the hierarchical order. The dimensional structure of the multidimensional encoding can then be determined based on various factors, such as the PCB board's process characteristics and testing requirements.

[0059] For example, one dimension can be set to represent horizontal position information, another dimension to represent vertical position information, and another dimension to represent the board surface layer identifier. Additional dimensions can also be set as needed to represent other process-related information, such as specific process area identifiers. The quantized position information and converted board surface layer identifiers are embedded into the corresponding dimensions according to the set dimensional structure to obtain the encoded information for each dimension. Finally, the encoded information for each dimension is integrated according to its weight in the process association to generate a position feature with process relevance.

[0060] In the embodiment of the present application, the position feature can reflect the relationship between the position of the enhanced candidate area in the PCB board and the process. For example, a specific position and layer may be more likely to have a specific type of defect in a certain process link, such as a short circuit defect may be more likely to occur in a specific position near the power layer during the power connection process. The technical effect of this part of the operation is that by extracting defect features with spatial perception capabilities, the spatial characteristics of the defect can be more accurately described, and the generation of position features with process relevance helps to establish the relationship between the defect and the process link in subsequent detection, thereby improving the accuracy and pertinence of the detection.

[0061] S105: Perform cross-modal fusion processing on the defect features and the position features to generate fusion features with process context awareness.

[0062] Cross-modal fusion is a fusion method that effectively integrates different types of features. Specifically, defect features can be integrated with position features. In one embodiment, a fusion mapping relationship is first constructed based on the process structure information and process flow information of the PCB board.

[0063] The PCB's process structure information can include circuit layouts at different levels, the distribution of different component types, and electrical connections in different areas. The process flow information covers every step from initial PCB substrate manufacturing to final component installation, including information on process steps such as printing circuits, etching, drilling, copper plating, and soldering.

[0064] For another example, in the etching process, the accuracy and depth of etching may affect the quality of the circuits in different areas and layers, and thus be associated with the types of defects that may occur. For different process structures and processes, different defect types and locations may have different correlations. The embodiment of the present application can analyze a large amount of PCB board sample data, which may contain various known defect types, defect locations, board levels, and corresponding process links, and then combine it with professional knowledge in the field of PCB board manufacturing to construct a fusion mapping relationship that can reflect the correlation weights of different defect characteristics and position characteristics in different process links. The fusion mapping relationship can be expressed as a matrix or a complex function model.

[0065] In one embodiment, after establishing a fusion mapping management system, a fusion operation can be performed on defect features and position features based on the established fusion mapping relationship. Assuming the defect feature vector is Fd, the position feature vector is Fb, and the fusion mapping relationship matrix is M, the fused feature vector Ff = M*[Fd:Fb] (using vector concatenation as an example), where [:] represents vector concatenation.

[0066] This embodiment uses a feature fusion operation to combine defect features and position features according to a predetermined weight relationship, so that the fused features can comprehensively consider the defect's spatial attributes and relationship with the process. For example, if a defect feature has a strong correlation with the position feature in a specific process, the corresponding weight in the fusion mapping relationship will be higher, so that this correlation can be fully reflected in the fusion operation.

[0067] Finally, the fused features are adjusted based on process context perception. Due to the complexity of the PCB board manufacturing process, different process links may have different requirements for the weights of defect features and position features in actual detection. For example, in the welding process link, the position feature may be more important for judging defects near the welding point. At this time, it is necessary to appropriately increase the weight ratio of the position feature in the fused feature. Through this adjustment, the generated fused feature can better reflect the defect situation of the current PCB board under a specific process link, and has a stronger process context perception capability. The embodiment of the present application can use cross-modal fusion processing to integrate the advantages of defect features and position features, and adjust according to the process context, so that the fused features finally generated can more accurately reflect the defect type of the PCB board and the associated process link, thereby improving the accuracy and reliability of the entire detection method and providing a more accurate and effective feature basis for subsequent defect detection processing.

[0068] S106 , performing defect detection processing on the PCB board according to the fusion feature to obtain the defect type of the PCB board and associated process link information.

[0069] In an embodiment of the present application, a classification model may be used to perform defect detection processing on the PCB board based on the fusion features, thereby improving the accuracy and reliability of defect detection.

[0070] A classification model is a model built on a machine learning algorithm. Its purpose is to classify data based on input features (here, fused features). For example, in image recognition, a classification model can determine whether an object in an image is a cat or a dog based on input image features. In PCB inspection, fused features can be used to determine the defect type and associated process steps.

[0071] There are many types of classification models, including support vector machines (SVMs), which separate different categories of data by finding an optimal hyperplane. For example, given two types of data points on a two-dimensional plane, an SVM will find a line (a hyperplane in higher-dimensional space) that separates the two types of points as closely as possible. A decision tree is also a classification model, similar to a tree-structured decision-making process. For example, factors such as weather (sunny, rainy, cloudy) and temperature (high, medium, low) can be used to determine whether outdoor activities are suitable. In PCB inspection, different elements of the fused features can be used to determine defect types and process steps.

[0072] In one embodiment, a pre-built and trained classification model can be constructed. This classification model is constructed based on a large amount of PCB sample data, which can include known fused features, corresponding defect types, and associated process information. For example, for a PCB sample with a short circuit defect caused by poor soldering during the soldering process, its corresponding fused features may include electrical features related to the short circuit (such as impedance anomalies in a specific area) and positional features related to the soldering process (such as features near solder joints). These fused features will be marked as related to the short circuit and the soldering process.

[0073] After the classification model is trained, the fused features of the PCB to be inspected can be input into the classification model. The model will analyze and judge the fused features based on its internally learned patterns. For example, if certain feature values in the fused features closely match previously learned feature values that indicate short circuit defects and specific process steps (such as soldering), for example, if the electrical features in the fused features indicate extremely low impedance in a certain area, and the position features indicate that the area is close to a solder joint, the model will determine that the current PCB board has a short circuit defect and that it is related to the soldering process.

[0074] The embodiments of the present application can accurately identify the defect types and associated process links of PCB boards by utilizing fusion features and a pre-trained classification model, thereby improving the production quality of PCB boards and reducing the generation of defective products. It also helps to improve the efficiency and reliability of the entire electronic equipment production process.

[0075] In one embodiment, reference Figure 3 , performing multi-dimensional embedding coding processing on the position information of the enhanced candidate area and the board surface layer identifier to generate a position feature with process relevance may specifically include:

[0076] S301: quantize the position information of the enhanced candidate area and convert the board layer identifier into a coding identifier.

[0077] In one embodiment, the enhanced position information of the candidate area refers to the positioning-related data of the area on the PCB board. For example, in an actual PCB board inspection scenario, the PCB board can be regarded as a two-dimensional plane with a length and width. If the lower left corner is the coordinate origin (0, 0) and the upper right corner coordinates are (X_max, Y_max), the candidate area position information may be the coordinate range of a rectangular area, such as the upper left corner coordinates are (x1, y1) and the lower right corner coordinates are (x2, y2).

[0078] Quantization is used to convert the position information into a form that facilitates subsequent operations. Because the original coordinate values are continuous real numbers, they are difficult to process and integrate with other features. This can be accomplished by dividing the PCB board's length and width into intervals with a specific step size. For example, X_max is divided into n intervals with a step size of Δx, and Y_max is divided into m intervals with a step size of Δy. Then, x1 and x2 are converted into interval numbers, and similarly, y1 and y2, to obtain quantized position information. The board layer identifier is used to distinguish different layers within a multilayer PCB board. For example, a four-layer PCB board may have different layers, such as signal layer, power layer, and ground layer. Simple codes such as 1 for signal layer and 2 for power layer can be used. To integrate this with the position information, it can be converted into an integer code ranging from 0 to L-1 based on the layer order or functional importance. At least two of these dimensions correspond to the enhanced candidate region's position information in at least two directions on the PCB board, and at least one dimension corresponds to the board layer identifier. This multi-dimensional design allows for comprehensive information integration. The technical benefit of this step is that different types of information can be processed within the same framework, laying the foundation for subsequent operations and improving information processing efficiency and accuracy.

[0079] In one embodiment, the dimensions include a horizontal dimension, a vertical dimension, and a layer identification dimension. The horizontal dimension corresponds to the position information of the enhanced candidate area in the horizontal direction of the PCB board, the vertical dimension corresponds to the position information of the PCB board in the vertical direction, and the layer identification dimension corresponds to the board surface layer identification. For example, in a PCB board with multiple parallel circuits, the horizontal position is related to factors such as the signal transmission direction and the spacing between adjacent circuits. If there is a short circuit or open circuit defect in the horizontal direction, the horizontal dimension information can determine the position of the defect in the horizontal direction, and then establish a connection with the process links such as horizontal circuit etching and welding. The horizontal dimension works in conjunction with the vertical dimension and the layer identification dimension. The vertical dimension determines the vertical position range. The combination of the two can accurately locate the position of the candidate area on the plane. Combined with the layer identification dimension, the specific level in the multi-layer PCB board can be clarified, thereby comprehensively describing the position and level information of the candidate area.

[0080] The vertical dimension corresponds to the vertical position information of the PCB board. In a multi-layer PCB board structure, the vertical position information involves the relationship between different layers, such as interlayer connections, via positions and other factors. If there is an interlayer short circuit or via blockage defect in the vertical direction, the vertical dimension information can determine the vertical position of the defect. This is very important for analyzing process links related to the vertical direction, such as lamination and drilling. The vertical dimension and the horizontal dimension jointly determine the position of the candidate area on the PCB board plane, and together with the board layer identification dimension, describe the position information in the entire PCB board structure.

[0081] S302: Embed the quantized position information and coding identifier into corresponding dimensions to obtain coding information of each dimension.

[0082] In this embodiment, the predetermined coding rules can be formulated based on multiple factors such as the structural characteristics of the PCB board, detection requirements, and subsequent processing convenience.

[0083] In one embodiment, for the horizontal dimension, if the PCB board is divided into n intervals horizontally, the quantized horizontal position information can be directly embedded according to the interval serial number. For example, in a specific detection, if the quantized horizontal position information indicates that the starting horizontal coordinate of the candidate area is in the third interval and the ending horizontal coordinate is in the fifth interval, then these two interval serial numbers are recorded in the horizontal dimension to accurately reflect the horizontal position characteristics of the candidate area.

[0084] The same principle applies to the vertical dimension. The quantized vertical position information is embedded according to the interval sequence number. For example, if the starting vertical coordinate is in the second interval and the ending vertical coordinate is in the fourth interval, these two interval sequence numbers are embedded in the vertical dimension. For the board layer identification dimension, the converted coding identifier is directly embedded. For example, if the coding identifier is 1, indicating the power layer, 1 is embedded in this dimension.

[0085] In the embodiment of the present application, the coded information of each dimension can be combined to fully describe the position and hierarchical information of the enhanced candidate area. The horizontal dimension reflects the horizontal position range, the vertical dimension reflects the vertical position range, and the board layer identification dimension indicates the board surface level. These coded information can construct the basis of position features with process relevance, organize different types of information in an orderly manner, provide an accurate data basis for the subsequent generation of position features, and help improve the analysis ability of the correlation between defect locations and process links.

[0086] S303 : Integrate the coded information of each dimension according to the weight of each dimension in the process association to generate a position feature with process association.

[0087] The weight of each dimension in the process association reflects the degree of correlation between that dimension and the PCB manufacturing process, and its determination can take into account multiple factors. For example, in certain process steps, horizontal position is more critical to defect generation and detection. For example, in the circuit printing process, when horizontal precision control of the printing equipment is difficult, the horizontal dimension may have a higher weight in defect detection related to the printing process. In the lamination process, when vertical alignment and interlayer connections are prone to problems, the vertical dimension may have a higher weight in the lamination process. The weight of the board layer identification dimension depends on the importance of each layer in the manufacturing process and the degree of defect susceptibility. For example, the power supply layer involves power supply, and defects have a greater impact. Therefore, the board layer identification dimension may be assigned a higher weight in the process steps related to power supply. The weight can be represented by a real number between 0 and 1 and is adjustable. The weight may change due to process improvements, changes in inspection requirements, or analysis of more sample data.

[0088] In one embodiment, the integration process can be performed using a weighted summation approach. Assuming the horizontal dimension encoding information is h, the vertical dimension is v, and the layer identification dimension is l, with weights a, b, and c, respectively, the integrated process-relevant position feature P = a*h+b*v+c*l. This generated position feature can better reflect the relationship between the position of the enhanced candidate region on the PCB and the process, thereby improving the accuracy and specificity of PCB inspection.

[0089] In this application, the coding dimension may include a horizontal dimension, a vertical dimension, and a layer identification dimension and their corresponding relationship. The horizontal dimension corresponds to the position information of the candidate area in the horizontal direction of the PCB board after enhancement. In the PCB board layout, the horizontal position information is very important for understanding the relationship between the candidate area and the circuits and components of the horizontal layout. For example, in a PCB board with multiple parallel circuits, the horizontal position is related to factors such as the signal transmission direction and the spacing between adjacent circuits. If there is a short circuit or open circuit defect in the horizontal direction, the horizontal dimension information can determine the position of the defect in the horizontal direction, and then establish a connection with the process links such as horizontal line etching and welding. The horizontal dimension works in conjunction with the vertical dimension and the layer identification dimension. The vertical dimension determines the vertical position range. The combination of the two can accurately locate the position of the candidate area on the plane. Combined with the layer identification dimension, the specific level in the multi-layer PCB board can be clarified, thereby comprehensively describing the candidate area position and level information. The vertical dimension corresponds to the position information in the vertical direction of the PCB board. In a multi-layer PCB board structure, the vertical position information involves the relationship between different layers, such as interlayer connections, via locations, and other factors. If there is an interlayer short circuit or via blockage defect in the vertical direction, the vertical dimension information can determine the vertical position of the defect. This is very important for analyzing process links related to the vertical direction, such as lamination and drilling. The vertical dimension and the horizontal dimension jointly determine the position of the candidate area on the PCB board plane, and together with the board layer identification dimension, describe the position information in the entire PCB board structure.

[0090] In one embodiment, reference Figure 4 , for step S103, the following methods can be used to implement context enhancement processing, thereby improving the accuracy and reliability of detection. Specifically including:

[0091] S401: Extracting impedance variation characteristics inside and outside a candidate region, and performing frequency domain analysis on the candidate region according to the impedance variation characteristics to determine a key signal path corresponding to the candidate region.

[0092] In this application, impedance is an electrical characteristic of a PCB circuit that reflects the circuit's resistance to alternating current. For a detailed explanation, please refer to the above description.

[0093] In an embodiment of the present application, after obtaining the impedance change characteristics, it is necessary to perform frequency domain analysis on the candidate area. In one embodiment, frequency domain analysis can be an analysis method that converts the signal from the time domain to the frequency domain. Among them, the time domain describes the change of the signal over time, while the frequency domain focuses on the different frequency components contained in the signal. In PCB board testing, by performing frequency domain analysis on the impedance change characteristics, a deeper understanding of the transmission characteristics of the signal in the circuit can be achieved. For example, a fast Fourier transform (FFT) algorithm is used to convert the time domain signal containing impedance change information into a frequency domain signal. In the frequency domain, the frequency distribution, amplitude and other information of the signal can be clearly seen. Through this analysis, the frequency components that have a significant impact on the circuit performance can be found, because different frequency components have different transmission characteristics on the PCB board and the degree to which they are affected by circuit components.

[0094] Based on the results of the above frequency domain analysis, the critical signal paths corresponding to the candidate regions can be further identified. Critical signal paths refer to signal transmission paths on a PCB that significantly impact circuit functionality and performance. For example, in high-frequency circuits, certain signal paths may involve important signal processing steps (such as high-frequency signal amplification and filtering), and changes in their impedance can significantly impact the performance of the entire circuit. By analyzing the relationship between different frequency components in the frequency domain and the PCB layout and components, these critical signal paths can be identified. For example, if the signal amplitude within a certain frequency range varies significantly in a specific region, and this region is connected to a critical circuit component (such as a chip pin), then this connection path is likely a critical signal path. Identifying critical signal paths provides an important basis for subsequent region enhancement processing, enabling more targeted processing of candidate regions containing critical signal paths, thereby improving detection accuracy and effectiveness.

[0095] In one embodiment, short-time Fourier transform processing may be performed on the impedance change characteristics of the candidate area to generate a time-frequency diagram; and energy analysis may be performed on the time-frequency diagram to generate a key signal path.

[0096] In this embodiment, the short-time Fourier transform (STFT) is a time-frequency analysis method used to analyze non-stationary signals. In the context of PCB board inspection, the impedance change characteristic of the candidate area is a signal that changes with time or spatial position. Because the circuit signals on the PCB board may have time-varying characteristics, for example, in different operating states or different circuit areas, the impedance changes will exhibit complex dynamic characteristics. The STFT divides the signal into shorter time segments (or spatial segments, depending on the signal representation method) and then performs a Fourier transform on each segment. Using this method, the time and frequency information of the signal can be obtained simultaneously, unlike the traditional Fourier transform, which only obtains the frequency information of the entire signal and loses the time information.

[0097] After performing STFT processing on the impedance change characteristics of the candidate area, the embodiment of the present application generates a time-frequency diagram. The time-frequency diagram is a two-dimensional representation, in which one dimension represents time (or spatial position) and the other dimension represents frequency. The amplitude of each point in the time-frequency diagram represents the signal strength at the corresponding time (or spatial position) and frequency. For example, in a PCB board with multi-layer wiring and multiple circuit functions, different circuit modules may generate signal changes of different frequencies at different times or different positions, and these changes are reflected in the impedance.

[0098] The time-frequency diagrams generated by the STFT in the embodiments of the present application can clearly demonstrate the distribution of these changes. For example, taking a PCB area containing multiple clock signals of different frequencies, the time-frequency diagram shows the energy concentration areas corresponding to the different clock frequencies, and the distribution of these areas in time (or space) is also accurately presented. This provides a comprehensive information foundation for subsequent energy analysis and the identification of critical signal paths.

[0099] In one embodiment, signals on critical signal paths within a PCB circuit typically have high energy. This is because critical signal paths often carry signals crucial to circuit functionality, such as those that play a key role in signal transmission and data processing. In a time-frequency graph, energy can be measured by calculating the square of the amplitude corresponding to each frequency point and time point (or spatial point). By performing energy analysis on the entire time-frequency graph, it is possible to determine which areas have high energy concentrations.

[0100] In the embodiment of the present application, after determining the energy distribution in the time-frequency diagram, the critical signal path can be generated. Specifically, the signal path corresponding to the area with higher energy concentration is likely to be the critical signal path. For example, in a PCB board with a complex power management circuit, the distribution and conversion signal of the power supply is very critical. In the time-frequency diagram, if the frequency area related to these power signals shows high energy, then the signal path traced along these energy concentration areas can be determined as the critical signal path. The method of generating critical signal paths based on energy analysis in the embodiment of the present application can accurately identify signal paths that are of great significance in terms of circuit function and performance, and provides an important basis for further PCB board detection, such as detecting potential defects on critical paths, thereby improving the pertinence and effectiveness of detection.

[0101] S402 : Construct an elliptical extension area according to the critical signal path and the component distribution density of the candidate area, wherein the long axis direction of the elliptical extension area is consistent with the main wiring direction corresponding to the candidate area.

[0102] In this embodiment, component distribution density is an important indicator for describing the characteristics of the candidate area. It reflects the distribution of components within a unit area. To calculate the component distribution density, it is necessary to first determine the area of the candidate area, as well as the number of components in the area or the number of valid features related to the components (for example, it can be indirectly represented by the number of valid pixels in the component edge detection results). For example, in a candidate area of a PCB board, if a large number of component outline pixels are detected through image analysis, and the area of the area is relatively small, then the component distribution density of this area is high.

[0103] Furthermore, the embodiment of the present application constructs an elliptical extension area based on consideration of the critical signal path and component distribution density. The construction of the elliptical extension area is based on a comprehensive consideration of the circuit characteristics and physical layout of the PCB board. The long axis direction of the elliptical extension area is consistent with the main wiring direction corresponding to the candidate area. This is because on the PCB board, the wiring direction is often closely related to the signal transmission direction. Setting the long axis direction to be consistent with the main wiring direction can better cover the area related to signal transmission. For example, if the critical signal path is wired in the horizontal direction, the long axis direction of the elliptical extension area will be set to the horizontal direction.

[0104] The ellipse shape is also adjusted based on component density. If the component density is high, the expanded area may need to be reduced to avoid including excessive irrelevant areas. If the density is low, the expanded area can be expanded to ensure that relevant areas where defects may be present are included. This construction method allows for more precise location and coverage of areas where defects may be present, improving inspection efficiency and accuracy.

[0105] In one embodiment, step S402 may be implemented by the following steps:

[0106] Performing principal component analysis on the key signal path to determine the main wiring direction of the candidate area;

[0107] Obtaining the minimum safe distance between adjacent lines in the target layer where the candidate area is located;

[0108] Constructing an initial elliptical extension area according to the main wiring direction and the minimum safety distance, wherein the major axis direction of the initial elliptical extension area is consistent with the main wiring direction, and the minor axis length is determined according to the minimum safety distance between adjacent lines;

[0109] The initial elliptical expansion area is adjusted according to the component distribution density of the candidate area to obtain an elliptical expansion area.

[0110] Principal component analysis (PCA) is a statistical method used to analyze the relationships between variables in a dataset. In PCB inspection scenarios, critical signal paths contain a wealth of data reflecting their layout characteristics on the PCB. PCA can be used to process data. For example, a matrix containing data related to the critical signal paths is first constructed. The data points may include information such as the coordinates and electrical characteristics of different locations along the path. The covariance matrix of this matrix is then calculated, reflecting the correlations between the data variables. The eigenvalues and corresponding eigenvectors of the covariance matrix are then calculated. The direction indicated by the eigenvector corresponding to the maximum eigenvalue is the primary routing direction of the candidate area.

[0111] In actual PCB layout, the primary routing direction indicates the main trend of circuit routing within a candidate area. For example, on a PCB with a specific functional module, a candidate area involving a specific functional circuit can be analyzed through principal component analysis of its key signal paths to determine the primary routing direction of the circuit routing in that area to achieve functional requirements and adhere to design rules. This provides an important basis for the subsequent construction of the elliptical expansion area, helping to accurately cover areas related to the circuit function and thus improve detection accuracy.

[0112] In PCB manufacturing and design, the minimum safe spacing between adjacent traces is a critical parameter for ensuring proper operation and preventing electrical failures. From an electrical performance perspective, excessively small spacing between adjacent traces can increase capacitive coupling, leading to signal crosstalk and compromised signal integrity. This is particularly true in high-speed digital circuits, where the rapid transitions of signal edges make small spacing highly susceptible to signal interference. From a manufacturing perspective, this spacing is limited by the precision of manufacturing equipment and process capabilities. If the designed spacing falls short of manufacturing capabilities, manufacturing defects can occur. Determining this minimum safe spacing requires reference to PCB design documents and manufacturing specifications. During the design phase, engineers determine this spacing based on circuit electrical parameters such as voltage, current, and signal frequency, as well as the manufacturing process. This information is stored in PCB design files, such as Gerber files. Some PCB inspection equipment can approximate this value through physical measurement, such as using optical or electron microscopes followed by image analysis software. However, this measurement is subject to various factors and should be verified by comparing it with the design documentation. When constructing an elliptical extension area, the minimum safe spacing between adjacent traces determines the length of the minor axis of the elliptical extension area, ensuring that the extension area does not violate safe spacing requirements and encompasses areas where defects may exist.

[0113] In the embodiment of the present application, the component distribution density of the candidate area reflects the density of the components in the area, and this density has a correlation with the adjustment of the elliptical expansion area. When the component distribution density is high, it means that there are many components in the area and the circuit structure is complex. At this time, the elliptical expansion area should not be over-expanded, because over-expansion may introduce circuit elements or lines that are not closely related to the current candidate area, increasing the detection complexity and false positive rate. On the contrary, when the component distribution density is low, the circuit structure is relatively simple, and there is more space where potential defects may exist. The elliptical expansion area needs to be appropriately expanded to ensure that all possible defect areas are covered. After the density levels such as high, medium, and low density areas are determined according to the component distribution density, the major axis or minor axis of the initial elliptical expansion area is adjusted. For example, the long axis or minor axis length may be reduced in high-density areas, and the long axis or minor axis length may be increased in low-density areas. The adjusted elliptical expansion area can better adapt to the component distribution density characteristics, thereby improving the accuracy and efficiency of detection.

[0114] In one embodiment, the calculation of component distribution density can be implemented in the following manner:

[0115] Obtaining the area and the number of valid pixels of the candidate area, wherein the number of valid pixels is the valid pixels in the component edge detection result of the candidate area;

[0116] The component distribution density of the candidate area is calculated according to the area and the number of effective pixels, wherein the component distribution density is the ratio of the number of component outline pixels per unit area.

[0117] The area of a candidate region is a geometric property used in PCB inspection, providing a basis for calculating component density. For example, in a PCB image represented as a pixel matrix, the area of the candidate region can be determined by defining the boundary pixels within the candidate region. By calculating the number of pixels within the boundary and combining this with the physical area represented by each pixel, the candidate region's area can be approximated.

[0118] For example, if each pixel represents a certain physical area, the area can be calculated once the number of pixels within the boundary is determined. The number of effective pixels refers to the number of pixels in the component edge detection results for the candidate area. During component edge detection, specific edge detection algorithms, such as the Canny edge detection algorithm, are used to identify component edges, and these edge pixels are considered effective pixels. This algorithm calculates the image gradient amplitude and direction, and then uses non-maximum suppression and double thresholding to obtain edge pixels. Counting these edge pixels yields the number of effective pixels. This number, combined with the area of the candidate area, can be used to accurately calculate component distribution density.

[0119] Among them, the component distribution density is defined as the proportion of the number of component outline pixels per unit area, and its calculation formula is to divide the number of effective pixels by the area of the candidate area. This calculation method can reflect the density of components in the candidate area. For example, given the area of the candidate area and the number of effective pixels, the component distribution density can be calculated by substituting them into the formula. In PCB board inspection, the component distribution density helps to gain a deeper understanding of the complexity of the circuit and the possibility of potential defect distribution in the candidate area. For example, in an embodiment, in an area with a high component distribution density, there are more interactions and connection points between components, and defects such as poor welding and short circuits are more likely to occur; in a low-density area, more attention is paid to the integrity of the line and the connection with surrounding components. Calculating the component distribution density can provide a basis for subsequent adjustment of the elliptical expansion area according to the density and defect detection.

[0120] In an embodiment of the present application, the density level of the candidate area can be determined based on the density of the component distribution, such as high, medium, and low density areas, to facilitate the classification and processing of different density situations. Among them, the division of density levels can be determined based on the type of PCB board, detection requirements, and empirical values. For example, for a specific multi-layer PCB board, through experiments and actual detection experience, when the component distribution density is higher than a certain value, it is a high-density area, in a certain range it is a medium-density area, and below a certain value it is a low-density area. The use of this density level division helps to simplify the subsequent adjustment operation of the elliptical expansion area. Different density levels correspond to different adjustment strategies, making the adjustment process more systematic and targeted.

[0121] Density levels are inherently linked to PCB design and functionality. Component density varies across different functional areas of a PCB. For example, the area surrounding the chip is denser due to pin connection requirements, while the power distribution area, primarily housing large capacitors and inductors, has a relatively lower density. By categorizing density levels, we can conduct inspections and analyses based on the characteristics of each area, improving accuracy and efficiency and providing a deeper understanding of the overall function and structure of the PCB.

[0122] After dividing the density levels, different strategies can be used to adjust the major axis or minor axis of the initial elliptical expansion area at different density levels. Specifically, in high-density areas, due to the dense components, in order to avoid the expansion area containing too many irrelevant areas, the major axis or minor axis will be shrunk and adjusted. For example, when the initial major axis and minor axis are specific values, the values of the major axis and minor axis may be reduced by a certain proportion in the high-density area. In the medium-density area, the adjustment range is relatively small, and the major axis or minor axis is appropriately fine-tuned according to the specific situation. In the low-density area, because there is more room for potential defects, the major axis or minor axis will be increased. Through this adjustment based on the density level, the resulting elliptical expansion area can better adapt to the component distribution characteristics of the candidate area, and more accurately locate areas where defects may exist in subsequent inspections, thereby improving detection accuracy.

[0123] S403: performing resolution optimization processing on the elliptical extended region to obtain an enhanced candidate region.

[0124] In this embodiment, the resolution of the elliptical extension area is optimized to improve detection accuracy within that area. In PCB inspection, resolution directly impacts the ability to detect minor defects. For example, lower resolution may prevent accurate identification of small broken circuits or damaged components. The elliptical extension area is constructed through the previous steps, and its initial resolution may not meet the requirements for accurate detection of potential defects due to approximations or considerations of various factors during the construction process.

[0125] Resolution optimization can be performed using a variety of methods, such as interpolation algorithms. Interpolation algorithms can add new pixels to existing ones, thereby increasing image resolution. For the elliptical expansion area, resolution optimization can more clearly display circuit details within the area, including wiring and component details. This helps to more accurately extract features from the area, resulting in an enhanced candidate region. The enhanced candidate region has improved resolution, enabling more effective detection of potential defects. For example, it can more clearly identify minor flaws in the circuit or minor damage on the surface of a component, thereby improving the sensitivity and accuracy of the entire PCB inspection method for defect detection.

[0126] The embodiment of the present application can also use an adaptive interpolation method to optimize the resolution of the elliptical extension area according to the density level of the candidate area to obtain an enhanced candidate area. The adaptive interpolation method is used to optimize the resolution of the elliptical extension area. The method can determine the interpolation method and parameters based on the local features of the image. Different regions in the PCB board image have different features. For example, the texture and contrast of the component area and the circuit area are different. The adaptive interpolation method can use a more accurate bilinear interpolation method in detail-rich areas such as the component edge and adjust parameters according to features such as edge gradient to retain details. In the smooth circuit area, a simple nearest neighbor interpolation method is used to improve computational efficiency. By resampling the pixels in the elliptical extension area and increasing the number of pixels to improve resolution, the enhanced candidate area obtained can more clearly display circuit details, which helps to detect various defects such as minor damage to the surface of components, minor defects in circuits, poor connection between components and circuits, etc., thereby improving the detection capability and accuracy of the entire PCB board detection method.

[0127] Accordingly, in order to better implement the above method, the embodiment of the present application also provides a PCB board detection device based on artificial intelligence. Figure 5As shown, the display defect detection device includes an image acquisition module 501, an image processing module 502, an enhancement module 503, a feature encoding module 504, a fusion module 505 and a detection module 506, which are specifically as follows:

[0128] Image acquisition module 501, for acquiring a set of original images of a PCB from multiple perspectives, each original image being associated with a perspective identifier and a board layer identifier. The perspective identifier represents the observation angle of the imaging device relative to the PCB, and the board layer identifier indicates the PCB layer currently being inspected.

[0129] An image processing module 502 is configured to process the target layer PCB image using an adaptive region generation network to generate a candidate region set containing potential defects, wherein the candidate region set includes at least one candidate region;

[0130] An enhancement module 503 is configured to perform multi-scale context enhancement processing on the candidate region according to impedance variation characteristics inside and outside the candidate region to obtain an enhanced candidate region;

[0131] A feature encoding module 504 is configured to extract features from the enhanced candidate area to extract defect features with spatial perception capabilities, and perform multi-dimensional embedding encoding processing on the position information of the enhanced candidate area and the panel layer identifier to generate position features with process relevance;

[0132] A fusion module 505 is configured to perform cross-modal fusion processing on the defect feature and the position feature to generate a fusion feature with process context awareness;

[0133] The detection module 506 is used to perform defect detection processing on the PCB board according to the fusion feature to obtain the defect type of the PCB board and related process link information.

[0134] In one embodiment, the feature encoding module 504 is configured to:

[0135] Quantizing the position information of the enhanced candidate region to obtain quantized position information in different directions of the PCB; and converting the board surface layer identifier into a coded identifier in a coding format; wherein at least two dimensions correspond to the position information of the enhanced candidate region in at least two directions of the PCB, and at least one dimension corresponds to the board surface layer identifier;

[0136] Embedding the quantized position information and the coding identifier into the corresponding dimension to obtain coding information of each dimension;

[0137] The coded information of each dimension is integrated according to the weight of each dimension in the process association to generate a position feature with process association, wherein the weight represents the importance of the corresponding dimension in the process association.

[0138] In one embodiment, the dimensions include a horizontal dimension, a vertical dimension, and a board layer identification dimension, the horizontal dimension corresponds to the position information of the enhanced candidate area in the horizontal direction of the PCB board, the vertical dimension corresponds to the position information of the PCB board in the vertical direction, and the board layer identification dimension corresponds to the board surface layer identification; the feature encoding module 504 is used to: embed the quantized position information and the encoding identification into the corresponding dimensions to obtain encoding information of each dimension, including: according to a predetermined encoding rule, embedding the quantized horizontal direction position information of the PCB board into the horizontal dimension, embedding the quantized vertical direction position information of the PCB board into the vertical dimension, and embedding the encoding identification into the layer identification dimension.

[0139] In one embodiment, the fusion module 505 is specifically configured to:

[0140] Constructing a fusion mapping relationship based on the process structure information and process flow information of the PCB board;

[0141] Performing a fusion operation on the defect feature and the position feature according to the fusion mapping relationship to obtain a fusion-operated feature;

[0142] Process context-aware adjustments are performed on the fused features to generate process context-aware fused features.

[0143] In one embodiment, the enhancement module 503 is specifically configured to:

[0144] Extracting impedance change characteristics inside and outside the candidate area, and performing frequency domain analysis on the candidate area based on the impedance change characteristics to determine a key signal path corresponding to the candidate area;

[0145] Constructing an elliptical extension area according to the critical signal path and the component distribution density of the candidate area, wherein the long axis direction of the elliptical extension area is consistent with the main wiring direction corresponding to the candidate area;

[0146] A resolution optimization process is performed on the elliptical extended region to obtain an enhanced candidate region.

[0147] In one embodiment, the enhancement module 503 is specifically configured to:

[0148] Performing short-time Fourier transform processing on the impedance change characteristics of the candidate area to generate a time-frequency graph;

[0149] Energy analysis is performed on the time-frequency diagram to generate key signal paths.

[0150] In one embodiment, the enhancement module 503 is specifically configured to:

[0151] Performing principal component analysis on the key signal path to determine the main wiring direction of the candidate area;

[0152] Obtaining the minimum safe distance between adjacent lines in the target layer where the candidate area is located;

[0153] Constructing an initial elliptical extension area according to the main wiring direction and the minimum safety distance, wherein the major axis direction of the initial elliptical extension area is consistent with the main wiring direction, and the minor axis length is determined according to the minimum safety distance between adjacent lines;

[0154] The initial elliptical expansion area is adjusted according to the component distribution density of the candidate area to obtain an elliptical expansion area.

[0155] In one embodiment, reference Figure 6 The device further includes a density calculation module 507, specifically configured to:

[0156] Obtaining the area and the number of valid pixels of the candidate area, wherein the number of valid pixels is the valid pixels in the component edge detection result of the candidate area;

[0157] The component distribution density of the candidate area is calculated according to the area and the number of effective pixels, wherein the component distribution density is the ratio of the number of component outline pixels per unit area.

[0158] In one embodiment, the enhancement module 503 is specifically used to determine the density level of the candidate area based on the component distribution density, and the density level includes: high, medium, and low density areas; adjust the major axis or minor axis of the initial elliptical extension area according to the density level to obtain an elliptical extension area; and use an adaptive interpolation method to optimize the resolution of the elliptical extension area according to the density level of the candidate area to obtain an enhanced candidate area.

[0159] The implementation of each of the above modules can be specifically referred to the above method embodiments, which will not be described in detail here. The technical effects achieved by each module and device can be referred to the description of the above method embodiments.

[0160] It should be noted that, in specific implementations, the above modules can be arbitrarily combined, integrated into one or more modules, or implemented as independent entities. Furthermore, the above modules can be implemented in the form of hardware or software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. The aforementioned storage medium can be a read-only memory, a magnetic disk, or an optical disk, etc.

[0161] like Figure 7 As shown, an embodiment of the present application further provides a computer device 60, characterized in that it includes a processor 601 and a memory 602, wherein the memory 602 stores a computer program, and when the computer program is executed by the processor 601, the processor 601 performs the steps of any of the methods described above.

[0162] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0163] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0164] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0165] In one aspect, an embodiment of the present application provides a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method provided in one aspect of the embodiment of the present application.

[0166] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "comprising" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0167] Those skilled in the art will appreciate that the modules and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0168] The methods and related devices provided by the embodiments of the present application are described with reference to the method flow charts and / or structural diagrams provided by the embodiments of the present application. Specifically, each process and / or block in the method flow charts and / or structural diagrams, as well as the combination of processes and / or blocks in the flow charts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including the instruction device, which implements the function specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the process. Figure 1 The flow or flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0169] The above disclosure is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A PCB board detection method based on artificial intelligence, characterized in that: The method comprises: Acquire a set of multi-view original images of a PCB board, each original image being associated with a view angle identifier and a board surface layer identifier, wherein the view angle identifier represents an observation angle of the imaging device relative to the PCB board, and the board surface layer identifier indicates a currently inspected PCB board layer; Using an adaptive region generation network to process the target layer PCB image to generate a candidate region set containing potential defects, wherein the candidate region set includes at least one candidate region; Performing multi-scale context enhancement processing on the candidate region according to impedance change characteristics inside and outside the candidate region to obtain an enhanced candidate region; Performing feature extraction on the enhanced candidate area to extract defect features with spatial perception capabilities, and performing multi-dimensional embedding coding processing on the position information of the enhanced candidate area and the panel surface layer identifier to generate position features with process relevance; Performing cross-modal fusion processing on the defect feature and the position feature to generate a fusion feature with process context awareness; Performing defect detection on the PCB board according to the fusion features to obtain the defect type of the PCB board and associated process link information; The position information of the enhanced candidate area and the board surface layer identification are subjected to multi-dimensional embedded coding processing to generate position features with process relevance, including: Quantizing the position information of the enhanced candidate region to obtain quantized position information in different directions of the PCB; and converting the board surface layer identifier into a coded identifier in a coding format; wherein at least two dimensions correspond to the position information of the enhanced candidate region in at least two directions of the PCB, and at least one dimension corresponds to the board surface layer identifier; Embedding the quantized position information and the coding identifier into the corresponding dimension to obtain coding information of each dimension; Integrating the coded information of each dimension according to its weight in the process association to generate a position feature with process association, wherein the weight represents the importance of the corresponding dimension in the process association; The dimensions include a horizontal dimension, a vertical dimension, and a board layer identification dimension. The horizontal dimension corresponds to the position information of the enhanced candidate area in the horizontal direction of the PCB board, the vertical dimension corresponds to the position information of the PCB board in the vertical direction, and the board layer identification dimension corresponds to the board surface layer identification. Embedding the quantized position information and the coding identifier into the corresponding dimension to obtain coding information of each dimension, including: according to a predetermined coding rule, embedding the quantized horizontal position information of the PCB board into the horizontal dimension, embedding the quantized vertical position information of the PCB board into the vertical dimension, and embedding the coding identifier into the hierarchical identifier dimension; The defect feature and the position feature are cross-modally fused to generate a fusion feature with process context awareness, including: Constructing a fusion mapping relationship based on the process structure information and process flow information of the PCB board; Performing a fusion operation on the defect feature and the position feature according to the fusion mapping relationship to obtain a fusion-operated feature; Performing process context-aware adjustments on the fused features to generate process context-aware fused features; The multi-scale context enhancement processing is performed on the candidate region according to the component distribution density at the location of the candidate region to obtain the enhanced candidate region, including: Extracting impedance change characteristics inside and outside the candidate area, and performing frequency domain analysis on the candidate area based on the impedance change characteristics to determine a key signal path corresponding to the candidate area; Constructing an elliptical extension area according to the critical signal path and the component distribution density of the candidate area, wherein the long axis direction of the elliptical extension area is consistent with the main wiring direction corresponding to the candidate area; A resolution optimization process is performed on the elliptical extended region to obtain an enhanced candidate region.

2. The PCB board detection method according to claim 1, wherein: Performing frequency domain analysis on the candidate area according to the impedance change characteristics to determine a key signal path, including: Performing short-time Fourier transform processing on the impedance change characteristics of the candidate area to generate a time-frequency graph; Energy analysis is performed on the time-frequency diagram to generate key signal paths.

3. The PCB board detection method according to claim 1, wherein: Constructing an elliptical expansion area according to the critical signal path and the component distribution density of the candidate area, including: Performing principal component analysis on the key signal path to determine the main wiring direction of the candidate area; Obtaining the minimum safe distance between adjacent lines in the target layer where the candidate area is located; Constructing an initial elliptical extension area according to the main wiring direction and the minimum safety distance, wherein the major axis direction of the initial elliptical extension area is consistent with the main wiring direction, and the minor axis length is determined according to the minimum safety distance between adjacent lines; The initial elliptical expansion area is adjusted according to the component distribution density of the candidate area to obtain an elliptical expansion area.

4. The PCB board detection method according to claim 3, wherein: The method further comprises: Obtaining the area and the number of valid pixels of the candidate area, wherein the number of valid pixels is the valid pixels in the component edge detection result of the candidate area; The component distribution density of the candidate area is calculated according to the area and the number of effective pixels, wherein the component distribution density is the ratio of the number of component outline pixels per unit area.

5. The PCB board detection method according to claim 4, wherein: The initial elliptical expansion area is adjusted according to the component distribution density of the candidate area to obtain the elliptical expansion area, including: Determine the density level of the candidate area according to the component distribution density, the density level including: high, medium and low density areas; Adjusting the major axis or the minor axis of the initial elliptical expansion area according to the density level to obtain an elliptical expansion area; Performing resolution optimization processing on the elliptical extended area to obtain an enhanced candidate area includes: performing resolution optimization processing on the elliptical extended area using an adaptive interpolation method according to a density level of the candidate area to obtain the enhanced candidate area.

6. A PCB board detection device based on artificial intelligence, characterized in that: The device comprises: An image acquisition module is configured to acquire a set of original images of a PCB from multiple perspectives, each original image being associated with a perspective identifier and a board layer identifier. The perspective identifier represents the observation angle of the imaging device relative to the PCB, and the board layer identifier indicates the PCB layer currently being inspected. An image processing module is configured to process a target-level PCB image using an adaptive region generation network to generate a candidate region set containing potential defects, wherein the candidate region set includes at least one candidate region; an enhancement module, configured to perform multi-scale context enhancement processing on the candidate region according to impedance variation characteristics inside and outside the candidate region to obtain an enhanced candidate region; a feature encoding module for performing feature extraction on the enhanced candidate area to extract defect features with spatial perception capability, and performing multi-dimensional embedding encoding processing on the position information of the enhanced candidate area and the panel surface layer identifier to generate position features with process relevance; A fusion module, configured to perform cross-modal fusion processing on the defect feature and the position feature to generate a fusion feature with process context awareness; a detection module, configured to perform defect detection on the PCB board according to the fusion feature, and obtain defect types and associated process information of the PCB board; The position information of the enhanced candidate area and the board surface layer identification are subjected to multi-dimensional embedded coding processing to generate position features with process relevance, including: Quantizing the position information of the enhanced candidate region to obtain quantized position information in different directions of the PCB; and converting the board surface layer identifier into a coded identifier in a coding format; wherein at least two dimensions correspond to the position information of the enhanced candidate region in at least two directions of the PCB, and at least one dimension corresponds to the board surface layer identifier; Embedding the quantized position information and the coding identifier into the corresponding dimension to obtain coding information of each dimension; Integrating the coded information of each dimension according to its weight in the process association to generate a position feature with process association, wherein the weight represents the importance of the corresponding dimension in the process association; The dimensions include a horizontal dimension, a vertical dimension, and a board layer identification dimension. The horizontal dimension corresponds to the position information of the enhanced candidate area in the horizontal direction of the PCB board, the vertical dimension corresponds to the position information of the PCB board in the vertical direction, and the board layer identification dimension corresponds to the board surface layer identification. Embedding the quantized position information and the coding identifier into the corresponding dimension to obtain coding information of each dimension, including: according to a predetermined coding rule, embedding the quantized horizontal position information of the PCB board into the horizontal dimension, embedding the quantized vertical position information of the PCB board into the vertical dimension, and embedding the coding identifier into the hierarchical identifier dimension; The defect feature and the position feature are cross-modally fused to generate a fusion feature with process context awareness, including: Constructing a fusion mapping relationship based on the process structure information and process flow information of the PCB board; Performing a fusion operation on the defect feature and the position feature according to the fusion mapping relationship to obtain a fusion-operated feature; Performing process context-aware adjustments on the fused features to generate process context-aware fused features; The multi-scale context enhancement process is performed on the candidate region according to the component distribution density at the location of the candidate region to obtain the enhanced candidate region, including: Extracting impedance change characteristics inside and outside the candidate area, and performing frequency domain analysis on the candidate area based on the impedance change characteristics to determine a key signal path corresponding to the candidate area; Constructing an elliptical extension area according to the critical signal path and the component distribution density of the candidate area, wherein the long axis direction of the elliptical extension area is consistent with the main wiring direction corresponding to the candidate area; A resolution optimization process is performed on the elliptical extended region to obtain an enhanced candidate region.

Images