A PCBA detection device and method based on image and model recognition

Abstract

The invention discloses a PCBA detection device and method based on image and model recognition, comprising: a host computer, an image recognition module, a test logic generation module, a detection controller, and a plurality of detection modules; the host computer is used to generate a model recognition result by using a plurality of PCBA circuit DES models according to the execution result, and output the detection result in combination with the image recognition result, and is used to input a detection instruction; the image recognition module uses the PCBA recognition model to perform defect recognition on a PCBA to be inspected through a generated port positioning frame and a component positioning frame, and returns the image recognition result to the host computer; the test logic generation module is used to automatically generate corresponding test logic and send it to the detection controller; the detection controller is used to control the execution of the plurality of detection modules according to the test logic, and returns the execution result to the host computer; the plurality of detection modules are respectively connected to the corresponding ports or devices of the PCBA to be inspected, and are used to simulate or detect various input and output signals.

Description

Technical Field

[0001] The present invention relates to the technical field of PCBA detection, and in particular to a PCBA detection device and method based on image and model recognition. Background Art

[0002] With the rapid development of military and aerospace industries, electronic products tend to be integrated and high-quality. Although the manufacturing process and reliability of electronic products have been steadily improved, there are more and more types of welding defects in the production and manufacturing process of electronic products. PCBA (Printed Circuit Board Assembly) is the popular name for PCB blank board after SMT (Surface Mount Technology) assembly or DIP (Dual In-line Package) plug-in, referred to as PCBA.

[0003] The quality of PCBA is a key factor that directly affects the reliability of electronic equipment. At the same time, the requirements for PCBA quality inspection are also increasing year by year. Therefore, how to achieve efficient and accurate inspection of PCBA production defects is a problem that is of general concern in PCBA production and needs to be solved urgently.

[0004] At present, most PCBA manufacturers use the following methods to detect PCBA production defects: manual inspection and online testing. Among them, manual inspection is manual visual inspection, which mainly uses visual inspection or magnifying glasses, microscopes and other equipment to find defects on the board. However, during long-term repetitive inspections, workers are prone to fatigue and interference from the surrounding environment, which leads to missed inspections and false inspections.

[0005] Online testing mainly includes two testing methods: bed of nails tester and flying probe tester. Its advantages are low cost for single board testing and strong performance for open circuit and short circuit testing, but its programming time is long and the testing speed is slow.

[0006] In summary, the existing PCBA defect detection methods have technical problems of low detection accuracy and efficiency. Summary of the invention

[0007] In order to overcome the shortcomings of the prior art, the present invention provides a PCBA detection device and method based on image and model recognition, which are used to solve the technical problems of low detection accuracy and detection efficiency in the existing PCBA defect detection method, thereby achieving the purpose of improving detection accuracy and detection efficiency.

[0008] To solve the above problems, the technical solution adopted by the present invention is as follows:

[0009] A PCBA detection device based on image and model recognition, comprising:

[0010] The host computer is used to generate model recognition results using several PCBA circuit DES models according to the execution results, output the detection results in combination with the image recognition results, and input the detection instructions;

[0011] The image recognition module uses the PCBA recognition model to identify defects on the PCBA to be inspected through the generated port positioning frame and component positioning frame, and returns the image recognition results to the host computer;

[0012] A test logic generation module is used to automatically generate corresponding test logic and send it to the detection controller;

[0013] A detection controller, used to control the execution of several detection modules according to the test logic and return the execution results to the host computer;

[0014] Several detection modules are connected to the corresponding ports or devices of the PCBA to be tested, and are used to simulate or detect various input and output signals;

[0015] The PCBA circuit DES models are associated with the PCBA identification model, and the PCBA circuit DES models are established according to a state transfer function, and the state transfer function is shown in Formula 1:

[0016] (1);

[0017] In the formula, is the state transfer function, For PCBA model, is the electrical response of ports and components in a fault-free state, is the fault state set, is a set of fault events, wherein a fault state corresponds to a fault event.

[0018] As a preferred embodiment of the present invention, the plurality of detection modules include: a switch output module, a switch input module;

[0019] The switch output module is used to output AC and DC switch signals to detect the AC and DC switch input ports of the PCBA to be inspected;

[0020] The switch quantity input module is used to receive AC and DC switch quantity signals to detect the AC and DC switch quantity output ports of the PCBA to be inspected.

[0021] As a preferred embodiment of the present invention, the several detection modules include: an analog detection module, a sound detection module, a display detection module, and a dual firmware burning module;

[0022] The analog detection module is used to output and receive analog signals to detect the analog input and output ports of the PCBA to be tested;

[0023] The sound and display detection module is used to receive sound and display signals to detect the sound and display components of the PCBA to be inspected;

[0024] The dual firmware burning module is used for performing firmware burning and function verification to detect the firmware burning port of the PCBA to be tested.

[0025] A PCBA detection method based on image and model recognition, using the above-mentioned PCBA detection device, includes the following steps:

[0026] In the host computer, several PCBA circuit DES models are established through simulation software, and the detection instructions containing the PCBA model to be tested are input and sent to the test logic generation module and the image recognition module;

[0027] The image recognition module is used to identify defects in the PCBA to be inspected, and the image recognition results are returned to the host computer;

[0028] The test logic generation module automatically generates the corresponding test logic according to the PCBA model to be tested and sends it to the detection controller;

[0029] The detection controller controls the execution of several detection modules according to the test logic, and returns the execution results to the host computer;

[0030] The host computer uses the several PCBA circuit DES models to generate model recognition results, and outputs detection results in combination with image recognition results.

[0031] As a preferred embodiment of the present invention, when establishing several PCBA circuit DES models, it includes:

[0032] Establish several PCBA circuit simulation models through simulation software, simulate the circuit in a fault-free state, and extract the first electrical response of ports and components;

[0033] Obtain the port and component failure modes contained in the PCBA circuit, analyze them, determine the parameters of the PCBA circuit in the DES model under different failure modes, obtain the fault state set, and establish the port and component failure model;

[0034] Injecting port and component fault models into corresponding ports and components in several PCBA circuit simulation models to obtain several circuit fault simulation models;

[0035] Perform simulation based on several circuit fault simulation models to extract the second electrical response of ports and components;

[0036] The first and second electrical responses are compared to obtain electrical responses that change under different fault modes, and a fault event set is constructed based on the changed electrical responses;

[0037] According to the fault state set and the fault event set, determine the state transfer function corresponding to the PCBA circuit in the DES model;

[0038] Establish several PCBA circuit DES models based on the state transfer function;

[0039] The PCBA circuit DES models include electrical responses under non-fault conditions, fault state sets and fault event sets corresponding to each type of PCBA.

[0040] As a preferred embodiment of the present invention, when automatically generating corresponding test logic, it includes:

[0041] Retrieve and load the corresponding test specifications from the preset test specification database according to the PCBA model to be tested;

[0042] Acquire the type of input and output signals, the test sequence, and the detection threshold from the test specification to generate the test logic, and send the test logic to the detection controller;

[0043] The test specification database supports dynamic updating, allowing test specifications to be added, modified or deleted according to actual needs; the plurality of PCBA circuit simulation models are associated with the test specifications, and simulation is performed according to the corresponding test specifications.

[0044] As a preferred embodiment of the present invention, when performing defect recognition on a PCBA to be inspected, it includes: constructing a PCBA recognition model, and performing defect recognition on the PCBA to be inspected based on the generated port positioning frame and component positioning frame;

[0045] Among them, when generating port and component positioning frames, it includes:

[0046] The image of the PCBA to be inspected is collected through the image recognition module to obtain the image of the PCBA to be inspected, and the image is preprocessed;

[0047] Obtain the prior conditions of the PCBA according to the model of the PCBA to be inspected and locate the Mark points;

[0048] Obtain PCBA port and component information files, and determine the relative relationship between the port and component coordinate systems and the image pixel coordinate system;

[0049] Convert port and component coordinates into image pixel coordinates and generate port and component positioning frames;

[0050] When generating the port positioning frame and the component positioning frame, the accuracy of the port positioning frame and the component positioning frame is determined by using the associated corresponding PCBA circuit DES model information.

[0051] As a preferred embodiment of the present invention, when constructing a PCBA recognition model, it includes:

[0052] Provide the YOLOv8 model and LSKNet Block module, and add the LSKNet Block module after the SPPF module in the Backbone layer of the YOLOv8 model;

[0053] Add the LSKNetBlock module between the last upsampling layer of the Neck layer of the YOLOv8 model and the final detection layer;

[0054] Add the LSKNet Block module to the connection of the Concat module before the three detection heads of the Head layer of the YOLOv8 model;

[0055] As a preferred embodiment of the present invention, when constructing a PCBA recognition model, it also includes:

[0056] The LSKNet Block module at the connection of the Concat module obtains the ratio of the area of ​​the expected selection receptive field of each target category to the area of ​​the port and component positioning box. , reducing the focus on irrelevant or redundant features and improving the model's expressiveness and discrimination, as shown in Formula 8:

[0057] (8);

[0058] In the formula, To include the target category The number of images, For the input image The sum of the spatially selective activations output by all LSKNetBlock modules in , The total pixel area of ​​all generated positioning boxes.

[0059] As a preferred embodiment of the present invention, when providing the LSKNet Block module, it includes:

[0060] The LSKNet Block module is constructed by a large kernel selection sub-block for dynamically adjusting the spatial receptive field, a feedforward network sub-block for channel mixing and feature refinement, and a Norm block;

[0061] The large core selection sub-block includes: a first fully connected layer, a first GELU activation, an LSK module, and a second fully connected layer connected in sequence;

[0062] The feedforward network sub-block includes: a third fully connected layer, a deep convolution, a second GELU activation, and a fourth fully connected layer connected in sequence.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] (1) The present invention automatically generates corresponding test logic through a test logic generation module according to a test instruction input by a user containing the PCBA model to be tested, so that the detection controller controls the execution of several detection modules according to the test logic and obtains the execution results. The host computer quickly and accurately identifies whether there is a circuit fault in the PCBA to be tested and the location of the circuit fault based on the execution results through several PCBA circuit DES models, thereby effectively improving the detection efficiency;

[0065] (2) The present invention uses an image recognition module to identify appearance defects of the PCBA to be inspected based on the PCBA recognition model, thereby effectively avoiding missed detection and false detection in long-term repetitive detection, thereby improving the detection accuracy; and combining the circuit fault recognition results generated by the DES model to finally determine whether the PCBA to be inspected is a qualified product;

[0066] (3) The PCBA circuit DES model constructed by the present invention is based on the state transfer function. Each type of PCBA has a corresponding electrical response, fault state set and fault event set in a fault-free state, so that after the upper computer receives the execution result fed back by the detection controller, it can quickly and accurately generate the model recognition result.

[0067] The present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a structural diagram of a PCBA detection device based on image and model recognition provided by the present invention;

[0069] Figure 2 It is a step diagram of the PCBA detection method based on image and model recognition provided by the present invention;

[0070] Figure 3 It is a structural schematic diagram of the Backbone layer of the PCBA identification model provided by the present invention;

[0071] Figure 4 It is a structural schematic diagram of the Neck layer of the PCBA identification model provided by the present invention;

[0072] Figure 5 It is a structural schematic diagram of the Head layer of the PCBA recognition model provided by the present invention;

[0073] Figure 6 It is a structural diagram of the LSKNet Block module provided by the present invention.

[0074] Description of the accompanying figures: 1. Host computer; 2. Detection machine control module; 3. Image recognition module; 4. Test logic generation module; 5. Detection controller; 6. AC switch output module; 7. DC switch output module; 8. AC switch input module; 9. DC switch input module; 10. Analog output module; 11. Analog input module; 12. Sound detection module; 13. Display detection module; 14. Dual firmware burning module; 15. PCBA to be inspected; 16. Detection machine; 17. Backbone layer; 18. Neck layer; 19. Head layer; 20. Stem Layer; 21. Stage Layer1; 22. Stage Layer2; 23. StageLayer3; 24. Stage Layer4; 25. Conv module; 26. C2f module; 27. SPPF module; 28. LSKNet Block module; 29. ​​Concat module; 30. Upsample module; 31. Conv2d module; 32. Bbox Loss module; 33, Cls Loss module; 34, Norm block; 35, large kernel selection sub-block; 36, feedforward network sub-block; 37, first fully connected layer; 38, first GELU activation; 39, LSK module; 40, second fully connected layer; 41, third fully connected layer; 42, deep convolution; 43, second GELU activation; 44, fourth fully connected layer. DETAILED DESCRIPTION

[0075] The PCBA detection device based on image and model recognition provided by the present invention is as follows: Figure 1 As shown, it includes: a host computer 1, a detection machine control module 2, an image recognition module 3, a test logic generation module 4, a detection controller 5 and several detection modules.

[0076] The host computer 1 is used to receive the execution result returned by the detection controller 5, generate a model recognition result using a number of PCBA circuit DES models, output the detection result in combination with the image recognition result, and input a detection instruction containing the model of the PCBA 15 to be detected.

[0077] The inspection machine control module 2 is used to receive the inspection instruction and control the inspection machine 16 to transport the PCBA 15 to be inspected to the inspection position.

[0078] The image recognition module 3 is used to receive the detection instruction, use the PCBA recognition model to perform defect recognition on the PCBA 15 to be inspected through the generated port positioning frame and component positioning frame, and return the image recognition result to the host computer 1.

[0079] The test logic generation module 4 is used to receive the detection instruction, automatically generate the corresponding test logic according to the model of the PCBA 15 to be detected, and send it to the detection controller 5.

[0080] The detection controller 5 is used to control the execution of several detection modules according to the test logic, and return the execution results of several detection modules to the host computer 1.

[0081] Several detection modules are respectively connected to corresponding ports or devices of the PCBA 15 to be detected, and are used to simulate or detect various input and output signals.

[0082] Among them, several PCBA circuit DES models are associated with the PCBA identification model, and several PCBA circuit DES models are established according to the state transfer function, and the state transfer function is shown in Formula 1:

[0083] (1);

[0084] In the formula, is the state transfer function, For PCBA model, is the electrical response of ports and components in a fault-free state, is the fault state set, is a set of fault events, wherein a fault state corresponds to a fault event.

[0085] Specifically, the present invention automatically generates corresponding test logic through the test logic generation module 4 according to the detection instruction containing the model of the PCBA 15 to be detected input by the user, so that the detection controller 5 controls the execution of several detection modules according to the test logic and obtains the execution result. The upper computer 1 quickly and accurately identifies whether there is a circuit fault in the PCBA 15 to be detected and the location of the circuit fault based on the execution result through several PCBA circuit DES models, thereby effectively improving the detection efficiency.

[0086] In addition, the present invention uses the image recognition module 3 to identify the appearance defects of the PCBA 15 to be inspected based on the PCBA recognition model, thereby effectively avoiding the situation of missed detection and false detection that are prone to occur in long-term repetitive detection, and improving the detection accuracy; and combined with the circuit fault recognition result generated by the DES model, it is finally determined whether the PCBA 15 to be inspected is a qualified product.

[0087] Specifically, when both the model recognition result and the image recognition result are qualified, the PCBA 15 to be inspected is considered to be a qualified product.

[0088] Specifically, the image recognition module 3 includes: a high-resolution camera, a lighting control unit, and an image processing unit.

[0089] A high-resolution camera for capturing a high-definition image of the PCBA 15 to be inspected;

[0090] Lighting control unit, used to adjust the lighting conditions when the camera is shooting to ensure image quality;

[0091] The image processing unit receives image data from the high-resolution camera and performs defect recognition on the PCBA 15 to be inspected.

[0092] The image recognition module 3 also supports a multi-viewing angle shooting function, which captures images of the PCBA 15 to be inspected at different viewing angles by adjusting the position or angle of the high-resolution camera, so as to comprehensively evaluate the defects of the PCBA.

[0093] Furthermore, the plurality of detection modules include: a switch quantity output module, a switch quantity input module, an analog quantity detection module, a sound detection module 12 , a display detection module 13 , and a dual firmware burning module 14 .

[0094] Furthermore, the switch quantity output module includes: an AC switch quantity output module 6 and a DC switch quantity output module 7 .

[0095] The AC switch quantity output module 6 is connected to the AC switch quantity input port of the PCBA 15 to be tested, and is used to output an AC switch quantity signal to detect the AC switch quantity input port;

[0096] The DC switch quantity output module 7 is connected to the DC switch quantity input port of the PCBA 15 to be inspected, and is used to output a DC switch quantity signal to detect the DC switch quantity input port.

[0097] Furthermore, the switch quantity input module includes: an AC switch quantity input module 8 and a DC switch quantity input module 9.

[0098] The AC switch quantity input module 8 is connected to the AC switch quantity output port of the PCBA 15 to be tested, and is used to receive the AC switch quantity signal to detect the AC switch quantity output port;

[0099] The DC switch quantity input module 9 is connected to the DC switch quantity output port of the PCBA 15 to be inspected, and is used to receive a DC switch quantity signal to detect the DC switch quantity output port.

[0100] Furthermore, the analog detection module includes: an analog output module 10, an analog input module 11;

[0101] The analog output module 10 is connected to the analog input port of the PCBA 15 to be tested, and is used to output an analog signal to detect the analog input port;

[0102] The analog input module 11 is connected to the analog output port of the PCBA 15 to be inspected, and is used to receive an analog signal to inspect the analog output port.

[0103] Furthermore, the sound detection module 12 is connected to the sound-generating device of the PCBA 15 to be detected, and is used to receive the sound signal to detect the sound-generating device.

[0104] Furthermore, the display detection module 13 is connected to the display device of the PCBA 15 to be detected, and is used to receive a display signal to detect the display device.

[0105] Furthermore, the dual firmware burning module 14 is connected to the firmware burning port of the PCBA 15 to be tested, and is used for performing firmware burning and function verification to detect the firmware burning port of the PCBA 15 to be tested.

[0106] Specifically, the present invention is equipped with corresponding detection modules for different ports and components of the PCBA 15 to be inspected, so the generated execution results are more comprehensive, so that the model recognition based on the execution results can obtain more accurate recognition results.

[0107] The PCBA detection method based on image and model recognition provided by the present invention adopts the PCBA detection device provided by the present invention, such as Figure 2 As shown, the following steps are included:

[0108] Step S1: Establishing several PCBA circuit DES models in the host computer 1 through simulation software;

[0109] Step S2: inputting a detection instruction including the model of the PCBA 15 to be detected through the host computer 1, and sending the detection instruction to the test logic generation module 4, the detection machine control module 2 and the image recognition module 3;

[0110] Step S3: the inspection machine control module 2 controls the inspection machine 16 to transport the PCBA 15 to be inspected to the inspection position, the image recognition module 3 performs defect recognition on the PCBA 15 to be inspected, and returns the image recognition result to the host computer 1;

[0111] Step S4: The test logic generation module 4 automatically generates corresponding test logic according to the model of the PCBA 15 to be tested, and sends it to the detection controller 5;

[0112] Step S5: controlling the execution of several detection modules according to the test logic through the detection controller 5, and returning the execution results of several detection modules to the host computer 1;

[0113] Step S6: The host computer 1 generates a model recognition result using several PCBA circuit DES models, and outputs a detection result in combination with the image recognition result.

[0114] In the above step S1, when establishing several PCBA circuit DES models, it includes:

[0115] Establish several PCBA circuit simulation models through simulation software, simulate the circuit in a fault-free state, and extract the first electrical response of ports and components;

[0116] Obtain the port and component failure modes contained in the PCBA circuit, analyze them, determine the parameters of the PCBA circuit in the DES model under different failure modes, obtain the fault state set, and establish the port and component failure model;

[0117] Injecting port and component fault models into corresponding ports and components in several PCBA circuit simulation models to obtain several circuit fault simulation models;

[0118] Perform simulation based on several circuit fault simulation models to extract the second electrical response of ports and components;

[0119] Comparing the first electrical response with the second electrical response to obtain electrical responses that change under different fault modes, and constructing a fault event set based on the changed electrical responses;

[0120] According to the fault state set and the fault event set, determine the state transfer function corresponding to the PCBA circuit in the DES model;

[0121] Several PCBA circuit DES models are established based on the state transfer function.

[0122] Specifically, the simulation software includes: Saber simulation software, MATLAB simulation software, PSpice simulation software, PSCAD simulation software, LTspice simulation software, and Altium Designer simulation software.

[0123] Preferably, Saber simulation software is used.

[0124] Specifically, the port and component failure modes include: open circuit, short circuit, parameter drift, no output, low gain, parameter out of tolerance, function failure, voltage discontinuity, input fixed "1", input fixed "0",

[0125] Output fixed at "1", output fixed at "0", poor contact, output too low.

[0126] Specifically, according to the manifestation or impact of the failure modes of ports and components in the circuit, port and component failure models are established, and according to the location of the failure in the circuit, the failure models are injected into the circuit simulation model to obtain the circuit failure simulation model. The circuit failure simulation model can reflect the impact of port and component failures.

[0127] When injecting a fault model, it includes: determining a suitable fault mode according to the types of fault ports and components and the circuit environment in which they are located;

[0128] Establish fault models of ports and components based on failure modes;

[0129] Based on the fault model, the simulation circuit in the circuit simulation model is modified to obtain a circuit fault simulation model.

[0130] The port fault injection method includes: modifying the properties of the port, modifying the model of the port, modifying the circuit schematic, and modifying the circuit netlist file.

[0131] The component fault injection method includes: modifying the properties of the component, modifying the model of the component, modifying the circuit schematic, and modifying the circuit netlist file.

[0132] Specifically, the state transfer function is shown in Formula 1:

[0133] (1);

[0134] In the formula, is the state transfer function, For PCBA model, is the electrical response of ports and components in a fault-free state, is the fault state set, is a set of fault events, wherein a fault state corresponds to a fault event.

[0135] Specifically, the PCBA circuit DES model constructed based on the state transfer function of the present invention has corresponding electrical responses, fault state sets and fault event sets in a fault-free state, so that after the host computer 1 receives the execution results fed back by the detection controller 5, it can quickly and accurately generate model recognition results.

[0136] In the above step S6, when generating the model identification result, if the execution result returned by the detection controller 5 is the electrical response of the port and component in the non-fault state, it is considered that the port and component are not faulty. If the execution result returned by the detection controller 5 is the electrical response of the port and / or component in the fault event set, it is considered that the port and / or component is faulty, and the fault location and fault type are determined according to the fault state set.

[0137] In the above step S4, when the corresponding test logic is automatically generated, it includes:

[0138] Retrieve and load the corresponding test specification from the preset test specification database according to the PCBA 15 model to be tested;

[0139] Obtain the type of input and output signals, test sequence, and detection threshold from the test specification to generate test logic, and send the test logic to the detection controller 5;

[0140] The test specification database supports dynamic updates, allowing users or system administrators to add, modify or delete test specifications according to actual needs to adapt to new PCBA models or changes in test requirements;

[0141] Several PCBA circuit simulation models are associated with test specifications in a test specification database, and simulation is performed according to the corresponding test specifications.

[0142] In the above step S3, when performing defect recognition on the PCBA 15 to be inspected, it includes: constructing a PCBA recognition model, and performing defect recognition on the PCBA 15 to be inspected based on the generated port positioning frame and component positioning frame;

[0143] Among them, when generating the port positioning frame and component positioning frame, it includes:

[0144] The image recognition module 3 is used to collect the image of the PCBA 15 to be inspected, obtain the image of the PCBA to be inspected, and perform image preprocessing;

[0145] Obtain the prior conditions of the PCBA according to the model of the PCBA to be inspected 15, and locate the Mark point;

[0146] Obtain PCBA port and component information files, and determine the relative relationship between the port coordinate system, component coordinate system, and image pixel coordinate system;

[0147] Convert port coordinates and component coordinates into image pixel coordinates, and generate port positioning frames and component positioning frames;

[0148] The prior conditions include: the position of the mark point on the PCBA, the shape of the PCBA, and the size of the PCBA.

[0149] Specifically, in the present invention, the Mark point is a circular Mark point located at a diagonal position, and its center is a circular pad without a via hole. When locating the Mark point, the detection methods used include: Hough transform, edge detection, template matching and least squares method.

[0150] Specifically, the PCBA port and component information file contains Mark point position coordinate information. The present invention obtains the relative relationship between the image pixel coordinate system and the port coordinate system and the component coordinate system by locating the Mark point and the Mark point position coordinate in the PCBA port and component information file, thereby realizing the positioning of each port and component.

[0151] Furthermore, several PCBA circuit DES models are associated with the PCBA identification model. When generating the port positioning frame and the component positioning frame, the accuracy of the port positioning frame and the component positioning frame is determined by using the associated corresponding PCBA circuit DES model information.

[0152] Specifically, the corresponding PCBA circuit DES model information includes: types of ports and components and the circuit environment in which they are located.

[0153] Specifically, the present invention helps to improve the accuracy of image recognition results generated by a PCBA recognition model by generating accurate port positioning frames and component positioning frames.

[0154] Specifically, the image preprocessing includes: image grayscale, image distortion correction and image denoising.

[0155] The image grayscale method includes: weighted average method, component method, single channel maximum method, three channel average method. Preferably, the weighted average method is used.

[0156] Image distortion correction specifically involves correcting the radial distortion and tangential distortion of the acquired PCBA image to be inspected, as shown in Formula 2 and Formula 3:

[0157] (2);

[0158] (3);

[0159] In the formula, , is the coordinate of the imaging point before correction, is the coordinate of the corrected imaging point, , , is the radial distortion parameter, , is the tangential distortion parameter.

[0160] Image denoising includes: obtaining the distortion-corrected image in pixels A local neighborhood window centered on The local average of the gradient and the local average of the absolute value of the gradient , as shown in Formula 4 and Formula 5:

[0161] (4);

[0162] (5);

[0163] In the formula, is the local neighborhood window The number of pixels in ; For the pixel The total variation gradient value at is based on The forward differential gradients in the horizontal and vertical directions are obtained.

[0164] According to the local average of the gradient and the local average of the absolute value of the gradient , construct a texture-structure perceptron, as shown in Formula 6:

[0165] (6);

[0166] Preferably, More preferably, .

[0167] The texture-structure perceptron is used as the weighting coefficient of the TV term to construct a total variation filter, as shown in Formula 7:

[0168] (7);

[0169] In the formula, ⊙ is the same or the same operation, and the subscript letter Indicates the horizontal form of the corresponding gradient, the subscript letter Represents the longitudinal form of the corresponding gradient.

[0170] Specifically, the present invention filters the distortion-corrected image through a total variation filter, thereby utilizing the local gradient characteristics to enhance the total variation gradient value of each pixel in the distortion-corrected image, thereby effectively improving the accuracy of the subsequent generation of port positioning frames and component positioning frames.

[0171] Furthermore, when constructing the PCBA recognition model, it includes:

[0172] Provide a YOLOv8 model and LSKNet Block module 28, and add LSKNet Block module 28 after SPPF module 27 of Backbone layer 17 of the YOLOv8 model to improve the defect attention of the model;

[0173] Add LSKNetBlock module 28 between the last upsampling layer of Neck layer 18 of the YOLOv8 model and the final detection layer to enhance the representation capability of the feature map;

[0174] Add LSKNetBlock module 28 at the connection of Concat module 29 before the three detection heads of Head layer 19 of YOLOv8 model to improve the model's attention to important local information;

[0175] Among them, the LSKNet Block module 28 at the connection of the Concat module 29 obtains the ratio of the area of ​​the expected selection receptive field of each target category to the area of ​​the port and component positioning box , reducing the focus on irrelevant or redundant features and improving the model's expressiveness and discrimination, as shown in Formula 8:

[0176] (8);

[0177] In the formula, To include the target category The number of images, For the input image The sum of the spatially selective activations output by all LSKNetBlock modules 28 in , The total pixel area of ​​all generated positioning boxes.

[0178] Specifically, add LSKNet Block module 28 after SPPF module 27 of Backbone layer 17 of YOLOv8 model, as Figure 3 The optimized Backbone layer 17 includes: Stem Layer 20, Stage Layer 1 21, Stage Layer 2 22, Stage Layer 3 23 and Stage Layer 4 24.

[0179] Stage Layer1 21, Stage Layer2 22, and Stage Layer3 23 all include: a Conv module 25 and a C2f module 26. Stage Layer4 24 includes: a Conv module 25, a C2f module 26, a SPPF module 27, and a LSKNet Block module 28.

[0180] The C2f module 26 in Stage Layer2 22 and Stage Layer3 23 is connected to the Concat module 29 of the Neck layer 18 .

[0181] The LSKNet Block module 28 in Stage Layer4 24 is connected to the Concat module 29 in the Neck layer 18.

[0182] Specifically, add LSKNet Block module 28 between the last upsampling layer and the final detection layer, as Figure 4 The optimized Neck layer 18 includes: a Conv module 25, a C2f module 26, a LSKNet Block module 28, an Upsample module 30, and a Concat module 29.

[0183] Specifically, add LSKNet Block module 28 at the connection of Concat module 29 before the three detection heads, as shown in Figure 4 , Figure 5 The Head layer 19 includes: a Conv module 25, a Conv2d module 31, a Bbox Loss module 32, and a Cls Loss module 33.

[0184] Furthermore, if Figure 6 As shown, when providing the LSKNet Block module 28, it includes:

[0185] The LSKNet Block module 28 is constructed by a large kernel selection sub-block 35 for dynamically adjusting the spatial receptive field, a feed-forward network sub-block 36 for channel mixing and feature refinement, and a Norm block 34;

[0186] The large core selection sub-block 35 includes: a first fully connected layer 37, a first GELU activation 38, an LSK module 39 and a second fully connected layer 40 connected in sequence;

[0187] The feedforward network sub-block 36 includes: a third fully connected layer 41, a depth convolution 42, a second GELU activation 43, and a fourth fully connected layer 44 which are connected in sequence.

[0188] Specifically, the present invention constructs an LSKNet Block module 28 that can dynamically adjust the network receptive field, and introduces the LSKNet Block module 28 at different positions of the YOLOv8 model, effectively improving the overall defect detection performance of the YOLOv8 model and more accurately detecting the defect information of the PCBA.

[0189] The above-mentioned embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and substitutions made by technicians in this field on the basis of the present invention shall fall within the scope of protection required by the present invention.

Claims

1. A PCBA detection device based on image and model recognition, characterized in that: include: The host computer is used to generate a model recognition result using a number of PCBA circuit DES models according to the execution result, and output a detection result in combination with the image recognition result, and is used to input a detection instruction containing the PCBA model to be detected; The image recognition module uses the PCBA recognition model to identify defects on the PCBA to be inspected through the generated port positioning frame and component positioning frame, and returns the image recognition results to the host computer; The test logic generation module is used to receive the test instructions, automatically generate the corresponding test logic according to the PCBA model to be tested, and send it to the test controller; A detection controller, used to control the execution of several detection modules according to the test logic and return the execution results to the host computer; Several detection modules are connected to the corresponding ports or devices of the PCBA to be tested, and are used to simulate or detect various input and output signals; The PCBA circuit DES models are associated with the PCBA identification model, and the PCBA circuit DES models are established according to a state transfer function, and the state transfer function is shown in Formula 1: （1）； In the formula, is the state transfer function, For PCBA model, is the electrical response of ports and components in a fault-free state, is the fault state set, is a set of fault events, wherein a fault state corresponds to a fault event.

2. The PCBA detection device based on image and model recognition according to claim 1, characterized in that: The plurality of detection modules include: a switch quantity output module and a switch quantity input module; The switch output module is used to output AC and DC switch signals to detect the AC and DC switch input ports of the PCBA to be inspected; The switch quantity input module is used to receive AC and DC switch quantity signals to detect the AC and DC switch quantity output ports of the PCBA to be inspected.

3. The PCBA detection device based on image and model recognition according to claim 1, characterized in that: The several detection modules include: an analog detection module, a sound detection module, a display detection module, and a dual firmware burning module; The analog detection module is used to output and receive analog signals to detect the analog input and output ports of the PCBA to be tested; The sound and display detection module is used to receive sound and display signals to detect the sound and display components of the PCBA to be inspected; The dual firmware burning module is used for performing firmware burning and function verification to detect the firmware burning port of the PCBA to be tested.

4. A PCBA detection method based on image and model recognition, characterized in that: The PCBA detection device according to any one of claims 1 to 3 comprises the following steps: In the host computer, several PCBA circuit DES models are established through simulation software, and the detection instructions containing the PCBA model to be tested are input and sent to the test logic generation module and the image recognition module; The image recognition module is used to identify defects in the PCBA to be inspected, and the image recognition results are returned to the host computer; The test logic generation module automatically generates the corresponding test logic according to the PCBA model to be tested and sends it to the detection controller; The detection controller controls the execution of several detection modules according to the test logic, and returns the execution results to the host computer; The host computer uses the several PCBA circuit DES models to generate model recognition results, and outputs detection results in combination with image recognition results.

5. The PCBA detection method based on image and model recognition according to claim 4, characterized in that: When establishing several PCBA circuit DES models, including: Establish several PCBA circuit simulation models through simulation software, simulate the circuit in a fault-free state, and extract the first electrical response of ports and components; Obtain the port and component failure modes contained in the PCBA circuit, analyze them, determine the parameters of the PCBA circuit in the DES model under different failure modes, obtain the failure state set, and establish the port and component failure model; Injecting port and component fault models into corresponding ports and components in several PCBA circuit simulation models to obtain several circuit fault simulation models; Perform simulation based on several circuit fault simulation models to extract the second electrical response of ports and components; The first and second electrical responses are compared to obtain electrical responses that change under different fault modes, and a fault event set is constructed based on the changed electrical responses; According to the fault state set and the fault event set, determine the state transfer function corresponding to the PCBA circuit in the DES model; Establish several PCBA circuit DES models based on the state transfer function; The PCBA circuit DES models include electrical responses under non-fault conditions, fault state sets and fault event sets corresponding to each type of PCBA.

6. The PCBA detection method based on image and model recognition according to claim 5, characterized in that: When automatically generating the corresponding test logic, it includes: Retrieve and load the corresponding test specifications from the preset test specification database according to the PCBA model to be tested; Acquire the type of input and output signals, the test sequence, and the detection threshold from the test specification to generate the test logic, and send the test logic to the detection controller; The test specification database supports dynamic updating, allowing test specifications to be added, modified or deleted according to actual needs; the plurality of PCBA circuit simulation models are associated with the test specifications, and simulation is performed according to the corresponding test specifications.

7. The PCBA detection method based on image and model recognition according to any one of claims 4 to 6, characterized in that: When performing defect identification on the PCBA to be inspected, it includes: constructing a PCBA identification model, and performing defect identification on the PCBA to be inspected based on the generated port positioning frame and component positioning frame; Among them, when generating port and component positioning frames, it includes: The image of the PCBA to be inspected is collected through the image recognition module to obtain the image of the PCBA to be inspected, and the image is preprocessed; Obtain the prior conditions of the PCBA according to the model of the PCBA to be inspected and locate the Mark points; Obtain PCBA port and component information files, and determine the relative relationship between the port and component coordinate systems and the image pixel coordinate system; Convert port and component coordinates into image pixel coordinates and generate port and component positioning frames; When generating the port positioning frame and the component positioning frame, the accuracy of the port positioning frame and the component positioning frame is determined by using the associated corresponding PCBA circuit DES model information.

8. The PCBA detection method based on image and model recognition according to claim 7, characterized in that: When building a PCBA identification model, include: Provide the YOLOv8 model and LSKNet Block module, and add the LSKNet Block module after the SPPF module in the Backbone layer of the YOLOv8 model; Add the LSKNet Block module between the last upsampling layer of the Neck layer of the YOLOv8 model and the final detection layer; Add the LSKNet Block module at the connection of the Concat module before the three detection heads of the Head layer of the YOLOv8 model.

9. The PCBA detection method based on image and model recognition according to claim 8, characterized in that: When building a PCBA identification model, it also includes: The LSKNet Block module at the connection of the Concat module obtains the ratio of the area of ​​the expected selection receptive field of each target category to the area of ​​the port and component positioning box. , reducing the focus on irrelevant or redundant features and improving the model's expressiveness and discrimination, as shown in Formula 8: （8）； In the formula, To include the target category The number of images, For the input image The sum of the spatial selection activations output by all LSKNet Block modules in , The total pixel area of ​​all generated positioning boxes.

10. The PCBA detection method based on image and model recognition according to claim 9, characterized in that: When providing the LSKNet Block module, it includes: The LSKNet Block module is constructed by a large kernel selection sub-block for dynamically adjusting the spatial receptive field, a feedforward network sub-block for channel mixing and feature refinement, and a Norm block; The large core selection sub-block includes: a first fully connected layer, a first GELU activation, an LSK module, and a second fully connected layer connected in sequence; The feedforward network sub-block includes: a third fully connected layer, a deep convolution, a second GELU activation, and a fourth fully connected layer connected in sequence.

Images