Circuit board fault detection method, device and system and storage medium
By using a pre-trained fault detection model to perform multi-scale feature extraction and receptive field optimization on circuit board images, the problem of low efficiency of circuit board fault detection is solved, and fast and accurate fault detection is achieved.
Patent Information
- Application Number
- CN202510889717.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-30
- Publication Date
- 2025-10-03
AI Technical Summary
In the prior art, circuit board fault detection relies on manual labor, resulting in low detection efficiency.
A pre-trained fault detection model is used to detect circuit board image information. The model includes a backbone network, a neck network, and a detection head. Through multi-scale feature extraction, receptive field optimization, and feature fusion, the location and type of burned components can be quickly identified.
It can quickly and accurately obtain the location and type of burned components in the circuit board, improve the efficiency of fault detection, and reduce the dependence on manual detection.
Smart Images
Figure CN120747618A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of electronic technology, and in particular to a circuit board fault detection method, device, system and storage medium. Background Art
[0002] With the rapid development of electronic technology, electronic devices are increasingly used in daily life and industrial production. From smartphones and personal computers to smart cars and medical devices, the complexity and integration of electronic devices are constantly increasing, placing increasing demands on printed circuit boards (PCBs). As the core of electronic devices, the quality of PCBs directly determines their performance and service life.
[0003] In the prior art, circuit board fault detection is usually performed manually, which has the problem of low detection efficiency. Therefore, how to improve the efficiency of circuit board fault detection has become a technical problem that needs to be solved urgently. Summary of the Invention
[0004] The present application provides a circuit board fault detection method, device, system and storage medium to solve the problem in the prior art that circuit board fault detection usually relies on manual labor and has low detection efficiency.
[0005] In a first aspect, an embodiment of the present application provides a circuit board fault detection method, the method comprising:
[0006] Obtain image information corresponding to the circuit board to be inspected;
[0007] The image information is input into a pre-trained fault detection model for fault detection to obtain a fault detection result corresponding to the circuit board to be detected, wherein the fault detection model is used to detect burned components of different scales in the image information to obtain the position and type of the burned components as the fault detection result for output.
[0008] Optionally, the fault detection model includes a backbone network, a neck network and a detection head;
[0009] The step of inputting the image information into a pre-trained fault detection model to perform fault detection and obtaining a fault detection result corresponding to the circuit board to be detected includes:
[0010] Inputting the image information into the backbone network for multi-scale feature extraction to obtain multiple initial feature maps of different scales;
[0011] Inputting the multiple initial feature maps into the neck network for receptive field optimization and feature fusion to obtain multiple fused feature maps;
[0012] The multiple fused feature maps are input into the detection head for fault detection to obtain the fault detection result.
[0013] Optionally, the backbone network includes a first convolution-batch normalization-activation function DBL module, a first residual module, a second residual module, a third residual module and a fourth residual module connected in sequence;
[0014] The image information is input into the backbone network for multi-scale feature extraction to obtain multiple initial feature maps of different scales, including:
[0015] Inputting the image information into the first DBL module, the first residual module and the second residual module in sequence to perform feature extraction to obtain a first initial feature map;
[0016] Inputting the first initial feature map into the third residual module for feature extraction to obtain a second initial feature map;
[0017] Inputting the second initial feature map into the fourth residual module for feature extraction to obtain a third initial feature map;
[0018] The scale of the first initial feature map is larger than that of the second initial feature map, and the scale of the second initial feature map is larger than that of the third initial feature map.
[0019] Optionally, the neck network includes a second DBL module, a first upsampling module, a first splicing module, a first receptive field module, a third DBL module, a second upsampling module, a second splicing module, a second receptive field module, a first DBL module group, a first convolution module, a third splicing module, a second DBL module group, a second convolution module, a third receptive field module, a fourth splicing module and a third DBL module group;
[0020] The multiple initial feature maps are input into the neck network for receptive field optimization and feature fusion to obtain multiple fused feature maps, including:
[0021] Inputting the third initial feature map into the second DBL module and the first upsampling module in sequence for processing to obtain a first intermediate feature map, inputting the second initial feature map into the first receptive field module for receptive field optimization to obtain a second initial feature map after receptive field optimization, and inputting the second initial feature map after receptive field optimization and the first intermediate feature map into the first splicing module for feature fusion to obtain a first fused feature map;
[0022] Inputting the first fused feature map into the third DBL module and the second upsampling module in sequence for processing to obtain a second intermediate feature map, and inputting the first initial feature map into the second receptive field module for receptive field optimization to obtain a first initial feature map after receptive field optimization, and inputting the first initial feature map after receptive field optimization and the second intermediate feature map into the second splicing module and the first DBL module group in sequence for feature fusion to obtain a second fused feature map;
[0023] Inputting the second fused feature map into the first convolution module for feature extraction to obtain a third intermediate feature map, and inputting the third intermediate feature map and the first fused feature map into the third splicing module and the second DBL module group in sequence for feature fusion to obtain a third fused feature map;
[0024] The third fused feature map is input into the second convolution module for feature extraction to obtain a fourth intermediate feature map, and the third initial feature map is input into the third receptive field module for receptive field optimization to obtain a third initial feature map after receptive field optimization, and the third initial feature map after receptive field optimization and the fourth intermediate feature map are sequentially input into the fourth splicing module and the third DBL module group to obtain a fourth fused feature map.
[0025] Optionally, the first receptive field module, the second receptive field module, and the third receptive field module each include a first convolution branch, a second convolution branch, a third convolution branch, and a fourth convolution branch, and the convolution result of the first convolution branch, the convolution result of the second convolution branch, and the convolution result of the third convolution branch are superimposed with the convolution result of the fourth convolution branch after splicing and convolution operations;
[0026] Among them, the first convolution branch includes a 1x1 convolution layer and a 3x3 convolution layer, the second convolution branch and the third convolution branch both include a 1x1 convolution layer, an asymmetric convolution layer and a hole convolution layer, and the fourth convolution branch includes a 1x1 convolution layer.
[0027] Optionally, the detection head includes a first detection branch, a second detection branch and a third detection branch;
[0028] Inputting the multiple fused feature maps into the detection head for fault detection to obtain the fault detection result includes:
[0029] Inputting the second fused feature map into the first detection branch to perform fault detection to obtain a first detection result;
[0030] Inputting the third fused feature map into the second detection branch to perform fault detection to obtain a second detection result;
[0031] Inputting the fourth fused feature map into the third detection branch to perform fault detection to obtain a third detection result;
[0032] The fault detection result is determined based on the first detection result, the second detection result, and the third detection result, wherein the first detection result, the second detection result, and the third detection result are detection results of burned components of different sizes respectively.
[0033] Optionally, before inputting the image information into a pre-trained fault detection model for fault detection to obtain a fault detection result corresponding to the circuit board to be detected, the method further includes:
[0034] Acquire a plurality of sample images, wherein each of the plurality of sample images is a circuit board image and is marked with a position and type of a burned component;
[0035] Dividing the multiple sample images into a training set and a test set;
[0036] The model parameters of the model to be trained are trained using the training set, and the trained model to be trained is tested using the test set to obtain the fault detection model, wherein the model structure of the model to be trained is the same as the model structure of the fault detection model.
[0037] In a second aspect, an embodiment of the present application further provides a circuit board fault detection device, the device comprising:
[0038] A first acquisition module is used to acquire image information corresponding to the circuit board to be inspected;
[0039] A detection module is used to input the image information into a pre-trained fault detection model for fault detection to obtain a fault detection result corresponding to the circuit board to be detected, wherein the fault detection model is used to detect burned components of different scales in the image information to obtain the position and type of the burned components as the fault detection result for output.
[0040] In a third aspect, an embodiment of the present application further provides a circuit board fault detection system, the system comprising a main control chip, a slave computer connected to the main control chip, a display module and an alarm module, and an image acquisition module connected to the slave computer;
[0041] The image acquisition module is used to acquire image information corresponding to the circuit board to be inspected and send the image information to the slave computer;
[0042] A pre-trained fault detection model is pre-deployed in the lower computer, and the lower computer is used to input the image information into the fault detection model to perform fault detection, and obtain a fault detection result corresponding to the circuit board to be detected, wherein the fault detection model is used to detect burned components of different scales in the image information to obtain the location and type of the burned components as the fault detection result for output;
[0043] The main control chip is used to obtain the fault detection result from the lower computer, transmit the fault detection result to the display module for display, and trigger the alarm module to issue an alarm according to the fault detection result.
[0044] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the circuit board fault detection method described in the first aspect is implemented.
[0045] The above technical solution provided by the embodiment of the present application has the following advantages over the prior art: The method provided by the embodiment of the present application obtains image information corresponding to the circuit board to be inspected; inputs the image information into a pre-trained fault detection model for fault detection, and obtains a fault detection result corresponding to the circuit board to be inspected, wherein the fault detection model is used to detect burnt components of different scales in the image information to obtain the position and type of the burnt components as the fault detection result for output. In this way, the fault detection model can be used to detect burnt components of different scales in the image information corresponding to the circuit board to be inspected, thereby quickly and accurately obtaining the position and type of each burnt component without relying on manual detection, thereby improving the efficiency of circuit board fault detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0048] One or more embodiments are exemplarily illustrated by pictures in the corresponding drawings. These exemplifications do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings are represented as similar elements. Unless otherwise stated, the figures in the drawings do not constitute proportional limitations.
[0049] Figure 1 A schematic diagram of a circuit board fault detection method provided in an embodiment of the present application;
[0050] Figure 2 A schematic diagram of the structure of a fault detection model provided in an embodiment of the present application;
[0051] Figure 3 A schematic diagram of the structure of a receptive field module provided in an embodiment of the present application;
[0052] Figure 4 A schematic structural diagram of a circuit board fault detection device provided in an embodiment of the present application;
[0053] Figure 5 A schematic diagram of the structure of a circuit board fault detection system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0054] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.
[0055] The disclosure below provides many different embodiments or examples for implementing different structures of the present application. In order to simplify the disclosure of the present application, the components and settings of specific examples are described below. Of course, these are merely examples and are not intended to limit the present application. In addition, the present application may repeat reference numbers and / or letters in different examples. Such repetition is for the purpose of simplicity and clarity and does not in itself indicate the relationship between the various embodiments and / or settings discussed.
[0056] In order to solve the problem in the prior art that circuit board fault detection usually relies on manual labor and has low detection efficiency, the present application provides a circuit board fault detection method, device, system and storage medium, which can improve the circuit board fault detection efficiency.
[0057] See also Figure 1 , Figure 1 This is a flow chart of a circuit board fault detection method provided in an embodiment of the present application. Figure 1 As shown, the circuit board fault detection method may include the following steps:
[0058] Step S101: Obtain image information corresponding to the circuit board to be inspected.
[0059] Specifically, the circuit board to be inspected can be any PCB that requires fault detection, such as a PCB in a smartphone, smart TV, air conditioner, computer, vehicle-mounted device, medical device, etc. The image information can include one or more images of the circuit board to be inspected, and the image can be any color image.
[0060] When acquiring image information corresponding to the circuit board to be inspected, one or more cameras can be used to capture the image from one or more angles of the circuit board to be inspected. If there is only one camera, the image captured by that camera can be directly used as the image information corresponding to the circuit board to be inspected. If there are multiple cameras, the images captured by these multiple cameras can be spliced together, and the spliced image can be used as the image information corresponding to the circuit board to be inspected. This embodiment of the present application does not impose any specific limitations.
[0061] Step S102: Input the image information into a pre-trained fault detection model for fault detection to obtain a fault detection result corresponding to the circuit board to be detected, wherein the fault detection model is used to detect burned components of different scales in the image information to obtain the location and type of the burned components as the fault detection result for output.
[0062] Specifically, the fault detection model can be pre-trained based on an improved YOLOv3, YOLOv4, YOLOv5, YOLOv6, YOLOv7, YOLOv8, or YOLOv9 network model, and is not specifically limited in the present embodiment. This fault detection model can quickly identify burned components of different scales in image information, thereby obtaining the location and type of the burned component and outputting it as a fault detection result, thereby assisting the user in subsequent fault analysis of the burned location.
[0063] Through the above method, the fault detection model can be used to detect burned components of different scales in the image information corresponding to the circuit board to be inspected, so as to quickly and accurately obtain the position and type of each burned component without relying on manual detection, thereby improving the fault detection efficiency of the circuit board.
[0064] In an optional embodiment, the fault detection model includes a backbone network, a neck network, and a detection head. The step S102, inputting the image information into a pre-trained fault detection model for fault detection, and obtaining a fault detection result corresponding to the circuit board to be detected, includes:
[0065] The image information is input into the backbone network for multi-scale feature extraction to obtain multiple initial feature maps of different scales;
[0066] Input multiple initial feature maps into the neck network for receptive field optimization and feature fusion to obtain multiple fused feature maps;
[0067] Multiple fused feature maps are input into the detection head for fault detection to obtain the fault detection results.
[0068] Specifically, the fault detection model can be implemented based on an improved YOLOv3 network model. The fault detection model can include a backbone network, a neck network, and a detection head. The backbone network can be an improved YOLOv3 backbone network that can perform multi-scale feature extraction to obtain an initial feature map. The neck network can be an improved YOLOv3 neck network that can be used to optimize the receptive field and fuse features to obtain a fused feature map. The detection head can be used to perform fault detection to obtain a fault detection result.
[0069] When inputting image information into a pre-trained fault detection model for fault detection and obtaining the fault detection result corresponding to the circuit board to be detected, the image information can be first input into the backbone network for multi-scale feature extraction to obtain multiple initial feature maps of different scales, and then the multiple initial feature maps are input into the neck network for receptive field optimization and feature fusion to obtain multiple fused feature maps, and then the multiple fused feature maps are input into the detection head for fault detection to obtain the fault detection result.
[0070] Through the above method, the fault detection model can be used to detect burned components of different scales in the image information corresponding to the circuit board to be inspected, so as to quickly and accurately obtain the position and type of each burned component without relying on manual detection, thereby improving the fault detection efficiency of the circuit board.
[0071] In an alternative embodiment, see Figure 2 , the backbone network includes a first convolution-batch normalization-activation function DBL module, a first residual module, a second residual module, a third residual module and a fourth residual module connected in sequence;
[0072] In the above steps, the image information is input into the backbone network for multi-scale feature extraction, and multiple initial feature maps of different scales are obtained, including:
[0073] The image information is sequentially input into the first DBL module, the first residual module, and the second residual module for feature extraction to obtain a first initial feature map;
[0074] Inputting the first initial feature map into the third residual module for feature extraction to obtain a second initial feature map;
[0075] Inputting the second initial feature map into the fourth residual module for feature extraction to obtain a third initial feature map;
[0076] The scale of the first initial feature map is larger than that of the second initial feature map, and the scale of the second initial feature map is larger than that of the third initial feature map.
[0077] Specifically, the backbone network can be an improved YOLOv3 backbone network, which may include a first convolution-batch normalization-activation function (Darknet Conv-BatchNorm-LeakyReLU, referred to as DBL) module, a first residual module (ResidualBlock, referred to as Res), a second residual module, a third residual module and a fourth residual module connected in sequence. Among them, the first DBL module may include a convolutional layer (Conv), a batch normalization layer (BN) and an activation function layer (i.e., Leaky ReLU activation function), which is mainly used for basic feature extraction to ensure nonlinear expression and normalization. The first residual module is a single residual module (i.e., RES1), which includes two DBLs and a cross-layer connection (Shortcut Connection). Its main function is to introduce residual connections to avoid information loss in shallow networks. The second residual module is two consecutive residual modules (i.e., RES2). Its main function is to gradually extract deeper features by stacking multiple residual modules. The third residual module and the fourth residual module are 8 consecutive residual modules (ie, RES8), which are mainly used to perform deep feature extraction and learn complex features through a large number of residual blocks.
[0078] When inputting image information into the backbone network for multi-scale feature extraction to obtain multiple initial feature maps of different scales, the image information can be first input into the first DBL module, the first residual module, and the second residual module for feature extraction, obtaining a first initial feature map feat0 of size (104, 104, 128). The first initial feature map is then input into the third residual module for feature extraction, obtaining a second initial feature map feat1 of size (52, 52, 256). The second initial feature map is then input into the fourth residual module for feature extraction, obtaining a third initial feature map feat2 of size (26, 26, 512). In this way, the three layers of feature maps feat0, feat1, and feat2 extracted from the backbone network are located in the middle, lower-middle, and bottom layers of the network, respectively. The first initial feature map feat0 is used to detect large-scale objects; the second initial feature map feat1 is used to detect medium-scale objects; and the third initial feature map feat2 is used to detect small-scale objects.
[0079] The main difference between the improved YOLOv3 backbone network and the traditional YOLOv3 backbone network (i.e., the Darknet-53 extraction network) is that the three initial feature maps output by the traditional YOLOv3 backbone network have sizes of (52, 52, 256), (26, 26, 512), and (13, 13, 1024), while the three initial feature maps output by the improved YOLOv3 backbone network are (104, 104, 128), (52, 52, 256), and (26, 26, 512). This shows that the improved YOLOv3 backbone network removes the initial feature map of size (13, 13, 1024) and instead outputs an initial feature map of size (104, 104, 128). Since the larger the initial feature map size, the smaller the receptive field, it can better capture finer-grained information and be used to detect very small targets. Therefore, compared with the traditional YOLOv3 backbone network, the improved YOLOv3 backbone network can better detect smaller burned components in circuit boards, thereby improving detection accuracy.
[0080] In an alternative embodiment, please continue to see Figure 2 , the neck network includes a second DBL module, a first upsampling module, a first splicing module, a first receptive field module, a third DBL module, a second upsampling module, a second splicing module, a second receptive field module, a first DBL module group, a first convolution module, a third splicing module, a second DBL module group, a second convolution module, a third receptive field module, a fourth splicing module and a third DBL module group;
[0081] In the above steps, multiple initial feature maps are input into the neck network for receptive field optimization and feature fusion, resulting in multiple fused feature maps, including:
[0082] Inputting the third initial feature map into the second DBL module and the first upsampling module for processing in sequence to obtain a first intermediate feature map, and inputting the second initial feature map into the first receptive field module for receptive field optimization to obtain a second initial feature map after receptive field optimization, and inputting the second initial feature map after receptive field optimization and the first intermediate feature map into the first splicing module for feature fusion to obtain a first fused feature map;
[0083] The first fused feature map is sequentially input into the third DBL module and the second upsampling module for processing to obtain a second intermediate feature map, and the first initial feature map is input into the second receptive field module for receptive field optimization to obtain a first initial feature map after receptive field optimization, and the first initial feature map and the second intermediate feature map after receptive field optimization are sequentially input into the second splicing module and the first DBL module group for feature fusion to obtain a second fused feature map;
[0084] Input the second fused feature map into the first convolution module for feature extraction to obtain a third intermediate feature map, and input the third intermediate feature map and the first fused feature map into the third splicing module and the second DBL module group in sequence for feature fusion to obtain a third fused feature map;
[0085] The third fused feature map is input into the second convolution module for feature extraction to obtain a fourth intermediate feature map, and the third initial feature map is input into the third receptive field module for receptive field optimization to obtain a third initial feature map after receptive field optimization, and the third initial feature map after receptive field optimization and the fourth intermediate feature map are sequentially input into the fourth splicing module and the third DBL module group to obtain a fourth fused feature map.
[0086] Specifically, the neck network can be an improved YOLOv3 neck network, which differs from the traditional YOLOv3 neck network in that the improved YOLOv3 neck network introduces multiple receptive field modules to optimize the receptive fields of multiple initial feature maps respectively. The neck network may include a second DBL module, a first upsampling module, a first splicing module, a first receptive field module, a third DBL module, a second upsampling module, a second splicing module, a second receptive field module, a first DBL module group, a first convolution module, a third splicing module, a second DBL module group, a second convolution module, a third receptive field module, a fourth splicing module and a third DBL module group, wherein the first DBL module group, the second DBL module group and the third DBL module group here refer to a combination of 5 consecutive residual modules (i.e., DBL*5). The first convolution module and the second convolution module here are used to perform a convolution operation with a step size of 2.
[0087] When multiple initial feature maps are input into the neck network for receptive field optimization and feature fusion to obtain multiple fused feature maps, the third initial feature map feat2 can be sequentially input into the second DBL module (for performing two convolution Conv2D 3x3 and Conv2D 1x1 operations to adjust the number of channels) and the first upsampling module (for performing upsampling operation) for processing to obtain a first intermediate feature map of size (52, 52, 256), and the second initial feature map feat1 is input into the first receptive field module (Receptive Field Block, referred to as RFB) for receptive field optimization to obtain a second initial feature map with a receptive field optimization of size (52, 52, 256), and the second initial feature map with receptive field optimization and the first intermediate feature map are input into the first splicing module for feature fusion to obtain a first fused feature map x1 with a size of (52, 52, 512). The first fused feature map x1 is then fed into the third DBL module and the second upsampling module for processing, resulting in a second intermediate feature map of size (104, 104, 256). The first initial feature map feat0 is then fed into the second receptive field module for receptive field optimization, resulting in a first initial feature map with receptive field optimization of size (104, 104, 128). The receptive field-optimized first initial feature map and the second intermediate feature map are then fed into the second concatenation module and the first DBL module for feature fusion, resulting in a second fused feature map x2 with size (104, 104, 384). The second fused feature map x2 is then fed into the first convolution module for feature extraction, resulting in a third intermediate feature map of size (52, 52, 768). The third intermediate feature map and the first fused feature map are then fed into the third concatenation module and the second DBL module for feature fusion, resulting in a third fused feature map x3 with size (52, 52, 768). Finally, the third fused feature map x3 is input into the second convolution module for feature extraction to obtain a fourth intermediate feature map with a size of (26, 26, 1024), and the third initial feature map feat2 is input into the third receptive field module for receptive field optimization to obtain a third initial feature map with a receptive field optimization of size (26, 26, 512), and the third initial feature map and the fourth intermediate feature map after receptive field optimization are input into the fourth splicing module and the third DBL module group in sequence to obtain a fourth fused feature map x4 with a size of (26, 26, 1024).
[0088] In this embodiment, the receptive field is optimized by the receptive field module, and the multiple initial feature maps after the receptive field optimization are subjected to feature fusion, so that the fault detection result can more effectively capture the features of small targets, thereby improving the detection accuracy.
[0089] In an alternative embodiment, see Figure 3 The first receptive field module, the second receptive field module and the third receptive field module all include a first convolution branch, a second convolution branch, a third convolution branch and a fourth convolution branch. The convolution result of the first convolution branch, the convolution result of the second convolution branch and the convolution result of the third convolution branch are concatenated and convolved, and then superimposed with the convolution result of the fourth convolution branch.
[0090] Among them, the first convolution branch includes a 1x1 convolution layer and a 3x3 convolution layer, the second convolution branch and the third convolution branch both include a 1x1 convolution layer, an asymmetric convolution layer and a hole convolution layer, and the fourth convolution branch includes a 1x1 convolution layer.
[0091] Specifically, the first receptive field module, the second receptive field module, and the third receptive field module all include 4 convolution branches, among which the first convolution branch includes a 1x1 convolution layer (i.e., Bconv1x1) and a 3x3 convolution layer (i.e., Bconv3x3), which are used to aggregate information and reduce the number of channels. The second convolution branch includes a 1x1 convolution layer (i.e., Bconv1x1), an asymmetric convolution layer (3x1 and 1x3), and a 3x3 hole convolution layer (with an expansion rate of 2). The second convolution branch includes a 1x1 convolution layer (i.e., Bconv1x1), an asymmetric convolution layer (3x1 and 1x3), and a 3x3 hole convolution layer (with an expansion rate of 3). The fourth convolution branch includes a 1x1 convolution layer (i.e., Bconv1x1). Finally, the results of the first three convolution branches can be spliced and convolved, and then spliced with the fourth convolution branch. This can optimize the receptive field of the initial feature map and improve detection accuracy.
[0092] In an alternative embodiment, please continue to see Figure 2 , the detection head includes a first detection branch, a second detection branch and a third detection branch;
[0093] In the above steps, multiple fused feature maps are input to the detection head for fault detection to obtain fault detection results, including:
[0094] Inputting the second fused feature map into the first detection branch to perform fault detection to obtain a first detection result;
[0095] Inputting the third fused feature map into the second detection branch to perform fault detection, thereby obtaining a second detection result;
[0096] Inputting the fourth fused feature map into the third detection branch to perform fault detection, thereby obtaining a third detection result;
[0097] A fault detection result is determined based on the first detection result, the second detection result, and the third detection result, wherein the first detection result, the second detection result, and the third detection result are detection results of burned components of different sizes respectively.
[0098] Specifically, the first detection branch, the second detection branch, and the third detection branch each include a DBL module and a convolution module.
[0099] When multiple fused feature maps are input into the detection head for fault detection to obtain a fault detection result, the second fused feature map x2 can be input into the first detection branch for fault detection to obtain a first detection result y1 of size (104, 104, 90), and the third fused feature map x3 can be input into the second detection branch for fault detection to obtain a second detection result y2 of size (52, 52, 90), and the fourth fused feature map x4 can be input into the third detection branch for fault detection to obtain a third detection result y3 of size (26, 26, 90). Subsequently, the fault detection result can be determined based on the first detection result y1, the second detection result y2, and the third detection result y3. In this way, different fused feature maps are detected in parallel through the three detection branches, thereby improving the overall detection efficiency of the fault detection model.
[0100] After obtaining the first detection result y1, the second detection result y2, and the third detection result y3, all predicted bounding boxes in y1, y2, and y3 can be collected into a list. Each predicted bounding box should contain a category, confidence, and bounding box coordinates (usually expressed as the coordinates of the upper left corner and the lower right corner, or the center point coordinates and width and height). Then calculate the comprehensive confidence of each predicted bounding box, which usually includes two parts: object confidence (predicting the probability that the area contains the target object) and category confidence (predicting the probability that the target object belongs to a specific category). The comprehensive confidence can be obtained by multiplying the object confidence by the category confidence. Then all predicted bounding boxes are sorted in descending order according to the comprehensive confidence, so that predicted bounding boxes with higher confidence are given priority in subsequent processing. Then apply non-maximum suppression (NMS for short), and the specific process is as follows:
[0101] 1. Initialize the retention list: Create an empty list to store the bounding boxes that are ultimately retained.
[0102] 2. Traverse the sorted bounding boxes:
[0103] (1) Start with the bounding box with the highest confidence and add it to the retained list.
[0104] (2) For the current bounding box, calculate the intersection over union (IoU) between it and all other remaining bounding boxes.
[0105] (3) If the IoU between a certain bounding box and the current bounding box is greater than the preset threshold of 0.5 (the standard setting is 0.5, and the threshold is debugged according to the test data), it is removed from the candidate list.
[0106] 3. Repeat until done: Continue processing the next bounding box with the highest confidence until all bounding boxes have been checked.
[0107] In this way, the bounding boxes in the retained list are the final detection results. These results can be used for subsequent visualization, storage or other processing.
[0108] Through the above steps, non-maximum suppression (NMS) can be effectively used to filter out overlapping bounding boxes, thereby obtaining clear and accurate detection results.
[0109] In an optional embodiment, before step S102, inputting the image information into a pre-trained fault detection model to perform fault detection and obtaining a fault detection result corresponding to the circuit board to be detected, the method further includes:
[0110] Acquire a plurality of sample images, wherein each of the plurality of sample images is a circuit board image and is marked with a position and type of a burned component;
[0111] Divide multiple sample images into training sets and test sets;
[0112] The model parameters of the model to be trained are trained using the training set, and the trained model to be trained is tested using the test set to obtain a fault detection model, wherein the model structure of the model to be trained is the same as the model structure of the fault detection model.
[0113] Specifically, when acquiring multiple sample images, a high-resolution camera can be used to capture images of circuit boards with burned components and circuit boards without burned components at different angles, different lighting conditions, and different distances. The collected sample images should cover a variety of fault types, including but not limited to rectifier bridge burnout, intelligent power module (IPM) module burnout, insulated gate bipolar transistor (IGBT) burnout, etc. It should be noted that in such specific application scenarios, since the sample images of circuit boards with burned components are limited, it is difficult to meet the needs of large-scale data sets based solely on the occurrence of after-sales failures. Therefore, noise, pixel value changes, inversion processing, Sobel derivative filtering, Gaussian filtering, etc. can be added to the sample images. By using the above methods, the sample images can be enriched to make subsequent model training more reliable. When annotating the sample images, image annotation tools such as labelImg can be used to manually annotate the sample images to annotate the areas of burned components in the sample images according to their position and size. The set label categories may include but are not limited to: normal and no fault, rectifier bridge burnout fault, IPM module burnout fault, IGBT1 burnout fault, IGBT2 burnout fault, etc. The label categories are not limited to the above categories and can be expanded with different fault types. Among them, for the obscured burnt area, only the visible part can be marked; for the burnt area that is too small or too blurred in the background, no marking is required. The generated annotation file format is the PASCAL VOC format based on the Extensible Markup Language (XML), and the input size of the image is unified.
[0114] After acquiring multiple sample images, you can divide them into training and test sets. For example, 85% of the sample images can be used as the training set, and 15% as the test set. Ensure that the distribution of samples across categories in the training and test sets is balanced to avoid bias during model training. Finally, you can use the training set to train the model parameters of the model to be trained, and use the test set to test the trained model to obtain a fault detection model.
[0115] During the model construction process, careful design of the model structure and hyperparameters allows for full utilization of hardware resources to further optimize final performance. The trained model is further enhanced and converted into a lightweight MobileNet neural network. This improves upon traditional convolution by employing depthwise separable convolutions, splitting the standard convolution operation into channel-by-channel and depthwise convolutions, and utilizing 1x1 convolutions to transform the number of channels. This design significantly reduces the number of model parameters and computational complexity, enabling efficient model training and inference. It can also be used on mobile devices, improving the real-time and convenience of circuit board fault detection.
[0116] During model deployment, the quantized and optimized TiLite model must be converted into the kmodel format supported by the slave computer (such as the K210 visual recognition module) using the ncc.exe tool. Finally, the generated kmodel is stored on the slave computer's Secure Digital Card (SD card). Once the model is deployed, it can be called from within the program. This allows the main control chip to communicate with the slave computer via serial ports for data transmission and command control. The main control chip controls the display and alarm modules, taking appropriate actions based on the inference results returned by the slave computer. The display module displays fault detection results, while the alarm module issues an alarm when a fault is detected. Finally, the entire system is tested to ensure that all modules are functioning properly.
[0117] Through the above method, a fault detection model can be trained and deployed, which facilitates the subsequent use of the fault detection model to detect burned components of different scales in the image information corresponding to the circuit board to be inspected, thereby quickly and accurately obtaining the position and type of each burned component without relying on manual detection, thereby improving the fault detection efficiency of the circuit board.
[0118] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of a circuit board fault detection device provided in an embodiment of the present application. Figure 4 As shown, the circuit board fault detection device 400 includes:
[0119] The first acquisition module 401 is used to acquire image information corresponding to the circuit board to be inspected;
[0120] Detection module 402 is used to input image information into a pre-trained fault detection model for fault detection, and obtain a fault detection result corresponding to the circuit board to be detected, wherein the fault detection model is used to detect burned components of different scales in the image information to obtain the location and type of the burned components as the fault detection result for output.
[0121] Furthermore, the fault detection model includes a backbone network, a neck network, and a detection head; the detection module 402 includes:
[0122] The feature extraction submodule is used to input image information into the backbone network for multi-scale feature extraction to obtain multiple initial feature maps of different scales;
[0123] The feature fusion submodule is used to input multiple initial feature maps into the neck network for receptive field optimization and feature fusion to obtain multiple fused feature maps;
[0124] The fault detection submodule is used to input multiple fused feature maps into the detection head for fault detection to obtain fault detection results.
[0125] Furthermore, the backbone network includes a first convolution-batch normalization-activation function DBL module, a first residual module, a second residual module, a third residual module and a fourth residual module connected in sequence; the feature extraction submodule includes:
[0126] A first feature extraction unit is used to sequentially input image information into a first DBL module, a first residual module, and a second residual module for feature extraction to obtain a first initial feature map;
[0127] A second feature extraction unit is used to input the first initial feature map into a third residual module for feature extraction to obtain a second initial feature map;
[0128] A third feature extraction unit is used to input the second initial feature map into the fourth residual module for feature extraction to obtain a third initial feature map;
[0129] The scale of the first initial feature map is larger than that of the second initial feature map, and the scale of the second initial feature map is larger than that of the third initial feature map.
[0130] Furthermore, the neck network includes a second DBL module, a first upsampling module, a first splicing module, a first receptive field module, a third DBL module, a second upsampling module, a second splicing module, a second receptive field module, a first DBL module group, a first convolution module, a third splicing module, a second DBL module group, a second convolution module, a third receptive field module, a fourth splicing module and a third DBL module group; the feature fusion submodule includes:
[0131] a first feature fusion unit, configured to sequentially input the third initial feature map into the second DBL module and the first upsampling module for processing to obtain a first intermediate feature map, input the second initial feature map into the first receptive field module for receptive field optimization to obtain a second initial feature map after receptive field optimization, and input the second initial feature map after receptive field optimization and the first intermediate feature map into the first splicing module for feature fusion to obtain a first fused feature map;
[0132] A second feature fusion unit is configured to sequentially input the first fused feature map into the third DBL module and the second upsampling module for processing to obtain a second intermediate feature map, and input the first initial feature map into the second receptive field module for receptive field optimization to obtain a first initial feature map after receptive field optimization, and sequentially input the first initial feature map and the second intermediate feature map after receptive field optimization into the second splicing module and the first DBL module group for feature fusion to obtain a second fused feature map;
[0133] A third feature fusion unit is used to input the second fused feature map into the first convolution module for feature extraction to obtain a third intermediate feature map, and input the third intermediate feature map and the first fused feature map into the third splicing module and the second DBL module group in sequence for feature fusion to obtain a third fused feature map;
[0134] The fourth feature fusion unit is used to input the third fused feature map into the second convolution module for feature extraction to obtain a fourth intermediate feature map, and input the third initial feature map into the third receptive field module for receptive field optimization to obtain a third initial feature map after receptive field optimization, and input the third initial feature map after receptive field optimization and the fourth intermediate feature map into the fourth splicing module and the third DBL module group in sequence to obtain a fourth fused feature map.
[0135] Furthermore, the first receptive field module, the second receptive field module, and the third receptive field module each include a first convolution branch, a second convolution branch, a third convolution branch, and a fourth convolution branch. The convolution result of the first convolution branch, the convolution result of the second convolution branch, and the convolution result of the third convolution branch are concatenated and convolved, and then superimposed with the convolution result of the fourth convolution branch.
[0136] Among them, the first convolution branch includes a 1x1 convolution layer and a 3x3 convolution layer, the second convolution branch and the third convolution branch both include a 1x1 convolution layer, an asymmetric convolution layer and a hole convolution layer, and the fourth convolution branch includes a 1x1 convolution layer.
[0137] Furthermore, the detection head includes a first detection branch, a second detection branch, and a third detection branch; the fault detection submodule includes:
[0138] a first fault detection unit, configured to input the second fused feature map into the first detection branch to perform fault detection and obtain a first detection result;
[0139] A second fault detection unit is used to input the third fused feature map into the second detection branch to perform fault detection and obtain a second detection result;
[0140] a third fault detection unit, configured to input the fourth fused feature map into the third detection branch to perform fault detection and obtain a third detection result;
[0141] The determination unit is used to determine a fault detection result based on the first detection result, the second detection result and the third detection result, wherein the first detection result, the second detection result and the third detection result are detection results of burned components of different sizes respectively.
[0142] Furthermore, the circuit board fault detection device 400 further includes:
[0143] a second acquisition module, configured to acquire a plurality of sample images, wherein each of the plurality of sample images is a circuit board image and is marked with a position and type of a burned component;
[0144] A partitioning module is used to partition multiple sample images into training sets and test sets;
[0145] The training module is used to train the model parameters of the to-be-trained model using the training set, and to test the trained to-be-trained model using the test set to obtain a fault detection model, wherein the model structure of the to-be-trained model is the same as the model structure of the fault detection model.
[0146] It should be noted that the circuit board fault detection device 400 can implement the circuit board fault detection method provided by any of the aforementioned method embodiments and can achieve the same technical effects, which will not be described in detail here.
[0147] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of a circuit board fault detection system provided in an embodiment of the present application. Figure 5 As shown, the circuit board fault detection system 500 includes: a main control chip 501, a lower computer 502 connected to the main control chip 501, a display module 503 and an alarm module 504, and an image acquisition module 505 connected to the lower computer 502;
[0148] The image acquisition module 505 is used to acquire image information corresponding to the circuit board to be inspected and send the image information to the slave computer 502;
[0149] A pre-trained fault detection model is pre-deployed in the lower computer 502. The lower computer 502 is used to obtain image information corresponding to the circuit board to be inspected, and input the image information into the fault detection model to perform fault detection, thereby obtaining a fault detection result corresponding to the circuit board to be inspected. The fault detection model is used to detect burned components of different scales in the image information, and obtain the location and type of the burned components as the fault detection result for output.
[0150] The main control chip 501 is used to obtain the fault detection result from the lower computer 502, transmit the fault detection result to the display module 503 for display, and trigger the alarm module 504 to alarm according to the fault detection result.
[0151] The main control chip 501 can be any control chip, and the main control chip 501 can execute the circuit board fault detection method provided by any of the above method embodiments. Since the circuit board fault detection method has been described in detail in the above embodiments, it will not be repeated here.
[0152] An embodiment of the present application further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the circuit board fault detection method provided in any of the aforementioned method embodiments is implemented.
[0153] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.
[0154] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a general hardware platform, or of course, by hardware. Based on this understanding, the above technical solution, in essence, or the part that contributes to the relevant technology, can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiment.
[0155] It should be understood that the terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "comprise", "include", "contain" and "have" are inclusive and therefore specify the presence of stated features, steps, operations, elements and / or parts, but do not exclude the presence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not to be construed as necessarily requiring them to be performed in the specific order described or illustrated, unless the order of execution is clearly indicated. It should also be understood that additional or alternative steps may be used.
[0156] The foregoing is merely a list of specific embodiments of the present application, intended to enable those skilled in the art to understand or implement the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the broadest scope consistent with the principles and novel features of the present application.
Claims
1. A circuit board fault detection method, characterized in that: The method comprises: Obtain image information corresponding to the circuit board to be inspected; The image information is input into a pre-trained fault detection model for fault detection to obtain a fault detection result corresponding to the circuit board to be detected, wherein the fault detection model is used to detect burned components of different scales in the image information to obtain the position and type of the burned components as the fault detection result for output.
2. The method according to claim 1, characterized in that The fault detection model includes a backbone network, a neck network and a detection head; The step of inputting the image information into a pre-trained fault detection model to perform fault detection and obtaining a fault detection result corresponding to the circuit board to be detected includes: Inputting the image information into the backbone network for multi-scale feature extraction to obtain multiple initial feature maps of different scales; Inputting the multiple initial feature maps into the neck network for receptive field optimization and feature fusion to obtain multiple fused feature maps; The multiple fused feature maps are input into the detection head for fault detection to obtain the fault detection result.
3. The method according to claim 2, characterized in that The backbone network includes a first convolution-batch normalization-activation function DBL module, a first residual module, a second residual module, a third residual module and a fourth residual module connected in sequence; The image information is input into the backbone network for multi-scale feature extraction to obtain multiple initial feature maps of different scales, including: Inputting the image information into the first DBL module, the first residual module and the second residual module in sequence to perform feature extraction to obtain a first initial feature map; Inputting the first initial feature map into the third residual module for feature extraction to obtain a second initial feature map; Inputting the second initial feature map into the fourth residual module for feature extraction to obtain a third initial feature map; The scale of the first initial feature map is larger than that of the second initial feature map, and the scale of the second initial feature map is larger than that of the third initial feature map.
4. The method according to claim 3, characterized in that The neck network includes a second DBL module, a first upsampling module, a first splicing module, a first receptive field module, a third DBL module, a second upsampling module, a second splicing module, a second receptive field module, a first DBL module group, a first convolution module, a third splicing module, a second DBL module group, a second convolution module, a third receptive field module, a fourth splicing module and a third DBL module group; The multiple initial feature maps are input into the neck network for receptive field optimization and feature fusion to obtain multiple fused feature maps, including: Inputting the third initial feature map into the second DBL module and the first upsampling module in sequence for processing to obtain a first intermediate feature map, inputting the second initial feature map into the first receptive field module for receptive field optimization to obtain a second initial feature map after receptive field optimization, and inputting the second initial feature map after receptive field optimization and the first intermediate feature map into the first splicing module for feature fusion to obtain a first fused feature map; Inputting the first fused feature map into the third DBL module and the second upsampling module in sequence for processing to obtain a second intermediate feature map, and inputting the first initial feature map into the second receptive field module for receptive field optimization to obtain a first initial feature map after receptive field optimization, and inputting the first initial feature map after receptive field optimization and the second intermediate feature map into the second splicing module and the first DBL module group in sequence for feature fusion to obtain a second fused feature map; Inputting the second fused feature map into the first convolution module for feature extraction to obtain a third intermediate feature map, and inputting the third intermediate feature map and the first fused feature map into the third splicing module and the second DBL module group in sequence for feature fusion to obtain a third fused feature map; The third fused feature map is input into the second convolution module for feature extraction to obtain a fourth intermediate feature map, and the third initial feature map is input into the third receptive field module for receptive field optimization to obtain a third initial feature map after receptive field optimization, and the third initial feature map after receptive field optimization and the fourth intermediate feature map are sequentially input into the fourth splicing module and the third DBL module group to obtain a fourth fused feature map.
5. The method according to claim 4, characterized in that The first receptive field module, the second receptive field module, and the third receptive field module each include a first convolution branch, a second convolution branch, a third convolution branch, and a fourth convolution branch, and the convolution result of the first convolution branch, the convolution result of the second convolution branch, and the convolution result of the third convolution branch are spliced and convolved, and then superimposed with the convolution result of the fourth convolution branch; Among them, the first convolution branch includes a 1x1 convolution layer and a 3x3 convolution layer, the second convolution branch and the third convolution branch both include a 1x1 convolution layer, an asymmetric convolution layer and a hole convolution layer, and the fourth convolution branch includes a 1x1 convolution layer.
6. The method according to claim 4, characterized in that The detection head includes a first detection branch, a second detection branch and a third detection branch; Inputting the multiple fused feature maps into the detection head for fault detection to obtain the fault detection result includes: Inputting the second fused feature map into the first detection branch to perform fault detection to obtain a first detection result; Inputting the third fused feature map into the second detection branch to perform fault detection to obtain a second detection result; Inputting the fourth fused feature map into the third detection branch to perform fault detection to obtain a third detection result; The fault detection result is determined based on the first detection result, the second detection result, and the third detection result, wherein the first detection result, the second detection result, and the third detection result are detection results of burned components of different sizes respectively.
7. The method according to claim 1, characterized in that Before inputting the image information into a pre-trained fault detection model to perform fault detection and obtaining a fault detection result corresponding to the circuit board to be detected, the method further includes: Acquire a plurality of sample images, wherein each of the plurality of sample images is a circuit board image and is marked with a position and type of a burned component; Dividing the multiple sample images into a training set and a test set; The model parameters of the model to be trained are trained using the training set, and the trained model to be trained is tested using the test set to obtain the fault detection model, wherein the model structure of the model to be trained is the same as the model structure of the fault detection model.
8. A circuit board fault detection device, characterized in that: The device comprises: A first acquisition module is used to acquire image information corresponding to the circuit board to be inspected; A detection module is used to input the image information into a pre-trained fault detection model for fault detection to obtain a fault detection result corresponding to the circuit board to be detected, wherein the fault detection model is used to detect burned components of different scales in the image information to obtain the position and type of the burned components as the fault detection result for output.
9. A circuit board fault detection system, characterized in that: The system includes a main control chip, a slave computer connected to the main control chip, a display module and an alarm module, and an image acquisition module connected to the slave computer; The image acquisition module is used to acquire image information corresponding to the circuit board to be inspected and send the image information to the slave computer; A pre-trained fault detection model is pre-deployed in the lower computer, and the lower computer is used to input the image information into the fault detection model to perform fault detection, and obtain a fault detection result corresponding to the circuit board to be detected, wherein the fault detection model is used to detect burned components of different scales in the image information to obtain the location and type of the burned components as the fault detection result for output; The main control chip is used to obtain the fault detection result from the lower computer, transmit the fault detection result to the display module for display, and trigger the alarm module to issue an alarm according to the fault detection result.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the circuit board fault detection method according to any one of claims 1 to 7 is implemented.