A visual detection method for intelligent segmentation of circuit board components

Abstract

The application belongs to the technical field of image detection, and particularly relates to a visual detection method for intelligent segmentation of components on a circuit board. The method comprises the following steps: collecting original images of the circuit board and performing manual labeling and correction to construct a component segmentation training data set; training a semantic segmentation model based on a deep convolutional neural network, adopting a cross-entropy and Dice mixed loss and dynamically adjusting the class weight to alleviate the class imbalance; inputting a to-be-detected image into the model after preprocessing to obtain a semantic segmentation mask; performing morphological post-processing on the segmentation mask to eliminate false areas and regularize the contour; and finally outputting the class, contour and position information of each component. Through standardized data set construction, preprocessing enhancement, mixed loss function and post-processing optimization, the application effectively improves the precision and stability of component segmentation under complex circuit board images, and especially improves the segmentation effect of low-frequency key components.

Description

Technical Field

[0001] This invention belongs to the field of image detection technology, specifically relating to a visual detection method for intelligent segmentation of circuit board components. Background Technology

[0002] Currently, the technological development in the field of circuit board component segmentation has gradually shifted from being dominated by traditional machine vision to being driven by deep learning. This evolution can be divided into two core stages: In the traditional stage, rule-based machine vision methods were the mainstream, mainly including classic algorithms such as threshold segmentation, edge detection, and region growing. These methods segment components by manually designing feature extraction rules and combining them with template matching. They were applicable to simple, low-density circuit board scenarios in the early days, and were relatively inexpensive and easy to deploy. However, with the development of the electronics industry, circuit board components have shown a trend towards high density, multiple types, and small sizes, making the limitations of traditional methods increasingly apparent. They are sensitive to changes in lighting and background noise, heavily reliant on manually designed features, and prone to problems such as edge breakage, incomplete segmentation, and missegmentation when facing complex scenarios with dense component layouts, small category differences, and surface reflections. This makes it difficult to meet the high-precision positioning requirements of automated spraying. Currently, traditional methods are mostly used as auxiliary preprocessing tools and can no longer independently support accurate segmentation tasks in complex scenarios. In the deep learning stage, segmentation models based on convolutional neural networks (CNNs) have gradually become the mainstream in research and application, significantly improving the accuracy of the technology in this field. U-Net and its improved architecture, with their superior semantic segmentation and boundary reconstruction capabilities, have been widely applied to circuit board component segmentation tasks. Through an encoder-decoder structure combined with techniques such as batch normalization, they effectively improve the feature extraction capability for minute components, achieving a validation accuracy of over 95% in the segmentation of common components such as resistors and capacitors. Meanwhile, fusion technologies such as panoramic semantic segmentation and instance segmentation are also being explored in the initial stages, attempting to address the dual needs of "category recognition + individual differentiation," providing new ideas for the accurate segmentation of high-density components. Furthermore, the application of transfer learning and data augmentation technologies has, to some extent, alleviated the problem of scarce circuit board annotation data and improved the model's generalization ability.

[0003] Although deep learning technology has been applied to circuit board image segmentation, the following problems still exist in practical applications: First, the types, sizes, and arrangement densities of components on different circuit board models vary greatly. If the training sample coverage is insufficient or the annotation is not standardized, it can easily affect the model's adaptability to various types of circuit boards. Second, circuit board images often contain interference such as uneven lighting, surface reflections, silkscreen printing, pads, and circuit backgrounds. The distinction between component boundaries and the background is insufficient, easily leading to missegmentation, missed segmentation, or discontinuous contours, affecting subsequent spraying path planning. Third, the number of samples for different types of components is unbalanced. Common categories such as resistors and capacitors have a large number of samples, while key low-frequency categories such as connectors and crystal oscillators have fewer samples. During model training, it is easy to favor high-frequency categories, resulting in insufficient segmentation accuracy for niche key components. Therefore, there is a need for a training method and visual detection method for intelligent segmentation models of circuit board components that can improve the standardization of training datasets, enhance the stability of complex image segmentation, and mitigate the impact of category imbalance. Summary of the Invention

[0004] In view of the problems and shortcomings of the existing technology, the present invention aims to provide a visual inspection method for intelligent segmentation of circuit board components.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of this invention provides a training method for an intelligent segmentation model of circuit board components, comprising the following steps: S1. Collect original circuit board images and construct component segmentation training dataset: Collect original circuit board images, manually annotate and correct the collected original circuit board images, generate part annotation images with component category, contour information and position information, and construct component segmentation training dataset based on the original circuit board images and the corresponding part annotation images. S2. Training the semantic segmentation model for circuit board components: Based on the component segmentation training dataset constructed in step S1, train a semantic segmentation model for intelligent segmentation of circuit board components, so that the semantic segmentation model can identify component categories, pads, sensitive areas and circuit board background.

[0006] Further, step S1 includes: S101. Use an industrial camera to acquire original images of the circuit board. The original images include image data of different types of circuit boards, different sizes of components, different arrangement densities, and different lighting conditions. S102. Perform format unification, size normalization and image quality screening on the original image to remove invalid images that are blurry, overexposed, underexposed or severely occluded. S103. Manually annotate the selected circuit board images. The annotation content includes the component category, component outline area, component outer frame and its position information in the image coordinate system. S104. Manually correct the manually labeled results, correcting samples with missing labels, incorrect labels, boundary offsets, and incorrect categories, to obtain standardized part labeled images; S105. Divide the original circuit board image and the corresponding component labeled image into training set and validation set to form a component segmentation training dataset.

[0007] Further, step S2 includes: S201. Count the number of samples of each type of component in the training set to obtain the sample distribution of different types of components. S202. Prioritize oversampling for key component categories with a small number of samples, and apply one or more data augmentation operations such as random rotation, horizontal flipping, brightness perturbation, and scale scaling to the training samples to reduce the impact of class imbalance on model training. S203. Construct a semantic segmentation model based on a deep convolutional neural network, including a feature extraction network and a pixel-level classification network, to output the category probability corresponding to each pixel. S204. The semantic segmentation model based on the deep convolutional neural network is trained using a hybrid loss function of cross-entropy loss and Dice loss, and the loss weight is dynamically adjusted according to the segmentation error of different categories during the training process, so that the low-frequency key component category receives higher training attention. S205. Use the validation set to verify the accuracy of the trained semantic segmentation model, and adjust the model parameters according to the validation results to obtain the trained intelligent segmentation model for circuit board components.

[0008] A second aspect of the present invention provides a visual inspection method for intelligent segmentation of circuit board components, comprising the following steps: (1) Preprocessing the circuit board image to be inspected: Preprocess the circuit board image to be inspected to reduce noise interference, enhance the edge features of components and unify the image input format to obtain the preprocessed circuit board image to be inspected. (2) Perform pixel-level segmentation using the trained model: Input the preprocessed circuit board image of the circuit board to be detected in step (1) into the intelligent segmentation model of circuit board components, output the pixel-level semantic segmentation result, and obtain the semantic segmentation mask of each component area, pad area, connector sensitive area and circuit board background area; wherein, the intelligent segmentation model of circuit board components is the intelligent segmentation model of circuit board components trained by the training method described in the first aspect above. (3) Post-processing the segmentation results: Post-process the semantic segmentation mask obtained in step (2) to remove mis-segmented areas, correct discontinuous boundary areas, and obtain regular component outlines. (4) Output component segmentation and detection results: Based on the regular component outline obtained in step (3), output the category information, outline information and position information of each component, and map this information to the circuit board image coordinate system to generate structured detection results.

[0009] Further, step (1) includes: (1a) Perform size normalization processing on the circuit board image to be detected so that it meets the input size requirements of the intelligent segmentation model of the circuit board components; (1b) Perform grayscale processing on the image to reduce the impact of image channel differences on subsequent segmentation; (1c) Perform Gaussian filtering, median filtering or bilateral filtering on the grayscale image to remove random noise generated during image acquisition; (1d) Use histogram equalization or adaptive contrast enhancement methods to improve the grayscale difference between the component area and the circuit board background; (1e) Perform geometric correction on the image based on the edge of the circuit board, positioning holes or preset reference points to correct image rotation, offset or perspective distortion, and obtain the preprocessed image of the circuit board to be inspected.

[0010] Further, step (2) includes: (2a) Input the preprocessed circuit board image to be detected into the trained intelligent segmentation model of the circuit board components, and extract the deep semantic features of components at different scales in the circuit board image through the feature extraction network. (2b) Based on deep semantic features, the probability of each pixel belonging to different categories is calculated through a pixel-level classification network to obtain a category probability map; (2c) Determine the category label of each pixel based on the category probability map and generate a semantic segmentation mask for circuit board components; (2d) Distinguish the component areas, pad areas, connector sensitive areas and circuit board background areas based on semantic segmentation masks.

[0011] Further, step (3) includes: (3a) Perform morphological opening and closing operations on the semantic segmentation mask to remove isolated noise, burrs and small missegmented regions; (3b) Fill the holes in the segmented area to correct the discontinuity problem inside the component area; (3c) Perform connected component analysis or contour separation on the adhesion area of ​​adjacent components to obtain independent component areas; (3d) Extract the contour of each independent component area and calculate its circumscribed rectangle, minimum circumscribed rectangle, center point coordinates, contour area and contour perimeter; (3e) Based on the preset area threshold, aspect ratio threshold and contour regularity threshold, false segmentation regions are eliminated to obtain regular component contours.

[0012] A third aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the training method for the intelligent segmentation model of circuit board components as described in the first aspect, and / or the visual detection method for intelligent segmentation of circuit board components as described in the second aspect.

[0013] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a computer processor, implements a training method for an intelligent segmentation model of circuit board components as described in the first aspect, and / or a visual detection method for intelligent segmentation of circuit board components as described in the second aspect.

[0014] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention improves the quality of the circuit board component segmentation training dataset. By collecting circuit board images of different models, layouts, and component distribution states, and manually annotating or correcting them, part-annotated images with component category, contour region, and location information are generated, constructing a component segmentation training dataset. This enables the training samples to cover various circuit board scenarios and improves the model's adaptability to multiple types and small batches of circuit boards.

[0015] 2. This invention improves the segmentation accuracy of niche key component categories. By prioritizing oversampling and data augmentation for low-frequency key component categories, and employing a hybrid loss function combining cross-entropy loss and Dice loss, the loss weights are dynamically adjusted based on the category segmentation error. This allows the model to focus more on low-frequency but technologically important component categories such as connectors, crystal oscillators, and sensors during training, reducing the problem of model bias towards high-frequency categories.

[0016] 3. This invention improves the segmentation stability of complex circuit board images. Through preprocessing methods such as size normalization, noise reduction, grayscale conversion, contrast enhancement, and geometric correction, the influence of uneven lighting, image noise, reflection, silkscreen printing, and background interference on the segmentation results is reduced, and the difference between the component area and the circuit board background is enhanced, thereby reducing missegmentation and missed segmentation. Attached Figure Description

[0017] Figure 1This is an image of the circuit board to be tested acquired in Embodiment 2 of the present invention; Figure 2 This is a mask diagram of the circuit board after segmentation according to Embodiment 2 of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Example 1

[0019] A training method for an intelligent segmentation model of circuit board components, the specific steps of which are as follows: S1. Acquire the original image of the circuit board, manually annotate and correct the acquired original circuit board image to generate component-annotated images with component category, contour information, and location information, and construct a component segmentation training dataset based on the original circuit board image and the corresponding component-annotated images. The specific operation process is as follows: S101. Use an industrial camera to acquire original images of the circuit board. The original images include image data of different types of circuit boards, different sizes of components, different arrangement densities, and different lighting conditions. S102. Perform format unification, size normalization and image quality screening on the original image to remove invalid images that are blurry, overexposed, underexposed or severely occluded. S103. Manually annotate the selected circuit board images. The annotation content includes the component category, component outline area, component outer frame and its position information in the image coordinate system. S104. Manually correct the manually labeled results, correcting samples with missing labels, incorrect labels, boundary offsets, and incorrect categories, to obtain standardized part labeled images; S105. Divide the original circuit board image and the corresponding component labeled image into training set and validation set to form a component segmentation training dataset.

[0020] S2. Based on the component segmentation training dataset constructed in step S1, train a semantic segmentation model for intelligent segmentation of circuit board components, enabling the semantic segmentation model to identify component categories, pads, sensitive areas, and the circuit board background. The specific operation process is as follows: S201. Count the number of samples of each type of component in the training set to obtain the sample distribution of different types of components. S202. Prioritize oversampling for key component categories with a small number of samples, and apply one or more data augmentation operations such as random rotation, horizontal flipping, brightness perturbation, and scale scaling to the training samples to reduce the impact of class imbalance on model training. S203. Construct a semantic segmentation model based on a deep convolutional neural network, including a feature extraction network and a pixel-level classification network, to output the category probability corresponding to each pixel. S204. The semantic segmentation model based on the deep convolutional neural network is trained using a hybrid loss function of cross-entropy loss and Dice loss, and the loss weight is dynamically adjusted according to the segmentation error of different categories during the training process, so that the low-frequency key component category receives higher training attention. S205. Use the validation set to verify the accuracy of the trained semantic segmentation model, and adjust the model parameters according to the validation results to obtain the trained intelligent segmentation model for circuit board components. Example 2

[0021] A visual inspection method for intelligent segmentation of circuit board components, the specific steps of which are as follows: (1) Preprocess the circuit board image to be inspected to reduce noise interference, enhance component edge features, and unify the image input format to obtain the preprocessed circuit board image to be inspected, such as... Figure 1 As shown. The specific operation process is as follows: (1a) Normalize the size of the circuit board image to meet the input size requirements of the intelligent segmentation model of circuit board components; (1b) Perform grayscale processing on the image to reduce the impact of image channel differences on subsequent segmentation; (1c) Perform Gaussian filtering, median filtering or bilateral filtering on the grayscale image to remove random noise generated during image acquisition; (1d) Use histogram equalization or adaptive contrast enhancement methods to improve the grayscale difference between the component area and the circuit board background; (1e) Perform geometric correction on the image based on the edge of the circuit board, positioning holes or preset reference points to correct image rotation, offset or perspective distortion, and obtain the preprocessed image of the circuit board to be inspected.

[0022] (2) Input the preprocessed circuit board image from step (1) into the intelligent segmentation model for circuit board components, and output pixel-level semantic segmentation results to obtain semantic segmentation masks for each component region, pad region, connector sensitive region, and circuit board background region; wherein, the intelligent segmentation model for circuit board components is the intelligent segmentation model for circuit board components trained using the training method described in Example 1 above. The specific operation process is as follows: (2a) Input the preprocessed circuit board image to be detected into the trained intelligent segmentation model of the circuit board components, and extract the deep semantic features of components at different scales in the circuit board image through the feature extraction network. (2b) Based on deep semantic features, the probability of each pixel belonging to different categories is calculated through a pixel-level classification network to obtain a category probability map; (2c) Determine the category label of each pixel based on the category probability map, and generate a semantic segmentation mask for circuit board components. The obtained mask image is shown below. Figure 2 As shown; (2d) Distinguish the component areas, pad areas, connector sensitive areas and circuit board background areas based on semantic segmentation masks.

[0023] (3) Post-process the semantic segmentation mask obtained in step (2) to remove mis-segmented regions and correct discontinuous boundary regions, thereby obtaining a regular component outline. The specific operation process is as follows: (3a) Perform morphological opening and closing operations on the semantic segmentation mask to remove isolated noise, burrs and small missegmented regions; (3b) Fill the holes in the segmented area to correct the discontinuity problem inside the component area; (3c) Perform connected component analysis or contour separation on the adhesion area of ​​adjacent components to obtain independent component areas; (3d) Extract the contour of each independent component area and calculate its circumscribed rectangle, minimum circumscribed rectangle, center point coordinates, contour area and contour perimeter; (3e) Based on the preset area threshold, aspect ratio threshold and contour regularity threshold, false segmentation regions are eliminated to obtain regular component contours.

[0024] (4) Based on the regular component outline obtained in step (3), output the category information, outline information and position information of each component, and map this information to the circuit board image coordinate system to generate structured detection results. Example 3

[0025] An electronic device includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement a training method for an intelligent segmentation model of circuit board components as described in Embodiment 1 above, and / or a visual detection method for intelligent segmentation of circuit board components as described in Embodiment 2 above. Example 4

[0026] A computer-readable storage medium storing a computer program, which, when executed by a processor, implements a training method for an intelligent segmentation model of circuit board components as described in Embodiment 1 above, and / or a visual detection method for intelligent segmentation of circuit board components as described in Embodiment 2 above.

[0027] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Those skilled in the art can modify or make equivalent substitutions to the technical solutions of the present invention based on the concept of the present invention, without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A training method for an intelligent segmentation model of circuit board components, characterized in that, Includes the following steps: S1. Acquire the original image of the circuit board, manually annotate and correct the acquired original image of the circuit board, generate part annotation images with component category, contour information and position information, and construct a component segmentation training dataset based on the original image of the circuit board and the corresponding part annotation images. S2. Based on the component segmentation training dataset constructed in step S1, train a semantic segmentation model for intelligent segmentation of circuit board components, so that the semantic segmentation model can identify component categories, pads, sensitive areas and circuit board background.

2. The training method according to claim 1, characterized in that, The specific operation of step S1 is as follows: S101. Use an industrial camera to acquire original images of the circuit board. The original images include image data of different types of circuit boards, different sizes of components, different arrangement densities, and different lighting conditions. S102. Perform format unification, size normalization and image quality screening on the original image to remove invalid images that are blurry, overexposed, underexposed or severely occluded. S103. Manually annotate the selected circuit board images. The annotation content includes the component category, component outline area, component outer frame and its position information in the image coordinate system. S104. Manually correct the manually labeled results, correcting samples with missing labels, incorrect labels, boundary offsets, and incorrect categories, to obtain standardized part labeled images; S105. Divide the original circuit board image and the corresponding component labeled image into training set and validation set to form a component segmentation training dataset.

3. The training method according to claim 1 or 2, characterized in that, The training steps for the semantic segmentation model used for intelligent segmentation of circuit board components include: S201. Count the number of samples of each type of component in the training set to obtain the sample distribution of different types of components. S202. Prioritize oversampling for key component categories with a small number of samples, and apply one or more data augmentation operations such as random rotation, horizontal flipping, brightness perturbation, and scale scaling to the training samples to reduce the impact of class imbalance on model training. S203. Construct a semantic segmentation model based on a deep convolutional neural network, including a feature extraction network and a pixel-level classification network, to output the category probability corresponding to each pixel. S204. The semantic segmentation model based on the deep convolutional neural network is trained using a hybrid loss function of cross-entropy loss and Dice loss, and the loss weight is dynamically adjusted according to the segmentation error of different categories during the training process, so that the low-frequency key component category receives higher training attention. S205. Use the validation set to verify the accuracy of the trained semantic segmentation model, and adjust the model parameters according to the validation results to obtain the trained intelligent segmentation model for circuit board components.

4. A visual inspection method for intelligent segmentation of circuit board components, characterized in that, Includes the following steps: (1) The circuit board image to be inspected is preprocessed to reduce noise interference, enhance the edge features of components and unify the image input format to obtain the preprocessed circuit board image to be inspected. (2) Input the preprocessed circuit board image of step (1) into the intelligent segmentation model of circuit board components, output the pixel-level semantic segmentation result, and obtain the semantic segmentation mask of each component area, pad area, connector sensitive area and circuit board background area; wherein, the intelligent segmentation model of circuit board components is the intelligent segmentation model of circuit board components trained by any of the training methods described in claims 1 to 3. (3) Post-process the semantic segmentation mask obtained in step (2) to remove mis-segmented regions, correct discontinuous boundary regions, and obtain regular component outlines. (4) Based on the regular component outline obtained in step (3), output the category information, outline information and position information of each component, and map this information to the circuit board image coordinate system to generate structured detection results.

5. The method according to claim 4, characterized in that, The specific operation of step (1) is as follows: (1a) Perform size normalization processing on the circuit board image to be detected so that it meets the input size requirements of the intelligent segmentation model of the circuit board components; (1b) Perform grayscale processing on the image to reduce the impact of image channel differences on subsequent segmentation; (1c) Perform Gaussian filtering, median filtering or bilateral filtering on the grayscale image to remove random noise generated during image acquisition; (1d) Use histogram equalization or adaptive contrast enhancement methods to improve the grayscale difference between the component area and the circuit board background; (1e) Perform geometric correction on the image based on the edge of the circuit board, positioning holes or preset reference points to correct image rotation, offset or perspective distortion, and obtain the preprocessed image of the circuit board to be inspected.

6. The method according to claim 4, characterized in that, The specific operation of step (2) is as follows: (2a) Input the preprocessed circuit board image to be detected into the intelligent segmentation model of circuit board components, and extract the deep semantic features of components at different scales in the circuit board image through the feature extraction network; (2b) Based on deep semantic features, the probability of each pixel belonging to different categories is calculated through a pixel-level classification network to obtain a category probability map; (2c) Determine the category label of each pixel based on the category probability map and generate a semantic segmentation mask for circuit board components; (2d) Distinguish the component areas, pad areas, connector sensitive areas and circuit board background areas based on semantic segmentation masks.

7. The method according to claim 4, characterized in that, The specific operation of step (3) is as follows: (3a) Perform morphological opening and closing operations on the semantic segmentation mask to remove isolated noise, burrs and small missegmented regions; (3b) Fill the holes in the segmented area to correct the discontinuity problem inside the component area; (3c) Perform connected component analysis or contour separation on the adhesion area of ​​adjacent components to obtain independent component areas; (3d) Extract the contour of each independent component area and calculate its circumscribed rectangle, minimum circumscribed rectangle, center point coordinates, contour area and contour perimeter; (3e) Based on the preset area threshold, aspect ratio threshold and contour regularity threshold, false segmentation regions are eliminated to obtain regular component contours.

8. An electronic device, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the training method for the intelligent segmentation model of circuit board components as described in any one of claims 1 to 3, and / or the visual detection method for intelligent segmentation of circuit board components as described in any one of claims 4 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a computer processor, implements the training method for the intelligent segmentation model of circuit board components as described in any one of claims 1 to 3, and / or the visual detection method for intelligent segmentation of circuit board components as described in any one of claims 4 to 7.

Images