A component positioning method based on image preprocessing and YOLO recognition

Abstract

The application discloses a component positioning method based on image preprocessing and YOLO recognition and belongs to the technical field of image recognition and processing. The input image is subjected to gray scale normalization and sliding window statistics, a global board surface background color estimation graph is constructed, and a contrast response graph is generated; the contrast response graph is input into a target detection network to obtain a component initial orientation detection frame; secondary background color modeling is carried out in a local area corresponding to the detection frame, a background substitution graph is generated based on local background samples, and a target significant residual graph is obtained through the difference between the original graph and the background substitution graph; direction-constrained edge fitting and geometric reconstruction are carried out on the significant residual graph to realize accurate positioning of the component; and a reliability judgment mechanism is constructed according to the parameter change before and after accurate positioning and the boundary continuity to execute rollback processing on low reliability results. The application realizes stable extraction of the real boundary of the component and improves the positioning accuracy and robustness in a complex pollution environment.

Description

Technical Field

[0001] This invention relates to the field of image recognition and image processing technology, specifically to a component positioning method based on image preprocessing and YOLO recognition, which is particularly suitable for high-precision positioning of components with surface dirt, silkscreen interference, and uneven background color in the scenario of waste circuit board recycling. Background Technology

[0002] With the rapid pace of electronic product upgrades, a massive amount of waste circuit boards are generated annually. These waste circuit boards contain large quantities of precious metals, rare metals, and recyclable electronic components, making their recycling highly profitable and environmentally valuable. Disassembling and reusing intact electronic components from circuit boards is a crucial link in the recycling industry chain. To achieve automated component disassembly (e.g., using robotic arms for precise heating and removal), the core technical challenge that needs to be addressed is the rapid and accurate positioning of components in complex visual environments.

[0003] Existing component localization methods can be mainly divided into three categories: methods based on traditional image processing, object detection methods based on deep learning, and combinations of the two.

[0004] First, there are methods based on traditional image processing. These methods typically utilize edge detection (such as the Canny operator), morphological processing, contour extraction, and shape matching. However, discarded circuit boards have the following significant characteristics: the board surface generally has contamination such as solder splatter, flux residue, and oxidation discoloration; the board surface background color (substrate color) is uneven, and the color of circuit boards from different manufacturers varies greatly; the board surface has a large number of dense silkscreen characters, wires, vias, and other pseudo-boundaries. These factors can seriously interfere with the edge extraction and contour fitting of traditional algorithms, leading to offset, breakage, or the inclusion of a large amount of noise in the positioning results, making it difficult to meet the accuracy requirements of automated dismantling.

[0005] Second, deep learning-based object detection methods. Deep learning methods, represented by YOLO and Faster R-CNN, have achieved great success in general object detection tasks. However, when directly applied to the localization of components on scrap circuit boards, the following problems exist: Deep learning networks abstract the overall features of the input image, outputting only approximate bounding boxes of the target. These bounding boxes have poor rotation invariance, typically only outputting axis-aligned rectangles, which cannot accurately fit the true pose of the components (especially components with arbitrary rotation angles); at the same time, the bounding boxes predicted by the network often include part of the background area, and when the edges of the components are covered by dirt, the bounding boxes are even more difficult to converge to the true physical boundaries. Simply relying on the output of the detection network cannot meet the requirements for sub-pixel or high-precision localization.

[0006] Third, existing methods combine deep learning and image processing. Some existing techniques attempt to use bounding boxes output by object detection networks to crop regions of interest (ROIs), and then perform traditional edge extraction and fitting within the ROI. This method narrows the processing range to some extent, but it still cannot effectively solve the problems of dirt, silkscreen printing, and uneven background color within the ROI. Because complex background interference also exists within the ROI, traditional local edge detection will still misidentify silkscreen edges, dirt edges, etc., as component boundaries, leading to distorted or inaccurate fitted quadrilaterals. Furthermore, existing methods lack self-evaluation and error correction mechanisms for localization results; once local processing fails, they directly output incorrect results, lacking robustness.

[0007] In summary, in the highly complex and unstructured visual environment of discarded circuit boards, existing technologies cannot effectively distinguish between the true boundaries of components and pseudo-boundaries such as stains and silkscreen printing, making it difficult to simultaneously meet the requirements of high accuracy, strong robustness, and real-time positioning. Therefore, a new component positioning method that can overcome the above-mentioned shortcomings is urgently needed. Summary of the Invention

[0008] The purpose of this invention is to provide a component localization method and system based on image preprocessing and YOLO recognition, to solve the technical problems of existing technologies, such as difficulty in distinguishing between real and false boundaries in complex backgrounds, insufficient localization accuracy, and sensitivity to dirt and uneven background color. The specific technical solution is as follows:

[0009] A component localization method based on image preprocessing and YOLO recognition includes the following steps:

[0010] Image preprocessing steps: Obtain the original image containing the components, perform grayscale normalization on the original image, construct a global board background color estimation map using the sliding window statistical method, and generate a contrast response map based on the original image and the global board background color estimation map;

[0011] Initial target localization step: Input the contrast response map into the pre-trained YOLO target detection network to obtain the initial orientation detection box of the component;

[0012] Local background subtraction step: Secondary background color modeling is performed in the local image region corresponding to the initial orientation detection box, a background replacement map is generated based on the local background samples, and a target salient residual map is obtained by subtracting the local image region from the background replacement map.

[0013] Precise positioning and reconstruction steps: Calculate the main direction of the initial orientation detection box, and combine the main direction to perform directional constraint edge fitting and geometric reconstruction on the target significant residual map to obtain the precise positioning contour of the component;

[0014] Credibility determination and rollback steps: Calculate the positioning credibility based on the parameter change rate of the precise positioning contour and the initial orientation detection box, as well as the continuity of the edge points. When the credibility is lower than a preset threshold, perform rollback processing and output the initial orientation detection box as the final positioning result.

[0015] Furthermore, the image preprocessing step includes: traversing the entire grayscale image using a sliding window of a specified size, calculating the average grayscale value and grayscale variance of all pixels in each window, fitting a surface to the average grayscale value of all windows, and using the fitted surface as the global panel background color estimation map.

[0016] The contrast response map is generated as follows: for each pixel in the original grayscale image, the grayscale value of the pixel is subtracted from the grayscale value of the corresponding pixel in the global background color estimation map to obtain a first difference; the grayscale value of the corresponding pixel in the global background color estimation map is added to a preset very small positive number to obtain a first sum; the ratio of the first difference to the first sum is used as the grayscale value of the contrast response map at that pixel.

[0017] Further, the local background subtraction step includes: within the initial directional detection box, selecting pixels near the edge of the detection box as a local background sample set, performing surface fitting or interpolation on the local background sample set to generate a background replacement image with the same size as the local image region; the target significant residual image is generated by: subtracting the pixel grayscale values ​​of the corresponding positions in the local image region and the background replacement image and taking the absolute value to obtain the grayscale difference value of each pixel, thus forming the target significant residual image.

[0018] Furthermore, the precise positioning and reconstruction steps include: performing principal component analysis on the coordinates of all pixels within the initial orientation detection box, and determining the direction of the first principal component as the estimated principal direction of the component; setting multiple parallel search lines on the target saliency residual map along a direction perpendicular to the estimated principal direction, and finding the location point where the gray value changes the most on each search line as a candidate edge point; grouping the candidate edge points belonging to the same boundary of the component into a group, and performing least squares line fitting on the four boundary point groups to obtain the four boundary lines of the component; calculating the intersection points of the four boundary lines to obtain the coordinates of the four vertices of the component, thus completing the geometric reconstruction.

[0019] Furthermore, in the credibility determination and rollback steps, the calculation method for the location credibility is as follows:

[0020] Calculate the rate of change of the area between the fine-positioning contour and the initial detection box, and the rate of change of the angle between the principal direction of the fine-positioning contour and the reference direction of the initial detection box;

[0021] Calculate the average continuity between adjacent edge points on the fine-positioned contour to obtain the edge continuity.

[0022] The first term is obtained by multiplying the difference between 1 and the area change rate by a preset first weighting coefficient, the second term is obtained by multiplying the edge continuity by a preset second weighting coefficient, and the third term is obtained by multiplying the angle change rate by a preset third weighting coefficient.

[0023] The first and second items are weighted and summed, and the difference between the weighted sum and the third item is used as the location confidence.

[0024] Furthermore, the YOLO object detection network is one of YOLOv5, YOLOv8, or YOLOx, and its training dataset contains component image samples under various poses, lighting conditions, and levels of contamination.

[0025] Furthermore, the rollback process includes: when the confidence level is lower than the preset threshold, abandoning the precise positioning contour, outputting the initial orientation detection box as the final positioning result, and sending a positioning degradation prompt message to the upper layer of the system.

[0026] Furthermore, the method also includes an image quality assessment step performed before the image preprocessing step: calculating the global contrast and signal-to-noise ratio of the original image, and performing histogram equalization or adaptive gamma correction on the original image when the quality index is below a threshold.

[0027] A component localization system based on image preprocessing and YOLO recognition, used to perform the method described in any of the above-mentioned embodiments, comprising:

[0028] Image acquisition module, used to acquire raw images containing components;

[0029] The image preprocessing module is used for grayscale normalization, global background color estimation, and contrast response map generation.

[0030] The target detection module, with a built-in YOLO network, is used to extract the initial orientation detection box of the component;

[0031] The local difference module is used to perform secondary background color modeling and difference operations to generate a salient residual map of the target.

[0032] The geometric reconstruction module is used for edge fitting and geometric reconstruction with directional constraints.

[0033] The credibility assessment module is used to calculate the credibility of the location and execute rollback decisions.

[0034] Furthermore, the system also includes an interaction and output module for visualizing the final positioning result and sending the positioning parameters of the components to the downstream robotic arm control system.

[0035] Compared with the prior art, the technical solution provided by the present invention has the following beneficial effects:

[0036] (i) By combining global and local "phased" background color inversion constraints, false boundary interference can be effectively suppressed.

[0037] This invention first constructs a background color estimation map of the entire circuit board through global sliding window statistics, and then uses a contrast response map to initially highlight the foreground of components from the overall background, reducing global illumination unevenness and background color differences. Furthermore, within the initial detection box provided by the YOLO network, a "secondary" refined background color modeling is performed: a high-fidelity background replacement map is generated using only edge background samples within local areas. Through difference operations, local backgrounds (such as grayscale changes caused by silkscreen printing or stains) are almost perfectly subtracted, leaving only the significant residuals of the components themselves. This "global coarse elimination, local fine elimination" strategy, unlike existing methods that "only use global preprocessing" or "only perform conventional edge detection within the ROI," greatly improves the anti-interference capability against complex backgrounds.

[0038] (ii) The collaborative mechanism of YOLO initial positioning and orientation constraint geometric reconstruction to achieve high-precision arbitrary angle positioning.

[0039] Existing detection networks such as YOLO can only output axis-aligned bounding boxes, failing to accurately describe components at arbitrary rotation angles. This invention fully utilizes the initial localization box output by YOLO: on one hand, this box is used to determine local areas requiring fine processing, significantly reducing computational load; on the other hand, the pixel distribution within this box is used to calculate the approximate principal orientation of the component through Principal Component Analysis (PCA). Under the constraint of this principal orientation, edge search and fitting are performed in orthogonal directions on the target salient residual map, giving edge extraction a clear "prior direction," effectively eliminating interference points in the same direction, and ultimately fitting the smallest bounding quadrilateral that accurately fits the true physical boundary of the component. This collaborative mechanism achieves sub-pixel level localization accuracy.

[0040] (iii) The introduction of credibility determination and rollback mechanisms ensures the overall robustness of the system.

[0041] The system of this invention constructs a quantitative reliability index based on the area change rate, angle change rate, and edge continuity of the precise positioning contour and the initial detection box. When excessive distortion of the precise positioning result is detected (e.g., due to severe component damage or extreme lighting), the system automatically performs a rollback process, outputting a more robust but slightly less accurate YOLO initial detection box and sending a prompt message. Through the design of "high precision first, robustness as a backup," the system ensures that the entire positioning system will not crash or output seriously erroneous results due to individual anomalies on the production line, greatly improving the reliability of actual deployment.

[0042] (iv) Specific optimizations for the waste circuit board scenario have reduced the dependence on the dataset.

[0043] This invention overcomes the dependence of pure deep learning solutions on massive amounts of high-quality labeled data through extensive image preprocessing and post-processing logic. Even if the YOLO network itself is trained on a small number of noisy training samples, as long as its output detection boxes can roughly encompass the components, subsequent refinement processes can correct the deviations of the detection boxes and accurately locate the boundaries through a physical model-driven approach. Therefore, this invention can be rapidly deployed and achieve the desired results even in industrial scenarios with limited labeled samples. Attached Figure Description

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0045] Figure 1 This is an overall flowchart of the component localization method based on image preprocessing and YOLO recognition of the present invention.

[0046] Figure 2 This is a schematic diagram of the global panel background color estimation and contrast response map generation of the present invention.

[0047] Figure 3 This is a schematic diagram of the local background difference and target salient residual map generation process of the present invention.

[0048] Figure 4 This is a schematic diagram of the edge fitting and geometric reconstruction process of the directional constraint of the present invention.

[0049] Figure 5 This is a schematic diagram of the credibility determination and rollback process of the present invention.

[0050] Figure 6This is a schematic diagram of the component positioning system based on image preprocessing and YOLO recognition of the present invention. Detailed Implementation

[0051] The technical solutions in the embodiments of the present invention will be clearly and completely described below. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0053] Example 1

[0054] Figure 1 This diagram illustrates the overall flowchart of a component positioning method based on image preprocessing and YOLO recognition, as provided in Embodiment 1 of the present invention. This embodiment can be applied to automated dismantling lines for waste circuit boards to accurately locate surface-mount resistors, capacitors, chips, and other components with surface contamination or silkscreen printing. This method can be executed by a component positioning system based on image preprocessing and YOLO recognition.

[0055] like Figure 1 As shown, the method specifically includes the following steps:

[0056] Step S110, Image Preprocessing Step: Obtain the original image containing the components, perform grayscale normalization on the original image, construct a global board background color estimation map using the sliding window statistical method, and generate a contrast response map based on the original image and the global board background color estimation map.

[0057] This step aims to eliminate or reduce global lighting unevenness and board surface background color differences, and generate a contrast response map that highlights the edges of components.

[0058] Figure 2 This is a schematic diagram illustrating the global panel background color estimation and contrast response map generation process provided in this embodiment. Specifically, as shown... Figure 2 As shown, the acquired original color image containing components is first converted into a grayscale image, and grayscale normalization is performed to reduce the computational burden caused by color information. Then, a sliding window of a specified size (e.g., 50×50 pixels) is used to traverse the entire grayscale image with a fixed step size (e.g., 10 pixels). At each window position, the average grayscale value and the variance of the grayscale values ​​of all pixels within the window are calculated. The average grayscale values ​​of all windows constitute a sparse matrix, and a global board background color estimation map with the exact same size as the original image is constructed through bicubic interpolation or least squares surface fitting. This estimation map depicts the macroscopic variation trend of the board background color.

[0059] Subsequently, a contrast response map is generated. The generation method is as follows: For each pixel in the original grayscale image, the grayscale value of that pixel is subtracted from the grayscale value of the corresponding pixel in the global background color estimation map to obtain a first difference; then, the grayscale value of the corresponding pixel in the global background color estimation map is added to a preset very small positive number (e.g., 0.000001) to obtain a first sum; finally, the ratio of the first difference to the first sum is used as the grayscale value of the contrast response map for that pixel. Specifically, the contrast response map can be calculated using the following formula:

[0060] ,

[0061] In the formula, Represents the contrast response diagram. Represents the original grayscale image. This represents the global background color estimation map. It is a very small positive number (e.g., 10). −6 To prevent division by zero errors, This refers to the coordinates of a pixel. The formula calculates the "relative contrast" of each pixel relative to the local background. In a uniform background area, the original grayscale value... Approximate base color estimate Therefore, the contrast response value Close to 0; at the edge of the component, the original grayscale value is close to 0. Significant deviation from the estimated background color contrast response value This produces a large positive or negative value, thus effectively enhancing the contrast between the foreground and the background.

[0062] For example, taking the positioning of surface-mount capacitors on a certain model of discarded mobile phone circuit board as an example, the surface of this circuit board has obvious uneven background color (e.g., the left side is darker and the right side is brighter), and at the same time, white silkscreen characters and wires are densely distributed on the board surface, which seriously interferes with the visual positioning of components. The present invention's step S110 is used for processing: First, the acquired original color image is converted into a grayscale image, and grayscale normalization is completed; then, a 50×50 pixel sliding window with a step size of 10 pixels is used to traverse the entire grayscale image, and the average grayscale value within each window is calculated; a global board surface background color estimation map is constructed through bicubic interpolation, which accurately reflects the macroscopic change trend of the background color of the circuit board from the lower left corner to the upper right corner. Subsequently, a contrast response map is generated, that is, for each pixel, the original grayscale value is subtracted from the background color estimation value, and then divided by the sum of the background color estimation value and a minimal constant.

[0063] In the processed contrast response map, global illumination unevenness is effectively suppressed, and the edges of the patch capacitors, which were originally difficult to discern due to the dark background, become clearly visible. Simultaneously, the grayscale response of the white silkscreen characters is suppressed to near the background value, eliminating false boundary interference. The contrast response value of the capacitor body area is significantly higher than the surrounding background, providing high-quality input for the subsequent YOLO network. This enables the network to quickly and accurately output the initial localization box of the capacitor, achieving an overall detection speed of 25 milliseconds per frame. Compared to using YOLO directly without preprocessing, the localization accuracy is improved by approximately 40%, and the false alarm rate is reduced by more than 60%.

[0064] Step S120, Initial Target Localization: The contrast response map is input into a pre-trained YOLO object detection network to obtain the initial orientation detection box of the component. The YOLO object detection network is, for example, one of YOLOv5, YOLOv8, or YOLOx, and its training dataset contains component image samples under various poses, lighting conditions, and contamination levels. The sample labels are the minimum bounding rectangles of the components. This step aims to utilize a deep learning object detection network to quickly locate the region of interest containing the component from the pre-processed contrast response map, outputting an axis-aligned rectangle that roughly surrounds the component, providing a reliable local processing range for subsequent fine-grained localization.

[0065] Specifically, the contrast response map generated in step S110 is input into a pre-trained YOLO object detection network. The network's forward propagation outputs a series of candidate boxes, each containing the target class confidence score and bounding box coordinates. The candidate box with the highest confidence score is selected as the initial orientation detection box for the component. This detection box is described by its center point coordinates, width, and height (since YOLO outputs axis-aligned rectangles, its rotation angle is 0 by default). This step leverages the powerful feature abstraction capabilities of deep learning to quickly locate the region of interest containing the component in the image.

[0066] The YOLO target detection network (taking YOLOv8n as an example) used in this embodiment mainly consists of the following three parts:

[0067] Backbone: Employing the CSPDarknet architecture, the backbone network contains multiple convolutional layers, batch normalization layers, and the SiLU activation function. It extracts multi-scale features from the image through progressive downsampling, outputting three feature maps at different scales (1 / 8, 1 / 16, and 1 / 32 of the input image size), corresponding to feature information with small, medium, and large receptive fields, respectively, for detecting targets of different sizes.

[0068] The Neck network employs a structure combining a Feature Pyramid Network (FPN) and a Path Aggregation Network (PAN). The FPN transmits high-level semantic features from top to bottom, while the PAN transmits low-level localization features from bottom to top, achieving bidirectional fusion of multi-scale features and enhancing the network's ability to detect small targets (such as patch resistors and capacitors).

[0069] The detection head employs a decoupled structure, separating the classification and regression tasks. For each grid point on each feature map, the network predicts the following information:

[0070] Bounding box coordinates: including the offset of the center point relative to the top left corner of the grid, width, and height (normalized);

[0071] Target confidence: Indicates the probability that a grid point contains a target;

[0072] Category probability: The softmax probability distribution for a predefined category (such as resistor, capacitor, chip, etc.).

[0073] In this embodiment, the input to the YOLO object detection network is the contrast response map (e.g., a grayscale image with a size of 640×640 pixels) generated in step S110. Before being input into the network, normalization processing is performed to map the pixel value range to the [0,1] interval.

[0074] In this embodiment, the output of the YOLO object detection network is as follows: after non-maximum suppression (NMS), the network outputs several candidate detection boxes. The parameters of each detection box include: the coordinates of the top-left and bottom-right corners of the bounding box (or the coordinates of the center point and width and height); the target confidence score (between 0 and 1); the category label and the corresponding category confidence score. The detection box with the highest confidence score is selected as the initial orientation detection box for the component, denoted as Box_init. This detection box is an axis-aligned rectangle (rotation angle defaults to 0 degrees), described by the center point coordinates (x_c, y_c), width w, and height h.

[0075] In this embodiment, the contrast response map is input into a pre-trained YOLO target detection network to obtain the initial orientation detection box of the component. The specific principle process includes:

[0076] Forward propagation: The contrast response map is input into the YOLOv8n network. After convolution, downsampling, and CSP modules in the backbone network, feature maps at three scales (P3, P4, P5) are extracted. The neck network performs bidirectional fusion of these three feature maps to obtain an enhanced feature map. The detection head predicts the bounding box offset, confidence score, and class probability at each grid point of each feature map.

[0077] Prior bounding box matching: The network pre-defines prior bounding boxes (anchor boxes) of different sizes and aspect ratios for each feature map. For each grid point, the network's predicted bounding box offset is decoded relative to the prior bounding box corresponding to that grid point to obtain the actual bounding box coordinates and size.

[0078] Confidence threshold filtering: For all predicted bounding boxes, filter them according to the target confidence score and retain candidate boxes with scores higher than a preset threshold (e.g., 0.5).

[0079] Non-maximum suppression (NMS): For the remaining candidate boxes after filtering, sort them from high to low confidence, calculate the intersection-union ratio (IoU) with other boxes one by one, delete redundant boxes with an overlap of more than a preset threshold (e.g., 0.45) with high-confidence boxes, and finally output a unique or a few optimal detection boxes.

[0080] Output selection: Select the candidate box with the highest confidence from the NMS output as the initial orientation detection box for the component.

[0081] For example, continuing the example of locating the chip capacitors on the circuit board of a certain model of discarded mobile phone in step S110:

[0082] In step S110, a contrast response map is obtained after preprocessing, in which global illumination unevenness is suppressed, silkscreen character interference is suppressed, and the contour of the patch capacitor is significantly enhanced. This contrast response map (size 640×640 pixels) is then input into a pre-trained YOLOv8n network.

[0083] After forward propagation, on the feature map at a 1 / 16 scale, the grid point corresponding to the capacitor region generated a high-confidence prediction. Specifically, the target confidence of this grid point was 0.92, and the probability of the "capacitor" class was 0.88. After decoding, the center coordinates of the initial directional detection box were obtained as (320, 240), with a width of 80 pixels and a height of 60 pixels. This detection box roughly enclosed the physical location of the capacitor.

[0084] During the non-maximum suppression stage, since there is only one target (the capacitor), there are no overlapping bounding box conflicts, and the detection box is directly preserved. The final output initial orientation detection box covers most of the capacitor's area. Although it does not precisely fit the capacitor's true rotation boundary (the capacitor has a tilt of about 15 degrees), it provides an accurate local processing range and a basis for estimating the principal direction for subsequent local background subtraction and orientation-constrained edge fitting.

[0085] Compared to directly performing YOLO detection on the original grayscale image, using the contrast response map as input increased the network detection confidence from 0.78 to 0.92, and reduced the center position deviation of the initial detection box from about 15 pixels to less than 5 pixels, effectively improving the accuracy of initial localization.

[0086] Step S130, Local Background Difference Step: Perform secondary background color modeling in the local image region corresponding to the initial orientation detection box, generate a background replacement map based on the local background sample, and obtain the target significant residual map through the difference operation between the local image region and the background replacement map.

[0087] This step is crucial for eliminating local false boundary interference. Within the local image region defined by the initial directional detection box obtained in step S120, secondary background color modeling is performed. For example... Figure 3 As shown, the specific process includes:

[0088] First, a local background sample set is constructed. Since components are typically located at the center of the detection frame, while the background (circuit board substrate, silkscreen printing, etc.) is mostly located at the edge of the detection frame, pixels within 10 pixels of the boundary in the local image region are selected as reliable local background sample points. These sample points are then fitted with a quadratic surface or interpolated using radial basis functions to generate a background replacement map with the exact same size as the local image region. This background replacement map can highly faithfully simulate the local background grayscale distribution under and around the components, even if silkscreen text or stains are present in that area.

[0089] Then, a target saliency residual map is generated. This is done by subtracting the grayscale values ​​of the corresponding pixels in the local image region from those in the background substitute image and taking the absolute value to obtain the grayscale difference for each pixel, thus forming the target saliency residual map. The result of the difference operation is that pixels similar to the background (such as areas belonging to silkscreen or uniform substrates) have their grayscale values ​​suppressed to near 0; while the component itself, due to its significant differences in grayscale and texture from the local background, retains its edge and internal features, forming a bright foreground residual. This residual map cleanly separates the target from the local background.

[0090] Step S140, Precise Positioning and Reconstruction Step: Calculate the principal direction of the initial orientation detection box, and perform directional constraint edge fitting and geometric reconstruction on the target salient residual map based on the principal direction to obtain the precise positioning contour of the component. This step aims to extract precise component edges from the target salient residual map and reconstruct the minimum bounding quadrilateral that fits the true physical boundary of the component through directional constraint fitting. Figure 4 As shown, the precise positioning and reconstruction steps specifically include:

[0091] First, the principal orientation of the initial orientation detection box is calculated. Principal component analysis is then performed on the coordinates of all pixels within the initial orientation detection box; specifically, the horizontal and vertical coordinates of each pixel within the detection box are collected. The covariance matrix of the coordinates is calculated, and the eigenvalues ​​and eigenvectors of this matrix are solved. The direction of the eigenvector corresponding to the largest eigenvalue is determined as the approximate axial direction of the component, denoted as the estimated principal direction. The direction of the second principal component is perpendicular to the direction of the first principal component. This principal direction provides prior direction information for subsequent edge search, effectively avoiding misjudging silkscreen characters or wires parallel to the principal direction as component boundaries.

[0092] Then, edge fitting with directional constraints is performed. In the target saliency residual map, multiple parallel search lines are set at equal intervals along the direction perpendicular to the estimated principal direction (i.e., the direction of the second principal component). The length of each search line covers the width range of the detection box in that direction. For each search line, the gray values ​​of pixels are scanned sequentially from one end to the other, and the position point where the gray value changes the most is recorded as the candidate edge point. Since components usually have four boundaries (top, bottom, left, and right), according to the position of the candidate edge point relative to the center of the detection box and the direction of the search line it is located on, the candidate edge points are assigned to four boundary point sets: the upper boundary point set (located on the positive side of the estimated principal direction), the lower boundary point set (located on the negative side of the estimated principal direction), the left boundary point set (located on the positive side of the second principal direction), and the right boundary point set (located on the negative side of the second principal direction).

[0093] Next, least-squares line fitting is performed on each of the four boundary point sets to obtain the linear equations of the four boundary lines. If the component boundary has significant curvature, polynomial curve fitting can be used to improve accuracy. The intersection points of the four boundary lines are calculated pairwise to obtain the coordinates of the four vertices of the component. Based on these four vertices, a quadrilateral that can accurately fit the actual physical boundary of the component at any rotation angle is reconstructed, which is called the precise positioning profile.

[0094] Example of a surface-mount capacitor continuing steps S110 / S120 / S130:

[0095] In the target saliency residual image obtained in step S130, the capacitor body area is highlighted, while the background is suppressed. Principal component analysis is performed on the coordinates of all pixels within the initial detection box, and the direction of the first principal component is calculated to be approximately 15 degrees, meaning the estimated principal direction of the capacitor is 15 degrees (the capacitor is tilted at 15 degrees). Thirty equally spaced parallel search lines are set along a direction perpendicular to 15 degrees (i.e., -75 degrees), with each search line covering the width of the detection box (approximately 80 pixels). Grayscale values ​​are scanned on each search line, and the position point with the largest jump is recorded. A total of 28 candidate points are obtained from the upper boundary point set, fitted with the linear equation y = 0.27x + 142; 30 candidate points are obtained from the lower boundary point set, fitted with the linear equation y = 0.27x + 198; 26 candidate points are obtained from the left boundary point set, fitted with the linear equation y = −3.73x + 1260; and 29 candidate points are obtained from the right boundary point set, fitted with the linear equation y = −3.73x + 1310. The intersection of the four straight lines was calculated, yielding the coordinates of the capacitor's four vertices as (285, 220), (355, 215), (360, 260), and (290, 265). This precisely positioned profile is a quadrilateral rotated approximately 15 degrees, perfectly matching the capacitor's actual physical boundaries. Compared to the initial axis-aligned box (from (280, 210) to (360, 270)), the area error was reduced from approximately 1200 pixels² to approximately 50 pixels², and the angle error was reduced from 15 degrees to less than 1 degree.

[0096] Step S150, Reliability Determination and Rollback Step: Based on the parameter change rate of the precise positioning contour and the initial orientation detection box, as well as the continuity of edge points, the positioning reliability is calculated. When the reliability is lower than a preset threshold, rollback processing is performed, and the initial orientation detection box is output as the final positioning result. This step aims to evaluate the reliability of the precise positioning result. When the fine-tuning process fails due to extreme interference (such as severe component damage, extreme abnormal lighting, or large-area stain coverage), it automatically rolls back to the more robust initial positioning result, ensuring the stability of the overall system output.

[0097] To prevent serious errors in output due to failure of fine-tuning under extreme interference, such as... Figure 5 As shown, this step introduces a quantitative credibility assessment and rollback mechanism, including the calculation of three evaluation metrics:

[0098] Area change rate : Calculate the area enclosed by the precisely located contour. Area of ​​the initial orientation detection box The absolute value of the difference is divided by the area of ​​the initial orientation detection box to obtain the area change rate, i.e.:

[0099] Area change rate This reflects the degree of area deviation between the fine positioning and the initial positioning. The smaller the area change rate, the more consistent the fine positioning contour is with the initial positioning frame in terms of size.

[0100] Rate of change of angle : Calculate the principal direction of the precise positioning profile The angle difference between the initial orientation detection box and the reference direction (default 0 degrees) is divided by 90 degrees (maximum possible angle deviation) to obtain the angle change rate, i.e.:

[0101] Rate of change of angle This reflects the rotational deviation in precise positioning. The smaller the rate of angular change, the more accurately the component's tilt angle is captured.

[0102] Edge continuity Sample N edge points at equal intervals along the four boundaries of the precisely located contour (e.g., take one point every 5 pixels), and calculate the average Euclidean distance between adjacent sampled points. Then, take its reciprocal as the edge continuity. That is:

[0103] This value reflects the smoothness and completeness of the fitted contour. The larger the value, the denser and more continuous the edge points, and the smoother and more complete the contour. For an ideal rectangular component, the edge continuity is close to 1.0.

[0104] Then calculate the location reliability. Using the preset first weighting coefficient Multiply by (1 minus the rate of change of area) The first term is obtained using a preset second weighting coefficient. The second term is obtained by multiplying by the edge continuity, and then using a preset third weighting coefficient. Multiplied by the rate of change of angle To obtain the third term, add the first term to the second term and then subtract the third term. The result is the location confidence level, i.e.:

[0105] Among them, three weighting coefficients It can be set based on experience and adjusted according to the actual scenario, for example, set to 0.4, 0.4, and 0.2 respectively. The smaller the area change rate, the higher the edge continuity, and the smaller the angle change rate, the higher the reliability.

[0106] When the positioning confidence level is greater than or equal to a preset threshold (e.g., 0.7), the fine positioning result is determined to be reliable, and the precise positioning contour obtained in step S140 is output as the final result.

[0107] When the positioning confidence level is lower than the threshold, the fine positioning is determined to be unsuccessful, and a rollback process is executed, including: abandoning the fine positioning result; outputting the initial orientation detection box obtained in step S120 (converted into center coordinates, width, height and zero angle information) as the final positioning result; and sending a "positioning downgrade" prompt message to the host computer or the upper layer of the system so that subsequent processes can perform corresponding processing (such as adjusting the robotic arm grasping strategy or manual intervention).

[0108] Embodiment 1 of the present invention, through the above steps S110 to S150, achieves accurate and robust positioning of components on waste circuit boards, and achieves the following significant advantages compared with the prior art:

[0109] 1) Effectively suppress false boundary interference by inverting the background color at both the global and local stages.

[0110] Step S110 uses sliding window statistics and surface fitting to construct a global board surface background color estimation map. The contrast response map (original grayscale minus background color estimation, divided by the background color estimation plus a minimal constant) eliminates global illumination unevenness and background color differences. Step S130 performs secondary background color modeling within the initial YOLO detection box, generating a background replacement map using edge background samples. Differential operations suppress the grayscale response of local silkscreen printing, stains, and other pseudo-boundaries to near zero. This strategy, combining global coarse elimination with local fine elimination, makes the component itself significantly prominent in the target significant residual map, fundamentally solving the problem of boundary mis-extraction caused by dirt and silkscreen printing in traditional methods.

[0111] (ii) YOLO initial localization and orientation constraint geometric reconstruction work together to achieve sub-pixel level arbitrary angle precision localization.

[0112] Step S120 utilizes a pre-trained YOLO network to quickly obtain an initial axis-aligned detection box from the contrast response map, providing an accurate local range for subsequent processing. Step S140 performs principal component analysis on the pixels within the detection box to obtain the estimated principal direction of the components, and sets a search line along the vertical direction to detect edge jumps. Finally, a precise rotated quadrilateral is reconstructed through least squares fitting and intersection calculation. This mechanism improves the positioning result from the coarse box of YOLO (axis-aligned, including background) to a quadrilateral with arbitrary angles that fits the real physical boundary. The angle error can be controlled within 1°, and the area error is reduced to less than 10% of the original box, meeting the high-precision requirements of automated disassembly.

[0113] (iii) Introduce a credibility assessment and rollback mechanism to ensure the overall robustness of the system.

[0114] Step S150 constructs a weighted reliability index based on the area change rate, angle change rate, and edge continuity between the fine positioning and the initial positioning. When the reliability is lower than a preset threshold (e.g., 0.7), the system automatically abandons unreliable fine positioning results, reverts to the YOLO initial detection frame, and sends a positioning degradation prompt. This "high precision first, robustness as a fallback" design avoids serious erroneous output contours due to extreme conditions (severe component damage, extreme lighting, large-area contamination), ensuring continuous and stable operation of the production line.

[0115] (iv) Reduce reliance on massive amounts of high-quality labeled data, facilitating rapid deployment.

[0116] This embodiment significantly reduces the dependence of pure deep learning solutions on large-scale, accurately labeled data through physical model-driven preprocessing (global background color estimation, local background subtraction) and geometric post-processing (orientation constraint fitting). Even if the YOLO network is trained with a small number of noisy training samples, as long as it can output detection boxes that roughly surround the components, subsequent steps can still correct deviations and achieve accurate localization through image processing and geometric constraints. This feature enables this method to be quickly deployed and achieve corresponding results in industrial recycling scenarios where labeling costs are high.

[0117] Example 2

[0118] like Figure 6 As shown, this embodiment provides a component localization system based on image preprocessing and YOLO recognition. This system is used to execute the method described in Embodiment 1. The system includes the following functional modules:

[0119] Image acquisition module: This module uses an industrial camera with a ring light source to acquire raw images containing the components. The output of this module is a high-resolution color image, which serves as the input to the entire positioning system.

[0120] Image preprocessing module: This module performs the image preprocessing function described in step S110 of Embodiment 1. Specifically, this module first converts the original color image into a grayscale image, completing grayscale normalization; then, it uses a sliding window of a specified size to traverse the entire grayscale image, calculating the average grayscale value of all pixels within each window, and constructing a global background color estimation map through surface fitting; finally, for each pixel, it subtracts the background color estimation value from the original grayscale value, and then divides by the sum of the background color estimation value and a minimal constant to generate a contrast response map. The output of this module is the contrast response map, which is sent to the target detection module.

[0121] Target Detection Module: This module incorporates a pre-trained YOLO target detection network (such as YOLOv5, YOLOv8, or YOLOx) to perform the initial target localization function described in step S120 of Embodiment 1. Taking the contrast response map as input, this module processes it through forward propagation, prior box matching, confidence threshold filtering, and non-maximum suppression. The output is an axis-aligned rectangle with the highest confidence level, serving as the initial orientation detection box for the component. This detection box is described by its center point coordinates, width, and height, providing the processing range for the subsequent local difference module.

[0122] Local Differentiation Module: This module performs the local background difference function described in step S130 of Embodiment 1. Within the local image region determined by the initial directional detection box, this module selects pixels near the edge of the detection box as a local background sample set. A background replacement map is generated through quadratic surface fitting or radial basis function interpolation. Then, the local image region is subtracted pixel-by-pixel from the background replacement map, and the absolute value is taken to generate a target significant residual map. This residual map effectively suppresses local background interference such as silkscreen printing and stains, highlighting the component itself.

[0123] Geometric Reconstruction Module: This module performs the precise positioning and reconstruction function described in step S140 of Embodiment 1. First, it performs principal component analysis on the coordinates of all pixels within the initial orientation detection box, determining the direction of the first principal component as the estimated principal direction of the component. Then, on the target saliency residual map, it sets multiple parallel search lines perpendicular to this principal direction, finding the point with the largest grayscale value jump on each search line as candidate edge points, and assigning these candidate edge points to four boundary point sets: top, bottom, left, and right. Next, it performs least-squares line fitting on each boundary point set to obtain four boundary lines. Finally, it calculates the intersection points of the boundary lines to obtain the coordinates of the four vertices of the component, reconstructing the precise positioning contour.

[0124] The credibility assessment module is used to execute the credibility determination and rollback processing function described in step S150 of Embodiment 1. This module calculates the area change rate and angle change rate of the precise positioning contour and the initial detection box, as well as the continuity of the edge points of the precise positioning contour; then, it calculates the positioning credibility using a weighted method: the first weight coefficient multiplied by (1 minus the area change rate) plus the second weight coefficient multiplied by the edge continuity, and then subtracting the third weight coefficient multiplied by the angle change rate. When the credibility is greater than or equal to a preset threshold (e.g., 0.7), the precise positioning contour is output; when the credibility is lower than the threshold, the initial orientation detection box is output as the final result, and a "positioning downgrade" prompt message is sent to the host computer.

[0125] Interaction and Output Module: This module visualizes the final positioning results and sends parameters such as the center coordinates, rotation angle, and coordinates of the four vertices of the component in the image coordinate system to the downstream robotic arm control or sorting system via TCP / IP protocol or serial communication, guiding the robotic arm to perform precise adsorption, gripping, or disassembly operations.

[0126] The above modules are connected in series to form a complete processing pipeline from image acquisition to localization result output. Among them, the image preprocessing module, target detection module, local difference module, geometric reconstruction module, and confidence evaluation module correspond to steps S110 to S150 in Embodiment 1. The specific working principle and data flow relationship of each module have been described in detail in Embodiment 1, and will not be repeated here.

[0127] This embodiment achieves automated, high-precision, and robust positioning of components on waste circuit boards through modular design, and can be easily integrated into electronic waste dismantling production lines.

[0128] The present invention has been described in detail above with reference to the accompanying drawings. Based on the technical concept of the present invention, the problem of difficult component positioning in complex backgrounds is effectively solved by using staged background color inversion and directional constraint edge fitting. It is understood that the above description is intended to illustrate the core idea of ​​the present invention, rather than to limit its scope of protection. Any modifications or equivalent substitutions of technical solutions that do not depart from the spirit of the present invention should be included within the scope of protection defined by the claims of the present invention.

[0129] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

Claims

1. A component localization method based on image preprocessing and YOLO recognition, characterized in that, Includes the following steps: Image preprocessing: Obtain the original image containing the components, perform grayscale normalization on the original image, construct a global board background color estimation map using the sliding window statistical method, and generate a contrast response map based on the original image and the global board background color estimation map; Initial target localization: The contrast response map is input into the pre-trained YOLO target detection network to obtain the initial orientation detection box of the component; Local background subtraction: Secondary background color modeling is performed in the local image region corresponding to the initial orientation detection box, a background replacement map is generated based on the local background samples, and a target salient residual map is obtained by subtracting the local image region from the background replacement map. Precise positioning and reconstruction: Calculate the main direction of the initial orientation detection box, and combine the main direction to perform directional constraint edge fitting and geometric reconstruction on the target significant residual map to obtain the precise positioning contour of the component; Confidence determination and rollback: Based on the parameter change rate of the precise positioning contour and the initial orientation detection box, as well as the continuity of edge points, the positioning confidence is calculated. When the confidence is lower than a preset threshold, rollback processing is performed, and the initial orientation detection box is output as the final positioning result.

2. The method according to claim 1, characterized in that, The image preprocessing step further includes: The entire grayscale image is traversed using a sliding window of a specified size. The average grayscale value and grayscale variance of all pixels in each window are calculated. The average grayscale value of all windows is fitted to a surface, and the fitted surface is used as the global panel background color estimation map. The contrast response map is generated as follows: for each pixel in the original grayscale image, the grayscale value of the pixel is subtracted from the grayscale value of the corresponding pixel in the global background color estimation map to obtain a first difference; the grayscale value of the corresponding pixel in the global background color estimation map is added to a preset very small positive number to obtain a first sum; the ratio of the first difference to the first sum is used as the grayscale value of the contrast response map at that pixel.

3. The method according to claim 1, characterized in that, The local background subtraction step further includes: Within the initial orientation detection box, pixels near the edge of the detection box are selected as a local background sample set. Surface fitting or interpolation is performed on the local background sample set to generate a background replacement image with the exact same size as the local image region. The target salient residual map is generated by subtracting the pixel grayscale value of the corresponding position in the local image region from that in the background replacement map and taking the absolute value to obtain the grayscale difference value of each pixel, thus forming the target salient residual map.

4. The method according to claim 1, characterized in that, The precise positioning and reconstruction step further includes: Principal component analysis is performed on the coordinates of all pixels within the initial orientation detection box, and the direction of the first principal component is determined as the estimated principal direction of the component. On the target significant residual map, multiple parallel search lines are set along the direction perpendicular to the estimated main direction. On each search line, the position point where the gray value jumps the most is found as the candidate edge point. The candidate edge points belonging to the same boundary of the component are grouped together, and the least squares line fitting or polynomial curve fitting is performed on the four boundary point groups respectively to obtain the four boundary lines of the component. Calculate the intersection points of the four boundary lines to obtain the coordinates of the four vertices of the component.

5. The method according to claim 1, characterized in that, In the credibility determination and rollback steps, the calculation method for the location credibility is as follows: Calculate the rate of change of the area between the fine-positioning contour and the initial detection box, and the rate of change of the angle between the principal direction of the fine-positioning contour and the reference direction of the initial detection box; Calculate the average continuity between adjacent edge points on the fine-positioned contour to obtain the edge continuity. The first term is obtained by multiplying the difference between 1 and the area change rate by a preset first weighting coefficient, the second term is obtained by multiplying the edge continuity by a preset second weighting coefficient, and the third term is obtained by multiplying the angle change rate by a preset third weighting coefficient. The first and second items are weighted and summed, and the difference between the weighted sum and the third item is used as the location confidence.

6. The method according to claim 1, characterized in that, The YOLO object detection network is one of YOLOv5, YOLOv8 or YOLOx, and its training dataset contains component image samples under various poses, lighting conditions and pollution levels. The sample label is the minimum bounding rectangle of the component.

7. The method according to claim 1, characterized in that, The rollback process includes: when the confidence level is lower than the preset threshold, determining that the precise positioning result is unreliable, abandoning the precise positioning contour, directly outputting the initial orientation detection box as the final positioning result, and sending a positioning degradation prompt message to the upper layer of the system.

8. The method according to claim 1, characterized in that, The method further includes an image quality assessment step performed before the image preprocessing step: calculating the global contrast and signal-to-noise ratio of the original image, and when the global contrast is lower than a preset contrast threshold or the signal-to-noise ratio is lower than a preset signal-to-noise ratio threshold, performing enhanced preprocessing of histogram equalization or adaptive gamma correction on the original image.

9. A component positioning system based on image preprocessing and YOLO recognition, characterized in that, For performing the method according to any one of claims 1 to 8, comprising: Image acquisition module, used to acquire raw images containing components; The image preprocessing module is used to perform grayscale normalization, global background color estimation, and contrast response map generation on the original image; The target detection module has a built-in pre-trained YOLO target detection network, which is used to extract the initial orientation detection box of the component from the contrast response map; The local difference module is used to perform secondary background color modeling and difference operations within the initial orientation detection box to generate a target saliency residual map; The geometric reconstruction module is used to calculate the main direction and perform edge fitting and geometric reconstruction with directional constraints on the target significant residual map; The credibility assessment module is used to calculate the credibility of the location and execute rollback decisions.

10. The system according to claim 9, characterized in that, The system also includes an interaction and output module, which is used to visualize the final positioning result and send the center coordinates, rotation angle and size parameters of the component in the image coordinate system to the downstream robotic arm control or sorting system.

Images