Visual inspection methods, devices, equipment and storage media for electronic components

Abstract

This invention relates to visual inspection technology, and more particularly to a visual inspection method, apparatus, device, and storage medium for electronic components. The method includes: acquiring an original image of the component under ordinary light source; establishing an original coordinate system in the original image based on a preset feature template to obtain the range of the area to be tested; then cropping an image of the range of the area to be tested from the original image as a local image; pre-training a position extraction model; inputting the local image into the pre-trained position extraction model for position extraction and coordinate prediction processing to obtain the coordinates of a first target to be detected; then comparing the coordinates of the first target to be detected with a preset standard coordinate threshold range to obtain a comparison result; and finally determining whether the coordinates of the first target to be detected are qualified based on the comparison result. This method can achieve accurate detection of objects with tiny structures such as connectors and circuit boards using only ordinary cameras and light sources, reducing detection costs and improving detection efficiency and accuracy.

Description

Technical Field

[0001] This invention relates to visual inspection technology, and more particularly to a visual inspection method, apparatus, device, and storage medium for electronic components. Background Technology

[0002] In the field of high-precision multi-target inspection, the inspection of products such as connectors, miniature LED displays, and semiconductor devices faces severe challenges. These targets often have three-dimensional structures or irregular shapes. Taking connector pin inspection as an example, the pin tips are tiny and irregular in shape, and their orientation is not fixed. In traditional image inspection technology, the shape differences of different pin tips under illumination conditions are significant. Camera sensors have difficulty accurately capturing the true position information of the pin tips, directly resulting in low detection accuracy and high false detection rate, which cannot meet the inspection requirements of high-precision scenarios.

[0003] To improve detection accuracy and precision, the industry is gradually adopting technologies such as 3D sensors and laser illumination. 3D sensors acquire 3D point cloud data of the target object, and subsequent point cloud processing can identify the true spatial position of the needle tip. However, these sensors have significant drawbacks. For example, structured light 3D sensors have a slow detection cycle, and laser scanning sensors require a high-precision displacement platform. Furthermore, 3D sensors are not only expensive to procure hardware, but also difficult to develop supporting algorithms. Laser illumination technology also suffers from complex algorithm development, insufficient adaptability, and low detection efficiency, making large-scale application in real-world production scenarios difficult. Summary of the Invention

[0004] In order to overcome the shortcomings of the prior art, the present invention aims to provide a visual inspection method for components, which can accurately inspect objects with tiny structures such as connectors and circuit boards using only ordinary cameras and light sources, thereby reducing inspection costs and improving inspection efficiency and accuracy.

[0005] The first aspect of this invention provides a visual inspection method for electronic components, comprising: acquiring an original image of the component under a normal light source; establishing an original coordinate system in the original image based on a preset feature template to obtain a region to be tested; cropping an image of the region to be tested from the original image as a local image; pre-training a position extraction model; inputting the local image into the pre-trained position extraction model for position extraction and coordinate prediction processing to obtain the coordinates of a first target to be detected; comparing the coordinates of the first target to be detected with a preset standard coordinate threshold range to obtain a comparison result; and determining whether the coordinates of the first target to be detected are qualified based on the comparison result.

[0006] Optionally, in a first implementation of the first aspect of the present invention, the step of acquiring the original image of the component under a normal light source and establishing an original coordinate system in the original image based on a preset feature template to obtain the range of the region to be measured includes: acquiring the original image of the component under a normal light source; performing template matching between the preset feature template and the original image to obtain the reference feature position in the original image corresponding to the preset feature template; establishing an original coordinate system based on the reference feature position; and performing a mapping operation in the original coordinate system based on the region parameters of the region to be measured recorded in the feature template that correspond to the reference feature position to obtain the range of the region to be measured.

[0007] Optionally, in a second implementation of the first aspect of the present invention, the pre-trained location extraction model includes: obtaining multiple initial image pairs, each initial image pair including a normal light source image and a laser light source image corresponding to the normal light source image; extracting multiple small region image pairs from the multiple initial image pairs, each small region image pair including a normal light source small image and a laser light source small image corresponding to the normal light source small image; performing image preprocessing on the multiple small region image pairs to obtain a training dataset; constructing an initial location extraction model based on a deep learning algorithm; and performing multiple rounds of iterative training on the initial location extraction model based on the training dataset to complete the pre-training of the location extraction model.

[0008] Optionally, in a third implementation of the first aspect of the present invention, the step of preprocessing the multiple pairs of small region images to obtain a training dataset includes: performing thresholding on each small image of a laser light source to obtain a binary image; performing connected component filtering on the binary image to extract the largest continuous closed set object and mark it to obtain a marked region; performing pixel processing on the marked region to obtain the coordinates of the center point of the marked region; and combining the coordinates of multiple center points and the small images of ordinary light sources corresponding to each center point coordinate to obtain the training dataset.

[0009] Optionally, in a fourth implementation of the first aspect of the present invention, the step of performing pixel processing on the marked region to obtain the center point coordinates of the marked region includes: obtaining the pixel coordinates of all pixels in the marked region; calculating the horizontal and vertical coordinate values ​​of multiple pixel coordinates to obtain a set of horizontal coordinates and a set of vertical coordinates; and performing coordinate mean calculation processing on the set of horizontal coordinates and the set of vertical coordinates respectively to obtain the center point coordinates of the marked region.

[0010] Optionally, in a fifth implementation of the first aspect of the present invention, comparing the coordinates of the first target to be detected with a preset standard coordinate threshold range to obtain a comparison result includes: obtaining the coordinates of a second target to be detected by manual detection; performing coordinate mean calculation processing on the coordinates of the first target to be detected and the second target to be detected to obtain the average coordinates of the target to be detected; and comparing the average coordinates of the target to be detected with a preset standard coordinate threshold range to obtain a comparison result.

[0011] Optionally, in a sixth implementation of the first aspect of the present invention, determining whether the coordinates of the first target to be detected are qualified based on the comparison result includes: if the average coordinates of the target to be detected are within a preset standard coordinate threshold range, then the coordinates of the first target to be detected are qualified; if the average coordinates of the target to be detected exceed the preset standard coordinate threshold range, then the deviation value between the coordinates of the first target to be detected and the coordinates of the second target to be detected is calculated; if the deviation value is less than or equal to a preset deviation tolerance threshold, then the coordinates of the first target to be detected are qualified; if the deviation value is greater than the preset deviation tolerance threshold, then the coordinates of the first target to be detected are unqualified.

[0012] A second aspect of the present invention provides a visual inspection device for electronic components, comprising: a region-to-be-tested determination module for acquiring an original image of the component under a normal light source, establishing an original coordinate system in the original image based on a preset feature template, and obtaining the range of the region to be tested; a local image acquisition module for extracting an image of the range of the region to be tested from the original image as a local image; a coordinate prediction module for pre-training a position extraction model, inputting the local image into the pre-trained position extraction model for position extraction and coordinate prediction processing, and obtaining the coordinates of a first target to be detected; a threshold comparison module for comparing the coordinates of the first target to be detected with a preset standard coordinate threshold range, and obtaining a comparison result; and a judgment module for judging whether the coordinates of the first target to be detected are qualified based on the comparison result.

[0013] A third aspect of the present invention provides a visual inspection device for electronic components, the visual inspection device comprising: a memory and at least one processor, the memory storing instructions; the at least one processor calling the instructions in the memory to cause the visual inspection device to perform the steps of the visual inspection method for electronic components described in any of the preceding claims.

[0014] A fourth aspect of the present invention provides a computer-readable storage medium storing instructions that, when executed by a processor, implement the steps of the visual inspection method for the components described in any of the preceding claims.

[0015] In the technical solution of this invention, the original image of the component under ordinary light source is first acquired. Based on the preset feature template, an original coordinate system is established in the original image to obtain the range of the area to be tested. Then, the image of the range of the area to be tested is cropped from the original image as a local image. A position extraction model is pre-trained. The local image is input into the pre-trained position extraction model for position extraction and coordinate prediction processing to obtain the coordinates of the first target to be detected. Then, the coordinates of the first target to be detected are compared with the preset standard coordinate threshold range to obtain the comparison result. Finally, based on the comparison result, it is determined whether the coordinates of the first target to be detected are qualified. It can achieve accurate detection of objects with tiny structures such as connectors and circuit boards using only ordinary cameras and light sources, reducing detection costs and improving detection efficiency and accuracy. Attached Figure Description

[0016] Figure 1 A logic flowchart of a visual inspection method for components provided in an embodiment of the present invention;

[0017] Figure 2 A schematic diagram of the structure of the visual inspection device for components provided in an embodiment of the present invention;

[0018] Figure 3 This is a schematic diagram of the structure of a visual inspection device for components provided in an embodiment of the present invention. Detailed Implementation

[0019] This invention provides a method, apparatus, device, and storage medium for visual inspection of electronic components. In this invention, the terms "first," "second," "third," "fourth," etc. (if present)," in the specification, claims, and accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in a sequence other than that illustrated or described herein. Furthermore, the terms "comprising" or "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or devices.

[0020] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 One embodiment of the visual inspection method for components in this invention includes:

[0021] 101. Obtain the original image of the component under ordinary light source, establish the original coordinate system in the original image based on the preset feature template, and obtain the range of the area to be measured;

[0022] In this embodiment, a complete original image of the component under ordinary light source is first acquired using an image acquisition device. This ensures that the image fully presents the overall structure and detailed features of the component. Common types of ordinary light sources, such as surface light sources, ring light sources, or strip light sources, can be used. Their core function is to clearly present the overall shape of the test area by directly illuminating the object under test. The versatility and ease of acquisition of ordinary light sources eliminate the need for complex and expensive optical equipment in acquiring the original image, significantly controlling the hardware cost of the detection system. Simultaneously, it adapts to different types of test objects and detection scenarios, enhancing the system's practicality. A preset feature template serves as a positioning reference, containing stable and unique reference feature information on the component. These reference features are typically inherent structures of the component, such as positioning holes, corner vertices, marking lines, or the outline of specific functional areas. Their position and shape remain fixed during manufacturing, making them suitable as positioning anchor points. In the matching stage between the original image and the feature template, a template matching algorithm accurately compares the two to locate the actual physical position in the original image corresponding to the template reference features. A primary coordinate system is established using the actual reference feature location as the core, thereby accurately delineating the target area to be detected in the original image, such as the pin tip area of ​​a connector or the functional unit area of ​​a semiconductor device, thus obtaining the range of the area to be tested. This effectively solves the problems of ambiguous location of the area to be tested and inconsistent references in traditional detection, significantly improving the consistency and accuracy of area positioning. Relying on the reference function of the preset feature template, the influence of interference factors such as background noise and illumination changes in the original image on area positioning can be avoided, ensuring that even with slight deviations in component placement, the area to be tested can still be locked through accurate matching of reference features. After establishing the primary coordinate system, subsequent operations such as target position extraction and size measurement are all carried out based on a unified spatial reference, providing spatial assurance for the accuracy of detection data and avoiding detection errors caused by inconsistent coordinate references.

[0023] 102. Extract the area to be tested from the original image as a local image;

[0024] In this embodiment, based on the determined range parameters of the area to be tested, a local image segment corresponding to the area to be tested is accurately extracted from the complete original image acquired under normal light source illumination, relying on the spatial positioning capability of the coordinate system. This local image ensures that the extracted local image can completely contain the target to be detected, while removing irrelevant background areas and non-detection parts from the original image, so that the local image focuses on the specific object to be detected.

[0025] 103. Pre-trained location extraction model: The local image is input into the pre-trained location extraction model for location extraction and coordinate prediction processing to obtain the coordinates of the first target to be detected;

[0026] In this embodiment, the obtained local image is input into the location extraction model. Based on the feature recognition capabilities learned during the training phase, the model extracts the location and predicts the coordinates of the target to be tested through a preset algorithm logic. This process can either first output a mask of the target location, and then convert it into specific location coordinates through geometric calculations, specifically by traversing all pixels belonging to the target region in the mask and recording their x and y coordinates in the local image coordinate system; then, the average of the x coordinates of these pixels is calculated to obtain the average x coordinate of the needle tip region, and the average of the y coordinates is calculated to obtain the average y coordinate; the two are combined to form the center coordinates of the needle tip in the local image, and then, based on the previously established global coordinate system, the spatial transformation of the center coordinates from the local image coordinate system to the global coordinate system is completed, finally obtaining the standardized first target coordinates; or the model can directly output the predicted coordinates of the target location, and then, based on the previously established global coordinate system, the spatial transformation of the predicted coordinates from the local image coordinate system to the global coordinate system is completed, finally obtaining the standardized first target coordinates. The application of the pre-trained location extraction model fully leverages the advantages of deep learning models in accurately identifying small and irregular targets. It effectively solves the location challenges of small objects like pinpoints in traditional image detection, which are difficult to locate due to their small size, variable orientation, and significant lighting effects. This significantly improves the accuracy and stability of predicting the position coordinates of small objects. Through the generalization capabilities accumulated during the pre-training phase, the model can adapt to detection needs under different working conditions, reducing the impact of environmental interference on detection results and enhancing the adaptability of the detection system. The mode of directly outputting coordinates or obtaining coordinates through mask transformation, combined with the coordinate system transformation process, ensures the consistency between the predicted coordinates and the overall detection benchmark, providing a standardized data foundation for subsequent multi-dimensional verification.

[0027] 104. Compare the coordinates of the first target to be detected with the preset standard coordinate threshold range to obtain the comparison result;

[0028] 105. Based on the comparison results, determine whether the coordinates of the first target to be detected are qualified.

[0029] In this embodiment, the preset standard coordinate threshold range is a benchmark range established based on the accuracy requirements of the detection task, the design parameters of the components, and the production process standards. It typically uses the theoretical standard coordinates of the target as the core, combined with the error tolerance in actual production, to define the allowable fluctuation ranges for both the horizontal and vertical coordinates, forming a complete threshold range. During the comparison process, the coordinates of the first target to be detected, obtained through a pre-trained model, are compared one by one with the horizontal and vertical coordinate ranges of the standard coordinate threshold range. This clarifies whether the coordinates are within the allowable range in both dimensions, thus forming a comparison result that includes the deviation of each dimension of the coordinates and a preliminary judgment of overall compliance. When conducting a conformity assessment based on this comparison result, if both the horizontal and vertical coordinates of the first target to be detected fall within the preset standard coordinate threshold range, the coordinates are deemed qualified, indicating that the actual position of the object, such as a needle tip, conforms to production and detection standards. If any dimension of the coordinate exceeds the standard threshold range, the coordinates are deemed unqualified, meaning that the position of the object, such as a needle tip, is deviated, and further investigation is needed to determine if there are product quality issues or errors in the detection process. By establishing a standardized coordinate comparison and conformity assessment mechanism, the positional deviation of the target to be inspected is effectively quantified, avoiding misjudgments caused by subjective factors in traditional manual assessment, and significantly improving the objectivity and accuracy of quality assessment. The preset standard coordinate threshold range is based on clear design and process requirements, providing a unified judgment benchmark for inspection and ensuring consistency of judgment standards across different batches and testing scenarios, thus facilitating the standardized implementation of quality control in the production process. Simultaneously, this assessment process can quickly locate non-conforming targets with out-of-range positions, providing accurate data support for subsequent quality traceability and production process optimization, helping to promptly identify and resolve deviations in the production process, and reducing the risk of non-conforming products entering the market.

[0030] This application discloses a visual inspection method for electronic components. First, the original image of the component under ordinary light source is acquired. Based on a preset feature template, an original coordinate system is established in the original image to obtain the range of the area to be tested. Then, the image of the range of the area to be tested is cropped from the original image as a local image. A pre-trained position extraction model is used, and the local image is input into the pre-trained position extraction model for position extraction and coordinate prediction to obtain the coordinates of the first target to be detected. Then, the coordinates of the first target to be detected are compared with a preset standard coordinate threshold range to obtain the comparison result. Finally, based on the comparison result, it is determined whether the coordinates of the first target to be detected are qualified. This method can accurately detect objects with tiny structures such as connectors and circuit boards using only ordinary cameras and light sources, reducing detection costs and improving detection efficiency and accuracy.

[0031] In this embodiment of the invention, the step of acquiring the original image of the component under a normal light source and establishing an original coordinate system in the original image based on a preset feature template to obtain the range of the area to be measured includes: acquiring the original image of the component under a normal light source; performing template matching between the preset feature template and the original image to obtain the reference feature position in the original image corresponding to the preset feature template; establishing an original coordinate system based on the reference feature position; and performing a mapping operation in the original coordinate system based on the parameters of the area to be measured recorded in the feature template that correspond to the reference feature position to obtain the range of the area to be measured.

[0032] In this embodiment, a high-resolution image acquisition device is used to acquire the original image of the component under ordinary light source. This ensures that the image fully preserves the overall structural features and detailed information of the component, providing a high-quality data foundation for subsequent feature matching. Then, a template matching operation is performed based on a preset feature template. This feature template pre-stores stable and unique reference feature information on the component. These reference features are mostly inherent structures of the component. By accurately comparing the feature template with the original image using a template matching algorithm, the actual reference feature position corresponding to the template in the original image can be accurately located. This achieves the association from template reference to actual image features, thereby locating the obtained feature. A primary coordinate system is established with the reference feature location as the core. Typically, the key points of the reference feature are selected as the origin of the coordinate system. The coordinate axes are defined by the geometric extension direction of the reference feature, and the coordinate units are set by combining the conversion relationship between image pixels and actual physical size, thus constructing a unified and accurate spatial measurement framework. Then, the feature template pre-records the spatial parameters of the area to be tested relative to the reference feature. These parameters clarify the relative offset, size boundary and other key information of the area to be tested. Based on these parameters, spatial mapping calculations are performed in the established primary coordinate system to convert the relative position parameters into specific absolute coordinate ranges, thereby accurately delineating the target area to be detected in the original image. It effectively solves the problems of ambiguous positioning of the test area and inconsistent benchmarks in traditional detection, significantly improving the accuracy and consistency of area positioning. Relying on the reference value of the preset feature template, it can effectively resist the influence of interference factors such as background noise and illumination changes in the original image. Even if there are slight deviations in the placement of components, the test area can still be locked through accurate matching of benchmark features, ensuring positioning stability. The establishment of the original coordinate system provides a unified spatial benchmark for subsequent operations such as target position extraction and size measurement, avoiding detection errors caused by coordinate system chaos, and providing a core guarantee for the accuracy of detection data.

[0033] In this embodiment of the invention, the pre-trained location extraction model includes: obtaining multiple initial image pairs, each initial image pair including a normal light source image and a laser light source image corresponding to the normal light source image; extracting multiple small region image pairs from the multiple initial image pairs, each small region image pair including a small image of a normal light source and a small image of a laser light source corresponding to the small image of the normal light source; performing image preprocessing on the multiple small region image pairs to obtain a training dataset; constructing an initial location extraction model based on a deep learning algorithm; and performing multiple rounds of iterative training on the initial location extraction model based on the training dataset to complete the pre-training of the location extraction model.

[0034] In this embodiment, a customized detection device is used, comprising two light source systems: a laser light source system and a conventional light source system, a high-precision detection camera, and an optional sample displacement stage to acquire multiple initial image pairs. The conventional light source system acquires images by generating images from multiple lasers capable of outputting horizontal laser lines. Each laser forms an extremely narrow light plane, projecting it onto the surface of the object under test in a near-horizontal direction. The laser light source system acquires images from the object under test. To achieve multi-angle coverage of the entire test plane, the laser light source system employs a ring or rectangular arrangement. This layout effectively reduces occlusion caused by multiple objects under test or the object's own structure, ensuring the target area is fully illuminated. Due to the extremely narrow nature of the laser light plane, only a portion of the object under test within a specific height range is illuminated and reflected. The reflected light is captured by the high-precision camera to form a clear laser light source image. By adjusting the height of the laser light source, specific targets can be precisely selected and focused, achieving efficient separation between the target area and the background. The sample displacement stage is mainly used to adapt to test objects of different sizes. By driving the test object to move precisely in the XY direction, the relative position of the test object with the camera and light source can be adjusted to ensure that all areas to be detected can be completely captured regardless of the size of the test object. This greatly improves the applicability of the device and meets the diverse needs of detection tasks.

[0035] In this embodiment, for components such as connectors, the extracted small region refers to the local image corresponding to objects with tiny structures such as a needle tip, ensuring that the image data is focused on the target to be detected and reducing background interference. Image preprocessing operations are performed on the small region image pairs to obtain a training dataset. An initial position extraction model is built based on a deep learning algorithm and pre-trained. The constructed model adopts an architecture adapted to image tasks, such as a convolutional neural network or a Transformer network. The output format of the model is determined according to the type of training dataset, i.e., outputting a mask image or a coordinate vector. The model parameters are optimized through multiple rounds of iterative training, enabling the model to accurately extract target position information from images with ordinary light sources. Focusing on the small region to be detected to extract image data effectively reduces the interference of irrelevant background information on model training, improves the model's learning efficiency of target features, and fully leverages the deep learning algorithm to capture the features of small and irregular targets, overcoming the limitations of traditional image detection methods. The pre-trained model can be directly applied to actual detection scenarios, quickly achieving automated extraction of target positions, significantly improving detection efficiency, and avoiding the high cost issues associated with 3D sensors and other equipment.

[0036] In this embodiment of the invention, the image preprocessing of multiple pairs of small region images to obtain a training dataset includes: thresholding each small image of a laser light source to obtain a binary image; performing connected component filtering on the binary image to extract the largest continuous closed set object and mark it to obtain a marked region; performing pixel processing on the marked region to obtain the center point coordinates of the marked region; and combining multiple center point coordinates and ordinary light source small images corresponding to each center point coordinate to obtain the training dataset.

[0037] In this embodiment, a thresholding operation is first performed on each small image of the laser source. A reasonable threshold is set based on the pixel grayscale difference between the target and the background in the small laser image. Regions with pixel values ​​higher or lower than this threshold are divided into target pixels and background pixels, respectively. This transforms the original grayscale laser image into a binary image containing only the target and background, achieving initial separation of the target region from the background. Subsequently, connected component filtering is performed on the obtained binary image. By identifying continuous regions where all pixels in the image are connected, scattered small regions caused by noise and light interference are eliminated. The largest continuous closed set of objects is selected and specifically marked, resulting in a marked region and a mask image. This marked region is the target region. The precise localization of the target area effectively eliminates interference from non-target factors. After obtaining the marked area, its center point coordinates are calculated through pixel-level data processing. Finally, multiple center point coordinates obtained through the above process are combined with the corresponding small images of ordinary light sources to construct the training dataset required for model training. Alternatively, multiple mask images obtained through the above process are combined with the corresponding small images of ordinary light sources to construct the training dataset required for model training. This effectively avoids the subjective errors and inefficiencies that may occur in manual annotation, significantly improving the efficiency and quality of training dataset preparation. The combined application of thresholding and connected component filtering can accurately separate the target from the background, especially suitable for small, irregularly shaped targets (such as connector pin tips) in high-precision scenes. It greatly reduces the impact of noise and other interference factors on target localization, ensuring the accuracy of center point coordinates. By associating small images of ordinary light sources with center point coordinates to construct a dataset, or by associating small images of ordinary light sources with mask images to construct a dataset, subsequent model training can learn the ability to directly predict the target location from images of ordinary lighting, without relying on complex laser imaging equipment.

[0038] In this embodiment of the invention, the step of performing pixel processing on the marked region to obtain the center point coordinates of the marked region includes: obtaining the pixel coordinates of all pixels in the marked region; calculating the horizontal and vertical coordinate values ​​of multiple pixel coordinates to obtain a set of horizontal coordinates and a set of vertical coordinates; and performing coordinate mean calculation on the set of horizontal coordinates and the set of vertical coordinates respectively to obtain the center point coordinates of the marked region.

[0039] In this embodiment, the core location of the target is quantitatively represented by statistically analyzing and averaging the pixel coordinates within the marked area, providing accurate location labels for subsequent model training. Specifically, the defined marked area is first traversed, and the pixel coordinates of all pixels within that area are collected in the image coordinate system. Each pixel coordinate contains unique horizontal and vertical coordinates, thus forming a coordinate dataset covering all pixels in the target area. Subsequently, the coordinate dataset was classified and statistically analyzed. The x-coordinate values ​​of all pixels were integrated into a x-coordinate set, and the y-coordinate values ​​of all pixels were integrated into a y-coordinate set, completing the dimensional splitting and classification of the coordinate data. Based on this, the mean calculation operation was performed on the x-coordinate set and the y-coordinate set respectively. That is, the sum of the values ​​of the x-coordinate set and the sum of the values ​​of the y-coordinate set were obtained by summing, and then dividing by the number of elements in the corresponding set, i.e., the number of pixels. The two means obtained are used as the x-coordinate and y-coordinate of the center point of the marked area, respectively. The two are combined to form the center point coordinates of the marked area. This coordinate can objectively reflect the geometric center position of the target area. Relying on pixel-level coordinate statistics and mean calculation, the center point coordinates are accurately solved. Compared with the traditional geometric estimation method, it makes full use of the position information of all pixels in the marked area, effectively avoids the center positioning deviation caused by the irregular shape of the target, and significantly improves the accuracy and stability of coordinate calculation. It is especially suitable for the position representation of small and irregular targets such as connector tips. It does not rely on complex hardware devices or algorithm models. The coordinate solution can be completed through basic statistical calculations, reducing the technical implementation cost.

[0040] In this embodiment of the invention, comparing the coordinates of the first target to be detected with a preset standard coordinate threshold range to obtain a comparison result includes: obtaining the coordinates of a second target to be detected through manual detection; performing coordinate mean calculation on the coordinates of the first target to be detected and the second target to be detected to obtain the average coordinates of the target to be detected; and comparing the average coordinates of the target to be detected with a preset standard coordinate threshold range to obtain a comparison result.

[0041] In this embodiment, reliable manually detected coordinates are obtained through traditional image processing methods, such as threshold extraction methods, to extract the location of the target region. After extracting the target region, the predicted location of the target region, i.e., the second target coordinate, can be obtained through geometric methods. This coordinate combines the stability of traditional algorithms with the accuracy of human experience, and has high reference value. Based on this, the coordinate mean of the first and second target coordinates is calculated, i.e., the arithmetic mean of the horizontal and vertical coordinates of the two coordinates is taken to form the average coordinate of the target. This average coordinate combines the automated output of the algorithm with the accurate result of manual verification, which can balance the possible deviations of a single data source and is closer to the true location of the target. Subsequently, the average coordinate is compared with a preset standard coordinate threshold range. The standard coordinate threshold range is determined based on product design parameters and process requirements. By judging whether the average coordinate falls within the threshold range, the final comparison result is formed, providing a basis for determining the qualification of the target location. By introducing manually detected coordinates based on traditional image processing, the stability of traditional methods in specific scenarios is maintained, while the reliability of coordinates is improved through manual verification. This provides a high-quality comparison benchmark for the first target coordinates output by the algorithm. The calculation of the coordinate mean realizes the organic integration of the two data sources, which not only preserves the efficiency of the algorithm detection, but also enhances the robustness of the results through manual experience correction.

[0042] In this embodiment of the invention, determining whether the coordinates of the first target to be detected are qualified based on the comparison result includes: if the average coordinates of the target to be detected are within a preset standard coordinate threshold range, then the coordinates of the first target to be detected are determined to be qualified; if the average coordinates of the target to be detected exceed the preset standard coordinate threshold range, then the deviation value between the coordinates of the first target to be detected and the coordinates of the second target to be detected is calculated; if the deviation value is less than or equal to a preset deviation tolerance threshold, then the coordinates of the first target to be detected are determined to be qualified; if the deviation value is greater than the preset deviation tolerance threshold, then the coordinates of the first target to be detected are determined to be unqualified.

[0043] In this embodiment, when the average coordinates of the target to be detected are within the preset standard coordinate threshold range, it indicates that the fusion result of the first target coordinates and the manually detected second target coordinates meets the product design and process requirements, thus directly determining that the first target coordinates are qualified. If the average coordinates of the target to be detected exceed the preset standard coordinate threshold range, deviation analysis needs to be further introduced. By calculating the deviation value between the first and second target coordinates, where the deviation value commonly uses quantitative indicators such as Euclidean distance, the degree of positional difference between the two is clarified. Subsequently, the final judgment is made based on the comparison result of the deviation value and the preset deviation tolerance threshold: if the deviation value is less than or equal to the preset deviation tolerance threshold, it indicates that although the first target coordinates differ from the manual coordinates to a certain extent, the difference is within an acceptable range, and the average coordinate exceeding the limit is most likely dominated by the small deviation of the manual coordinates, so the first target coordinates are qualified; if the deviation value is greater than the preset deviation tolerance threshold, it indicates that the first target coordinates not only deviate from the standard range, but also differ significantly from the manual coordinates used as a reference, and has a large positioning error, so it is determined to be unqualified. By implementing case-by-case verification, the coordinates of the first target to be inspected are accurately evaluated, effectively avoiding misjudgments such as directly determining non-compliance due to average coordinates exceeding limits. Simultaneously, it can accurately identify non-compliance cases where the coordinates themselves have significant deviations. Introducing a deviation tolerance threshold fully respects the reference value of manually inspected coordinates while providing a reasonable error tolerance margin for the algorithm's output coordinates, adapting to minor errors that may exist in high-precision inspection scenarios.

[0044] The core basis for setting the preset standard coordinate threshold range is the product's design and process requirements. It must strictly adhere to the component design drawings and technical specifications, using the theoretical design coordinates of the target position as a benchmark, while also considering the actual error range in the production process, such as injection molding accuracy, welding deviation, and assembly tolerances, to define the threshold range. For the preset deviation tolerance threshold, the core basis for setting it is first to consider the error characteristics of manually inspected coordinates. Although manually inspected coordinates serve as a reference, they are inherently flawed due to factors such as the operator's experience and the limitations of traditional image processing algorithms. The threshold must cover these reasonable errors to avoid errors caused by minor deviations in manually inspected coordinates. The first step is to determine the qualified coordinates output by the error detection algorithm. Secondly, based on the positioning accuracy level of the algorithm model and combined with the model training and validation data, it is necessary to analyze its positioning error distribution under different working conditions. If the model has a small positioning error in stable scenarios, the threshold can be appropriately narrowed. If the model is less adaptable to complex scenarios, the threshold can be appropriately relaxed without affecting product quality, ensuring the objectivity of the judgment results. At the same time, it is necessary to consider the enterprise's quality control risk tolerance. Enterprises adopting strict quality control models and having zero tolerance for market risks need to strictly limit the threshold; enterprises that tend to balance quality and cost can relax the threshold within a controllable risk range to reduce the waste of qualified products caused by excessive testing.

[0045] The visual inspection method for components in the embodiments of the present invention has been described above. The visual inspection device for components in the embodiments of the present invention will be described below. Please refer to [link / reference]. Figure 2 One embodiment of the visual inspection device for components in this invention includes:

[0046] The test area determination module 201 is used to acquire the original image of the component under a normal light source, establish an original coordinate system in the original image based on a preset feature template, and obtain the range of the test area.

[0047] Local image acquisition module 202: used to extract the area to be tested from the original image as a local image;

[0048] Coordinate prediction module 203: used to pre-train the position extraction model, input the local image into the pre-trained position extraction model to perform position extraction and coordinate prediction processing, and obtain the coordinates of the first target to be detected;

[0049] Threshold comparison module 204: used to compare the coordinates of the first target to be detected with a preset standard coordinate threshold range to obtain a comparison result;

[0050] Judgment module 205: used to determine whether the coordinates of the first target to be detected are qualified based on the comparison results.

[0051] Based on the same ideas as the methods in the above embodiments, the apparatus provided in this application can implement the methods in the above embodiments.

[0052] above Figure 2 The visual inspection device for components in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The visual inspection equipment for components in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0053] Figure 3 This is a schematic diagram of the structure of a visual inspection device for components provided in an embodiment of the present invention. The visual inspection device 300 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown in the diagram), each module including a series of instruction operations on the visual inspection device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations in the storage media 330 on the visual inspection device 300 to implement the steps of the visual inspection method for components provided in the above-described method embodiments.

[0054] The visual inspection device 300 for components may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art will understand that... Figure 3 The illustrated structure of the visual inspection equipment for the components does not constitute a limitation on the visual inspection equipment for the components. It may include more or fewer components than illustrated, or combine certain components, or have different component arrangements.

[0055] The present invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium, wherein the computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of a visual inspection method for components.

[0056] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0057] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0058] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A visual inspection method for electronic components, characterized in that, include: Acquire the original image of the component under ordinary light source, establish an original coordinate system in the original image based on a preset feature template, and obtain the range of the area to be measured. The image of the region to be tested is cropped from the original image and used as a local image; Multiple initial image pairs are obtained, each initial image pair including a normal light source image and a laser light source image corresponding to the normal light source image; multiple small region image pairs are extracted from the multiple initial image pairs, each small region image pair including a small image of a normal light source and a small image of a laser light source corresponding to the small image of a normal light source; thresholding is performed on each small image of a laser light source to obtain a binary image; connected component filtering is performed on the binary image to extract the continuous closed set object with the largest area and mark it to obtain a marked region; pixel processing is performed on the marked region to obtain the center point coordinates of the marked region; multiple center point coordinates and the small images of normal light sources corresponding to each center point coordinate are combined to obtain a training dataset; an initial position extraction model is constructed based on a deep learning algorithm, and the initial position extraction model is trained iteratively multiple times based on the training dataset to complete the pre-training of the position extraction model; the local image is input into the pre-trained position extraction model for position extraction and coordinate prediction processing to obtain the coordinates of the first target to be detected; The coordinates of the first target to be detected are compared with a preset standard coordinate threshold range to obtain the comparison result; Based on the comparison results, it is determined whether the coordinates of the first target to be detected are qualified.

2. The visual inspection method for components according to claim 1, characterized in that, The process involves acquiring the original image of the component under normal light source, establishing an original coordinate system in the original image based on a preset feature template, and obtaining the range of the area to be measured, including: Acquire raw images of components under normal light sources; The preset feature template is matched with the original image to obtain the reference feature position in the original image corresponding to the preset feature template. Establish an original coordinate system based on the aforementioned reference feature positions; Based on the parameters of the region to be measured recorded in the feature template and corresponding to the reference feature position, a mapping operation is performed in the original coordinate system to obtain the range of the region to be measured.

3. The visual inspection method for components according to claim 1, characterized in that, The step of performing pixel processing on the marked region to obtain the center point coordinates of the marked region includes: Obtain the pixel coordinates of all pixels in the marked region; By statistically analyzing the x-coordinate and y-coordinate values ​​of multiple pixel coordinates, a set of x-coordinates and a set of y-coordinates are obtained. The coordinates of the center point of the marked region are obtained by calculating the mean of the coordinates of the x-coordinate set and the y-coordinate set respectively.

4. The visual inspection method for components according to claim 1, characterized in that, The coordinates of the first target to be detected are compared with a preset standard coordinate threshold range to obtain the comparison results, including: Obtain the coordinates of the second target to be detected by manual inspection; The average coordinates of the first and second targets to be detected are calculated to obtain the average coordinates of the targets to be detected. The average coordinates of the target to be detected are compared with the preset standard coordinate threshold range to obtain the comparison result.

5. The visual inspection method for components according to claim 4, characterized in that, The step of determining whether the coordinates of the first target to be detected are qualified based on the comparison result includes: If the average coordinates of the target to be detected are within the preset standard coordinate threshold range, then the coordinates of the first target to be detected are deemed qualified. If the average coordinates of the target to be detected exceed the preset standard coordinate threshold range, then the deviation value between the coordinates of the first target to be detected and the coordinates of the second target to be detected is calculated; If the deviation value is less than or equal to the preset deviation tolerance threshold, the coordinates of the first target to be detected are deemed qualified. If the deviation value is greater than the preset deviation tolerance threshold, the coordinates of the first target to be detected are determined to be unqualified.

6. A visual inspection device for electronic components, characterized in that, include: Test area determination module: used to acquire the original image of the component under ordinary light source, establish an original coordinate system in the original image based on a preset feature template, and obtain the range of the test area; Local image acquisition module: used to extract the area to be tested from the original image as a local image; The coordinate prediction module is used to obtain multiple initial image pairs, each initial image pair including a normal light source image and a laser light source image corresponding to the normal light source image; extract multiple small region image pairs from the multiple initial image pairs, each small region image pair including a small image of a normal light source and a small image of a laser light source corresponding to the small image of a normal light source; perform thresholding on each small image of a laser light source to obtain a binary image; perform connected component filtering on the binary image to extract the continuous closed set object with the largest area and mark it to obtain a marked region; perform pixel processing on the marked region to obtain the center point coordinates of the marked region; combine multiple center point coordinates and the small images of normal light sources corresponding to each center point coordinate to obtain a training dataset; construct an initial position extraction model based on a deep learning algorithm; perform multiple rounds of iterative training on the initial position extraction model based on the training dataset to complete the pre-training of the position extraction model; input the local image into the pre-trained position extraction model for position extraction and coordinate prediction processing to obtain the coordinates of the first target to be detected; Threshold comparison module: used to compare the coordinates of the first target to be detected with a preset standard coordinate threshold range to obtain the comparison result; Judgment module: used to determine whether the coordinates of the first target to be detected are qualified based on the comparison results.

7. A visual inspection device for electronic components, characterized in that, The visual inspection device for the components includes: a memory and at least one processor, wherein the memory stores instructions; At least one of the processors invokes the instructions in the memory to cause the visual inspection device of the component to perform the steps of the visual inspection method of the component as described in any one of claims 1-5.

8. A computer-readable storage medium storing instructions thereon, characterized in that, When the instructions are executed by the processor, they implement the steps of the visual inspection method for the components as described in any one of claims 1-5.

Images