Component offset detection method and device and electronic equipment

Abstract

The invention relates to the technical field of semiconductor manufacturing, and discloses a component offset detection method and device and electronic equipment. The method comprises the steps of obtaining a target image of a to-be-detected component, and performing positioning detection according to the target image and a template image to obtain a reference point image and a pin image on the target image; according to the reference point image and the pin image, detecting whether the position offset of the reference point and the pin is within a preset range; and if the position offset is within the preset range, carrying out image line segment detection on the pin image to obtain an angle of a pin in the to-be-detected component, and determining whether the to-be-detected component deviates or not based on the angle of the pin. The method does not depend on a color extraction algorithm, errors caused by difficult color processing are reduced, and the accuracy and stability of offset detection are improved; the method is suitable for various types of components, more direct and accurate deviation judgment is provided for angle information, and flexibility is achieved so as to adapt to design and manufacturing of different PCB cards.

Description

Technical Field

[0001] The present application relates to the field of semiconductor manufacturing technology, and in particular to a component offset detection method, device and electronic equipment. Background Art

[0002] In the integrated circuit manufacturing process, factors such as inaccurate placement of materials by the placement machine or thermal stress during welding may cause slight deviations or errors in the position of circuit components on the chip relative to the design specifications. Such slight deviations or errors may affect the performance, reliability, power consumption and manufacturing cost of the chip. Therefore, in the integrated circuit manufacturing process, it is very important to accurately control the accuracy of component positions.

[0003] The related technology extracts the color of the component body through a color algorithm, obtains the position of the component body by fitting an external rectangle, and compares the center point offset of the component body position with the image area to be detected to determine the offset position.

[0004] In the process of implementing the embodiments of the present application, the inventors found that the related technology has at least the following technical problems: the color extraction algorithm has a high operational complexity, which may cause edge burrs and color loss, thereby affecting the accuracy of the fitted circumscribed rectangle; especially for black IC boards, the extraction effect is usually poor, and the position information of the IC body may not be accurately obtained. In addition, the related technology depends on the positioning accuracy of the PCB board. If the positioning of the PCB board is inaccurate, false alarms are likely to occur. These problems may lead to inaccurate offset detection results. Summary of the invention

[0005] The main technical problem solved by the implementation methods of the present application is how to improve the accuracy, stability and flexibility of component offset detection.

[0006] In order to solve the above technical problems, a technical solution adopted in the implementation mode of the present application is: to provide a method for detecting the offset of components, including: obtaining a target image corresponding to the component to be detected, performing positioning detection based on the target image and the template image, and obtaining a reference point image and a pin image on the target image; based on the reference point image and the pin image, detecting whether the position offset between the reference point and the pin is within a preset range; if the position offset is within the preset range, performing image line segment detection on the pin image, obtaining the angle of the pin in the component to be detected, and determining whether the component to be detected is offset based on the angle of the pin.

[0007] The offset detection method of the component directly obtains the position information of the component through the positioning detection of the reference point and the pin, without relying on the color extraction algorithm, which provides more accurate and stable position information, reduces the error caused by the difficulty of color processing, and thus improves the accuracy and stability of the offset detection. In addition, since the method does not rely on the color extraction algorithm, it can be applied to various types of components, including black IC boards with difficult color processing, which expands the applicability of the method, so that it can be used for offset detection on more different types of components, and improves the versatility and practicality of the method. Moreover, the reference points are fixed during the design and manufacturing process, and their positions will not be affected by the placement of components. Therefore, using reference points and pins for offset calculation can provide more accurate results, especially when the shape of the component body changes or the size difference is large. Furthermore, determining the offset by detecting the angle of the pin provides more direct and accurate offset information compared to the method of fitting the circumscribed rectangle. Such angle information can more accurately reflect the actual offset of the component, further improving the stability and reliability of the detection results. Finally, this method performs positioning and offset detection based on the characteristics of the components themselves, and does not rely on the accurate positioning or marking of the PCB board. Instead, it performs positioning and judgment by analyzing the reference points and pin characteristics of the components. This method is more flexible and can cope with changes in the design and manufacturing of different PCB boards, thus having better adaptability and flexibility, and being able to cope with diverse component detection needs. Therefore, the above-mentioned component offset detection method has significant beneficial effects in terms of accuracy, stability, applicability and flexibility. It provides more accurate and stable position information, is suitable for various types of components, provides more direct and accurate offset judgment through angle information, and has the flexibility to adapt to different PCB board designs and manufacturing. These advantages make this method a reliable and efficient component offset detection solution.

[0008] Optionally, performing image line segment detection on the pin image to obtain the angle of the pin in the component to be detected includes: performing line segment detection on the pin image to obtain the straight line segment in the edge of the pin; obtaining the angle corresponding to each straight line segment; counting the angles of all straight line segments; identifying the angle with the highest frequency as the angle of the pin, and the angle does not include zero and ninety degrees. Among them, excluding the angles of zero and ninety degrees is to exclude horizontal, vertical and right-angled line segments, because they usually indicate a non-offset situation. By excluding these angles, the actual angle of the pin can be estimated more accurately; by identifying the angle with the highest frequency as the angle of the pin, the main direction of the pin can be accurately determined.

[0009] Optionally, the positioning detection is performed according to the target image and the template image to obtain the reference point image and the pin image on the target image, including: obtaining the template image corresponding to the target image, determining the size of the sliding window according to the template image, and determining the sliding step length; according to the sliding step length, using the sliding window to traverse the target image, and in the traversal process, obtaining the region of interest image generated by each sliding position of the sliding window; calculating the similarity metric value between each region of interest image and the template image; comparing the similarity metric values, and finding the region of interest image corresponding to the maximum similarity metric value; when the template image is a reference point template image, determining the region of interest image corresponding to the maximum similarity metric value as the reference point image; when the template image is a pin template image, determining the region of interest image corresponding to the maximum similarity metric value as the pin image. Among them, positioning the reference point and the pin by means of a sliding window can simply and intuitively determine the position of the reference point and the pin, which provides accurate reference information for subsequent offset detection of components.

[0010] Optionally, the positioning detection is performed based on the target image and the template image to obtain the reference point image and the pin image on the target image, including: obtaining the template image corresponding to the target image; creating a first pyramid image corresponding to the target image and a second pyramid image corresponding to the template image; performing similarity measurement on the first pyramid image and the second pyramid image to obtain the target area; when the template image is a reference point template image, determining that the image of the target area is a reference point image; when the template image is a pin template image, determining that the image of the target area is a pin image. Wherein, through similarity measurement and pyramid matching, the target area can be accurately positioned to improve the accuracy and reliability of detection.

[0011] Optionally, the performing similarity measurement on the first pyramid image and the second pyramid image to obtain the target area includes: obtaining an image of a region of interest at the highest level in the first pyramid image; performing similarity measurement on the image of the region of interest at the highest level and a template image at the highest level in the second pyramid image to obtain a maximum similarity measurement value of the highest level; transferring an area corresponding to the maximum similarity measurement value of the highest level to a level below the highest level in the first pyramid image to determine a local area of ​​the next level; obtaining an image of the region of interest corresponding to the local area; performing similarity measurement on the image of the region of interest corresponding to the local area and a template image at a level below the highest level in the second pyramid image to obtain a maximum similarity measurement value of the next level; based on the first pyramid image, transferring an area corresponding to the maximum similarity measurement value of the next level to the next level, performing local scan matching of the current level, and repeating the process of obtaining the maximum similarity measurement value of the current level and the local scan matching until the lowest level of the first pyramid image is reached, and taking an area corresponding to the maximum similarity measurement value obtained at the lowest level as the target area. Among them, the pyramid matching algorithm can locate the target in a shorter time, because the algorithm will first match on the higher-level pyramid image to quickly determine the rough position, and then gradually refine the match until the lowest level is reached. This can greatly reduce the number of pixels that need to be searched and matched, thereby speeding up the entire matching process, thereby improving the offset detection efficiency of components.

[0012] Optionally, the detecting whether the position offset between the reference point and the pin is within a preset range according to the reference point image and the pin image includes: obtaining a reference point detection frame corresponding to the reference point image, and obtaining a first center point corresponding to the reference point detection frame; obtaining a pin detection frame corresponding to the pin image, and obtaining a second center point corresponding to the pin detection frame; calculating the offset between the first center point and the second center point, the offset being the position offset between the reference point and the pin; comparing the offset with the preset range to determine whether the position offset between the reference point and the pin is within the preset range. Among them, by calculating the offset between the center points, the influence of noise or interference factors in some images on the position offset calculation can be eliminated, the stability of the position offset result is improved, and the offset detection is more reliable. At the same time, the method has strong flexibility and is applicable to reference points and pins of various types and shapes, which enhances the adaptability and practicality of the method.

[0013] Optionally, the template image includes a reference point template image and a pin template image, and the method further includes: acquiring the reference point template image and the pin template image; acquiring the reference point template image and the pin template image includes: determining a reference point area and a pin area according to the original image, the original image includes multiple reference points and multiple pins; searching for all reference points and all pins in the original image based on the reference point area and the pin area; wherein all reference points constitute a reference point template image, and the reference point template image includes the position information and grayscale information of the reference points; and all pins constitute a pin template image, and the pin template image includes the position information and grayscale information of the pins. wherein, by acquiring the reference point template image and the pin template image, and ensuring that all reference point and pin information are contained therein, the method realizes an accurate and automated template acquisition process. the method improves the accuracy, automation and comprehensiveness of the template, and provides a reliable basis for subsequent processing and analysis.

[0014] In order to solve the above technical problems, another technical solution adopted in the embodiment of the present application is: to provide a component offset detection device, including: a target image acquisition and target positioning module, used to acquire the target image corresponding to the component to be detected, perform positioning detection according to the target image and the template image, and obtain the reference point image and the pin image on the target image; a target detection module, used to detect whether the position offset between the reference point and the pin is within a preset range according to the reference point image and the pin image; an offset detection module, used to perform image line segment detection on the pin image if the position offset is within the preset range, obtain the angle of the pin in the component to be detected, and determine whether the component to be detected is offset based on the angle of the pin. The component offset detection device has the same beneficial effects as the above component offset detection method.

[0015] To solve the above technical problems, another technical solution adopted by the embodiment of the present application is: to provide an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above method. The electronic device has the same beneficial effects as the above component offset detection method.

[0016] In order to solve the above technical problems, another technical solution adopted by the embodiment of the present application is: providing a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by an electronic device, the electronic device executes the above method. The non-volatile computer-readable storage medium has the same beneficial effects as the above component offset detection method.

[0017] In order to solve the above technical problems, another technical solution adopted in the embodiment of the present application is: providing a computer program product, the computer program product includes a computer program stored on a non-volatile computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by an electronic device, the electronic device executes the above method. The computer program product has the same beneficial effects as the above component offset detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] One or more embodiments are exemplarily described by corresponding drawings, which do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and the figures in the drawings do not constitute proportional limitations unless otherwise stated.

[0019] Figure 1 is a flow chart of a component offset detection method provided in an embodiment of the present application;

[0020] Figure 2 is a flow chart of a method for obtaining a reference point image and a pin image provided by an embodiment of the present application;

[0021] Figure 3 is a flowchart of a method for obtaining a reference point image and a pin image provided by another embodiment of the present application;

[0022] Figure 4 It is a flow chart of a method for determining the position offset of a reference point detection frame and a pin detection frame according to a reference point image and a pin image provided by an embodiment of the present application;

[0023] Figure 5a to Figure 5d It is a schematic diagram of the process of obtaining a pad template image and a pin template image provided in an embodiment of the present application;

[0024] Figure 6 It is a structural schematic diagram of a component offset detection device provided in an embodiment of the present application;

[0025] Figure 7 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0026] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device schematic diagram or the flow chart. Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those generally understood by technicians in the technical field of the present application. The terms used in the specification of the present application are only for the purpose of describing specific implementation methods and are not used to limit the present application.

[0027] IC chip, full name Integrated Circuit Chip, is a special component that integrates multiple electronic components (such as transistors, resistors, capacitors, etc.) on a single semiconductor chip. IC chip realizes complex circuit functions by imprinting, depositing and connecting electronic components on the chip. It is a key core component of modern electronic devices and is widely used in various fields such as computers, mobile phones, televisions, automotive electronics, etc.

[0028] Component offset refers to the offset phenomenon that may occur during the design, manufacturing and assembly of integrated circuits (ICs), including slight changes in component size and position. Component offset can be caused by a variety of factors, including manufacturing process, temperature changes, and slight changes in the relative position between pads and chip pins.

[0029] Among them, inaccurate control of parameters such as soldering temperature, time, and pressure during component assembly may cause slight shifts in the positions of pads and pins. During the assembly process, due to temperature changes, materials such as chips, substrates, and solder may expand and contract, causing slight changes in the relative positions of pads and pins. Mechanical stress applied during assembly, such as pressure during soldering or bending of the substrate, may also cause slight shifts in pads and pins.

[0030] This offset may have a certain impact on the performance of the chip. For example, the electrical connection may deteriorate, resulting in a decrease in the quality of signal transmission; poor contact between the pad and the pin may cause increased resistance; and may even cause problems such as short circuits or open circuits.

[0031] In order to reduce the impact of component offset on performance, designers usually perform precise modeling, simulation and testing during the design and manufacturing process. They also use various technologies and methods to compensate or calibrate the offset in the chip to ensure that the chip can work properly and meet the design requirements under different working conditions. In this process, it is very necessary to first detect whether the components have the above-mentioned offset.

[0032] Related technologies use color algorithms to extract the color of the component body, which involves image processing techniques such as color space conversion and threshold segmentation. Through this step, the algorithm distinguishes the component body from the background. After extracting the color of the component body, a circumscribed rectangle can be fitted to obtain the position of the component body, which can be achieved through edge detection, contour extraction and other technologies. The purpose of fitting the circumscribed rectangle is to find the minimum rectangular area that surrounds the component body. After obtaining the position of the component body, it is compared with the center point offset of the image area to be detected, which involves calculating the distance or offset between the two center points. By comparison, it can be determined whether the component has shifted in position, and the direction and degree of the shift can be determined.

[0033] However, this solution has some problems. First, the color extraction algorithm has a high operational complexity, which may cause edge burrs and color missing, thus affecting the accuracy of the fitted circumscribed rectangle. Especially for black IC boards, the extraction effect is usually poor, and the position information of the component body may not be accurately obtained. Secondly, this solution has high requirements for the positioning accuracy of the PCB board. If the positioning of the PCB board is inaccurate, such as position offset or rotation, it may cause false alarms or missed alarms, which cannot meet the requirements of actual use. The above problems may affect the accuracy and stability of the solution, resulting in unreliable offset detection results. Therefore, in practical applications, it may be necessary to consider improving the algorithm or adopting other more reliable detection methods to solve these problems.

[0034] Based on this, the embodiment of the present application provides a method and device for detecting the offset of components, by acquiring a target image of the component to be detected, and then performing positioning detection according to the target image and the template image, to obtain a reference point image and a pin image on the target image. Next, according to the reference point image and the pin image, it is detected whether the position offset between the reference point and the pin is within a preset range; if the position offset is within the preset range, the pin image is subjected to image line segment detection to obtain the angle of the pin in the component to be detected, and whether the component to be detected is offset according to the angle of the pin. The method and device for detecting the offset of components in the embodiment of the present application directly obtain the position information of the component through positioning detection of the reference point and the pin, without relying on the color extraction algorithm, which provides more accurate and stable position information, reduces the error caused by the difficulty of color processing, and thus improves the accuracy and stability of the offset detection. In addition, since the method does not rely on the color extraction algorithm, it can be applied to various types of components, including black IC boards with difficult color processing, etc., which expands the applicability of the method, so that it can be used for offset detection on more different types of components, and improves the versatility and practicality of the method. Moreover, the reference points are fixed during the design and manufacturing process, and their positions will not be affected by the placement of components. Therefore, using reference points and pins for offset calculation can provide more accurate results, especially when the shape of the component body changes or the size difference is large. Furthermore, determining the offset by detecting the angle of the pin provides more direct and accurate offset information compared to the method of fitting the circumscribed rectangle. Such angle information can more accurately reflect the actual offset of the component, further improving the stability and reliability of the detection results. Finally, this method performs positioning and offset detection based on the characteristics of the component itself, and does not rely on the accurate positioning or marking of the PCB board. Instead, it performs positioning and judgment by analyzing the reference points and pin characteristics of the component. This method is more flexible and can cope with changes in the design and manufacturing of different PCB boards, so it has better adaptability and flexibility and can cope with diverse component detection needs. Therefore, the component offset detection method and device of the embodiment of the present application have significant beneficial effects in terms of accuracy, stability, applicability and flexibility. It provides more accurate and stable position information, is applicable to various types of components, provides more direct and accurate offset judgment through angle information, and has the flexibility to adapt to different PCB board designs and manufacturing. These advantages make the method and device a reliable and efficient component offset detection solution.

[0035] For details, please refer to Figure 1 , Figure 1 1 is a flow chart of a component offset detection method provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0036] S11, obtaining a target image corresponding to the component to be inspected.

[0037] The components to be detected (such as integrated circuit chips) can be devices that integrate multiple electronic components on a small silicon chip, such as transistors, capacitors, resistors, etc. These electronic components are connected together by wires to form a complete circuit function. In this embodiment, the components to be detected are components with one or more groups of pins in a row, and these pins are arranged outside the components according to a certain layout. Such components with multiple pins in a row can be various electronic components, such as integrated circuit chips, connectors, sockets, etc.

[0038] The user can determine the components to be tested by checking product documents, logos or contacting suppliers.

[0039] The method for obtaining the target image corresponding to the component to be detected may include: obtaining the target image corresponding to the component from a website or relevant technical documents according to the identification information of the component to be detected. The target image refers to the image of the component, which may be a photo or schematic diagram of a chip.

[0040] In this embodiment, a method based on AOI (Automated Optical Inspection) can be used to obtain a target image of the component to be inspected. AOI is a method for automatic detection and inspection using an optical system and image processing technology, which can be used to detect and verify components and connections on a printed circuit board (PCB). Assuming that the component is an IC chip, during the manufacturing and testing process of the component, the AOI system can be used to detect features such as the pins, logos, and appearance of the chip. By using a high-resolution camera and image processing algorithm, the AOI system can capture the target image of the chip and analyze and process the image. The steps of obtaining the target image of the components to be inspected based on the AOI method may include: first, preparing the test device to ensure that the AOI system operates normally and has a light source, camera and image processing software suitable for chip inspection; then, positioning the chip, that is, placing the chip to be inspected in the appropriate position of the test device to ensure that the chip is accurately aligned and positioned; next, the AOI system uses a camera to capture the image of the chip to be inspected, wherein the camera may take multiple images at different angles and lighting conditions to obtain a comprehensive perspective and detailed information; the AOI system performs image processing and analysis on the captured chip image, which includes processing steps such as denoising, image enhancement, edge detection, feature extraction, etc. to extract the target features of the chip; finally, through image processing and analysis, the AOI system generates a target image of the chip to be inspected, which may include the appearance, pin layout, pin definition and other relevant features of the chip.

[0041] S12. Perform positioning detection according to the target image and the template image to obtain a reference point image and a pin image on the target image.

[0042] Among them, the corresponding template image is obtained according to the target image, and the template image is a known image used as a reference. In this embodiment, the template image includes a reference point template image and a pin template image, which are used to compare and match with the image to be detected. When performing image positioning detection, the role of the template image is to provide a known, standard image sample for comparison and matching with the image to be detected. By comparing the template image and the image to be detected, the accurate position and characteristics of the reference point and the pin can be determined, thereby realizing positioning detection and analysis.

[0043] Among them, a neural network model can be used to input a target image and a template image, and output a reference point image and a pin image. Specifically, a training data set containing a target image and a corresponding template image is prepared, and each sample in the training data set includes a pair of target images and template images, and the position and features of the reference point image and the pin image in each sample can be obtained. A neural network model is determined, such as a convolutional neural network (CNN) model or a model combining convolution and recurrent neural network (RNN), etc. Then, using the prepared training data set, the target image and the template image are used as inputs, and the reference point image and the pin image are used as target outputs to train the neural network model, wherein during the training process, the model parameters are optimized by minimizing the difference between the output image and the annotated image. Next, the performance of the model is evaluated using a validation set or a test set, and it is tuned as needed. Various indicators, such as mean square error or structural similarity index, can be used to evaluate the similarity between the output image and the annotated image, thereby obtaining a trained model. When applied, the trained model is used for inference, the target image and the template image are input into the trained model, and then the reference point image and the pin image are extracted from the output of the model. Among them, the neural network model is used to match and locate the target image and template image, which has the advantages of automatic feature learning, robustness, generalization ability, high-precision positioning and scalability.

[0044] Reference points are fixed position points used in the circuit design and electronic manufacturing process to determine and locate the positions of other components or assemblies. These reference points are generally predetermined on the base plate (circuit board, multilayer board, etc.), and they are not affected by the placement of components. Reference points are used to determine the relative position relationship of components during the circuit design stage to ensure the correct connection and function of the circuit. In the electronic manufacturing process, reference points are used to locate and align the correct position of components for precise welding or connection.

[0045] In this embodiment, the reference point is a relatively (pad layer) fixed point in the target image. Reference points include pads (soldering joints), vias between multilayer boards, silk screens on identification boards, etc. Pads can be used as reference points because their positions are relatively fixed and will not be affected by the placement of components. Vias between multilayer boards can also be used as reference points because they are usually fixed during the design and manufacturing process. The silk screen (printed mark) of the identification board can also be used as one of the reference points to provide a reference position for component placement. By using a known fixed reference point as a reference, the influence of possible changes in chip body shape or size differences in template matching can be eliminated, thereby improving the accuracy of positioning.

[0046] Reference point images are used to indicate the location and layout of reference points. Reference point images can include layout diagrams of pads, location diagrams of vias between multilayer boards, images of silk screens identifying boards, etc. These images can indicate the reference locations for component placement. The form of reference point images can vary depending on specific design and manufacturing requirements. They can be in the form of plan views, schematic diagrams, stack-up diagrams, etc., in order to clearly indicate the location and layout of reference points.

[0047] Among them, the layout diagram of the pad is used to indicate the layout and position of the soldering joints. The pad is a metal disc used to connect the pins of electronic components and the circuit board. They are electrically connected by welding. The layout diagram of the pad shows the position, shape and size of the pads, as well as the spacing and connection method between them. The location diagram of the vias between multilayer boards is used to indicate the position and layout of the vias between multilayer circuit boards. Multilayer circuit boards can be composed of multiple stacked circuit boards. Vias are used to connect circuits between different layers. The location diagram of vias shows the position, shape and size of the vias, as well as their relationship with other electronic components or circuits. The image of the silk screen of the identification board is used to indicate the text, symbols or graphics of the identification board. Silk screens are generally printed on circuit boards to identify important information such as the location of electronic components, pin direction, part number, etc. The image of the silk screen of the identification board shows the location, size and font of these logos.

[0048] Pin images are images used to locate and determine the location of chip pins. Pins are the metal leads of a chip that are used to connect to other components on a PCB. Pin images are used to detect and analyze the location, arrangement, shape, and other features of pins. By analyzing pin images, the exact location of the pins in the image can be determined for subsequent pin detection, pin connection verification, or other related tasks.

[0049] In the process of performing positioning detection based on the target image and the template image to obtain the reference point image and the pin image on the target image, the reference point image is used to determine the relative position relationship of the components to ensure the correct connection and function realization of the circuit. The target image can be compared and matched with the template image to locate the area in the target image that matches the template image, and the reference point image can be extracted from it, so that the position and layout of the reference point on the target image can be accurately determined. The pin image is used to represent the position and layout of the pins of the components. By matching the target image and the template image, the area in the target image that matches the template image can be found, and the pin image can be extracted from it, so that the position and layout of the pins in the target image can be determined.

[0050] By determining the position and layout of the reference points and pins, an accurate reference frame can be established in the target image for subsequent component offset detection. Compared with the traditional template matching scheme, the method for determining the reference point and pin position has higher flexibility, robustness and scalability. They can adapt to a wider range of changes and noise interference and provide more accurate and reliable positioning results. Among them, the traditional template matching scheme may require the use of exactly the same template image for matching. However, in practical applications, there may be some differences between the target image and the template image, such as scale changes, rotation, noise, etc. Through the method of the embodiment of the present application, these differences can be overcome to a certain extent, and more flexible positioning and matching can be achieved. The traditional template matching scheme is sensitive to factors such as illumination changes, noise interference and perspective changes, while the method of the embodiment of the present application can extract more robust features, so that a high matching accuracy can be maintained when there is a certain degree of change and interference. In addition, different components and target images may have different appearances and features. Traditional template matching is difficult to adapt to such changes, while the method of the embodiment of the present application can adapt to different components and target images.

[0051] In this embodiment, when the template image includes a reference point template image and a pin template image, the reference point image and the pin image can be obtained in the following two ways. The reference point template image and the pin template image are known images used as references, and are used to compare and match with the image to be detected to determine the accurate positions and features of the reference points and pins.

[0052] One method is to use a sliding window to locate and detect pads and pins. Figure 2 As shown, performing positioning detection according to the target image and the template image to obtain the reference point image and the pin image on the target image includes:

[0053] S101. Acquire a template image corresponding to the target image, determine the size of the sliding window according to the template image, and determine the sliding step length.

[0054] The size of the template image may be obtained first, and the size of the sliding window may be suitable for the size of the template image, thereby ensuring that the sliding window can completely contain the template image. Therefore, the size of the sliding window may be the same as or larger than the size of the template image.

[0055] The size and step length of the sliding window will affect the degree of overlap between windows. A smaller step length will result in more overlap between windows, while a larger step length will reduce overlap. In this embodiment, an appropriate degree of overlap, i.e., a suitable sliding step length, can be selected according to application requirements. In addition, the size and step length of the sliding window can also take into account the requirements of computing resources and processing speed, and an appropriate sliding window size and step length can be selected according to the specific situation to obtain accurate and efficient results in the sliding window positioning process.

[0056] S102: traverse the target image using the sliding window according to the sliding step size, and during the traversal process, obtain an image of a region of interest generated at each sliding position of the sliding window.

[0057] Among them, the region of interest (ROI) image refers to the area defined as interesting or important in the entire image, which is the area where further analysis, processing or feature extraction is desired. The ROI image can be a sub-image or image area extracted from the original image, which is defined by specifying its position and size in the original image. By extracting the ROI image, attention can be focused on a specific area, reducing the amount of data processed and the computational cost, and helping to improve the efficiency and accuracy of processing.

[0058] Among them, the sliding window can be moved on the image with a specified sliding step starting from the starting position of the target image. Each time the window is moved, the coordinates of the current sliding position are recorded. At each sliding position, the region of interest image is extracted from the target image according to the position and size of the current window. The region of interest image is the part covered by the sliding window, and this image will contain the target or specific area within the window. After obtaining the region of interest image, it can be further processed and analyzed, such as feature extraction, target detection, classification, etc. Repeat the above process until the sliding window traverses the entire target image and covers all possible positions. By using the sliding window to traverse the target image and obtain the region of interest image, images in multiple windows can be obtained at different positions, that is, multiple region of interest images are obtained.

[0059] S103: Calculate a similarity measure between each of the region of interest images and the template image.

[0060] The similarity measurement method of the image can be used to calculate the similarity measurement value between each region of interest image and the template image, such as Mean Squared Error (MSE) and Structural Similarity Index (SSIM). The specific calculation process can be referred to the records of the relevant technology.

[0061] The similarity metric is a value used to measure the similarity between the region of interest image and the template image. If the mean square error or structural similarity index method is used to calculate the similarity metric, a lower MSE value or a higher SSIM value indicates that the similarity between the two images is higher, while a higher MSE value or a lower SSIM value indicates that the similarity between the two images is lower. Other similarity measurement methods can also be selected, not limited to the above two methods.

[0062] S104: Compare the similarity measurement values, and find the region of interest image corresponding to the maximum similarity measurement value.

[0063] Among them, the corresponding value can be calculated and recorded for the similarity measure between each region of interest image and the template image. For each region of interest image, its similarity measure value is compared with the maximum similarity measure value recorded. If the similarity measure value of the current region of interest image is greater than the maximum similarity measure value recorded, the maximum similarity measure value is updated to the current value, and the corresponding region of interest image is recorded. Continue to compare the similarity measure value of the next region of interest image, and repeat the above process until all region of interest images are traversed. After the traversal is completed, the region of interest image corresponding to the maximum similarity measure value is obtained. Through this process, the region of interest image that is most similar to the template image can be found, and the image has the largest similarity measure value, which can help locate and identify the area in the target image that is most similar to the template image.

[0064] It should be noted that the above maximum similarity metric value represents the region of interest image that is most similar to the template image. For example, if the similarity metric value is calculated using the structural similarity index method, the larger the similarity metric value, the higher the similarity. If the mean square error method is used to calculate the similarity metric value, the smaller the similarity metric value, the higher the similarity. In this step, the minimum similarity metric value is obtained to determine the region of interest image that is most similar to the template image. Therefore, the region of interest image that is most similar to the template image can be obtained according to the corresponding similarity metric value calculation method.

[0065] S105: When the template image is a reference point template image, determine the region of interest image corresponding to the maximum similarity measure value as the reference point image.

[0066] S106: When the template image is a pin template image, determine that the region of interest image corresponding to the maximum similarity measure value is a pin image.

[0067] After obtaining the region of interest image that is most similar to the template image according to the above steps, if the template image is a reference point template image, determine the region of interest image corresponding to the maximum similarity metric value as the reference point image; if the template image is a pin template image, determine the region of interest image corresponding to the maximum similarity metric value as the pin image. The reference point image and the pin image are images used to locate the reference point and the pin in the target image, respectively. Among them, the reference point image includes the layout diagram of the pad, the location diagram of the vias between the multilayer boards, and the image of the silk screen of the identification board. The above method can also be used to obtain the pad image, the via image and the silk screen image in the target image, respectively, and then any one of the pad image, the via image and the silk screen image is used to perform offset detection with the pin image.

[0068] The above method of locating and detecting reference points and pins in the target image by sliding window is simple, intuitive and widely applicable. By adjusting the size and step size of the sliding window, the target can be searched at different scales to adapt to reference points and pins of different sizes or numbers. However, when the target image is large, a complete traversal of the entire target image may result in a long processing time. To solve this problem, a pyramid-based template matching method can be used. The pyramid-based template matching method is a multi-scale target detection method that can speed up the processing speed by matching on image pyramids of different scales.

[0069] Therefore, another method provided by the embodiment of the present application is to locate the reference points and pins on the target image using a pyramid-based template matching method. Figure 3 As shown, performing positioning detection according to the target image and the template image to obtain the reference point image and the pin image on the target image includes:

[0070] S201 : Acquire a template image corresponding to the target image, and create a first pyramid image corresponding to the target image and a second pyramid image corresponding to the template image.

[0071] The pyramid-based template matching method is a multi-scale target detection method used to locate and match targets on images of different scales. By performing template matching on images of different scales, the pyramid-based method can improve the robustness and accuracy of target detection.

[0072] First, the number of pyramid levels can be set, and then the target image and the template image are downsampled or upsampled multiple times according to the set number of pyramid levels to generate a series of images of different scales, wherein each pyramid level can be obtained by downsampling (or upsampling) the image of the previous level, so that a series of images with different resolutions can be obtained, and these images with different resolutions constitute a pyramid image.

[0073] Among them, for the target image, a series of images with gradually reduced resolutions can be obtained by multiple downsampling (i.e., downsampling), i.e., from the bottom to the top of the pyramid, to form the first pyramid image. Each pyramid level is obtained by downsampling the image of the previous level, which can be achieved by reducing the width and height of the image, so that a series of target images with different resolutions can be generated. Similarly, for the template image, multiple downsampling can be performed in the same way to obtain a series of template images of different scales corresponding to the target image pyramid, i.e., the second pyramid image. Among them, when performing the downsampling operation, jagged image edges often appear, which is caused by the loss of high-frequency information. In order to reduce the jagged effect, a smoothing filter can be applied after downsampling to smooth the image.

[0074] S202: Acquire a region of interest image at the highest level in the first pyramid image.

[0075] For the highest layer in the first pyramid image, the above sliding window method can be used to obtain the region of interest image corresponding to the layer image, as detailed in the above steps S101 and S102. Among them, in the highest layer of the first pyramid image, the image of the layer has the lowest resolution, that is, the size is relatively small, and in the process of obtaining the region of interest image of the highest layer, a higher processing speed and efficiency can be obtained.

[0076] S203: Perform similarity measurement on the region of interest image at the highest level and the template image at the highest level in the second pyramid image to obtain a maximum similarity measurement value at the highest level.

[0077] In the pyramid-based template matching method, the highest-level region of interest image and the highest-level template image in the second pyramid image can be measured for similarity to obtain the highest-level maximum similarity measurement value, which is similar to the above-mentioned process of calculating the similarity measurement value of each region of interest image and the template image. In the highest-level pyramid image, the region of interest image with the maximum similarity measurement value can be selected as the positioning result, and the position corresponding to the maximum similarity measurement value can be considered as the target positioning result of the highest level.

[0078] S204: Transfer the area corresponding to the maximum similarity measure value of the highest layer to a layer next to the highest layer in the first pyramid image to determine a local area of ​​the next layer.

[0079] That is, the target positioning result of the highest layer is transferred to the next layer corresponding to the highest layer of the pyramid to determine the local area of ​​the next layer. This is achieved by mapping the positioning result of the highest layer to the same position of the next layer. Usually, the image resolution of the next layer is higher. Therefore, when mapping the area of ​​the highest layer to the next layer, pixel-level matching can be performed through interpolation or upsampling, so that the local area of ​​the next layer can be determined for further target positioning and matching.

[0080] S205: Acquire a region of interest image corresponding to the local area.

[0081] The process is similar to the process of obtaining an image of a region of interest based on a sliding window. A suitable sliding window can be selected according to the local region.

[0082] S206 , performing similarity measurement on the image of the region of interest corresponding to the local region and the template image of a layer next to the highest layer in the second pyramid image to obtain a maximum similarity measurement value of the next layer.

[0083] The similarity measurement between the region of interest image and the template image is performed based on the same level of the first pyramid image and the second pyramid image. The specific process can refer to the above embodiment, thereby obtaining the target positioning result of the current level. The target positioning result can be passed to the next level to determine the local area of ​​the next level.

[0084] S207: Based on the first pyramid image, transfer the area corresponding to the maximum similarity metric value of the next layer to the next layer, perform local scan matching on the current layer, and repeat the process of obtaining the maximum similarity metric value of the current layer and the local scan matching until the lowest layer of the first pyramid image is reached, and use the area corresponding to the maximum similarity metric value obtained at the lowest layer as the target area.

[0085] By passing the positioning results to the next layer, target detection and matching can be performed at a higher resolution. This layer-by-layer passing process can be repeated until the lowest layer of the pyramid is reached. By passing and adjusting the positioning results at different levels, the accuracy and robustness of target detection can be improved.

[0086] S208: When the template image is a reference point template image, determine the image of the target area as a reference point image.

[0087] S209: When the template image is a pin template image, determine that the image of the target area is a pin image.

[0088] In this embodiment, the target can be located in a shorter time through the pyramid matching algorithm, because the algorithm will first match on the higher level pyramid image to quickly determine the rough location, and then gradually refine the match until the lowest level is reached. This can greatly reduce the number of pixels that need to be searched and matched, thereby speeding up the entire matching process.

[0089] In addition to the pyramid-based template matching method, other methods can also be used to obtain pad positioning images and pin images. For example, the feature extraction algorithm in computer vision can be used to extract the features of reference points and pins, and locate them by feature matching. The edge information of reference points and pins can also be extracted by edge detection algorithms, and then contour fitting algorithms (such as polygon fitting, minimum enclosing rectangle, etc.) can be used to estimate their position and shape.

[0090] S13. Detecting whether the position offset between the reference point and the pin is within a preset range according to the reference point image and the pin image.

[0091] like Figure 4 As shown, the detecting, based on the reference point image and the pin image, whether the position offset between the reference point and the pin is within a preset range includes:

[0092] S301. Obtain a reference point detection frame corresponding to the reference point image, and obtain a first center point corresponding to the reference point detection frame.

[0093] The reference point detection box is a rectangular box or a bounding box used to frame and identify the position and size of the reference point.

[0094] Among them, the reference point image can be preprocessed, including image denoising, adjusting brightness and contrast, and other operations; then, image processing algorithms, such as edge detection, corner detection, and other methods are used to extract features in the reference point image in order to detect and locate the reference points; different methods are used to locate the reference point detection frame based on the extracted features, for example, a contour-based method or a template matching-based method can be used to detect and locate reference points such as pads, vias, or silk screens; after obtaining the reference point detection frame, the first center point can be obtained by calculating the geometric features of the detection frame, such as the center point coordinates or the center of gravity coordinates of the frame.

[0095] For example, the reference point is a pad, and obtaining a reference point detection frame corresponding to the reference point image and obtaining a first center point corresponding to the detection frame include: preprocessing the pad image, such as denoising, contrast enhancement and other operations; using an image processing algorithm to detect the pad to locate the position and shape of the pad; screening and positioning based on the detected pad edge or feature point, wherein screening can be performed based on information such as the size, shape and position of the pad to exclude non-pad interference, and then geometric calculation or fitting algorithms, such as the least squares method or the minimum circumscribed rectangle algorithm, can be used to determine the position and bounding box of the pad; next, based on the position and bounding box of the pad, a reference point detection frame is constructed, and the reference point detection frame can be a bounding box or a rectangular box for identifying the position and size of the pad; obtaining the first center point from the reference point detection frame, the first center point can be obtained by calculating the center coordinates or the center of gravity coordinates of the rectangular box, for example, for a rectangular box, the coordinates of the upper left corner and the lower right corner of the rectangular box can be calculated, and then the coordinates of the center point between them can be calculated.

[0096] S302: Obtain a pin detection frame corresponding to the pin image, and obtain a second center point corresponding to the pin detection frame.

[0097] The pin detection box is a rectangular box or bounding box used to frame and identify the position and size of the pin.

[0098] Among them, obtaining the pin detection frame corresponding to the pin image and obtaining the second center point corresponding to the pin detection frame includes: preprocessing the pin image; using the image processing algorithm to detect the pin; screening and locating according to the detected pin edge or feature point to determine the position and boundary frame of the pin; constructing the pin detection frame according to the position and boundary frame of the pin; and obtaining the second center point from the pin detection frame. This process is similar to the above process of obtaining the reference point detection frame and the first center point.

[0099] S303: Calculate an offset between the first center point and the second center point, where the offset is a position offset between the reference point and the pin.

[0100] Among them, the coordinate offset of the two center points can be calculated by subtracting the coordinates of the first center point from the coordinates of the second center point to obtain the offset vector; after obtaining the offset vector, the modulus (length) of the vector can be used to represent the size of the position offset, or the numerical values ​​of the x-axis and y-axis components can be obtained separately to represent the position offset.

[0101] S304: Compare the offset with a preset range to determine whether the position offset between the reference point and the pin is within the preset range.

[0102] When the position offset is less than a preset threshold, it is determined that the positions of the reference point and the pin are within a preset range; otherwise, it is determined that the positions of the reference point and the pin are not within the preset range.

[0103] Among them, if the shapes of the reference point (such as the pad) and the pin have certain regularity and recognizability, the template matching method can be used to obtain their center points. According to the above embodiment, the pad template image and the pin template image can be obtained. In this embodiment, the pad template image is matched with the pad positioning image to find the nearest matching position, which is the first center point; similarly, the nearest matching position between the pin template image and the pin image is found, which is the second center point.

[0104] For reference points and pins that are not regular and identifiable, feature extraction algorithms can be used to extract feature points of reference points and pins respectively, and then feature matching algorithms can be used to match feature points in reference point images and pin images with corresponding template feature points to obtain the location of the center point. Deep learning models (such as convolutional neural networks) can also be used to detect and locate reference points and pins. By training a network model, the locations of reference points and pins can be directly predicted, and then the corresponding center points can be extracted. This process can also be used to obtain the above-mentioned reference point images and pin images.

[0105] The position offset can be used to measure the position difference between the reference point and the pin. The value of the position offset can be obtained by calculating the Euclidean distance between the two center points or other suitable distance measurement methods.

[0106] If the reference point and the pin are within the preset range, that is, the reference point and the pin are very close to the design requirements without obvious deviation or error, the following step S14 is performed. If the reference point and the pin are not within the preset range, it means that the reference point and the pin are incorrect, and the current detection can be terminated.

[0107] This embodiment can eliminate the influence of some errors or noise by determining whether the positions of the reference points and the pins are within a preset range, thereby improving the stability of the position offset results and making the offset detection more reliable; at the same time, the method has strong flexibility and is suitable for reference points and pins of various types and shapes, enhancing the adaptability and practicality of the method.

[0108] In addition to the IC body, there are usually other elements such as silk screen marks on the PCB board. Silk screen is a text, symbol or graphic printed on the PCB board to identify the location of components, the function of pins and other information. However, the presence of silk screen may affect the accuracy of IC offset detection. Therefore, the following steps select image line segment detection for the located pin area, so as to focus on the area of ​​IC pins to eliminate the interference of silk screen. By detecting the line segments in the image, the pins and silk screen can be better distinguished, and the attention can be focused on the analysis and processing of the pin area.

[0109] S14. If the position offset is within a preset range, image line segment detection is performed on the pin image to obtain the angle of the pin in the component to be detected, and whether the component to be detected is offset is determined based on the angle of the pin.

[0110] Among them, the edge detection algorithm can be used to detect the edge of the pin image to find the obvious edges and contours in the pin image; the line segment detection algorithm is applied to the edge detection result to obtain the straight line segment in the pin edge; for each detected straight line segment, its length and angle can be calculated, the length can be calculated by the starting point and end point coordinates of the straight line segment, and the angle can be calculated by the slope of the line or by the starting point and end point coordinates of the straight line segment. In the pin area, the angles of all line segments are counted, excluding the angles of zero and ninety degrees, and the angle with the highest frequency is identified, and then the angle with the highest frequency is used as the offset angle of the pin, which represents the offset degree of the pin relative to a reference direction (such as the horizontal direction of the image).

[0111] Optionally, the pin image is downsampled to obtain a downsampled image; wherein the pin image is downsampled to reduce the amount of calculation and improve detection efficiency. Detect line segments in the downsampled image; a line segment detection algorithm, such as Hough transform or Canny edge detection combined with straight line fitting and other methods, can be used to detect line segments in the downsampled image. Extract information of the line segment, the information includes the starting point, end point, angle and length of the line segment; for each detected line segment, information such as the starting point, end point, angle and length can be extracted, and this information can be obtained by calculating the difference in the endpoint coordinates of the line segment. According to the information, the main angle of the pin in the pin image is obtained, and the main angle is the angle of the pin in the component to be detected.

[0112] Among them, for each line segment in the extracted line segment information, its angle is calculated, and these angles are statistically analyzed. The number of times each angle appears can be recorded in the form of a histogram or frequency count. After the angle distribution is statistically analyzed, the angle with the most occurrences is found. This angle with the most occurrences can be considered as the main angle of the pin. Optionally, in order to improve accuracy, after the angle distribution is statistically analyzed, further screening can be performed based on some prior knowledge or rules. For example, line segments that obviously deviate from the main angle can be excluded, or angles of zero and ninety degrees that will not be offset can be excluded. Among them, the line segments in the extracted line segment information include the edges of the pin frame, the edges of the pin itself, and lines in other images, such as Figure 5d As shown, the lead frame is composed of a rectangle, and each side of the rectangle can be included in the extracted line segment information. Among them, the edge of the lead frame refers to the continuous line segments that constitute the shape of the lead frame, such as the four sides of the rectangle. The edge of the pin itself refers to the outer contour line segments of the pin, which are located inside the lead frame and represent the shape of the pin. The lines in other images may be lines that are unrelated to the lead frame or the pin itself, and may be formed by noise, interference or other structures in the image. These lines may be caused by light changes, limitations of image acquisition equipment, artifacts in image processing, etc. The lines in other images can be independent line segments or line segments that intersect or connect with the lead frame or the pin itself.

[0113] The angle of the pin in the component to be detected can be obtained through the above steps. Considering the diversity of pins in components, it is also possible to try to detect multiple main angles instead of just a single main angle based on the above steps, and finally determine the angle of the pin in the component to be detected based on multiple main angles. Specifically, the extracted line segment information is statistically analyzed according to the above steps to obtain the frequency of each angle; according to the requirements, a threshold is set to filter the first main angle with a frequency higher than the threshold, wherein the threshold can be adjusted according to the actual situation to balance the accuracy and diversity of the detection results; according to the first main angles filtered out, for each first main angle, the above steps can be used to perform line segment detection and angle extraction, thereby obtaining the pin line segment information corresponding to multiple first main angles; according to the pin line segment information corresponding to the multiple first main angles, multiple second main angles are calculated respectively; the average value of the multiple second main angles is calculated, or the weighted average value is calculated, and the calculation result is determined as the final main angle, which is the angle of the pin in the component to be detected.

[0114] In this embodiment, image line segment detection is performed based on the pin image and the main angles of the pin are obtained, which can provide pin angle information, increase the accuracy of pin positioning, and achieve the effect of quickly obtaining the pin angle. This information and angles are helpful for subsequent component offset detection tasks.

[0115] If the angle of the pin is greater than the threshold, it means that the component to be detected has shifted, and the corresponding shift result can be output, such as rotation information, including rotation direction and angle, etc. If the angle of the pin is less than or equal to the threshold, it means that the position and direction of the component to be detected are accurate, and the detection result that no shift has occurred can be output to the user.

[0116] The offset detection method of components provided in the embodiment of the present application first locates the reference point and pad of the component to be detected by template matching, determines whether the reference point and pad of the component to be detected are within a preset range, and if they are within the preset range, further obtains the pin angle of the component to be detected by edge detection to determine whether the component is offset. Among them, the template matching method can help accurately locate the reference point and pad position of the component to be detected, which helps to ensure that the subsequent edge detection process is performed on the correct area. Through the edge detection method, the pin angle information of the component to be detected can be extracted, which is very important for determining whether the component is offset. By comprehensively locating the pin and reference point and detecting the pin angle, it is possible to more accurately determine whether the component is offset. In addition, the method of this embodiment can also be applied to an automated component offset detection system. Through the combination of template matching and edge detection, rapid detection of large quantities of components can be achieved, thereby improving production efficiency.

[0117] In some embodiments, the reference point template image and the pin template image can be obtained in advance. The reference point template image can be an image sample of a reference point (such as a pad) with a clear shape and features. The pin template image can be an image containing one or more pins, with the shape, arrangement and features of the pins. Pins are metal leads of a chip that are used to connect to other components on a PCB. The pin template image can be used to determine the position, arrangement and other relevant features of the pins.

[0118] Acquiring the reference point template image and the pin template image includes: determining a reference point area and a pin area according to the original image, wherein the original image includes multiple reference points and multiple pins; searching for all reference points and all pins in the original image based on the reference point area and the pin area. Among them, all the reference points constitute a reference point template image, and the reference point template image includes the position information and grayscale information of the reference points; all the pins constitute a pin template image, and the pin template image includes the position information and grayscale information of the pins. Among them, the original image can be an image containing pads and pins at accurate positions. This embodiment realizes a precise and automated template acquisition process, improves the accuracy, automation and comprehensiveness of the template, and provides a reliable basis for the processing and analysis of component offset detection.

[0119] For example, assuming that the reference point is a pad and the component is an IC chip, the IC chip is offset detected based on the pad and pin. For details, please refer to Figure 5a to Figure 5d ,exist Figure 5a In the image, select a pad area (dashed box) and a pin area (dashed box) as templates. Then use the template matching algorithm to find areas similar to the template in the entire image to find all pads and pins. The search results are as follows: Figure 5b As shown. Based on the search results, we can determine Figure 5c The pad template image shown in Figure 5d The pin template image shown. Obtain the target image corresponding to the IC chip to be detected, perform positioning detection according to the target image and the pad template image and the pin template image respectively, and obtain the pad image and the pin image in the target image; then detect whether the positions of the pads and the pins are within the preset range according to the pad image and the pin image; if they are within the preset range, perform image line segment detection according to the pin image to obtain the angle of the pin in the IC chip to be detected, and then determine whether the IC chip to be detected is offset according to the angle of the pin. Among them, based on the pad template image and the pin template image, a sliding window method or a pyramid matching algorithm can be used to obtain the pad image and the pin image in the target image, and then calculate the position offset of the pad image and the pin image, and judge whether it is within the preset range according to the position offset. If it is within the preset range, perform image line segment detection on the pin image to obtain the angle of the pin, and judge whether the IC chip is offset according to the angle.

[0120] The method directly obtains the position information of the IC chip through the positioning detection of the pads and pins, without relying on the color extraction algorithm, which provides more accurate and stable position information and reduces the error caused by the difficulty of color processing. Since the method does not rely on the color extraction algorithm, it can be applied to various types of IC chips, including black IC boards and other situations where color processing is difficult, which expands the applicability of the method and enables it to perform offset detection on more different types of IC chips. The offset is determined by detecting the angle of the pin, which provides more direct and accurate offset information compared to the method of fitting the circumscribed rectangle. Such angle information can more accurately reflect the actual offset of the IC chip and improve the stability of the detection result. Finally, the method performs positioning and offset detection based on the characteristics of the IC chip itself. It does not rely on the accurate positioning or marking of the PCB board, but locates and judges by analyzing the characteristics of the pads and pins of the IC chip. This method is more flexible and can cope with changes in the design and manufacturing of different PCB boards.

[0121] See also Figure 6 , Figure 6is a schematic diagram of the structure of a component offset detection device provided in an embodiment of the present application, such as Figure 6 As shown, the component offset detection device 60 includes: a target image acquisition and target positioning module 61, a target detection module 62 and a offset detection module 63.

[0122] Among them, the target image acquisition and target positioning module 61 is used to acquire the target image corresponding to the component to be detected, perform positioning detection based on the target image and the template image, and obtain the reference point image and the pin image on the target image; the target detection module 62 is used to detect whether the position offset between the reference point and the pin is within a preset range based on the reference point image and the pin image; the offset detection module 63 is used to perform image line segment detection on the pin image if the position offset is within the preset range, obtain the angle of the pin in the component to be detected, and determine whether the component to be detected is offset based on the angle of the pin.

[0123] It should be noted that the above-mentioned component offset detection device can execute the component offset detection method provided in the embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in the embodiment of the component offset detection device, please refer to the component offset detection method provided in the embodiment of the present application.

[0124] Figure 7 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application, and the electronic device can be used to execute the above-mentioned component offset detection method. Figure 7 As shown, the electronic device 70 includes: one or more processors 71 and a memory 71, Figure 7 A processor 71 is taken as an example. The processor 71 and the memory 72 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0125] The memory 72 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the offset detection method of components in the embodiment of the present application. The processor 71 executes various functional applications and data processing of the electronic device by running the non-volatile software programs, instructions and modules stored in the memory 72, that is, the offset detection method of components in the above method embodiment is implemented.

[0126] The memory 72 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required by at least one function; the data storage area may store data created according to the use of the offset detection device of the component, etc. In addition, the memory 72 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 72 may optionally include a memory remotely arranged relative to the processor 71, and these remote memories may be connected to the offset detection device of the component via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0127] The one or more modules are stored in the memory 72, and when executed by the one or more processors 71, the component offset detection method in any of the above method embodiments is executed.

[0128] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0129] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: ultra-mobile personal computer devices, which belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. Such terminals include: PDA, MID and UMPC devices, etc.; mobile communication devices, such as smart phones, functional phones, etc.; servers, devices that provide computing services; and other electronic devices with data interaction functions.

[0130] The embodiment of the present application provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors, such as Figure 7 A processor 71 in the embodiment may enable the one or more processors to execute the component offset detection method in any of the above method embodiments.

[0131] An embodiment of the present application provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by the electronic device, the electronic device is enabled to execute the component offset detection method in any of the above method embodiments.

[0132] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] Through the description of the above implementation methods, ordinary technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Ordinary technicians in this field can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0134] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as described above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting the offset of a component, characterized in that: include: Acquire a target image corresponding to the component to be detected, perform positioning detection based on the target image and the template image, and obtain a reference point image and a pin image on the target image; According to the reference point image and the pin image, detecting whether the position offset between the reference point and the pin is within a preset range; If the position offset is within a preset range, image line segment detection is performed on the pin image to obtain the angle of the pin in the component to be detected, and based on the angle of the pin, it is determined whether the component to be detected is offset.

2. The method according to claim 1, characterized in that: The performing image line segment detection on the pin image to obtain the angle of the pin in the component to be detected includes: Performing line segment detection on the pin image to obtain straight line segments in the pin edge; Obtaining the angle corresponding to each of the straight line segments; Count the angles of all straight line segments; The angle at which the highest frequency occurs is identified as the angle of the pin, which angle does not include zero degrees and ninety degrees.

3. The method according to claim 1, characterized in that The performing positioning detection according to the target image and the template image to obtain the reference point image and the pin image on the target image includes: Obtaining a template image corresponding to the target image, determining the size of the sliding window according to the template image, and determining the sliding step length; According to the sliding step length, the target image is traversed using the sliding window, and during the traversal process, an image of a region of interest generated at each sliding position of the sliding window is obtained; Calculating a similarity measure between each of the region of interest images and the template image; Comparing the similarity metric values, and finding the region of interest image corresponding to the maximum similarity metric value; When the template image is a reference point template image, determining the region of interest image corresponding to the maximum similarity measure value as the reference point image; When the template image is a pin template image, it is determined that the region of interest image corresponding to the maximum similarity measure value is a pin image.

4. The method according to claim 1, characterized in that The performing positioning detection according to the target image and the template image to obtain the reference point image and the pin image on the target image includes: Obtaining a template image corresponding to the target image; Creating a first pyramid image corresponding to the target image and a second pyramid image corresponding to the template image; Performing similarity measurement on the first pyramid image and the second pyramid image to obtain a target area; When the template image is a reference point template image, determining the image of the target area as a reference point image; When the template image is a pin template image, the image of the target area is determined to be a pin image.

5. The method according to claim 4, characterized in that The performing similarity measurement on the first pyramid image and the second pyramid image to obtain the target area includes: Acquire an image of a region of interest at the highest level in the first pyramid image; Performing similarity measurement on the region of interest image at the highest level and the template image at the highest level in the second pyramid image to obtain a maximum similarity measurement value at the highest level; Transferring the area corresponding to the maximum similarity measure value of the highest layer to a layer below the highest layer in the first pyramid image to determine a local area of ​​the next layer; Acquire an image of a region of interest corresponding to the local area; Performing similarity measurement on the image of the region of interest corresponding to the local region and the template image of the next layer of the highest layer in the second pyramid image to obtain a maximum similarity measurement value of the next layer; Based on the first pyramid image, the area corresponding to the maximum similarity measure value of the next layer is transferred to the next layer, and local scan matching of the current layer is performed, and the process of obtaining the maximum similarity measure value of the current layer and local scan matching is repeated until the lowest layer of the first pyramid image is reached, and the area corresponding to the maximum similarity measure value obtained at the lowest layer is used as the target area.

6. The method according to claim 1, characterized in that The detecting, based on the reference point image and the pin image, whether the position offset between the reference point and the pin is within a preset range comprises: Obtain a reference point detection frame corresponding to the reference point image, and obtain a first center point corresponding to the reference point detection frame; Obtain a pin detection frame corresponding to the pin image, and obtain a second center point corresponding to the pin detection frame; Calculating an offset between the first center point and the second center point, the offset being a position offset between the reference point and the pin; The offset is compared with the preset range to determine whether the position offset between the reference point and the pin is within the preset range.

7. The method according to any one of claims 1 to 6, characterized in that: The template image includes a reference point template image and a pin template image, and the method further includes: acquiring the reference point template image and the pin template image; The acquiring of the reference point template image and the pin template image comprises: Determine a reference point area and a pin area according to an original image, wherein the original image includes a plurality of reference points and a plurality of pins; Searching for all reference points and all pins in the original image based on the reference point area and the pin area; Among them, all the reference points constitute a reference point template image, and the reference point template image includes the position information and grayscale information of the reference points; all the pins constitute a pin template image, and the pin template image includes the position information and grayscale information of the pins.

8. A device for detecting the displacement of a component, characterized in that: include: The target image acquisition and target positioning module is used to acquire the target image corresponding to the component to be detected, perform positioning detection based on the target image and the template image, and obtain the reference point image and the pin image on the target image; An object detection module, used to detect whether the position offset between the reference point and the pin is within a preset range according to the reference point image and the pin image; The offset detection module is used to perform image line segment detection on the pin image if the position offset is within a preset range, obtain the angle of the pin in the component to be detected, and determine whether the component to be detected is offset based on the angle of the pin.

9. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

10. A non-volatile computer-readable storage medium, characterized in that: The non-volatile computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by an electronic device, the electronic device executes the method according to any one of claims 1 to 7.

Images