Component offset detection method and device and electronic equipment

Abstract

The invention relates to the technical field of semiconductor manufacturing, and discloses a component offset detection method and device and electronic equipment. The method comprises the following steps: acquiring a target image containing a to-be-detected component; identifying the target image, determining a component and a bonding pad in the target image, and obtaining a rotation angle corresponding to the component; screening out a plurality of edge bonding pads corresponding to the to-be-detected component from the target image, and obtaining a minimum enclosing rectangle of the edge bonding pads; calculating the offset of the central point of the component to be detected relative to the central point of the minimum enclosing rectangle of the edge bonding pad in the horizontal direction and the vertical direction; and determining offset information of the to-be-detected component according to the offset and the rotation angle corresponding to the to-be-detected component. According to the automatic component offset detection method, the offset condition between the component and the bonding pad can be rapidly and accurately detected, and due to the fact that a template does not need to be added, the method has good flexibility and applicability and can adapt to components of different types and sizes.

Description

Technical Field

[0001] The present application relates to the field of semiconductor manufacturing technology, and in particular to a component offset detection method, device and electronic equipment. Background Art

[0002] Assembling printed circuit boards (PCBs) usually consists of components and circuit boards. Components refer to various electronic components used to build circuits, such as resistors, capacitors, diodes, transistors, integrated circuits, etc. These electronic components have different functions, such as controlling current, storing data, amplifying signals, etc. Components are fixed to the designated positions of the circuit board by welding or other connection methods. Once the components are installed, the PCB can be connected to other electronic devices, such as computers, mobile phones, TVs, etc., to complete specific functions or tasks.

[0003] The location and positioning of components on a PCB is critical because circuit design relies on the correct connection and relative position between components. If components are offset, it may cause short circuits or poor contact between components. Therefore, detecting component offset is important to ensure proper circuit function, avoid short circuits and contact problems, ensure manufacturing quality, and improve production efficiency.

[0004] The related technology uses a template matching solution to detect the offset of components, which mainly includes: creating a reference template, image acquisition, extracting the features of the components from the acquired image, matching the extracted features with the reference template, and judging whether the components are offset based on the matching results.

[0005] In the process of implementing the embodiments of the present application, the inventors found that the related technology has at least the following technical problems: the template matching method needs to add a template image for each component and perform programming, which takes time and effort and is inefficient. In addition, if the components of the plate or PCB change, such as changes in shape, size or layout, it is necessary to add a new template image, which increases the cost of maintenance and updates, and may cause the detection results to be inaccurate for frequently changing production lines. Summary of the invention

[0006] The main technical problem solved by the implementation methods of the present application is how to improve the efficiency and accuracy of component offset detection.

[0007] In order to solve the above technical problems, a technical solution adopted in the implementation mode of the present application is: providing a component offset detection method, including: obtaining a target image containing components to be detected; identifying the target image, determining the components and pads in the target image, and obtaining the rotation angle corresponding to the components; screening out a number of edge pads corresponding to the components to be detected from the target image, and obtaining the minimum circumscribed rectangle covering the edge pads; calculating the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the minimum circumscribed rectangle of the edge pad; determining the offset information of the component to be detected according to the offset and the rotation angle corresponding to the component to be detected, and the offset information includes angle offset and position offset. Among them, by automatically identifying the components and pads in the image and extracting the rotation angle of the components, manual intervention can be reduced and the accuracy of identification can be improved; by calculating the offset between the center point of the component to be detected and the center point of the minimum circumscribed rectangle of the pad, the offset of the component to be detected can be quantified and a quantitative offset measurement can be provided; combining the offset and the rotation angle of the component to be detected, the offset information of the component to be detected is obtained, so that it can be determined whether the component to be detected is offset based on the offset information, so that offset detection can be performed automatically and offset problems can be found quickly and accurately. Overall, this method comprehensively utilizes image processing, target feature analysis, and displacement calculation to improve the efficiency and accuracy of component offset detection.

[0008] Optionally, the target image is identified, the components and pads in the target image are determined, and the rotation angle corresponding to the components is obtained, including: preprocessing the target image obtained; inputting the preprocessed target image into a preset recognition model, performing target detection and classification, and outputting a category label for each detected target, wherein the category label is used to indicate the type of component or pad; and outputting the rotation angle corresponding to the component, wherein the rotation angle is represented by the rotation angle of a rectangular box; the loss function of the recognition model is calculated based on the projection difference of the Gaussian distribution. Among them, by identifying the target image and obtaining the rotation angle and category label of the component, automated detection and classification can be achieved, thereby improving the efficiency of offset detection; in addition, the target object is located and identified, and rotation angle information is provided, which provides a basis for data analysis and quality control.

[0009] Optionally, the method further includes: detecting whether the rotation angle corresponding to the component is within a preset angle range; if the rotation angle corresponding to the component is within the preset angle range, then expressing the currently obtained rotation angle as the rotation angle of the component; if the rotation angle corresponding to the component is not within the preset angle range, then rotating the horizontal axis in the coordinate system corresponding to the rectangular frame of the component counterclockwise, and recording the rotation angle when the horizontal axis rotates counterclockwise until the rectangular frame of the component is first encountered to obtain a new rotation angle, and the new rotation angle is used to express the rotation angle of the component. Among them, by setting the preset angle range and the corresponding processing steps, the accuracy and robustness of target detection can be improved, while simplifying the representation and processing process of the rotation angle, so that the algorithm is more adaptable to application scenarios with different rotation angle ranges.

[0010] Optionally, the step of selecting a number of edge pads corresponding to the components to be detected from the target image and obtaining the minimum circumscribed rectangle covering the edge pads includes: calculating the distance between the center point of each component and the center point of the target image according to the components determined in the target image; determining the component corresponding to the minimum distance as the component to be detected; determining all pads corresponding to the components to be detected according to the position information of the components to be detected, and obtaining the minimum circumscribed rectangle of all pads. By determining the components to be detected and then determining all pads from the determined components to be detected, the accuracy of identification and positioning can be improved, the ability to understand and analyze components in the image can be enhanced, and more comprehensive detection and quality assessment can be provided.

[0011] Optionally, the method of determining all pads corresponding to the component to be detected according to the position information of the component to be detected includes: when the number of pads corresponding to the component to be detected is two or three, determining pad one that is closest to the component to be detected, and determining the position information of other pads according to the position information of pad one and the position information of the component to be detected; when the number of pads corresponding to the component to be detected is greater than three, determining all pads corresponding to the component to be detected based on the preset number of pads, the topological structure between the pads, and the position information of the component to be detected. Wherein, given the position information of the component to be detected, the position information of the pad is determined according to specific conditions, so that the positions of all pads corresponding to the component to be detected can be more accurately inferred and determined, thereby improving the accuracy and reliability of pad detection; and, the pad positions are inferred according to the number and topological structure of the pads of the component to be detected, as well as preset parameters, which makes the method flexible and can adapt to the detection requirements of pads of different numbers and arrangements.

[0012] Optionally, the method of determining all pads corresponding to the component to be detected based on a preset number of pads, a topological structure between pads and the position information of the component to be detected includes: if the preset number of pads is equal to the number of pads identified in the target image, the pads are all pads corresponding to the component to be detected; if the preset number of pads is less than the number of pads identified from the target image, determining pad one closest to the component to be detected, and determining the topological structure of pad one and candidate pads according to the position information of pad one and the preset number of pads; obtaining the distance between the center of the topological structure of pad one and the candidate pad and the center of the component to be detected, and determining all pads corresponding to the component to be detected according to the distance. Among them, the pads corresponding to the components to be detected are determined by step-by-step inference, taking into account the topological structure and positional relationship between the pads, thereby improving the accuracy of the pad position; when the preset number of pads is less than the number of pads identified in the target image, it can adapt to the situation of different numbers of pads, and by calculating the distance between the topological centers and judging according to the set threshold, all the pads corresponding to the components to be detected can be flexibly determined.

[0013] Optionally, the recognition model also outputs the rotation angle corresponding to each pad. After screening out a number of edge pads corresponding to the component to be detected from the target image and obtaining the minimum circumscribed rectangle covering the edge pads, the method further includes: calculating the angular deviation between the rectangular frame of the pad determined from the target image and the minimum circumscribed rectangle; when the angular deviation is greater than a preset threshold, calculating the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the preset frame, and determining the offset information of the component to be detected based on the offset and the rotation angle; when the angular deviation is less than or equal to the preset threshold, executing the steps of calculating the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the minimum circumscribed rectangle of the edge pad, and determining the offset information of the component to be detected based on the offset and the rotation angle corresponding to the component to be detected. Among them, when the pad identified according to the target image is not the pad of the patch to be tested, that is, the pad is not fully recognized by the model, the pad information is not used for offset calculation. Instead, the preset picture frame is used for offset calculation, so as to obtain accurate offset information of the component.

[0014] Optionally, before the step of calculating the horizontal and vertical offsets of the center point of the component relative to the center point of the minimum circumscribed rectangle of the pad, the method further includes: detecting whether the component to be detected is a collision part; if the component to be detected is not a collision part, performing the step of calculating the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the minimum circumscribed rectangle of the edge pad. A collision part refers to a situation where a component has been successfully installed on a pad during normal soldering, but later the component is lost or damaged due to an accidental impact or collision. By detecting collision parts, damage can be discovered early, product quality and reliability can be improved, and the workload of component offset detection can be reduced.

[0015] Optionally, the detection of whether the component to be detected is a collision part includes: obtaining a first color of the board background corresponding to the component to be detected; obtaining a second color within a preset range of the center point of the component to be detected; calculating a similarity metric value between the first color and the second color; when the similarity metric value is greater than a preset threshold, determining that the component to be detected is a collision part, otherwise, determining that the component to be detected is not a collision part. By comparing the similarity between the color around the component and the board background color, it is possible to assist in determining whether the component is a collision part, thereby providing additional information to assist the model in making the judgment, thereby improving the accuracy of the detection.

[0016] In order to solve the above technical problems, another technical solution adopted in the implementation mode of the present application is: to provide a component offset detection device, including: an image acquisition module, used to acquire a target image containing components to be detected; a component and pad detection module, used to identify the target image, determine the components and pads in the target image, and obtain the rotation angle corresponding to the components; a pad detection module, used to screen out a number of edge pads corresponding to the components to be detected from the target image, and obtain the minimum circumscribed rectangle covering the edge pads; an offset calculation module, used to calculate the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the minimum circumscribed rectangle of the edge pad; an offset detection module, used to determine the offset information of the component to be detected according to the offset and the rotation angle corresponding to the component to be detected, and the offset information includes angle offset and position offset. Among them, by automatically identifying the components and pads in the image and extracting their rotation angles, manual intervention can be reduced and the accuracy of identification can be improved; by calculating the offset between the center point of the component to be detected and the center point of the minimum circumscribed rectangle of the pad, the offset of the component to be detected can be quantified and a quantitative offset measurement can be provided; combining the offset and the rotation angle of the component to be detected, the offset information of the component to be detected is obtained, so that it can be determined whether the component to be detected is offset based on the offset information, so that offset detection can be performed automatically and offset problems can be found quickly and accurately. Overall, the device comprehensively utilizes image processing, target feature analysis, and displacement calculation to improve the efficiency and accuracy of component offset detection.

[0017] To solve the above technical problems, another technical solution adopted by the embodiment of the present application is: to provide an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above method. The electronic device has the same beneficial effects as the above component offset detection method.

[0018] In order to solve the above technical problems, another technical solution adopted by the embodiment of the present application is: providing a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by an electronic device, the electronic device executes the above method. The non-volatile computer-readable storage medium has the same beneficial effects as the above component offset detection method.

[0019] In order to solve the above technical problems, another technical solution adopted in the embodiment of the present application is: providing a computer program product, the computer program product includes a computer program stored on a non-volatile computer-readable storage medium, the computer program includes program instructions, and when the program instructions are executed by an electronic device, the electronic device executes the above method. The computer program product has the same beneficial effects as the above component offset detection method. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] One or more embodiments are exemplarily described by corresponding drawings, which do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and the figures in the drawings do not constitute proportional limitations unless otherwise stated.

[0021] Figure 1 is a flow chart of a component offset detection method provided in an embodiment of the present application;

[0022] Figure 2 It is a flow chart of a method for determining components and pads in a target image and obtaining a rotation angle corresponding to the components provided in an embodiment of the present application;

[0023] Figure 3 is a flowchart of a method for determining components and pads in a target image and obtaining a rotation angle corresponding to the components, provided by another embodiment of the present application;

[0024] Figure 4 is a flow chart of a method for detecting collision parts provided in an embodiment of the present application;

[0025] Figure 5 is a schematic diagram of a target image provided in an embodiment of the present application;

[0026] Figure 6 It is a structural schematic diagram of a component deviation detection device provided in an embodiment of the present application;

[0027] Figure 7 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. It should be noted that, if there is no conflict, the various features in the embodiments of the present application can be combined with each other, all within the scope of protection of the present application. In addition, although the functional modules are divided in the device schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described can be performed in a sequence different from the module division in the device schematic diagram or the flow chart. Unless otherwise defined, all technical and scientific terms used in this specification have the same meaning as those generally understood by technicians in the technical field of the present application. The terms used in the specification of the present application are only for the purpose of describing specific implementation methods and are not used to limit the present application.

[0029] As you can know, components refer to various electronic components used to build circuits, such as resistors, capacitors, diodes, transistors, integrated circuits, etc. Component offset refers to the error or movement of components relative to their intended position during the assembly process. This offset can be caused by a variety of factors, including manufacturing errors, positioning problems during assembly, temperature changes, etc. Component offset may have a negative impact on the function and performance of the circuit. For example, on a circuit board, if the position of components deviates from the designed position, they may not be properly connected to other components or circuit paths, causing the circuit to not work or function malfunction. In addition, offset may also cause short circuits or other unexpected circuit connections between components on the circuit board.

[0030] In order to reduce the impact of component offset, related technologies use template matching to detect component offset. First, a template image is created for each component, which is a visual representation of the component at a predetermined position. During the assembly process, an image acquisition device (such as a camera, etc.) is used to obtain the actual assembled image, and then the actual image is compared with the pre-created template image through a template matching algorithm. Next, through template matching, the components in the actual image can be found and their positions can be determined. Then, by calculating the difference between the center point of the component after offset and the center point of the component at the predetermined position, it is determined whether an offset has occurred.

[0031] However, the template matching method requires the creation of a template image for each component, which requires a certain amount of work and time, especially for complex circuit boards or a large number of different types of components. Multiple template images need to be created and maintained, which may lead to difficulties in programming and maintenance. The template matching method needs to add a new template image when there is a partial change in the board type or PCB material. This means that for circuit board designs with large changes or changes on the production line, the template matching method may not be flexible enough and requires additional work to update the template image. In addition, the template matching method cannot effectively detect the angular offset or rotation of components; and the accuracy of the template matching method is limited by the overall accuracy of the machine equipment. If the mechanical error is large, it may seriously affect the accuracy of the offset detection. In order to ensure accuracy, high-precision machine equipment needs to be used and calibrated and maintained.

[0032] Based on the above, the inventors consider providing a more flexible and accurate component offset detection method, which can reduce manual intervention and improve the accuracy of identification by automatically identifying components and pads in the image and extracting their position information; through collision detection, it can be determined whether there is a collision in the component, and the collision is removed from the component set, which can remove interference elements and improve the accuracy of subsequent analysis; by calculating the offset between the center point of the component and the center point of the pad, the offset of the component can be quantified and a quantitative offset measurement can be provided; combining the offset and the position information of the component, it can be determined whether the component is offset, so that offset detection can be performed automatically and offset problems can be found quickly and accurately. The component offset detection method provided in the embodiment of the present application can improve the efficiency and accuracy of component offset detection by comprehensively utilizing image processing, target feature analysis, collision detection, and displacement calculation.

[0033] Specifically, see Figure 1 , Figure 1 1 is a flow chart of a component offset detection method provided in an embodiment of the present application. Figure 1 As shown, the method includes:

[0034] S11, obtaining a target image containing components to be inspected.

[0035] Components to be tested refer to components that need to be tested for offset, including various electronic components on a circuit board or PCB. Components are connected to pads on the circuit board by welding. Pads are metal contacts used to connect components to circuit boards, which provide electrical and mechanical connections between electronic components and circuit boards. The pins or pads of components are connected to each other through welding processes, such as hot melt welding or surface mount technology.

[0036] A camera or other image acquisition device can be used to obtain a target image containing components to be detected. In this embodiment, a method based on AOI (Automated Optical Inspection) can be used to obtain a target image containing components to be detected. AOI is a method for automatic detection and inspection using an optical system and image processing technology, which can be used to detect and verify components and connections on a printed circuit board (PCB). The steps of obtaining a target image containing components to be detected according to AOI may include: placing the circuit board to be detected on the workbench of the AOI system and ensuring that the position of the circuit board is correct; setting the scanning parameters of the AOI system according to the components to be detected and application requirements, including resolution, exposure time, image processing options, etc.; starting the AOI system and letting it scan automatically, and the high-resolution camera or image acquisition device in the system can capture the image on the circuit board; the AOI system can also perform real-time image processing on the captured image, including denoising, contrast enhancement, brightness adjustment and color balance, etc., so as to help optimize image quality and improve the accuracy of subsequent component detection; based on the results of image processing, the AOI system uses image processing algorithms and techniques to detect and locate the components to be detected; according to the results of component detection, the AOI system extracts a target image containing components to be detected.

[0037] The target image may be in the form of a picture. The target image may be a cropped circuit board area, or a partial image containing only the components to be detected. The target image may be saved for subsequent component offset detection, analysis and processing.

[0038] S12: Identify the target image, determine the components and pads in the target image, and obtain the rotation angles corresponding to the components.

[0039] In this embodiment, a preset recognition model is used to recognize the target image to obtain components and pads in the target image, where the components refer to all components contained in the target image, including components to be detected. Components to be detected refer to specific components that need to be detected or processed, which can specifically be components that are fully displayed in the target image.

[0040] Deep learning methods can be used to identify components and pads in the target image. For example, collect image datasets containing components and pads and annotate them. The annotations can use bounding boxes or pixel-level annotations to mark the locations of components and pads. Then preprocess the collected image data, including image scaling, normalization, denoising, etc. Preprocessing helps improve the stability and accuracy of the model. Next, select a deep learning model architecture suitable for the task, such as a convolutional neural network (CNN), an object detection model (such as Faster R-CNN, YOLO, SSD, etc.) or a semantic segmentation model (such as U-Net, Mask R-CNN, etc.). The dataset is then divided into a training set and a validation set for model training and evaluation. The training set is used to train the deep learning model, and the accuracy of the model is improved by iteratively optimizing the model parameters; the validation set is used to evaluate the trained model, and the model's precision, recall and other indicators are calculated; based on the evaluation results, the model architecture, hyperparameters or data enhancement methods can be adjusted to further improve the model performance. Finally, the trained model is used to detect or segment components and pads in the target image, where the model can output the location information of components and pads respectively, which can be bounding box coordinates or pixel-level masks.

[0041] In addition to using deep learning-based methods to identify components and pads in the target image, other methods can also be used for identification, such as using feature extraction algorithms in the field of computer vision (such as SIFT, SURF, HOG, etc.) to extract features in the target image, and then using traditional machine learning algorithms (such as support vector machines, random forests, K nearest neighbors, etc.) for classification or positioning.

[0042] The position information includes the rotation angles of the components and pads, and the coordinates, width and height of the circumscribed rectangular frames of the components and pads. The rotation angles can be expressed as the rotation angles of the circumscribed rectangular frames of the components and pads.

[0043] The method of this embodiment can predict the rotation angle of components and pads. For square components and pads, since the width and height are equal or similar at this time, the loss function calculated in the process of using the model will be very small, resulting in the inability to correctly predict the rotation angle of the components or pads.

[0044] It is understandable that the loss function used to regress the rotation angle can be a difference-based metric, such as mean square error or mean absolute error. When the width and height are equal or similar, the component or pad presents a square shape, which means that the rotation angle can be multiple possibilities such as 0 degrees, 90 degrees, 180 degrees or 270 degrees, but the values ​​of width and height are equal or similar, which causes the model to have difficulties in regressing the rotation angle. Since the calculation of the loss function is usually based on the difference between the predicted value and the true value, when the width and height are equal or similar, the model may predict a rotation angle so that the predicted bounding rectangle is very close to the true bounding rectangle, resulting in a very small loss function. However, this predicted rotation angle is not necessarily the true rotation angle, because a square component or pad can have multiple reasonable rotation angles. Therefore, due to the small loss function, the model cannot accurately distinguish the difference between different rotation angles, resulting in the inability to correctly predict the rotation angle of the square component or pad.

[0045] In this embodiment, the loss function of the recognition model is calculated based on the projection difference of the Gaussian distribution. For example, the loss function can be re-derived by adjusting the conversion method of the two-dimensional Gaussian distribution to solve the above problem. For example, in the training stage, for square components and pads, the center point coordinates (x, y), width (w) and height (h) are normally predicted; half of the height (h) (h / 2) is converted to the width (w) plus half of the height (w+h / 2), so that the converted value is not equal to the width (w), thereby changing the projection shape of the two-dimensional Gaussian distribution; the converted width and height are used to calculate the two-dimensional Gaussian distribution, wherein the two-dimensional Gaussian distribution is used to model the uncertainty of the position and size of the target, and the accuracy of the predicted box is evaluated by calculating the projection of the Gaussian distribution; finally, the loss function is calculated based on the predicted center point coordinates and the converted width and height of the two-dimensional Gaussian distribution, wherein when calculating the loss function, the difference between the predicted box and the real box, as well as the difference in the projection of the two-dimensional Gaussian distribution and other indicators can be used to measure the accuracy of the prediction. Through the above method, the transformation method of the two-dimensional Gaussian distribution is adjusted so that the projection of square components and pads no longer presents a circular shape. This can better handle the angle prediction problem of square targets and improve the performance of the model on square components and pads, enabling it to predict the rotation angle more accurately.

[0046] In this example, see Figure 2 , the above step S12 specifically includes:

[0047] S121, preprocessing the obtained target image.

[0048] Preprocessing the target image includes: scaling or cropping the target image to fit a specific resolution or region of interest; using filters or denoising algorithms to remove noise from the target image to improve the clarity and quality of the target image; using edge detection algorithms to detect edge structures in the image to extract edge information of target components and pads; etc. Preprocessing operations can improve the quality of the target image, reduce noise, enhance image features, etc., for subsequent component and pad identification.

[0049] S122. Input the preprocessed target image into a preset recognition model to perform target detection and classification, output the category label of each detected target, and the category label is used to indicate the type of component or pad; and output the rotation angle corresponding to the component, and the rotation angle is represented by the rotation angle of the rectangular box.

[0050] The recognition model may also output a rotation angle corresponding to the pad, and the rotation angle may also be represented by a rotation angle of a rectangular frame.

[0051] The preset recognition model may be a deep learning model for target detection and classification, such as Faster R-CNN (Region-based Convolutional Neural Networks), SSD (Single Shot MultiBox Detector), etc. The loss function of the recognition model is calculated based on the projection difference of Gaussian distribution.

[0052] In the recognition model, the target detection algorithm is used to detect components and pads in the preprocessed target image. The target detection algorithm locates and marks the position information of components and pads in the image, and the position information can be represented by rectangular boxes enclosed by components and pads respectively. The enclosed rectangular box is a rectangle that surrounds components or pads and represents their position and range. Optionally, the target detection algorithm can also output a confidence score for each detected target. For each detected target, the recognition model will classify it and determine its category label, that is, the type of component or pad. Finally, the position information (coordinates, width, height and rotation angle of the rectangular box) and category label are obtained based on the target detection.

[0053] The coordinates of the rectangular box may specifically be the coordinates of the upper left corner of the rectangular box, including the coordinate values ​​of the horizontal axis (x-axis) and the vertical axis (y-axis), and the position of the rectangular box may be determined according to the coordinate values. The width and height of the rectangular box are used to indicate the size of the target area selected by the rectangular box. The rotation angle is used to indicate the rotation angle of the rectangular box. The category is used to indicate whether the rectangular box is a component or a pad.

[0054] In this embodiment, by identifying the target image and obtaining the position information and category labels of components and pads, automatic detection and classification can be achieved, thereby improving the efficiency of offset detection; in addition, the target object is located and identified, and the rotation angle information is provided, which provides a basis for data analysis and quality control.

[0055] Optionally, see Figure 3 , the above step S12 specifically also includes:

[0056] S123, detecting whether the rotation angle corresponding to the component is within a preset angle range.

[0057] Among them, the preset angle range can be (-45°, 45°], because the rotation or offset of the components is usually not very large, limiting the preset angle range to meet the current application scenario. The selection of the preset angle range can be adjusted according to the specific application scenario and data distribution. If there is a larger rotation angle in a specific application, the preset angle range can be expanded according to the actual situation. In actual applications, based on statistical analysis and experimental verification of the data, selecting a suitable preset angle range can make the model more accurate and efficient in target detection.

[0058] If the rotation angle is within the preset angle range, the following step S124 is executed.

[0059] S124: Express the currently obtained rotation angle as the rotation angle of the component.

[0060] If the rotation angle is within the preset angle range, the side of the rectangular frame that the x-axis first encounters when rotating clockwise is used as the width, and the other side of the rectangular frame is used as the height. In this process, the rotation angle of the x-axis is the currently obtained rotation angle.

[0061] If the rotation angle is not within the preset angle range, the following step S125 is executed.

[0062] S125. Rotate the horizontal axis in the coordinate system corresponding to the rectangular frame of the component counterclockwise. During the rotation, regard the width of the original rectangular frame as the new height, regard the height of the original rectangular frame as the new width, and record the rotation angle of the horizontal axis when it rotates counterclockwise until it first encounters the edge of the rectangular frame of the component to obtain a new rotation angle, which is used to represent the rotation angle of the component.

[0063] Among them, the height and width of the original rectangular frame can be the height and width of the rectangular frame determined when the rotation angle is within the preset angle range. When the rotation angle is not within the preset angle range, the rectangular frame may be rotated to a certain extent. In order to adjust it to a position parallel to the x-axis, the coordinate system of the rectangular frame is rotated counterclockwise until the edge of the rectangular frame is encountered for the first time, so that the width of the original rectangular frame is regarded as the new height, and the height of the original rectangular frame is regarded as the new width, so as to obtain new position information. During the rotation process, the rotation angle is recorded, and this rotation angle is the currently obtained rotation angle. By rotating the x-axis in the coordinate system corresponding to the rectangular frame counterclockwise, the direction of the rectangular frame can be adjusted to correspond to the new width and height, thereby obtaining the correct size and angle of the rotated rectangular frame to accurately represent the position information of the target.

[0064] In this embodiment, the position information of the target object is determined according to the rotation angle and the preset angle range. If the rotation angle is within the preset range, the width, height and rotation angle of the original rectangular box are directly used. If the rotation angle is not within the preset range, the width and height are adjusted by rotating the rectangular box in the coordinate system, and the rotation angle is recorded to obtain new position information. This embodiment can improve the accuracy and robustness of target detection, while simplifying the representation and processing of position information, making the algorithm more adaptable to application scenarios with different rotation angle ranges.

[0065] S13, selecting a plurality of edge pads corresponding to the components to be detected from the target image, and obtaining a minimum circumscribed rectangle covering the edge pads.

[0066] A component usually includes multiple pads. Edge pads are pads located at the edge or bottom of the component and are used to connect with solder joints on the circuit board. For each edge pad selected, its minimum enclosing rectangle is calculated. The minimum enclosing rectangle is the smallest rectangle that surrounds all edge pads. It can be obtained by calculating the bounding box of the pad or by a preset algorithm.

[0067] It is understandable that after the components and pads in the image are identified according to the preset recognition model, some components may be incomplete in the target image, for example Figure 5 As shown. In this case, it is possible to exclude or mark these incomplete components and pads as untrustworthy areas, which can avoid misleading or errors in the incomplete or unreliable component data during subsequent analysis and processing. Therefore, the components to be inspected can be further determined, and then the offset of the components to be inspected can be determined, thereby improving the accuracy of identification and positioning, enhancing the understanding and analysis capabilities of components in the image, and providing more comprehensive inspection and quality assessment.

[0068] Specifically, a plurality of edge pads corresponding to the components to be detected are screened out from the target image to obtain a minimum circumscribed rectangle covering the edge pads, including: calculating the distance between the center point of each component and the center point of the target image according to the components determined in the target image; determining the component corresponding to the minimum distance as the component to be detected; determining all pads corresponding to the components to be detected according to the position information of the components to be detected, and obtaining the minimum circumscribed rectangle of all the pads.

[0069] There may be multiple components corresponding to the minimum distance. Since the distances between multiple components in the target image and the center point of the target image are relatively close, or their distances are almost equal, in this case, one or more components corresponding to the minimum distance can be selected as the components to be detected according to specific needs and application scenarios. The distance between the center point of the component and the center point of the target image can be measured using the Euclidean distance or the Manhattan distance.

[0070] Wherein, determining all the pads corresponding to the component to be detected according to the position information of the component to be detected includes: when the number of pads corresponding to the component to be detected is two or three, determining pad one that is closest to the component to be detected, and determining the position information of other pads according to the position information of pad one and the position information of the component to be detected. When the number of pads corresponding to the component to be detected is greater than three, determining all the pads corresponding to the component to be detected based on the preset number of pads, the topological structure between the pads, and the position information of the component to be detected. Wherein, when the number of pads corresponding to the component to be detected is two or three, by determining pad one that is closest to the component to be detected, and combining its position information with the position information of the component to be detected, the position information of other pads can be inferred, thereby improving the accuracy of the overall pad position. When the number of pads corresponding to the components to be detected is greater than three, all pads are determined based on the preset number of pads, the topological structure between the pads, and the position information of the components to be detected. This method takes into account the estimation of the number of pads and the topological relationship between the pads, and can more accurately infer and determine the positions of all pads corresponding to the components to be detected, thereby improving the accuracy and reliability of pad detection. Moreover, this embodiment infers the pad positions based on the number and topological structure of the pads of the components to be detected, as well as preset parameters, which makes this embodiment flexible and can adapt to the detection requirements of pads of different numbers and arrangements.

[0071] Wherein, when the number of pads corresponding to the component to be detected is greater than three, if the preset number of pads is equal to the number of pads identified in the target image, then the pads are all pads corresponding to the component to be detected; if the preset number of pads is less than the number of pads identified from the target image, the following steps are performed: determine pad one that is closest to the component to be detected; determine the topological structure of pad one and candidate pads according to the position information of pad one and the preset number of pads; obtain the distance between the center of the topological structure of pad one and candidate pads and the center of the component to be detected; determine all pads corresponding to the component to be detected according to the distance. Wherein, according to the position information of pad one and the preset number of pads, the topological structure between pad one and candidate pads can be determined, which includes determining the connection mode between pads, such as linear arrangement, circular arrangement or other forms of connection.

[0072] When processing components with three or more pads, all pads corresponding to the components to be detected are determined based on the preset number of pads, the topological structure between the pads, and the position information of the components to be detected, specifically including: comparing the number of identified pads with the preset number of pads, if the number of pads is equal, directly using the identified pad position as the final result. If the number of pads is not equal (for example, the number of identified pads is greater than the preset number of pads), the pad closest to the component to be detected is selected as pad one, wherein the distance between each pad and the center point of the component to be detected can be calculated, and the pad closest to the center point is selected as pad one.

[0073] Then, according to the position information of pad one and the preset number of pads, the position of the alternate pad is determined, and then the connection relationship between all pads is determined, so as to obtain the topological structure of pad one and the alternate pad. Wherein, if the preset number of pads is n, (n-1) / 2 alternate pads can be selected on both sides of pad one along the direction of pad one, so that the linear arrangement of pads can be maintained. If the preset number of pads is n, n-1 alternate pads can be selected around pad one according to the equidistant interval between the position of pad one and the alternate pad, forming a ring arrangement. Other topological structures can also be selected according to specific needs and design requirements, such as rectangular arrangement, triangular arrangement, etc. In this case, according to the position information of pad one and the preset number of pads, the accurate positions of other alternate pads are calculated. Finally, the connection relationship between pads can be determined, and the connection relationship between pad one and the alternate pad can be determined according to the selected topological structure, which includes determining the distance, angle or other geometric relationship between pads. For example, when arranged linearly, there may be a horizontal or vertical connection relationship between the pads; when arranged in a ring, there may be an equiangular connection relationship between the pads, and so on.

[0074] After obtaining the topological structure of pad 1 and candidate pads as described above, the distance between the center of the topological structure and the center of the component to be detected is obtained, and then all pads corresponding to the component to be detected are determined based on the distance. Among them, the center coordinates of the topological structure of pad 1 and candidate pads can be calculated according to the type of topological structure and pad position information; the center coordinates of the component to be detected can be calculated according to the shape and position information of the component to be detected; and then a suitable distance measurement method (such as Euclidean distance) is used to calculate the distance between the center of the topological structure of pad 1 and candidate pads and the center of the component to be detected; finally, according to the obtained center distance, the pads within a certain range of the center distance of the component to be detected are classified as pads of the component to be detected. A threshold can be set to determine whether the pad is associated with the component to be detected. If the distance is less than the threshold, the pad is classified as a pad of the component to be detected.

[0075] This embodiment considers the preset number of pads and the topological structure between the pads, as well as the position information of the components to be detected, so as to more accurately determine the positions of all pads corresponding to the components to be detected, thereby improving the accuracy of pad position detection. In addition, when the preset number of pads is less than the number of pads identified in the target image, all pads corresponding to the components to be detected are further determined by determining the topological structure of pad one and the candidate pads, and calculating the distance between the center of the topological structure of pad one and the candidate pads and the center of the components to be detected. This method enables this embodiment to adapt to the situation of different numbers of pads, thereby improving flexibility and adaptability.

[0076] In some embodiments, after executing the above step S13, the method further includes: detecting whether the component to be detected is a collision component. A collision component refers to a situation where a component has been successfully installed on a pad during a normal soldering process, but later the component is lost or damaged due to an accidental impact or collision. This defect is usually caused by external forces, such as improper human operation, equipment failure, or accidental collision. A collision component may cause serious damage to the circuit board and the component itself, resulting in circuit failure or causing problems such as a circuit short circuit.

[0077] In the process of identifying the components to be inspected through the above steps, if the image of the collision part contains the outline of the component to be inspected and the outline of the electrode, this may cause difficulties for the recognition model to distinguish between the collision part and the component to be inspected. Since the collision part has been connected to the pad during the soldering process, its outline may be very similar to the outline of the normal soldered component, making it difficult for the model to accurately distinguish it, and the collision part may be identified as the component to be inspected. Therefore, it is very important to detect whether there is a collision part among the components to be inspected. By detecting collision parts, damage can be discovered early, product quality and reliability can be improved, and the workload of component offset detection can be reduced.

[0078] See also Figure 4 , detecting whether the component to be detected is a collision component specifically includes:

[0079] S131, obtaining a first color of the board background corresponding to the component to be detected;

[0080] S132, obtaining a second color within a preset range of the center point of the component to be detected;

[0081] S133, calculating a similarity measurement value between the first color and the second color;

[0082] S134: When the similarity measurement value is greater than a preset threshold, it is determined that the component to be detected is a collision component; otherwise, it is determined that the component to be detected is not a collision component.

[0083] First, the board background color of the location of the component to be detected needs to be obtained. The first color can be obtained by collecting a certain number of background pixel samples around the component to be detected and calculating their average color. Then, the color within the preset range around the center point of the component to be detected is obtained. The second color can be obtained by collecting a certain number of pixel samples around the center point of the component to be detected and calculating their average color.

[0084] The similarity measurement value between the first color and the second color can be calculated using a similarity measurement method in a CIE (Commission Internationale de l'Eclairage) color space, such as CIE76, CIE94 or CIEDE2000. These measurement methods can be calculated based on the LAB value of the color to measure the degree of difference between the colors.

[0085] This embodiment can assist in determining whether the component to be detected is a collision component by comparing the similarity between the color around the component to be detected and the color of the board background. This method can provide additional information to assist the judgment of the AI ​​model, thereby improving the accuracy of detection.

[0086] If it is determined that the component to be detected is a collision part, the component to be detected can be reported as a collision part so that appropriate measures can be taken to deal with the collision problem. If the component is not a collision part, the following step S14 is performed.

[0087] S14, calculating the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the minimum circumscribed rectangle of the edge pad.

[0088] Specifically, the minimum bounding rectangle of the edge pad fitting is obtained; the center point of the minimum bounding rectangle is determined; and the offset between the center point of the component to be detected and the center point of the minimum bounding rectangle in the horizontal direction and the vertical direction is calculated.

[0089] Among them, the components to be tested are electrically connected to the pads through their pins or solder balls. During the circuit board manufacturing process, the pins or solder balls of the components to be tested are accurately placed at the positions corresponding to the pads, and then the two are connected through the welding process. A component can correspond to one or more pads, depending on the number and layout of the pins of the component to be tested.

[0090] In this embodiment, the geometric shape information of the pad corresponding to the component to be detected is obtained, and the shape information of the pad can be extracted from the image or outline of the pad by image processing technology, such as edge detection algorithm or morphological operation. Then the minimum bounding rectangle of the pad is calculated using the shape information, which can be achieved by applying a rotating rectangle fitting algorithm or a minimum area rectangle fitting algorithm, which can calculate the minimum bounding rectangle that can completely surround the edge pad. Next, the center point coordinates of the rectangle are calculated according to the boundary coordinates of the minimum bounding rectangle. Then the center point coordinates of the component can be determined by the geometric information of the component or image processing technology. The horizontal coordinate of the center point of the component is subtracted from the horizontal coordinate of the center point of the minimum bounding rectangle, and the vertical coordinate of the center point of the component is subtracted from the vertical coordinate of the center point of the minimum bounding rectangle, so as to obtain the offset between the center point of the component and the center point of the minimum bounding rectangle.

[0091] In this embodiment, the position offset information of the component to be detected on the pad can be obtained by calculating the offset, and based on the information, it can be determined whether the component to be detected is correctly installed on the pad.

[0092] S15. Determine the offset information of the component to be detected according to the offset and the rotation angle corresponding to the component to be detected, wherein the offset information includes an angle offset and a position offset.

[0093] Specifically, the position offset of the component to be detected relative to the pad is obtained according to the offset; and the angular offset of the component is obtained according to the rotation angle. After obtaining the offset information including the angular offset and the position offset, the offset of the component to be detected is detected based on the offset information. Among them, the position offset and rotation of the component to be detected are comprehensively considered to determine whether the component to be detected is offset, so that the offset of the component to be detected can be comprehensively evaluated, thereby improving the accuracy and reliability of offset detection.

[0094] Among them, the offset in the horizontal direction (dx) can be compared with the preset threshold. If the offset exceeds the threshold, that is, |dx|>threshold, it can be judged that the component to be detected has horizontal displacement. The offset in the vertical direction (dy) can also be compared with the preset threshold. If the offset exceeds the threshold, that is, |dy|>threshold, it can be judged that the component to be detected has vertical displacement. The rotation angle of the component to be detected is also compared with the preset threshold. If the rotation angle exceeds the threshold, that is, |rotation angle|>threshold, it can be judged that the component to be detected has rotational offset. The above comprehensive consideration of the results of horizontal displacement, vertical displacement and rotational offset, if any one of the offsets exceeds the corresponding threshold, it can be judged that the component to be detected has corresponding offset. By classifying the offset types, the offset of the component to be detected can be described more accurately.

[0095] Optionally, in order to more intuitively display the offset of the components to be detected, the offset information can be visualized. The offset components or pads to be detected can be marked on the target image, or arrows, text or color coding can be used to indicate the direction and degree of the offset, which can facilitate users to observe and understand the offset of the components to be detected.

[0096] The offset of the component to be detected is comprehensively judged based on the obtained horizontal displacement, vertical displacement and rotational displacement. If the position offset and rotational offset are both within the preset threshold range, it can be judged that the component to be detected has no obvious offset. If at least one of the position offset and rotational offset exceeds the preset threshold, it can be judged that the component to be detected has the corresponding offset.

[0097] The component offset detection method of this embodiment can detect the offset of one component in the target image, or can detect the offset of multiple components in the target image at the same time. When detecting the offset of multiple components, the above steps S11 to S15 can also be used.

[0098] The component offset detection method provided in the embodiment of the present application can reduce manual intervention and improve the accuracy of identification by automatically identifying components and pads in the image and extracting their position information; through collision detection, it can be determined whether there is a collision in the component, and the collision is removed from the component set, so that interference elements can be removed and the accuracy of subsequent analysis can be improved; by calculating the offset between the center point of the component to be detected and the center point of the minimum circumscribed rectangle of the pad, the offset of the component to be detected can be quantified, and a quantitative offset measurement can be provided; the angle offset and position offset of the component to be detected are obtained by combining the offset and the position information of the component to be detected, so as to determine whether the component to be detected is offset, so that offset detection can be performed automatically, and the offset problem can be found quickly and accurately. In general, the embodiment of the present application comprehensively utilizes image processing, target feature analysis, collision detection and displacement calculation, etc., to improve the efficiency and accuracy of component offset detection, and provide important information for subsequent quality control and adjustment. At the same time, since there is no need to add a template, it also has good flexibility and applicability, and can adapt to components of different types and sizes.

[0099] In some embodiments, the preset recognition model can also output the rotation angle corresponding to the pad. After executing the above step S13, the method further includes: calculating the angle deviation between the rectangular frame of the pad determined from the target image and the minimum circumscribed rectangle; when the angle deviation is greater than a preset threshold, calculating the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the preset picture frame, and determining the offset information of the component to be detected based on the offset and the rotation angle. When the angle deviation is less than or equal to the preset threshold, the above steps S14 to S15 are executed. Among them, the rectangular frame of the pad determined from the target image can be a circumscribed rectangular frame corresponding to a single pad, and the circumscribed rectangular frame is used to calculate the angle offset of the minimum circumscribed rectangle of the edge pad obtained by fitting the step S13. The rotation angle corresponding to the pad output by the recognition model can be used in the calculation process. If the calculated angle deviation is greater than the preset threshold, it means that the pad is not the pad corresponding to the component to be detected, that is, the pad is not fully recognized by the model. At this time, the preset picture frame and the component to be detected output by the model can be used to calculate the offset, thereby determining the offset information of the component to be detected. In this embodiment, by calculating the angle deviation between the rectangular frame of the pad and the minimum circumscribed rectangle, it is determined whether the model correctly identifies the pad. When the angle deviation exceeds the preset threshold, it means that the model does not fully identify the pad. At this time, the offset information of the component is calculated by using the preset picture frame. This method can improve the accuracy and robustness of component positioning, especially when there is uncertainty or error in the model's recognition of the pad, it can be corrected by the preset picture frame, so as to obtain more reliable component offset information. The preset frame can be a rectangular frame manually drawn by a human operator to surround the component. In this case, when the human operator observes that the pad is not fully recognized by the model or there is an error, he manually draws a rectangular frame to accurately surround the component. The manually drawn rectangular frame is compared with the result output by the model, and the offset information of the component is determined according to the horizontal and vertical offsets of its center point and the rotation angle.

[0100] See also Figure 6, the embodiment of the present application provides a component offset detection device. The component offset detection device 60 includes: an image acquisition module 61, which is used to acquire a target image containing components to be detected; a component and pad detection module 62, which is used to identify the target image, determine the components and pads in the target image, and obtain the rotation angle corresponding to the components; a pad detection module 63, which is used to filter out a number of edge pads corresponding to the components to be detected from the target image, and obtain the minimum circumscribed rectangle covering the edge pads; an offset calculation module 64, which is used to calculate the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the minimum circumscribed rectangle of the edge pad; an offset detection module 65, which is used to determine the offset information of the component to be detected according to the offset and the rotation angle corresponding to the component to be detected, and the offset information includes angle offset and position offset.

[0101] Optionally, the component offset detection device 60 further includes a collision detection module 66. Before realizing the function of the offset calculation module 64, the collision detection module 66 detects whether the component to be detected is a collision component. If the component to be detected is not a collision component, the offset calculation module 64 is controlled to realize its corresponding function. The collision detection module 66 is specifically used to: obtain the first color of the board background corresponding to the component to be detected; obtain the second color within a preset range of the center point of the component to be detected; calculate the similarity metric value between the first color and the second color; when the similarity metric value is greater than a preset threshold, it is determined that the component to be detected is a collision component, otherwise, it is determined that the component to be detected is not a collision component.

[0102] Among them, the component and pad detection module 62 is specifically used to: preprocess the obtained target image; input the preprocessed target image into a preset recognition model to perform target detection and classification, and output the category label of each detected target, wherein the category label is used to indicate the type of component or pad; and output the rotation angle corresponding to the component, wherein the rotation angle is represented by the rotation angle of a rectangular box; the loss function of the recognition model is calculated based on the projection difference of the Gaussian distribution.

[0103] Optionally, the component and pad detection module 62 is also used to: detect whether the rotation angle corresponding to the component is within a preset angle range; if the rotation angle corresponding to the component is within the preset angle range, then the currently obtained rotation angle is represented as the rotation angle of the component; if the rotation angle corresponding to the component is not within the preset angle range, then the horizontal axis in the coordinate system corresponding to the rectangular frame of the component is rotated counterclockwise, and the rotation angle of the horizontal axis when it is rotated counterclockwise until the rectangular frame of the component is first encountered is recorded to obtain a new rotation angle, and the new rotation angle is used to represent the rotation angle of the component.

[0104] Wherein, the pad detection module 63 is specifically used for: calculating the distance between the center point of each component and the center point of the target image according to the components in the determined target image; determining the component corresponding to the minimum distance as the component to be detected; determining all pads corresponding to the component to be detected according to the position information of the component to be detected, and obtaining the minimum circumscribed rectangle of all pads. Wherein, determining all pads corresponding to the component to be detected according to the position information of the component to be detected includes: when the number of pads corresponding to the component to be detected is two or three, determining pad one closest to the component to be detected, and determining the position information of other pads according to the position information of pad one and the position information of the component to be detected; when the number of pads corresponding to the component to be detected is greater than three, determining all pads corresponding to the component to be detected based on the preset number of pads, the topological structure between the pads, and the position information of the component to be detected. Among them, the method of determining all pads corresponding to the component to be detected based on a preset number of pads, a topological structure between pads and the position information of the component to be detected includes: if the preset number of pads is equal to the number of pads identified in the target image, the pads are all pads corresponding to the component to be detected; if the preset number of pads is less than the number of pads identified from the target image, determining pad one closest to the component to be detected, and determining the topological structure of pad one and candidate pads according to the position information of pad one and the preset number of pads; obtaining the distance between the center of the topological structure of pad one and candidate pads and the center of the component to be detected, and determining all pads corresponding to the component to be detected according to the distance.

[0105] Optionally, the recognition model also outputs the rotation angle corresponding to each pad. After executing the function of the pad detection module 63, the offset detection module 65 is used to: calculate the angular deviation between the rectangular frame of the pad determined from the target image and the minimum circumscribed rectangle; when the angular deviation is greater than a preset threshold, calculate the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the preset picture frame, and determine the offset information of the component to be detected based on the offset and the rotation angle.

[0106] When the angle deviation is less than or equal to the preset threshold, the function of the above-mentioned offset calculation module 64 is executed, and then the offset detection module 65 is used to determine the offset information of the component to be detected according to the offset and the rotation angle corresponding to the component to be detected, and the offset information includes angle offset and position offset.

[0107] It should be noted that the component offset detection device of the embodiment of the present application can execute the component offset detection method provided in the embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method. For technical details not fully described in the embodiment of the component offset detection device, please refer to the component offset detection method provided in the embodiment of the present application.

[0108] Figure 7 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application, and the electronic device can be used to perform the above-mentioned component offset detection method. Figure 7 As shown, the electronic device 70 includes: one or more processors 71 and a memory 71, Figure 7 A processor 71 is taken as an example. The processor 71 and the memory 72 may be connected via a bus or other means. Figure 7 The example of connecting through bus is taken in the following.

[0109] The memory 72 is a non-volatile computer-readable storage medium that can be used to store non-volatile software programs, non-volatile computer executable programs and modules, such as program instructions / modules corresponding to the component offset detection method in the embodiment of the present application. The processor 71 executes various functional applications and data processing of the electronic device by running the non-volatile software programs, instructions and modules stored in the memory 72, that is, the component offset detection method in the above method embodiment is implemented.

[0110] The memory 72 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the component offset detection device, etc. In addition, the memory 72 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 72 may optionally include a memory remotely arranged relative to the processor 71, and these remote memories may be connected to the component offset detection device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0111] The one or more modules are stored in the memory 72, and when executed by the one or more processors 71, the component offset detection method in any of the above method embodiments is executed.

[0112] The above-mentioned product can execute the method provided in the embodiment of the present application, and has the functional modules and beneficial effects corresponding to the execution method. For technical details not fully described in this embodiment, please refer to the method provided in the embodiment of the present application.

[0113] The electronic devices of the embodiments of the present application exist in various forms, including but not limited to: ultra-mobile personal computer devices, which belong to the category of personal computers, have computing and processing functions, and generally also have mobile Internet access features. Such terminals include: PDA, MID and UMPC devices, etc.; mobile communication devices, such as smart phones, functional phones, etc.; servers, devices that provide computing services; and other electronic devices with data interaction functions.

[0114] The embodiment of the present application provides a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are executed by one or more processors, such as Figure 7 A processor 71 in the embodiment may enable the one or more processors to execute the component offset detection method in any of the above method embodiments.

[0115] An embodiment of the present application provides a computer program product, which includes a computer program stored on a non-volatile computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by the electronic device, the electronic device is enabled to execute the component offset detection method in any of the above method embodiments.

[0116] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0117] Through the description of the above implementation methods, ordinary technicians in this field can clearly understand that each implementation method can be implemented by means of software plus a general hardware platform, and of course, it can also be implemented by hardware. Ordinary technicians in this field can understand that all or part of the processes in the above-mentioned embodiment method can be completed by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0118] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Under the concept of the present application, the technical features in the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other changes in different aspects of the present application as described above, which are not provided in detail for the sake of simplicity. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some of the technical features can be replaced by equivalents. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A component offset detection method, characterized in that: include: Acquire a target image containing components to be inspected; Identify the target image, determine the components and pads in the target image, and obtain the rotation angle corresponding to the components; Filtering out a number of edge pads corresponding to the components to be detected from the target image, and obtaining a minimum circumscribed rectangle covering the edge pads; Calculate the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the minimum circumscribed rectangle of the edge pad; The offset information of the component to be detected is determined according to the offset and the rotation angle corresponding to the component to be detected, and the offset information includes an angle offset and a position offset.

2. The method according to claim 1, characterized in that The identifying the target image, determining the components and pads in the target image, and obtaining the rotation angles corresponding to the components, includes: Preprocessing the obtained target image; The preprocessed target image is input into a preset recognition model for target detection and classification, and the category label of each detected target is output, wherein the category label is used to indicate the type of component or pad; and the rotation angle corresponding to the component is output, wherein the rotation angle is represented by the rotation angle of a rectangular box; the loss function of the recognition model is calculated based on the projection difference of the Gaussian distribution.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: Detecting whether the rotation angle corresponding to the component is within a preset angle range; If the rotation angle corresponding to the component is within the preset angle range, the currently obtained rotation angle is represented as the rotation angle of the component; If the rotation angle corresponding to the component is not within the preset angle range, the horizontal axis in the coordinate system corresponding to the rectangular frame of the component is rotated counterclockwise, and the rotation angle of the horizontal axis when it is rotated counterclockwise until it first encounters the rectangular frame of the component is recorded to obtain a new rotation angle, and the new rotation angle is used to represent the rotation angle of the component.

4. The method according to claim 1, characterized in that The step of selecting a plurality of edge pads corresponding to the components to be detected from the target image and obtaining a minimum circumscribed rectangle covering the edge pads includes: According to the determined components in the target image, calculating the distance between the center point of each component and the center point of the target image; Determine the component corresponding to the minimum distance as the component to be detected; All pads corresponding to the components to be detected are determined according to the position information of the components to be detected, and the minimum circumscribed rectangle of all the pads is obtained.

5. The method according to claim 4, characterized in that The step of determining all pads corresponding to the components to be detected according to the position information of the components to be detected includes: When the number of pads corresponding to the component to be detected is two or three, determine the pad one closest to the component to be detected, and determine the position information of other pads according to the position information of pad one and the position information of the component to be detected; When the number of pads corresponding to the component to be detected is greater than three, all pads corresponding to the component to be detected are determined based on the preset number of pads, the topological structure between the pads, and the position information of the component to be detected.

6. The method according to claim 5, characterized in that The determining all pads corresponding to the components to be detected based on the preset number of pads, the topological structure between the pads, and the position information of the components to be detected includes: If the preset number of pads is equal to the number of pads identified in the target image, then the pads are all pads corresponding to the components to be detected; If the preset number of pads is less than the number of pads identified from the target image, determine pad 1 which is closest to the component to be detected, and determine the topological structure of pad 1 and candidate pads according to the position information of pad 1 and the preset number of pads; The distance between the center of the topological structure of the first pad and the candidate pad and the center of the component to be detected is obtained, and all pads corresponding to the component to be detected are determined according to the distance.

7. The method according to claim 2, characterized in that The recognition model further outputs the rotation angle corresponding to each pad, and after screening out a number of edge pads corresponding to the components to be detected from the target image and obtaining the minimum circumscribed rectangle covering the edge pads, the method further includes: Calculating an angle deviation between a rectangular frame of a pad determined from the target image and the minimum circumscribed rectangle; When the angle deviation is greater than a preset threshold, the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the preset picture frame are calculated, and the offset information of the component to be detected is determined according to the offset and the rotation angle; When the angle deviation is less than or equal to a preset threshold, the steps of calculating the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the minimum circumscribed rectangle of the edge pad, and determining the offset information of the component to be detected based on the offset and the rotation angle corresponding to the component to be detected are performed.

8. The method according to claim 1, characterized in that: Before the step of calculating the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the minimum circumscribed rectangle of the edge pad, the method further includes: Detecting whether the component to be detected is a collision component; If the component to be detected is not a collision part, the step of calculating the horizontal and vertical offsets of the center point of the component to be detected relative to the center point of the minimum circumscribed rectangle of the edge pad is performed.

9. The method according to claim 8, characterized in that The detecting whether the component to be detected is a collision part comprises: Obtaining a first color of the board background corresponding to the component to be detected; Acquire a second color within a preset range of the center point of the component to be detected; Calculating a similarity measure between the first color and the second color; When the similarity measurement value is greater than a preset threshold, it is determined that the component to be detected is a collision component; otherwise, it is determined that the component to be detected is not a collision component.

10. An electronic device, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the component displacement detection method according to any one of claims 1 to 9.

Images