Chip image correction method and device, computer equipment and readable storage medium

Abstract

The application relates to a chip image correction method and device, computer equipment and a readable storage medium. A template matching algorithm and a deep learning target detection network are used to perform target detection on a to-be-processed image to obtain a chip image, a SAM model is used to perform graph correction on the chip image to obtain an initial correction image, the initial correction image is rotated based on a plurality of preset rotation angles, and a target rotation image with the maximum matching similarity is determined from the plurality of rotation images. The template matching and the deep learning target detection network are combined to position the chip image, so that missing detection is avoided. The SAM model is introduced to realize lossless correction of the chip image at the pixel level. The initial correction image is orthogonally rotated in four quadrants, the highest fitting degree direction is locked by using a matching scoring mechanism, and the direction difference problem caused by tray mixing and inversion is solved. High-precision, high-robustness and full-automatic chip image correction is realized.

Description

Technical Field

[0001] This application relates to the field of chip image processing technology, and in particular to a chip image correction method, apparatus, computer equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] In the modern semiconductor manufacturing and packaging industry chain, the appearance quality of chips directly affects the reliability and lifespan of the final electronic devices and smart terminals. The primary prerequisite for high-quality appearance defect assessment is obtaining standardized, distortion-free, and precisely positioned images of individual chips.

[0003] In traditional technologies, on actual industrial automated production lines, images of chip tray arrays are typically acquired using optical acquisition equipment. However, due to mechanical vibrations and nozzle placement accuracy errors, chips in the tray can experience random positional shifts and angular deflections. Traditional template matching algorithms that rely solely on geometric contours are extremely prone to positional drift or even extraction failure when faced with slight chip rotation, uneven lighting, or complex background reflections.

[0004] Furthermore, after acquiring an image of a single chip, the presence of an initial deflection angle means that directly performing image subtraction or pixel-level comparison will produce significant errors (commonly known as "pseudo-defects"). Although some traditional techniques exist for image rotation registration, they are often computationally complex and rely on clear edge corners. When the chip edge is chamfered and reflective or has burrs, it is easy to cause errors in the calculation of the rotation correction angle. Summary of the Invention

[0005] Therefore, it is necessary to provide a chip image correction method, apparatus, computer device, computer-readable storage medium, and computer program product to address the aforementioned technical problems.

[0006] In a first aspect, this application provides a chip image correction method, the method comprising: Acquire a processing image based on a chip tray, wherein the processing image includes an image corresponding to at least one chip in the chip tray; A template matching algorithm and a deep learning object detection network are used to perform object detection on the image to be processed, and at least one chip image is identified in the image to be processed. The chip image is image-corrected based on the SAM model to obtain an initial corrected image. The initial corrected image is rotated based on multiple preset rotation angles to obtain multiple rotated images; The target rotated image with the highest matching similarity to the chip template image is determined from the plurality of rotated images. The target rotated image is used to characterize the target chip image after character orientation correction. The chip template image is a standard image with standard character orientation corresponding to a flawless chip.

[0007] In one embodiment, the step of using a template matching algorithm and a deep learning object detection network to perform object detection on the image to be processed, and determining at least one chip image in the image to be processed, includes: Based on the template matching algorithm, the chip in the image to be processed is identified, and when the chip is identified, the first pixel coordinate information of the chip is determined. The chip in the image to be processed is identified based on a deep learning object detection network, and the second pixel coordinate information of the identified chip is extracted. The pixel crossover ratio is determined based on the first pixel coordinate information and the second pixel coordinate information of the chip; If the pixel crossover ratio is determined to be greater than a preset threshold, the first pixel coordinate information and the second pixel coordinate information are determined to correspond to the same chip region; The chip image of the chip is determined based on the first pixel coordinate information.

[0008] In one embodiment, the method further includes: If the chip is not identified, a deep learning target detection network is used to identify the chip in the image to be processed, and the second pixel coordinate information and corresponding confidence score of the identified chip are extracted. If the confidence level is determined to be greater than the confidence level threshold, the chip image of the chip is determined based on the second pixel coordinate information.

[0009] In one embodiment, the step of performing image correction on the chip image based on the SAM model to obtain an initial corrected image includes: Based on the center coordinates of the chip image as the cue point, the chip image is input into the SAM model to obtain the mask prediction set of the chip image output by the SAM model; The mask with the largest area is determined from the mask prediction set. If the area of ​​the mask meets a preset area threshold, the mask is determined as the target mask. The target mask is converted into a grayscale binary image, and morphological smoothing and rectangle fitting are performed on the grayscale binary image to obtain the corner coordinates and rotation angle of the fitted rectangle. Based on the rotation angle, a rotation moment is constructed to rotate the chip image, thereby obtaining an initial corrected image after graphic correction of the chip image.

[0010] In one embodiment, the method further includes: If the area of ​​the mask does not meet the preset area threshold, update the position of the prompt point; Based on the updated cue points, the chip image is input into the SAM model to obtain the mask prediction set of the chip image output by the SAM model, and then the step of determining the mask with the largest area from the mask prediction set is returned.

[0011] In one embodiment, determining the target rotated image with the highest matching similarity to the chip template image from the plurality of rotated images includes: Calculate the SSD value between each rotated image and the chip template image; The rotated image with the smallest SSD value is determined as the target rotated image with the highest matching similarity.

[0012] In one embodiment, the preset multiple rotation angles include 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0013] In one embodiment, the method further includes: The spatial coordinate information of characters in the target chip image is extracted and stored based on the bimodal binarization algorithm and the connected component detection algorithm.

[0014] In one embodiment, the step of extracting and storing the spatial coordinate information of characters in the target chip image based on the bimodal binarization algorithm and the connected component detection algorithm includes: If it is determined that the chip in the target chip image does not have pins, the brightness distribution of the target chip image is statistically analyzed by histogram, and two peak brightness values ​​are extracted. A binarization threshold is determined based on the average of two peak brightness values, and a binarized image of the target chip image is determined based on the binarization threshold. After denoising the binarized image and performing connected component detection, the connected components obtained, excluding the background connected components, are determined as character fields. The coordinates of the minimum bounding rectangle of each character field are determined as the spatial coordinate information of the character in the target chip image and stored.

[0015] In one embodiment, the method further includes: If it is determined that the chip in the target chip image has pins, the pin portion in the target chip image is eliminated to obtain the chip body image after the pins are eliminated; The brightness distribution of the chip's main image is statistically analyzed using histograms, and two peak brightness values ​​are extracted. A binarization threshold is determined based on the average of two peak brightness values, and a binarized image of the chip's main image is determined based on the binarization threshold. After denoising the binarized image and performing connected component detection, the connected components obtained, excluding the background connected components, are determined as character fields. The coordinates of the minimum bounding rectangle of each character field are determined as the spatial coordinate information of the character in the target chip image and stored.

[0016] In one embodiment, the method further includes: Based on the brightness of each pixel in the target chip image, obtain the pixel area of ​​the pixel whose brightness reaches the first brightness threshold; If the pixel area is determined to be greater than the pixel area threshold, it is determined that the chip in the target chip image has pins; If the pixel area is determined to be less than or equal to the pixel area threshold, it is determined that the chip in the target chip image does not have pins.

[0017] In one embodiment, the step of removing pin portions from the target chip image to obtain a chip body image after pin removal includes: The target chip image is binarized based on the second brightness threshold to obtain the corresponding binarized image; The binarized image is dilated based on a preset morphological kernel to obtain a dilated binarized image. The binarized image after dilation is scanned column by column from left to right, and the brightness ratio of each column of pixels is counted. If the brightness ratio is determined to be greater than a set ratio threshold, it is determined that the corresponding column contains pins, and the column coordinates of the corresponding column are recorded. If the brightness ratio of the first column of pixels is detected to have decreased to less than a set threshold, the first column of pixels is determined to be the boundary between the pin and the chip body. The dilated binarized image is scanned from right to left, top to bottom, and bottom to top to determine the boundaries between the pins and the chip body in each direction. The target chip image is cropped according to the determined boundary to obtain the chip body image after removing the pins.

[0018] Secondly, this application also provides a chip image correction apparatus, the apparatus comprising: An image acquisition module is used to acquire an image to be processed based on a chip tray, wherein the image to be processed includes an image corresponding to at least one chip in the chip tray; The target detection module is used to perform target detection on the image to be processed using a template matching algorithm and a deep learning target detection network, and to determine at least one chip image in the image to be processed; The image correction module is used to perform image correction on the chip image based on the SAM model to obtain an initial corrected image after image correction. An orientation correction module is used to rotate the initial correction image based on multiple preset rotation angles to obtain multiple rotated images; and to determine the target rotated image with the highest matching similarity to the chip template image from the multiple rotated images. The target rotated image is used to represent the target chip image after character orientation correction. The chip template image is a standard image with standard character orientation corresponding to a flawless chip.

[0019] Thirdly, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described in the first aspect.

[0020] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0021] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0022] The aforementioned chip image correction method, apparatus, computer equipment, computer-readable storage medium, and computer program product acquire an image to be processed based on a chip tray, employ a template matching algorithm and a deep learning object detection network to perform object detection on the image to be processed, identifying at least one chip image in the image to be processed, performing graphic correction on the chip image based on a SAM model to obtain an initial corrected image, rotating the initial corrected image based on multiple preset rotation angles to obtain multiple rotated images, and determining the target rotated image with the highest matching similarity to the chip template image from the multiple rotated images. By combining template matching with a deep learning object detection network for chip image localization, it avoids missed detections. By introducing a SAM model for graphic correction of the chip image, it can achieve pixel-level lossless correction of the chip image. By performing orthogonal rotation of the initial corrected image in four quadrants and using a matching scoring mechanism to lock the direction of the highest fit, it solves the problem of directional differences caused by tray mixing and inversion, achieving high-precision, high-robustness, and fully automatic chip image correction. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a flowchart illustrating a chip image correction method in one embodiment; Figure 2 This is a schematic diagram of the image to be processed in one embodiment; Figure 3 This is a schematic diagram illustrating the rotation of an initial corrected image in one embodiment; Figure 4 This is a flowchart illustrating the target detection steps in one embodiment; Figure 5 This is a flowchart illustrating the graphic correction steps in one embodiment; Figure 6 This is a schematic diagram of graphic correction in one embodiment; Figure 7 This is a flowchart illustrating the steps for extracting character spatial coordinate information in one embodiment; Figure 8 This is a schematic diagram of the bimodal distribution of the histogram in one embodiment; Figure 9 This is a flowchart illustrating the pin elimination step in one embodiment; Figure 10 This is a schematic diagram of pin elimination in one embodiment; Figure 11 This is a structural block diagram of a chip image correction device in one embodiment; Figure 12 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0025] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0026] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0027] Traditional template matching algorithms that rely solely on geometric contours are extremely prone to positional drift or even extraction failure when faced with slight chip rotation, uneven lighting, or complex background reflections. Furthermore, after acquiring an image of a single chip, the initial deflection angle can lead to significant errors (commonly known as "pseudo-defects") when directly performing image subtraction or pixel-level comparison. While some traditional techniques exist for image rotation registration, they are often computationally complex and dependent on sharp edge points. When chip edges are chamfered and reflective or have burrs, rotation correction angle calculations are prone to errors. Moreover, when the chip has edge burrs or local defects, edge-based matching algorithms often fail to converge.

[0028] Secondly, for pin-packaged chips (such as QFP and SOP), the peripheral metal pins will produce strong high-intensity reflections under industrial light sources, and the pins themselves are prone to slight physical deformation. This not only seriously interferes with the template matching accuracy based on global pixels, but also introduces huge noise interference to the subsequent independent analysis of colloidal surface features or silkscreen characters.

[0029] Therefore, traditional preprocessing methods based on simple image comparison or single feature extraction struggle to achieve high-precision, fully automated chip image extraction and attitude correction when dealing with changes in lighting, multiple packaging forms, and target rotation. This technical bottleneck in the extraction and correction stages severely limits the accuracy ceiling of subsequent defect detection algorithms, thereby affecting the yield rate and overall intelligent management level of semiconductor packaging production lines.

[0030] Based on this, this application provides a chip image correction method, aiming to solve the problems of positioning failure and limited contrast caused by random chip deflection, pin reflection interference, and inconsistent orientation when dealing with chip tray array images using traditional techniques. This achieves high-precision, high-robustness, and fully automated chip image correction.

[0031] In one exemplary embodiment, such as Figure 1As shown, a chip image correction method is provided, which can be applied to Automated Optical Inspection (AOI) equipment in the semiconductor packaging testing process. Specifically, it may include the following steps: Step 102: Obtain the image to be processed based on the chip tray.

[0032] The image to be processed includes an image corresponding to at least one chip in the chip tray. Specifically, the image to be processed of the chip tray can be acquired through an optical image acquisition module (such as an industrial camera). This image typically contains multiple chips arranged neatly but with different orientations, and these multiple chips are usually chips of the same model from the same batch to be tested.

[0033] For example, after receiving the loading signal from the control mechanism, the optical image acquisition module prepares to acquire images. When the tray filled with the chip under test is placed into the center of the optical image acquisition module's field of view, the module initiates continuous image capture, calculates the sharpness gradient of the current image, and performs closed-loop micron-level positioning adjustments based on the real-time image from the camera. When the tray touches the limit sensor and the visual image reaches the positioning calibration, the tray is controlled to enter a locked braking state, and the image acquisition module performs high-definition image capture with an extremely short exposure time to obtain a high-definition image to be processed. For example, the image to be processed can be as follows: Figure 2 As shown, it includes images of multiple neatly arranged chips.

[0034] The optical image acquisition module is responsible for acquiring high dynamic range, high-definition, distortion-free images of the chip tray. In industrial-grade implementations, this module requires a high-resolution, multi-megapixel industrial camera (such as a CMOS sensor) combined with a customized coaxial shadowless light source and a high-angle ring light source system. This effectively suppresses specular reflections from the silicon crystal or encapsulated material. In some scenarios, a telecentric lens can also be fitted to the front of the camera to eliminate perspective distortion caused by the thickness of the tray edge from a physical optical perspective, thereby ensuring the consistency of image extraction size.

[0035] Step 104: Use template matching algorithm and deep learning object detection network to perform object detection on the image to be processed, and determine at least one chip image in the image to be processed.

[0036] The template matching algorithm is a pattern recognition method that locates the region in the image to be processed that is most similar to the chip template image, in order to identify the chip region in the image to be processed. The chip template image is a standard image corresponding to a flawless chip with the same chip model as the chip in the image to be processed.

[0037] Deep learning object detection networks are one of the core tasks in computer vision, aiming to locate objects of interest, such as chips, in images and generate candidate regions, i.e., chip regions. For example, a deep learning object detection network can be a convolutional neural network from the YOLO family.

[0038] In this embodiment, in order to accurately locate each chip in the image to be processed, a dual-fusion detection strategy is adopted: that is, combining template matching algorithm and deep learning object detection network to perform object detection in the image to be processed, so as to accurately locate each chip image in the image to be processed.

[0039] Step 106: Perform graphic correction on the chip image based on the SAM model to obtain the initial corrected image after graphic correction.

[0040] Among them, the SAM model (Segment Anything Model) is a general-purpose image segmentation model for artificial intelligence, focusing on image segmentation tasks. Its technical architecture employs an encoder-decoder design, achieving efficient segmentation of arbitrary objects by fusing image features with cue information (such as points, boxes, and text). In this embodiment, the SAM model can ignore background interference and edge blurring. Using point cues, it can accurately extract the chip object from the background, generating a mask. With this complete chip shape, subsequent algorithms can easily calculate its rotation angle and perform image correction.

[0041] Pattern calibration is the correction of random positional offsets and angular deflections of chips in the tray caused by mechanical vibration, nozzle placement accuracy errors, etc.

[0042] Specifically, after determining at least one chip image in the image to be processed based on the above steps, the SAM model can be used to perform graphic correction on each chip image. With its powerful generalization ability, the SAM model can generate a high-precision chip mask based on the center point prompt. Then, based on the rotation angle fitted by the chip mask, an affine transformation can be performed on the original chip image to obtain an initial corrected image that is geometrically straight.

[0043] Step 108: Rotate the initial corrected image based on multiple preset rotation angles to obtain multiple rotated images.

[0044] The preset rotation angles can include 0 degrees, 90 degrees, 180 degrees and 270 degrees.

[0045] Considering that during the chip front-end manufacturing process, the chip may be upside down when placed on the tray (e.g., by 90°, 180°, or 270°), even after the aforementioned SAM correction and leveling, the character orientation remains unknown. In this embodiment, to further address the issue of inconsistent silkscreen character orientation in the chip image—for example, a chip with a standard character orientation may be geometrically upright after a 180-degree rotation, but the text may be upside down—this embodiment also introduces an orthogonal template matching mechanism. Specifically, such as… Figure 3 As shown, by constructing rotation mapping operators in four directions, the two-dimensional matrix of the initial corrected image is rotated clockwise by 0 degrees, 90 degrees, 180 degrees, and 270 degrees sequentially, resulting in multiple rotated images. These include a 0-degree rotated image, a 90-degree rotated image, a 180-degree rotated image, and a 270-degree rotated image. Based on subsequent steps, a target rotated image with the same orientation as the silkscreen characters in the chip template image is determined from these images.

[0046] Step 110: Determine the target rotated image with the highest matching similarity to the chip template image from multiple rotated images.

[0047] The target rotation image is used to characterize the target chip image after character orientation correction. The chip template image is the standard image with standard character orientation corresponding to a flawless chip, where the character orientation is the normal orientation.

[0048] For example, the SSD value (Sum of Squared Differences) between each rotated image and the chip template image can be calculated separately, and the rotated image with the smallest SSD value can be determined as the target rotated image with the highest matching similarity.

[0049] For example, the NCC (Normalized Cross Correlation) value between each rotated image and the chip template image can be calculated separately, and the rotated image with the largest NCC value can be determined as the target rotated image with the highest matching similarity.

[0050] In one scenario, the silkscreen characters on a chip are typically located within the adhesive layer. For pin-packaged chips (such as QFP and SOP), the peripheral metal pins generate strong high-intensity reflections under industrial light sources, which can interfere with the template matching accuracy based on global pixels. Therefore, when calculating the SSD or NCC values, only the central region of the rotated image and the chip template image (i.e., the adhesive region excluding the pins) can be considered. This avoids noise interference from the pins, improves matching accuracy, and ensures the accuracy of the correction.

[0051] In the aforementioned chip image correction method, an image to be processed based on a chip tray is acquired. A template matching algorithm and a deep learning object detection network are used to detect targets in the image, identifying at least one chip image. The chip image is then image-corrected using a SAM model to obtain an initial corrected image. This initial corrected image is then rotated at multiple preset rotation angles to obtain multiple rotated images. From these rotated images, the target rotated image with the highest matching similarity to the chip template image is determined. This method combines template matching with a deep learning object detection network for chip image localization, thus avoiding missed detections. By introducing the SAM model for image correction, pixel-level lossless correction of the chip image is achieved. Furthermore, by performing orthogonal rotation of the initial corrected image across four quadrants and using a matching scoring mechanism to lock the direction of the highest fit, the method solves the directional difference problem caused by tray mixing and inversion, achieving high-precision, high-robustness, and fully automatic chip image correction.

[0052] In one exemplary embodiment, such as Figure 4 As shown, in step 104, a template matching algorithm and a deep learning object detection network are used to perform object detection on the image to be processed, identifying at least one chip image in the image to be processed. Specifically, this may include: Step 402: Identify the chip in the image to be processed based on the template matching algorithm. If the chip is identified, determine the first pixel coordinate information of the chip.

[0053] The template matching algorithm is a sliding window matching algorithm based on the chip template image. For example, assume the chip template image is... (size is) The image to be processed is Then a window of the same size as the chip template image is used in the image. Slide one by one And calculate the relationship between each sliding window and the chip template image. The response matrix is ​​obtained by normalizing the cross-correlation (NCC) or squared difference (SSD) between the two values ​​to find local maxima clusters in the response matrix. For example, the coordinates of a sliding window with an NCC greater than a preset threshold (e.g., 0.6) are determined as the first pixel coordinates of the chip, or the coordinates of a sliding window with an SSD less than a preset threshold (e.g., 0.4) are determined as the first pixel coordinates of the chip. In other words, the coordinates of the identified sliding window that meets the requirements are determined as the first pixel coordinates of the identified chip.

[0054] Step 404: Based on the deep learning object detection network, target recognition is performed on the chip in the image to be processed, and the second pixel coordinate information of the recognized chip is extracted.

[0055] Since template matching has certain limitations, in this embodiment, to overcome background noise (such as scratches or reflections on the tray mesh), a deep learning object detection network can be used simultaneously to extract the second pixel coordinate information of the chip from the image to be processed. After the image is processed by the backbone network to extract deep features, the bounding box prediction results of all chips are directly regressed in the detection head, which is the second pixel coordinate information of the chip (including the center coordinates). , ,Width ,high and confidence level wait).

[0056] The deep learning object detection network can be obtained by training a YOLO convolutional neural network based on a large number of chip patterns. The chip patterns can be labeled with specific location tags for the chip regions, such as the coordinates of the chip region's center point, height, and width.

[0057] In one scenario, the chip template image can be a pre-configured sample image with the same chip model as the image to be processed. It has standard character orientation, and the characters are clear, flawless, and the encapsulation colloid is smooth and without any defects.

[0058] In another scenario, the chip template image can also be a detection target determined by a deep learning object detection network after identifying the chip in the image to be processed, and the target has a confidence level that meets the requirements (e.g., confidence level greater than 95%), or a manually selected detection target. It has standard character orientation, and the characters are clear, flawless, and the encapsulating colloid is smooth and without defects.

[0059] Step 406: Determine the pixel crossover ratio based on the first pixel coordinate information and the second pixel coordinate information of the chip.

[0060] Specifically, the identification results of steps 402 and 404 are fused to finally determine the identified chip image.

[0061] For example, the first pixel coordinate information of the chip determined in step 402 and the second pixel coordinate information of the chip extracted in step 404 can be transformed to unify the coordinate system. Then, for any pair of candidate boxes that are close to each other in space (i.e., the candidate box corresponding to the first pixel coordinate information and the candidate box corresponding to the second pixel coordinate information), their intersection-union ratio (IoU, which represents the result obtained by dividing the pixel intersection area of ​​the two candidate boxes by the pixel union area) is calculated.

[0062] Step 408: If the pixel crossover ratio is greater than a preset threshold, determine that the first pixel coordinate information and the second pixel coordinate information correspond to the same chip region.

[0063] The preset threshold can be a pre-set conditional threshold used to determine whether two candidate boxes belong to the same chip region. Its value can be set based on the needs of the actual scenario. For example, if the preset threshold is 0.9, then if the pixel intersection-union ratio of the two candidate boxes calculated above is... If so, the target can be considered to have been double-confirmed, which means that the candidate box corresponding to the first pixel coordinate information and the candidate box corresponding to the second pixel coordinate information correspond to the same chip region.

[0064] Step 410: Determine the chip image of the chip region based on the first pixel coordinate information.

[0065] For example, to ensure edge sharpness, the sub-pixel coordinates, i.e., the first pixel coordinate information, obtained through template matching, can be used to determine the chip image of the aforementioned chip region. Based on this, the image to be processed is cropped to obtain each cropped chip image. This method, by fusing template matching and a deep learning object detection network to perform object detection on the image to be processed, significantly improves the robustness of the detection.

[0066] In one scenario, if in step 402, the entire image to be processed has been traversed... If no matching sliding window is found, it indicates that the template matching algorithm failed to identify the chip in the image. Therefore, a deep learning object detection network is used to identify the chip in the image, extracting the second pixel coordinates and corresponding confidence scores of the identified chips. The second pixel coordinates with confidence scores greater than a threshold (e.g., 0.7) are used to determine the chip image. This provides a fallback option in case template matching fails, using the high-confidence output of the deep learning object detection network to prevent missed detections.

[0067] In one exemplary embodiment, such as Figure 5 As shown, in step 106, the chip image is subjected to image correction based on the SAM model to obtain an initial corrected image. This may specifically include the following steps: Step 502: Based on the center coordinates of the chip image as the prompt point, input the chip image into the SAM model to obtain the mask prediction set of the chip image output by the SAM model.

[0068] Because chips may have physical placement deviations in the tray, chip images are highly likely to be rotated at arbitrary angles. If this is not corrected, it will have a disastrous impact on subsequent meshing and defect comparison. Therefore, a SAM model can be used to perform image correction on the chip images.

[0069] Specifically, the geometric center coordinates of the chip image can be extracted and used as "point cues" to input into the SAM model. The SAM model then uses its powerful image segmentation capabilities and generalization to output a set of mask predictions for the chip image.

[0070] Step 504: Determine the mask with the largest area from the mask prediction set.

[0071] In this embodiment, to ensure the accuracy of segmentation, the mask prediction set can be traversed, and the mask with the largest area can be selected as the candidate. The area of ​​the mask can be determined based on the number of non-zero pixels in the mask matrix.

[0072] Step 506: Determine whether the area of ​​the mask with the largest area meets the preset area threshold.

[0073] The preset area threshold can be a pre-configured conditional threshold for selecting the target mask. For example, the preset area threshold can be configured based on the pixel area of ​​the chip template image; for instance, the preset area threshold can be in the range of 85% to 110% of the pixel area of ​​the chip template image.

[0074] In this embodiment, if the area of ​​the largest mask meets the preset area threshold, i.e., it is within the threshold range, step 508 is executed; if the area of ​​the largest mask does not meet the preset area threshold, i.e., it is outside the threshold range, step 514 is executed.

[0075] Step 508: The mask with the largest area is determined as the target mask.

[0076] Specifically, if the area of ​​the mask with the largest area meets the preset area threshold, then the mask with the largest area is determined as the target mask.

[0077] Step 510: Convert the target mask into a grayscale binary image, perform morphological smoothing and rectangle fitting on the grayscale binary image, and obtain the corner coordinates and rotation angle of the fitted rectangle.

[0078] For example, the target mask defined above can first be converted into a standard binary matrix to obtain a grayscale binary image. However, since the mask may contain noise or pixel-level jagged edges (such as physical burrs on a chip), morphological smoothing can be used to smooth the mask edges (such as a closing operation that first dilates and then erodes), for example, by using a constructed... Structural elements (such as Gaussian kernel filter matrices) undergo a closing operation (dilation followed by erosion) to remove noise. Then, the smoothed mask is fitted using a rotating caliper algorithm or a minimum bounding rectangle algorithm to calculate the coordinates of the four corner points of the fitted rectangle and the rotation angle relative to the horizontal axis.

[0079] Step 512: Based on the rotation angle, construct a rotation moment to rotate the chip image to obtain an initial corrected image after graphic correction of the chip image.

[0080] Based on the rotation angles obtained above, a rotation matrix is ​​constructed to rotate the chip image, thereby obtaining the initial corrected image after graphic correction. For example, bilinear interpolation or cubic convolution interpolation algorithms can be used to rotate the pixel matrix of the chip image. After rotation, the image is cropped based on the calculated orthogonal corner boundaries, removing excess black fill areas after rotation, ultimately returning a strictly horizontally aligned corrected chip image, i.e., the initial corrected image. A correction illustration can be found in [reference needed]. Figure 6 The image on the left is the chip image, the image in the middle is the target mask, and the image on the right is the initial corrected image obtained by calculating the rotation angle of the target mask and mapping it to the original image for correction, so as to achieve pixel-level lossless correction.

[0081] Step 514: Update the position of the cue points, and input the chip image into the SAM model based on the updated cue points to obtain the mask prediction set of the chip image output by the SAM model.

[0082] If the area of ​​the largest mask determined above is not within the range due to noise or colloid damage caused by force majeure, i.e., it does not meet the preset area threshold, then the position of the prompt point is updated, for example, by spirally shifting it downwards or outwards by a preset number of pixels (such as 1 or 2 pixels) to redetermine the prompt point. Based on the updated prompt point, the SAM model is called again, that is, the chip image is input into the SAM model again based on the new prompt point to obtain the mask prediction set of the chip image output by the SAM model, and then the process returns to step 504, which is the step of determining the largest mask from the mask prediction set, until a target mask with an area that meets the expectation is obtained.

[0083] In an exemplary embodiment, the above method may further include: extracting and storing the spatial coordinate information of characters in the target chip image based on a bimodal binarization algorithm and a connected component detection algorithm, so as to facilitate subsequent defect detection.

[0084] For example, such as Figure 7 As shown, the spatial coordinate information of characters in the target chip image is extracted and stored based on the bimodal binarization algorithm and the connected component detection algorithm. Specifically, this may include: Step 702: If it is determined that there are no pins in the chip in the target chip image, the brightness distribution of the target chip image is statistically analyzed by histogram, and two peak brightness values ​​are extracted.

[0085] Because metals are highly reflective in industrial settings, they can introduce noise interference to character extraction. Therefore, in this embodiment, when extracting characters from the target chip image, the presence of pins in the chip image is first detected. Character extraction is only performed if it is determined that no pins are present.

[0086] For example, when it is determined that the chip in the target chip image does not have pins, the brightness distribution of the target chip image is statistically analyzed using a histogram to extract two peak brightness values. Since the chip's colloidal background is typically dark, while silkscreened characters are typically bright, such as... Figure 8 As shown, the histogram will show a clear bimodal distribution. By finding the local maxima of the derivative function, the main peak position (extremely low gray value segment) corresponding to the colloidal background and the secondary peak position (high gray value segment) corresponding to the silkscreen characters can be accurately extracted.

[0087] Step 704 determines the binarization threshold based on the average of the two peak brightness values, and determines the binarized image of the target chip image based on the binarization threshold.

[0088] For example, based on the two peak brightnesses extracted above, their average value is taken as the optimal binarization threshold, and then the binarized image of the target chip image is determined according to the binarization threshold. This method can adapt to the image contrast under different lighting conditions, avoid the problem of fixed threshold failure caused by the attenuation of light source brightness between batches, and has strong illumination robustness.

[0089] Step 706: After denoising the binarized image, perform connected component detection and determine the connected components other than the background connected components as character fields.

[0090] Specifically, after determining the binarized image of the target chip image based on the above steps, it can be denoised (e.g., little kernel structure denoising). Then, connected component detection is performed, and each detected non-background connected component (i.e., connected components other than background connected components) is regarded as a potential character domain.

[0091] Step 708: Determine and store the coordinates of the minimum bounding rectangle of each character field as the spatial coordinate information of the character in the target chip image.

[0092] Specifically, by calculating the minimum bounding rectangle of each character field and recording its top-left corner coordinates, width, and height, this coordinate information becomes the spatial coordinate information of the character in the target chip image. Storing this information allows subsequent defect detection algorithms to focus only on these specific areas, significantly improving detection efficiency and accuracy.

[0093] In one scenario, if pins are detected in the target chip image, the pin portion of the target chip image is removed to obtain a pin-free chip body image. Then, the brightness distribution of the chip body image is statistically analyzed using a histogram, and two peak brightness values ​​are extracted. A binarization threshold is determined based on the average of the two peak brightness values, and the binarized image of the chip body image is determined according to the binarization threshold. After denoising the binarized image, connected component detection is performed, and the connected components obtained (excluding the background connected components) are identified as character domains. The coordinates of the minimum bounding rectangle of each character domain are determined as the spatial coordinate information of the character in the target chip image and stored. It can be understood that the character domain extraction process in this embodiment is similar to... Figure 7 Similar to the above, I will not elaborate further here.

[0094] In an exemplary embodiment, detecting whether a chip in a target chip image has pins may specifically include: obtaining the pixel area of ​​a pixel whose brightness reaches a first brightness threshold based on the brightness of each pixel in the target chip image; determining that a chip in the target chip image has pins if the pixel area is greater than the pixel area threshold; and determining that a chip in the target chip image does not have pins if the pixel area is less than or equal to the pixel area threshold.

[0095] In industrial environments, metal pins often exhibit high brightness and high reflectivity. Therefore, based on the brightness of each pixel in the target chip image, a pre-set first brightness threshold can be used to iterate through each pixel. The pixel area of ​​pixels whose brightness reaches the first brightness threshold is counted and compared to a pixel area threshold. If the pixel area of ​​pixels reaching the first brightness threshold is greater than the pixel area threshold, it can be determined that the chip in the target chip image has pins. If the pixel area of ​​pixels reaching the first brightness threshold is less than or equal to the pixel area threshold, it can be determined that the chip in the target chip image does not have pins. The first brightness threshold can be a high grayscale value, such as greater than or equal to 250. The pixel area threshold can be a certain proportion of the pixel area of ​​the chip template image, such as 10% of the pixel area of ​​the chip template image.

[0096] In one exemplary embodiment, such as Figure 9 As shown, the pin portion of the target chip image is removed to obtain the main chip image after pin removal. Specifically, this process may include the following steps: Step 902: Binarize the target chip image based on the second brightness threshold to obtain the corresponding binarized image.

[0097] Since pins will produce strong high-intensity reflections under light sources, in order to avoid noise interference caused by this, when it is determined that pins exist in the target chip image, the pin portion of the target chip image can be eliminated to obtain the main chip image after pin elimination.

[0098] Specifically, when eliminating the pin area, the target chip image can first be binarized based on a second brightness threshold to obtain a corresponding binarized image. The second brightness threshold can be a threshold with a grayscale value of 200. After binarization, the pin area becomes white, as shown below. Figure 10 As shown on the left.

[0099] Step 904: Dilate the binarized image based on a preset morphological kernel to obtain a dilated binarized image.

[0100] The preset morphological kernel can be a pre-defined large-scale structural element (such as a 5x5 or 7x7 rectangular kernel). In this embodiment, the binarized image is dilated by using the preset morphological kernel, so that the discontinuous pin regions are connected into a sheet, thereby obtaining the dilated binarized image.

[0101] Step 906: Scan the dilated binarized image column by column from left to right and count the brightness percentage of each column of pixels.

[0102] Step 908: If the brightness ratio is greater than the set ratio threshold, determine that the corresponding column contains pins and record the column coordinates of the corresponding column.

[0103] Step 910: If the brightness ratio of the first column of pixels is detected to have decreased to less than the set ratio threshold, the first column of pixels is determined to be the boundary between the pin and the chip body.

[0104] For example, the expanded image is scanned column by column from left to right, and the proportion of white pixels in each column is counted. When the proportion of white pixels in a certain column (i.e., the ratio of the number of white pixels in a column to the number of pixels in that column) exceeds a set proportion threshold (e.g., 20%), that column is marked as a pin area; when the proportion of white pixels suddenly drops to below the proportion threshold in the first column of pixels, the position of that column of pixels is identified as the boundary between the pin and the chip body (i.e., the colloid part).

[0105] Step 912: Scan the expanded binarized image from right to left, from top to bottom, and from bottom to top to determine the boundaries between the pins and the chip body in each direction.

[0106] Repeat the scanning process described above, from right to left, from top to bottom, and from bottom to top, to determine the boundaries between the chip body and the pins in four directions.

[0107] Step 914: Crop the target chip image according to the determined boundaries to obtain the chip body image after removing the pins.

[0108] Based on the boundaries defined in the four directions, the target chip image is cropped to remove the peripheral pin areas, retaining only the main body of the chip, such as... Figure 10 As shown on the right. This effectively eliminates the interference of pin reflection and deformation on subsequent silkscreen character detection.

[0109] This embodiment automatically detects the presence or absence of pins based on the area characteristics of brightness distribution. When the presence of pins is determined, a row and column scanning method is used to physically truncate the influence of the pins. The entire algorithm does not require manual pre-setting of CAD dimension drawings for various chips, and can intelligently handle various leaded package forms such as SOP and QFP. It removes the pin areas that are extremely prone to sudden changes in light intensity, providing a clean data source for analyzing chip colloids.

[0110] In one scenario, to avoid interference from the high-gloss reflection of the pins under the light source on the character orientation correction, the pin portion in the initial correction image can be eliminated first in the manner described above before character orientation correction, so as to further improve the accuracy of character orientation correction.

[0111] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0112] Based on the same inventive concept, this application also provides a chip image correction apparatus for implementing the chip image correction method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more chip image correction apparatus embodiments provided below can be found in the limitations of the chip image correction method described above, and will not be repeated here.

[0113] In one exemplary embodiment, such as Figure 11 As shown, a chip image correction device is provided, including: an image acquisition module 1102, a target detection module 1104, an image correction module 1106, and an orientation correction module 1108, wherein: Image acquisition module 1102 is used to acquire an image to be processed based on a chip tray, wherein the image to be processed includes an image corresponding to at least one chip in the chip tray; The target detection module 1104 is used to perform target detection on the image to be processed using a template matching algorithm and a deep learning target detection network, and to determine at least one chip image in the image to be processed; The image correction module 1106 is used to perform image correction on the chip image based on the SAM model to obtain an initial corrected image after image correction. The orientation correction module 1108 is used to rotate the initial correction image based on multiple preset rotation angles to obtain multiple rotated images; and to determine the target rotated image with the highest matching similarity to the chip template image from the multiple rotated images. The target rotated image is used to represent the target chip image after character orientation correction. The chip template image is a standard image with standard character orientation corresponding to a flawless chip.

[0114] In an exemplary embodiment, the target detection module is specifically configured to: identify a chip in the image to be processed based on a template matching algorithm; determine the first pixel coordinate information of the chip when the chip is identified; perform target recognition on the chip in the image to be processed based on a deep learning target detection network, and extract the second pixel coordinate information of the identified chip; determine the pixel intersection-union ratio (IUU) based on the first pixel coordinate information and the second pixel coordinate information; determine the first pixel coordinate information and the second pixel coordinate information corresponding to the same chip region when the IUU is determined to be greater than a preset threshold; and determine the chip image of the chip based on the first pixel coordinate information.

[0115] In an exemplary embodiment, the target detection module is further configured to: if the chip is not identified, perform target recognition on the chip in the image to be processed based on a deep learning target detection network, extract the second pixel coordinate information and the corresponding confidence level of the identified chip; and if it is determined that the confidence level is greater than a confidence level threshold, determine the chip image of the chip based on the second pixel coordinate information.

[0116] In an exemplary embodiment, the image correction module is specifically configured to: input the chip image into a SAM model based on the center coordinates of the chip image as a cue point, and obtain a mask prediction set of the chip image output by the SAM model; determine the mask with the largest area from the mask prediction set, and determine the mask as the target mask if the area of ​​the mask meets a preset area threshold; convert the target mask into a grayscale binary image, perform morphological smoothing and rectangle fitting on the grayscale binary image, and obtain the corner coordinates and rotation angle of the fitted rectangle; construct a rotation moment based on the rotation angle and rotate the chip image to obtain an initial corrected image after image correction of the chip image.

[0117] In an exemplary embodiment, the image correction module is further configured to: update the position of the prompt point when it is determined that the area of ​​the mask does not meet the preset area threshold; input the chip image into the SAM model based on the updated prompt point to obtain the mask prediction set of the chip image output by the SAM model, and return to perform the step of determining the mask with the largest area from the mask prediction set.

[0118] In an exemplary embodiment, the orientation correction module is further configured to: calculate the SSD value between each rotated image and the chip template image respectively; and determine the rotated image with the smallest SSD value as the target rotated image with the highest matching similarity.

[0119] In one exemplary embodiment, the preset multiple rotation angles include 0 degrees, 90 degrees, 180 degrees, and 270 degrees.

[0120] In an exemplary embodiment, the device further includes a character extraction module for extracting and storing the spatial coordinate information of characters in the target chip image based on a bimodal binarization algorithm and a connected component detection algorithm.

[0121] In an exemplary embodiment, the character extraction module is further configured to: when it is determined that the chip in the target chip image does not have pins, extract two peak brightness values ​​by statistically analyzing the brightness distribution of the target chip image using a histogram; determine a binarization threshold based on the average of the two peak brightness values, and determine a binarized image of the target chip image according to the binarization threshold; denoise the binarized image and perform connected component detection, and determine the connected components other than the background connected components as character domains; determine and store the coordinates of the minimum bounding rectangle of each character domain as the spatial coordinate information of the character in the target chip image.

[0122] In an exemplary embodiment, the character extraction module is further configured to: when it is determined that the chip in the target chip image has pins, perform pin removal processing on the pin portion of the target chip image to obtain a chip body image after pin removal; extract two peak brightness values ​​by statistically analyzing the brightness distribution of the chip body image using a histogram; determine a binarization threshold based on the average of the two peak brightness values, and determine a binarized image of the chip body image according to the binarization threshold; denoise the binarized image and perform connected component detection, and determine the connected components other than the background connected components as character domains; determine and store the coordinates of the minimum bounding rectangle of each character domain as the spatial coordinate information of the character in the target chip image.

[0123] In one embodiment, the device further includes a pin identification module, configured to: obtain the pixel area of ​​a pixel whose brightness reaches a first brightness threshold based on the brightness of each pixel in the target chip image; determine that the chip in the target chip image has pins if the pixel area is greater than the pixel area threshold; and determine that the chip in the target chip image does not have pins if the pixel area is less than or equal to the pixel area threshold.

[0124] In one embodiment, the device further includes a pin removal module, configured to: binarize the target chip image based on a second brightness threshold to obtain a corresponding binarized image; dilate the binarized image based on a preset morphological kernel to obtain a dilated binarized image; scan the dilated binarized image column by column from left to right and count the brightness percentage of each column of pixels; if the brightness percentage is determined to be greater than a set ratio threshold, determine that the corresponding column contains pins and record the column coordinates of the corresponding column; if the brightness percentage is detected to have decayed to less than the set ratio threshold in the first column of pixels, determine that the first column of pixels is the boundary between the pin and the chip body; scan the dilated binarized image from right to left, from top to bottom, and from bottom to top respectively to determine the boundary between the pin and the chip body in each direction; and crop the target chip image according to the determined boundary to obtain a chip body image after pin removal.

[0125] Each module in the aforementioned chip image correction device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0126] In one exemplary embodiment, a computer device is provided, the internal structure of which can be as shown in the figure. Figure 12As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a chip image correction method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0127] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0128] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0129] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0130] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0131] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0132] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0133] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0134] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A chip image correction method, characterized in that, The method includes: Acquire a processing image based on a chip tray, wherein the processing image includes an image corresponding to at least one chip in the chip tray; A template matching algorithm and a deep learning object detection network are used to perform object detection on the image to be processed, and at least one chip image is identified in the image to be processed. The chip image is image-corrected based on the SAM model to obtain an initial corrected image. The initial corrected image is rotated based on multiple preset rotation angles to obtain multiple rotated images; The target rotated image with the highest matching similarity to the chip template image is determined from the plurality of rotated images. The target rotated image is used to characterize the target chip image after character orientation correction. The chip template image is a standard image with standard character orientation corresponding to a flawless chip.

2. The method according to claim 1, characterized in that, The step of using a template matching algorithm and a deep learning object detection network to perform object detection on the image to be processed, and determining at least one chip image in the image to be processed, includes: Based on the template matching algorithm, the chip in the image to be processed is identified, and when the chip is identified, the first pixel coordinate information of the chip is determined. The chip in the image to be processed is identified based on a deep learning object detection network, and the second pixel coordinate information of the identified chip is extracted. The pixel crossover ratio is determined based on the first pixel coordinate information and the second pixel coordinate information of the chip; If the pixel crossover ratio is determined to be greater than a preset threshold, the first pixel coordinate information and the second pixel coordinate information are determined to correspond to the same chip region; The chip image of the chip region is determined based on the first pixel coordinate information.

3. The method according to claim 2, characterized in that, The method further includes: If the chip is not identified, a deep learning target detection network is used to identify the chip in the image to be processed, and the second pixel coordinate information and corresponding confidence score of the identified chip are extracted. If the confidence level is determined to be greater than the confidence level threshold, the chip image of the chip is determined based on the second pixel coordinate information.

4. The method according to claim 1, characterized in that, The step of performing image correction on the chip image based on the SAM model to obtain an initial corrected image includes: Based on the center coordinates of the chip image as the cue point, the chip image is input into the SAM model to obtain the mask prediction set of the chip image output by the SAM model; The mask with the largest area is determined from the mask prediction set. If the area of ​​the mask meets a preset area threshold, the mask is determined as the target mask. The target mask is converted into a grayscale binary image, and morphological smoothing and rectangle fitting are performed on the grayscale binary image to obtain the corner coordinates and rotation angle of the fitted rectangle. Based on the rotation angle, a rotation moment is constructed to rotate the chip image, thereby obtaining an initial corrected image after graphic correction of the chip image.

5. The method according to claim 4, characterized in that, The method further includes: If the area of ​​the mask does not meet the preset area threshold, update the position of the prompt point; Based on the updated cue points, the chip image is input into the SAM model to obtain the mask prediction set of the chip image output by the SAM model, and then the step of determining the mask with the largest area from the mask prediction set is returned.

6. The method according to claim 1, characterized in that, The step of determining the target rotated image with the highest matching similarity to the chip template image from the plurality of rotated images includes: Calculate the SSD value between each rotated image and the chip template image; The rotated image with the smallest SSD value is determined as the target rotated image with the highest matching similarity.

7. The method according to claim 1, characterized in that, The preset rotation angles include 0 degrees, 90 degrees, 180 degrees and 270 degrees.

8. The method according to any one of claims 1 to 7, characterized in that, The method further includes: The spatial coordinate information of characters in the target chip image is extracted and stored based on the bimodal binarization algorithm and the connected component detection algorithm.

9. The method according to claim 8, characterized in that, The step of extracting and storing the spatial coordinate information of characters in the target chip image based on the bimodal binarization algorithm and the connected component detection algorithm includes: If it is determined that the chip in the target chip image does not have pins, the brightness distribution of the target chip image is statistically analyzed by histogram, and two peak brightness values ​​are extracted. A binarization threshold is determined based on the average of two peak brightness values, and a binarized image of the target chip image is determined based on the binarization threshold. After denoising the binarized image and performing connected component detection, the connected components obtained, excluding the background connected components, are determined as character fields. The coordinates of the minimum bounding rectangle of each character field are determined as the spatial coordinate information of the character in the target chip image and stored.

10. The method according to claim 9, characterized in that, The method further includes: If it is determined that the chip in the target chip image has pins, the pin portion in the target chip image is eliminated to obtain the chip body image after the pins are eliminated; The brightness distribution of the chip's main image is statistically analyzed using histograms, and two peak brightness values ​​are extracted. A binarization threshold is determined based on the average of two peak brightness values, and a binarized image of the chip's main image is determined based on the binarization threshold. After denoising the binarized image and performing connected component detection, the connected components obtained, excluding the background connected components, are determined as character fields. The coordinates of the minimum bounding rectangle of each character field are determined as the spatial coordinate information of the character in the target chip image and stored.

11. The method according to claim 10, characterized in that, The method further includes: Based on the brightness of each pixel in the target chip image, obtain the pixel area of ​​the pixel whose brightness reaches the first brightness threshold; If the pixel area is determined to be greater than the pixel area threshold, it is determined that the chip in the target chip image has pins; If the pixel area is determined to be less than or equal to the pixel area threshold, it is determined that the chip in the target chip image does not have pins.

12. The method according to claim 10, characterized in that, The step of removing pin portions from the target chip image to obtain a chip body image after pin removal includes: The target chip image is binarized based on the second brightness threshold to obtain the corresponding binarized image; The binarized image is dilated based on a preset morphological kernel to obtain a dilated binarized image. The dilated binarized image is scanned column by column from left to right, and the brightness ratio of each column of pixels is counted. If the brightness ratio is determined to be greater than a set ratio threshold, it is determined that the corresponding column contains pins, and the column coordinates of the corresponding column are recorded. If the brightness ratio of the first column of pixels is detected to have decreased to less than a set threshold, the first column of pixels is determined to be the boundary between the pin and the chip body. The dilated binarized image is scanned from right to left, top to bottom, and bottom to top to determine the boundaries between the pins and the chip body in each direction. The target chip image is cropped according to the determined boundary to obtain the main chip image after removing the pins.

13. A chip image correction device, characterized in that, The device includes: An image acquisition module is used to acquire an image to be processed based on a chip tray, wherein the image to be processed includes an image corresponding to at least one chip in the chip tray; The target detection module is used to perform target detection on the image to be processed using a template matching algorithm and a deep learning target detection network, and to determine at least one chip image in the image to be processed; The image correction module is used to perform image correction on the chip image based on the SAM model to obtain an initial corrected image after image correction. An orientation correction module is used to rotate the initial correction image based on multiple preset rotation angles to obtain multiple rotated images; and to determine the target rotated image with the highest matching similarity to the chip template image from the multiple rotated images. The target rotated image is used to represent the target chip image after character orientation correction. The chip template image is a standard image with standard character orientation corresponding to a flawless chip.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 12.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 12.

16. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 12.

Images