Wafer image splicing method and device and electronic equipment

Abstract

The invention relates to the technical field of semiconductors, and provides a wafer image splicing method, a wafer image splicing device and electronic equipment, which are used for improving the splicing precision of wafer images. In consideration of the influence of jitter of a motion system where a camera is located on wafer image splicing, a local wafer image sequence is divided into four quadrant image arrays by selecting three reference image arrays for a local wafer image array collected by the camera, and the three reference image arrays and the quadrant image arrays are respectively used as a whole. Splicing is independently carried out according to different splicing sequences far away from the center of a local wafer image array, so that the global splicing error is reduced, meanwhile, when two adjacent images in each whole are spliced pairwise according to the corresponding sequence, splicing in the transverse dimension and the longitudinal dimension can be carried out, the jitter influence of a motion system is further weakened, and the splicing efficiency is improved. And the local splicing error is reduced.

Description

Background Art

[0002] With the development of artificial intelligence (AI) technology, it has been widely used in various fields. In the field of wafer quality inspection in the semiconductor industry, due to the improvement of production technology, the size of the core particles is getting smaller and smaller, resulting in higher and higher requirements for wafer quality inspection. This requires the use of high-resolution and high-precision industrial cameras for visual quality inspection. Generally, the resolution of industrial cameras is required to reach an accuracy of 10um / pixel.

[0003] Due to the small size of the core particles, ordinary cameras cannot directly capture the complete image of the entire wafer. Industrial cameras are required to cooperate with the movement of the robotic arm to perform multi-angle shooting row by row and column by column, which requires the wafer image to be stitched.

[0004] Traditional stitching technology searches for local feature key points in a partial wafer image, aligns multiple partial wafer images by solving a transformation matrix, and then processes the overlapping boundaries to stitch together a complete wafer image. This stitching method is effective for large-sized cores or several partial wafer images. It is not applicable to small-sized cores, especially when there are too many cores and there are many shooting angles. This is because the wafer surface has a periodic pattern. During the process of matching feature key points, there will be periodic misalignment. When the size of the overlapping boundary is smaller than the period of the pattern, the local wafer image cannot be accurately aligned, thereby reducing the stitching quality.

[0005] Therefore, improving the stitching quality of wafer images plays an important role in the field of wafer quality inspection. Summary of the invention

[0006] The embodiments of the present application provide a wafer image stitching method, device and electronic device for improving the stitching accuracy of wafer images.

[0007] In a first aspect, an embodiment of the present application provides a wafer image stitching method, comprising:

[0008] Acquire a sequence of local wafer images taken by the camera according to a preset moving trajectory;

[0009] According to the number of rows and columns of the local wafer image sequence, a first reference image array, a second image reference array and a third image reference array are selected from the local wafer image sequence; wherein the first reference image array runs through the local wafer image array with the same row number or column number, and the second reference image array and the third reference image array are vertically distributed on both sides of the first reference image array;

[0010] dividing the remaining images in the local wafer image sequence into four quadrant image arrays according to the first reference image array, the second image reference array, and the third image reference array;

[0011] For the first reference image array, the second image reference array, the third image reference array and the four quadrant image arrays, two adjacent images are stitched in sequence from the horizontal and vertical dimensions in different stitching orders away from the center of the local wafer image array; wherein the two adjacent images contain overlapping areas.

[0012] The beneficial effects of the above technical solution are: taking into account the influence of the jitter of the motion system in which the camera is located on the wafer image stitching, the local wafer image sequence captured by the camera is divided into four quadrant image arrays by selecting three reference image arrays. The three reference image arrays and the quadrant image array are respectively taken as a whole and independently stitched in different stitching orders away from the center of the local wafer image array, thereby reducing the global stitching error of the local wafer image array. At the same time, when two adjacent images in each whole are stitched in pairs in the corresponding order, they are stitched in both the horizontal and vertical dimensions, thereby further weakening the influence of the jitter of the motion system and reducing the local stitching error of the local wafer image array.

[0013] Optionally, the first reference image array, the second image reference array, the third image reference array, and the four quadrant image arrays are respectively stitched in different stitching orders away from the center of the local wafer image array, and stitching two adjacent images in two dimensions horizontally and vertically in sequence, including:

[0014] For the first reference image array, taking the center point of the local wafer image array as the center image of the first reference image array, taking the center image as a reference and moving in a first stitching direction away from the center image, stitching two adjacent images in two dimensions, horizontally and vertically;

[0015] For any reference image array of the second reference image array and the third reference image array, sequentially stitching two adjacent images in a second stitching direction perpendicular to the first stitching direction and away from the central image in two dimensions, horizontally and vertically;

[0016] For any one of the four quadrant image arrays, two adjacent images are sequentially stitched in two dimensions, horizontally and vertically, in a third stitching direction perpendicular to the first stitching direction and the second stitching direction and away from the central image.

[0017] The beneficial effects of the above technical solution are as follows: since the second stitching direction is perpendicular to the first stitching direction and away from the central image, and the third stitching direction is perpendicular to the first stitching direction and the second stitching direction and away from the central image, therefore, stitching two adjacent images in the first reference image array in the first stitching direction in both horizontal and vertical dimensions can reduce the global stitching error of the local wafer image array in the first stitching direction, and stitching two adjacent images in the second reference image array and the third reference image array in both horizontal and vertical dimensions in the second stitching direction can reduce the global stitching error of the local wafer image array in the second stitching direction, and stitching two adjacent images in the four quadrant image arrays in both horizontal and vertical dimensions in the third stitching direction can reduce the global stitching error of the local wafer image array in the global coordinate system composed of the first stitching direction and the second stitching direction.

[0018] Optionally, sequentially splicing two adjacent images in horizontal and vertical dimensions includes:

[0019] Determining a reference image and a search image in the two adjacent images according to a target stitching direction corresponding to the image array where the two adjacent images are located;

[0020] Determining a search order in both horizontal and vertical dimensions according to the reference image;

[0021] Keeping the reference image stationary, the search image is gradually moved closer to the reference image in the horizontal and vertical dimensions according to the search order with a preset step length, until the preset search range of the corresponding dimension is reached and the movement is stopped; wherein, during each movement, the number of matching pixels in the region of interest of the reference image and the search image is counted, the range of the region of interest is less than or equal to the range of the overlapping region, and the preset search range is less than the range of the region of interest;

[0022] According to the number of matched pixels of the multiple movements corresponding to the horizontal and vertical dimensions, the movement distances in the corresponding dimensions are determined respectively;

[0023] The reference image and the search image are spliced ​​according to the corresponding moving distances in the horizontal and vertical dimensions.

[0024] The beneficial effect of the above technical solution is that during the local stitching process, the regions of interest of two adjacent images are matched pixel by pixel in four directions: up, down, left, and right, so as to reduce the local stitching errors of the two adjacent images in the horizontal and vertical directions respectively.

[0025] Optionally, after determining the movement distances in the corresponding dimensions according to the numbers of matched pixels of the multiple movements corresponding to the horizontal and vertical dimensions, the method further includes:

[0026] Keeping the reference image stationary, the search image is moved according to the moving distance corresponding to the next direction dimension in the search order;

[0027] The moved search image is gradually moved closer to the reference image according to the previous direction dimension in the search order with a preset step length until the search image reaches the preset search range of the previous direction dimension and stops moving;

[0028] Counting again the number of new matching pixels in the region of interest of the reference image and the search image at each movement, and re-determining the moving distance corresponding to the previous direction dimension according to the position corresponding to the maximum number of new matching pixels;

[0029] Replace the moving distance in the corresponding direction dimension with the re-determined moving distance.

[0030] The beneficial effect of the above technical solution is: considering the influence of periodic characteristics, there may be pixel matching errors in the search and matching process of local stitching. Therefore, adding a search check can further improve the local stitching accuracy of the wafer image.

[0031] Optionally, keeping the reference image stationary and gradually moving the search image closer to the reference image in both horizontal and vertical dimensions according to the search order with a preset step length until reaching a preset search range of a corresponding dimension and stopping the movement includes:

[0032] Keeping the reference image stationary, the search image is gradually moved closer to the reference image from a target direction dimension with a preset step length until it reaches a first preset search range and stops moving; wherein the target direction is horizontal or vertical;

[0033] The reference image is kept stationary, and the search image is gradually moved closer to the reference image from both sides perpendicular to the target direction with a preset step length until it reaches a second preset search range and stops moving.

[0034] The beneficial effect of the above technical solution is that when two adjacent images are locally stitched, one image is moved based on the other image, and a pixel-by-pixel matching search process is performed in the horizontal and vertical directions respectively, thereby reducing the cumulative error in a direction perpendicular to the direction when the robotic arm moves in one direction, and improving the local stitching accuracy.

[0035] Optionally, determining the movement distance in the corresponding dimension according to the number of matched pixels of the multiple movements corresponding to the horizontal and vertical dimensions respectively includes:

[0036] For any directional dimension of the horizontal and vertical dimensions, the number of matching pixels that have been moved multiple times in the dimension is sorted, and the position corresponding to the maximum number of matching pixels is determined as the moving distance corresponding to the directional dimension.

[0037] The beneficial effect of the above technical solution is: determining the moving distance by comparing the matching pixel numbers, ensuring that two adjacent images have the most identical pixels in the overlapping area in the horizontal and vertical directions, thereby improving the horizontal and vertical stitching accuracy.

[0038] Optionally, the adopting different stitching orders to stitch each two adjacent images respectively included in the first reference image array, the second image reference array, the third image reference array and the quadrant image array in two dimensions, horizontally and vertically, includes:

[0039] For any first partial wafer image in the first reference image array, determine the global coordinates of the first partial wafer image in the partial wafer image array according to the horizontal and vertical movement distances between the first partial wafer image and the first adjacent image, respectively; wherein the first adjacent image is an adjacent image of the first partial wafer image in the first stitching direction;

[0040] For any second partial wafer image in the second reference image array and the third reference image array, determine the global coordinates of the second partial wafer image in the partial wafer image array according to the horizontal and vertical movement distances between the second partial wafer image and the second adjacent image, respectively; wherein the second adjacent image is an adjacent image of the first partial wafer image in the second stitching direction;

[0041] For any third local wafer image in the four quadrant image arrays, the first global coordinate of the third local wafer image is determined according to the horizontal and vertical movement distances between the third local wafer image and the third adjacent image respectively, and the second global coordinate of the third local wafer image is determined according to the horizontal and vertical movement distances between the third local wafer image and the fourth adjacent image respectively, and the global coordinates of the third local wafer image in the local wafer image sequence are determined according to the first global coordinates and the second global coordinates; wherein the third adjacent image and the fourth adjacent image are adjacent images of the third local wafer image in the direction close to the center image of the array.

[0042] The beneficial effect of the above technical solution is: for each local wafer image, based on the stitching order corresponding to the image array where the local wafer image is located, its global coordinates are calculated in a corresponding manner, thereby reducing the stitching gap of each local wafer image during global stitching and improving the stitching accuracy.

[0043] Optionally, selecting a first reference image array, a second image reference array, and a third image reference array from the local wafer image sequence according to the number of rows and columns of the local wafer image sequence includes:

[0044] Determining a center point of the partial wafer image sequence according to the number of rows and columns of the partial wafer image sequence;

[0045] Using a plurality of local wafer images having the same row number or column number as the center point as the first reference image array;

[0046] A plurality of local wafer images on both sides perpendicular to the first reference image array and having the same column number as the center point are respectively used as the second reference image array and the third reference image array.

[0047] The beneficial effect of the above technical solution is: by dividing the local wafer image array in the above manner, the quadrant image array is divided more evenly, thereby ensuring that the splicing speed of the four quadrant image arrays as a whole is more balanced, thereby improving the splicing effect.

[0048] In a second aspect, an embodiment of the present application provides a wafer image stitching device, comprising:

[0049] An acquisition module is used to acquire a sequence of local wafer images taken by a camera according to a preset moving trajectory;

[0050] an array selection module, configured to select a first reference image array, a second image reference array, and a third image reference array from the local wafer image sequence according to the number of rows and columns of the local wafer image sequence; wherein the first reference image array runs through the local wafer image array with the same row number or column number, and the second reference image array and the third reference image array are vertically distributed on both sides of the first reference image array;

[0051] an array division module, configured to divide the remaining images in the local wafer image sequence into four quadrant image arrays according to the first reference image array, the second image reference array and the third image reference array;

[0052] An image stitching module is used to stitch two adjacent images in the horizontal and vertical dimensions in sequence in different stitching orders away from the center of the local wafer image array for the first reference image array, the second image reference array, the third image reference array and the four quadrant image arrays; wherein the two adjacent images include overlapping areas.

[0053] In a third aspect, an embodiment of the present application provides an electronic device, including a processor, a memory, and a communication interface, wherein the communication interface, the memory, and the processor are connected via a bus;

[0054] The communication interface is used to send and receive data;

[0055] The memory stores a computer program, and the processor executes the steps of any one of the wafer image stitching methods provided in the first aspect according to the computer program.

[0056] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed, the steps of any one of the wafer image stitching methods provided in the first aspect can be implemented.

[0057] The technical effects brought about by any one of the implementation methods in the second to fourth aspects can refer to the technical effects brought about by the corresponding implementation method in the first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0059] Figure 1 A flow chart of a wafer image stitching method provided in an embodiment of the present application;

[0060] Figure 2 A schematic diagram of dividing a local wafer image sequence provided in an embodiment of the present application;

[0061] Figure 3A Another schematic diagram of dividing a local wafer image sequence provided in an embodiment of the present application;

[0062] Figure 3B Another schematic diagram of dividing a local wafer image sequence provided in an embodiment of the present application;

[0063] Figure 4 A schematic diagram of splicing errors provided in an embodiment of the present application;

[0064] Figure 5 A flow chart of the global stitching method provided in the embodiment of the present application;

[0065] Figure 6 A schematic diagram of the global splicing sequence provided in an embodiment of the present application;

[0066] Figure 7 A flow chart of a local splicing method provided in an embodiment of the present application;

[0067] Figure 8 A schematic diagram of a local splicing sequence provided in an embodiment of the present application;

[0068] 9A to 9D Schematic diagram of four partial splicing situations provided in the embodiments of the present application;

[0069] Fig.10 A flow chart of a local splicing search and verification method provided in an embodiment of the present application;

[0070] Fig.11A and Fig. 11B The left and right and top and bottom partial splicing effect diagrams provided in the embodiments of the present application;

[0071] Fig.12 The effect diagram of global splicing provided by the embodiment of the present application;

[0072] Fig.13 A structural diagram of a wafer image stitching device provided in an embodiment of the present application;

[0073] Fig.14 A structural diagram of a positioning device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the technical solution of the present application, rather than all of the embodiments. Based on the embodiments recorded in the application documents, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the technical solution of the present application.

[0075] Based on the exemplary embodiments shown in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application. In addition, although the disclosure in this application is introduced according to one or several exemplary examples, it should be understood that each aspect of these disclosures can also constitute a complete technical solution separately.

[0076] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0077] The terms "first", "second", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented in an order other than those given in the diagrams or descriptions of the embodiments of this application.

[0078] In addition, the terms "include" and "have" and any variations thereof are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to those components expressly listed but may include other components not expressly listed or inherent to such products or devices.

[0079] The term "module" as used in this application refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0080] The following is an overview of the design concept of the embodiments of the present application in conjunction with application scenarios.

[0081] In the semiconductor wafer production process, the wafer must go through the process steps of substrate, photosensitive resin coating, photolithography, corrosion, passivation, silver (nickel) plating, laser cutting, splitting, screening, etc. before it can become a qualified wafer. Any process link in this process is crucial. If there is a slight abnormality, different types of defects will be left on the surface of the wafer. In order to control the production quality of wafers, the industry mainly uses optical quality inspection equipment based on automatic optical inspection (AOI) to collect wafer images and perform AI visual defect re-judgment on wafer images to improve the accuracy and efficiency of wafer quality inspection. Among them, the wafer image is mainly taken by the microscopic imaging system in conjunction with the movement of the robotic arm row by row and column by row to take multiple local wafer images (generally small-sized core particles will reach hundreds of images), and then the complete image is obtained by image stitching. Since the movement data of the robotic arm cannot be obtained in the AI ​​visual defect re-judgment scenario, both the stitching accuracy and the stitching efficiency need to be considered when stitching images.

[0082] At present, conventional stitching methods (such as template-based stitching, feature point-based stitching, phase correlation-based stitching, image matching-based stitching, etc.) cannot be effectively stitched in wafer scenes due to various periodic features existing in the wafer, especially for small-sized core particles, where the number of shooting angles is very large and the effective stitching area (i.e., overlapping area) of two images with adjacent angles is very small (generally less than 2%), resulting in large stitching errors. The stitching method for wafer images of large-sized core particles generally only considers horizontal stitching and is not applicable to the stitching of wafer images of small-sized core particles.

[0083] In view of this, the embodiment of the present application provides a wafer image stitching method, and provides a global and local fusion stitching method for wafer stitching scenarios with small core size, multiple shooting angles, and very small effective stitching areas between two adjacent images. Among them, three reference image arrays are selected for global stitching. Since the row number or column number of the first reference image array is the same, the second reference image array and the third reference image array are vertically distributed on both sides of the first reference image array. In this way, the local wafer image sequence collected by the camera can be divided into four quadrant image arrays. These three reference image arrays and quadrant image arrays are respectively regarded as a whole, and are independently stitched in different stitching orders away from the center of the local wafer image array, thereby reducing the impact of the jitter of the motion system where the camera is located on the wafer image stitching and reducing the global stitching error; in each whole, when two adjacent images are stitched locally, they are stitched from both the horizontal and vertical dimensions, thereby reducing the impact of the slight jitter of the motion system and improving the local stitching accuracy.

[0084] See also Figure 1 , is a flow chart of the wafer splicing method provided in an embodiment of the present application, and the process mainly includes the following steps:

[0085] S101: Acquire a sequence of local wafer images taken by a camera according to a preset moving trajectory.

[0086] In practical applications, when using a camera to collect wafer images from multiple perspectives, the camera is usually fixed on a motion system (such as a robotic arm) for regular image acquisition. Each time the camera moves, it captures an entire partial wafer image, thereby obtaining a wafer image sequence. Therefore, the camera's shooting trajectory is the movement trajectory of the motion system. The movement trajectory of the motion system is generally set and can be obtained or deduced through automated optical inspection (AOI).

[0087] It should be noted that the embodiments of the present application do not impose any restrictive requirements on the moving trajectory of the motion system. For example, the moving trajectory may be a Z-shaped trajectory, a linear moving trajectory, a row-by-row scanning trajectory, a column-by-column scanning trajectory, etc.

[0088] During the movement of the motion system, the machine will have a fixed overlap distance by default (approximately the size of a core particle). In fact, there will be an error of several pixels. In this way, when the camera takes local wafer images from two adjacent perspectives, there will be a small overlap area between the two adjacent images.

[0089] S102: Selecting a first reference image array, a second image reference array, and a third image reference array from the partial wafer image sequence according to the number of rows and columns of the partial wafer image sequence.

[0090] The first reference image array runs through the local wafer image array with the same row number or column number, and the second reference image array and the third reference image array are vertically distributed on both sides of the first reference image array.

[0091] In one example, the selection process of the three reference image arrays includes:

[0092] Firstly, according to the number of rows and columns of the partial wafer image sequence, the center point of the partial wafer image sequence is determined.

[0093] Then, a plurality of local wafer images having the same row number or column number as the center point are used as a first reference image array.

[0094] Finally, a plurality of local wafer images on both sides perpendicular to the first reference image array and having the same column number as the center point are respectively used as the second reference image array and the third reference image array.

[0095] Taking the Z-shaped moving trajectory as an example, the local wafer image sequence collected from 300 viewing angles is as follows: Figure 2 As shown in the figure, each number represents a local wafer image captured from a viewing angle. The resolution of each image is 5120*5120 (pixels). The whole is a wafer image. The number of cores on a wafer image can reach hundreds of thousands. The entire local wafer image sequence contains 19 rows and 19 columns of images, among which the local wafer image (sequence number 152) located at the 10th row and 10th column is the center point of the local image sequence. The 19 local wafer images in the 10th row (serial numbers 143-161) are used as the first reference image array, the 9 local wafer image arrays in the 10th column and vertically distributed on the upper side of the first reference image array (serial numbers 4, 14, 28, 42, 59, 76, 95, 114, 133) are used as the second reference image array, and the 9 local wafer image arrays in the 10th column and vertically distributed on the lower side of the first reference image array (serial numbers 171, 190, 209, 227, 245, 261, 275, 287, 296) are used as the third reference image array.

[0096] For example, Figure 3AAs shown, the 19 local wafer images in the 10th column (serial numbers 4, 14, 28, 42, 59, 76, 95, 114, 133, 152, 171, 190, 209, 227, 245, 261, 275, 287, 296) can also be used as the first reference image array, the 9 local wafer images in the 10th row (serial numbers 143-151) can be used as the second reference image array, and the 9 local wafer images in the 10th row (serial numbers 153-161) can be used as the third reference image array.

[0097] It should be noted that Figure 2 and Figure 3A These are just three examples of reference image array selection methods. Other selection methods may also be used, such as Figure 3B As shown, the second reference image array and the third reference image array are offset perpendicularly to two sides of the first reference image.

[0098] S103: Divide the remaining images in the local wafer image sequence into four quadrant image arrays according to the first reference image array, the second image reference array, and the third image reference array.

[0099] Since the row number or column number of the first reference image array is the same, the second reference image array and the third reference image array are vertically distributed on both sides of the first reference image array, so that the local wafer image sequence can be divided into four quadrant image arrays, such as Figures 2 to 3B shown.

[0100] Optionally, when the second reference image array and the third image reference array are respectively based on the center of the first reference image array and are vertically distributed on both sides of the center of the first reference image array with the same row number or column number, the local wafer image array can be evenly divided into four quadrant image arrays, thereby ensuring that the stitching speed of the four quadrant image arrays when taken as a whole is more balanced, thereby improving the stitching effect.

[0101] S104: For the first reference image array, the second image reference array, the third image reference array and the four quadrant image arrays, sequentially stitch two adjacent images in the horizontal and vertical dimensions in different stitching orders away from the center of the local wafer image array.

[0102] Due to the error in the movement of the motion system, if the splicing order is directly followed from left to right and then up and down, there will be a large horizontal splicing error between the rows, for example Figure 4 When the partial wafer images of the third row with serial numbers 21-34 are horizontally spliced ​​left and right, since each partial wafer image of serial numbers 21-34 also has a vertical displacement, each row of the image array will actually be misaligned up and down, so when the next row of images is spliced, cracks will be generated.

[0103] Therefore, in an embodiment of the present application, taking into account the impact of the jitter of the motion system in which the camera is located on the wafer image stitching, three reference image arrays are selected based on the coordinate system of the horizontal X-axis and the vertical Y-axis, wherein the row number or column number of the first reference image array is the same, and the second reference image array and the third reference image array are based on the center of the first reference image and are vertically distributed on both sides of the first reference image with the same column number or row number. In this way, the local wafer image sequence captured by the camera can be divided into four quadrant image arrays, and the three reference image arrays and the quadrant image arrays are respectively taken as a whole and independently stitched in different stitching orders, thereby reducing the global stitching errors in the horizontal and vertical directions.

[0104] Specifically, the splicing order used by the reference image array and the quadrant image array is as follows: Figure 5 As shown, it mainly includes the following:

[0105] S1041: For the first reference image array, the center point of the local wafer image array is used as the center image of the first reference image array, and two adjacent images are stitched in sequence from the horizontal and vertical dimensions in a first stitching direction away from the center image based on the center image.

[0106] by Figure 2 Taking the division method shown as an example, the serial numbers of the first reference image array are 143-161, wherein the central image is the local wafer image No. 152, which is used as the origin of the local wafer image array, and the first reference image array is divided into sub-array 1 (serial numbers are 143-152) and sub-array 2 (serial numbers are 152-161). The first stitching direction of sub-array 1 away from the central image is horizontally to the right, and the first stitching direction of sub-array 2 away from the central image is horizontally to the left, as shown in FIG. Figure 6 Therefore, every two adjacent images in subarray 1 are stitched horizontally to the right in sequence (i.e., 152 and 151 are stitched, 151 and 150 are stitched, 150 and 149 are stitched, 149 and 148 are stitched, 148 and 17 are stitched, 147 and 146 are stitched, 146 and 145 are stitched, 145 and 144 are stitched, and 144 and 143 are stitched), and every two adjacent images in subarray 2 are stitched horizontally to the left in sequence (i.e., 152 and 153 are stitched, 153 and 154 are stitched, 154 and 155 are stitched, 155 and 156 are stitched, 156 and 157 are stitched, 157 and 158 are stitched, 158 and 159 are stitched, 159 and 160 are stitched, and 160 and 161 are stitched). When each two adjacent images are spliced ​​together, they are spliced ​​from two dimensions: horizontal (i.e., horizontal X-axis) and vertical (i.e., vertical Y-axis).

[0107] If two adjacent images in the first reference image array are stitched according to the first stitching direction, at this time, for any first local wafer image in the first reference image array, the global coordinates of the first local wafer image in the local wafer image array are determined according to the horizontal and vertical movement distances between the first local wafer image and the first adjacent image respectively, wherein the first adjacent image is the adjacent image of the first local wafer image in the first stitching direction.

[0108] For example, taking the local wafer image with serial number 152 as the origin (0,0), the serial number of the first local wafer image is 153, and the first adjacent image adjacent to it is 152. When calculating the global coordinates of 153 in the local wafer image array, the X-axis coordinate of 153 is determined according to the lateral movement distance when 153 and 152 are spliced, and the Y-axis coordinate of 153 is determined according to the longitudinal movement distance when 153 and 152 are spliced.

[0109] For another example, taking the local wafer image with serial number 152 as the origin (0,0), the serial number of the first local wafer image is 151, and the first adjacent image adjacent to it is 150. When calculating the global coordinates of the local wafer image 151 in the local wafer image array, the X-axis coordinate of 151 is determined according to the lateral movement distance when 151 and 152 are spliced, and the Y-axis coordinate of 151 is determined according to the longitudinal movement distance when 151 and 152 are spliced.

[0110] By analogy, the position of any local wafer image in the first reference image array can be obtained by gradually relying on the positions of the local wafer images that have been determined nearby.

[0111] S1042: For any reference image array of the second reference image array and the third reference image array, sequentially stitch two adjacent images in a second stitching direction perpendicular to the first stitching direction and away from the central image in two dimensions, horizontally and vertically.

[0112] Still Figure 2 Taking the division method shown in FIG. 1 as an example, the sequence numbers of the second reference image array are 4, 14, 28, 42, 59, 76, 95, 114, and 133, and the second stitching direction perpendicular to the first stitching direction and away from the first stitching direction is vertically upward, such as Figure 6As shown. Therefore, every two adjacent images in the second reference image array are sequentially spliced ​​vertically upward (i.e., 152 and 133 are spliced, 133 and 124 are spliced, 124 and 95 are spliced, 95 and 76 are spliced, 76 and 59 are spliced, 59 and 42 are spliced, 42 and 28 are spliced, 28 and 14 are spliced, and 14 and 4 are spliced). The serial numbers of the third reference image array are 171, 190, 209, 227, 245, 261, 275, 287, and 296, and the second splicing direction perpendicular to the first splicing direction and away from the first splicing direction is vertically downward, as shown. Figure 6 As shown. Therefore, every two adjacent images in the third reference image array are sequentially spliced ​​vertically downward (i.e., 152 and 171 are spliced, 171 and 190 are spliced, 190 and 209 are spliced, 209 and 227 are spliced, 227 and 245 are spliced, 245 and 261 are spliced, 261 and 275 are spliced, 275 and 287 are spliced, and 287 and 296 are spliced). Wherein, when every two adjacent images are spliced, they are spliced ​​from two dimensions, horizontally (i.e., horizontal X axis) and vertically (i.e., vertical Y axis).

[0113] If the two adjacent images contained in the second reference image array and the third reference image array are respectively stitched according to the second stitching direction, for any second local wafer image of the two, the global coordinates of the second local wafer image in the local wafer image array are determined according to the horizontal and vertical movement distances between the second local wafer image and the second adjacent image respectively; wherein the second adjacent image is the adjacent image of the first local wafer image in the second stitching direction.

[0114] For example, taking the local wafer image with serial number 152 as the origin (0,0), the serial number of the second local wafer image is 133, and the serial number of the second adjacent image adjacent to it is 152. When calculating the global coordinates of the local wafer image 133 in the local wafer image array, the X-axis coordinate of 133 is determined according to the lateral movement distance when 133 and 152 are spliced, and the Y-axis coordinate of 133 is determined according to the longitudinal movement distance when 133 and 152 are spliced.

[0115] For another example, taking the local wafer image with serial number 152 as the origin (0,0), the serial number of the second local wafer image is 171, and the serial number of the second adjacent image is 152. When calculating the global coordinates of the local wafer image 171 in the local wafer image array, the X-axis coordinate of 171 is determined according to the lateral movement distance when 171 and 152 are spliced, and the Y-axis coordinate of 171 is determined according to the longitudinal movement distance when 171 and 152 are spliced.

[0116] By analogy, the position of any local wafer image in the second reference image array and the third reference image array can be obtained by gradually relying on the positions of the local wafer images that have been determined nearby.

[0117] S1043: For any one of the four quadrant image arrays, in a third stitching direction perpendicular to the first stitching direction and the second stitching direction and away from the central image, stitching two adjacent images in sequence from two dimensions, horizontally and vertically.

[0118] Still Figure 2 Taking the division method shown as an example, the sequence numbers of a quadrant image array are 5-13, 29-41, 60-75, 96-113, 134-14, and the third stitching direction perpendicular to the first stitching direction (horizontally right and horizontally left) and the second stitching direction (vertically upward and vertically upward) and away from the center image is the upper right direction, as shown in FIG. Figure 6 Therefore, any local wafer image in a quadrant image array can be stitched with two adjacent images close to the center image, such as 134 in the 9th row is stitched with 133 and 151, 135 is stitched with 134 and 150, 136 is stitched with 135 and 149, ..., 142 is stitched with 141 and 143, 113 in the 8th row is stitched with 114 and 134, 112 is stitched with 113 and 135, ..., 105 is stitched with 106 and 142, 96 in the 7th row is stitched with 95 and 113, 97 is stitched with 96 and 112, ..., 104 is stitched with 103 and 105, and so on. The serial numbers of the two-quadrant image array are 0-3, 15-27, 43-58, 77-94, 115-132, and the third stitching direction perpendicular to the first stitching direction (horizontally right and horizontally left) and the second stitching direction (vertically upward and vertically upward) and away from the center image is the upper left, such as Figure 6Therefore, any local wafer image in the two-quadrant image array can be stitched with two adjacent images close to the center image, such as 132 in the 9th row is stitched with 133 and 153, 131 is stitched with 132 and 154, 130 is stitched with 131 and 155, ..., 124 is stitched with 125 and 123, 115 in the 8th row is stitched with 114 and 132, 116 is stitched with 115 and 131, ..., 123 is stitched with 122 and 124, 94 in the 7th row is stitched with 95 and 115, 93 is stitched with 94 and 116, ..., 86 is stitched with 87 and 123, and so on. The sequence numbers of the three-quadrant image array are 162-170, 191-208, 228-244, 262-274, 288-295, and the third stitching direction perpendicular to the first stitching direction (horizontally right and horizontally left) and the second stitching direction (vertically upward and vertically upward) and away from the center image is the lower left, as shown in FIG. Figure 6 Therefore, any local wafer image in the three-quadrant image array can be stitched with two adjacent images close to the center image, such as 170 in the 11th row is stitched with 171 and 153, 169 is stitched with 170 and 154, 168 is stitched with 169 and 155, ..., 162 is stitched with 163 and 161, 191 in the 12th row is stitched with 190 and 170, 192 is stitched with 191 and 169, ..., 199 is stitched with 198 and 162, 208 in the 13th row is stitched with 209 and 191, 207 is stitched with 208 and 192, ..., 200 is stitched with 201 and 199, and so on. The sequence numbers of the four-quadrant image array are 172-189, 210-226, 246-260276-286, 297-299, and the third stitching direction perpendicular to the first stitching direction (horizontally right and horizontally left) and the second stitching direction (vertically upward and vertically upward) and away from the center image is the lower right, such as Figure 6Therefore, any local wafer image in the four-quadrant image array can be stitched with two adjacent images close to the center image, such as 172 in the 11th row stitched with 171 and 151, 173 stitched with 172 and 150, 174 stitched with 173 and 149, ..., 180 stitched with 179 and 143, 189 in the 12th row stitched with 190 and 172, 188 stitched with 189 and 173, ..., 181 stitched with 182 and 180, 210 in the 13th row stitched with 209 and 189, 211 stitched with 210 and 188, ..., 218 stitched with 217 and 181. When each two adjacent images are spliced ​​together, they are spliced ​​from two dimensions: horizontal (i.e., horizontal X-axis) and vertical (i.e., vertical Y-axis).

[0119] If the two adjacent images contained in each of the four quadrant image arrays are stitched respectively according to the third stitching direction, for any third local wafer image among the four, the first global coordinate of the third local wafer image is determined according to the moving distance between the third local wafer image and the third adjacent image in the horizontal and vertical directions respectively, and the second global coordinate of the third local wafer image is determined according to the moving distance between the third local wafer image and the fourth adjacent image in the horizontal and vertical directions respectively, and the global coordinate of the third local wafer image in the local wafer image sequence is determined according to the first global coordinate and the second global coordinate; wherein the third adjacent image and the fourth adjacent image are adjacent images of the third local wafer image in the direction close to the center image of the array.

[0120] For example, taking the local wafer image with serial number 152 as the origin (0,0), the serial number of the third local wafer image is 132, the serial number of the third adjacent image is 152, and the serial number of the fourth adjacent image is 153. When calculating the global coordinates of the local wafer image 132 in the local wafer image array, the first global coordinate of 132 is determined according to the lateral and longitudinal movement distances when 132 and 133 are spliced, and the second global coordinate of 132 is determined according to the lateral and longitudinal movement distances when 132 and 153 are spliced. Finally, the global coordinate of 132 is (first global coordinate + second global coordinate) / 2.

[0121] By analogy, the position of any local wafer image in the four quadrant image arrays can be obtained by gradually relying on the positions of the local wafer images that have been determined nearby.

[0122] After obtaining the positions of all partial wafer images in the global coordinate system, the global stitching of all images is completed. During the global stitching process, for each partial wafer image, based on the stitching order corresponding to the image array where each partial wafer image is located, its global coordinates are calculated in a corresponding manner, thereby reducing the stitching gap of each partial wafer image during global stitching and improving the stitching accuracy.

[0123] In an embodiment of the present application, after the local wafer image array is divided into three reference image arrays and four quadrant image arrays, each part is spliced ​​as a whole in a different splicing order away from the center of the local wafer image array, wherein two adjacent images in the first reference image array are spliced ​​in the horizontal and vertical dimensions in the first splicing direction, which can reduce the local wafer image array in the first splicing direction (such as Figure 2 The global stitching error on the horizontal X-axis in the middle) is obtained by stitching two adjacent images in the second reference image array and the third reference image array in the second stitching direction in the horizontal and vertical dimensions, thereby reducing the local wafer image array in the second stitching direction (such as Figure 2 The global stitching error on the vertical Y axis) is obtained by stitching two adjacent images in the four quadrant image arrays in the third stitching direction in the horizontal and vertical dimensions, thereby reducing the global coordinate system (such as Figure 2 The global stitching error in the XY coordinate system.

[0124] For each of the three reference image arrays and the four quadrant image arrays, when locally stitching two adjacent images in the corresponding stitching order, stitching is performed from the horizontal and vertical dimensions respectively, thereby further weakening the jitter effect of the motion system and reducing the local stitching error.

[0125] In practical applications, considering that the motion system where the camera is located is moving horizontally, there is actually still a vertical movement error. This vertical movement error will cause the stitching error to accumulate and increase. Therefore, each time two adjacent images are partially stitched, whether it is left-right stitching or top-down stitching, stitching in both horizontal and vertical dimensions will be performed. Among them, the idea of ​​horizontal and vertical stitching is the same, which is mainly to set the area of ​​interest of the two adjacent images (mainly the overlapping area), and then use one image as a reference to continuously move the area of ​​interest of the other image, so as to match the areas of interest of the adjacent perspective images pixel by pixel.

[0126] Taking the robotic arm as an example, when the camera collects a local wafer image array, the overlapping area of ​​two adjacent images is generally set with reference to the movement range of the robotic arm. The range of the set area of ​​interest does not exceed the range of the overlapping area, and the search range of pixel matching is smaller than the range of the area of ​​interest. In this way, the search range can be guaranteed not to be affected by periodic characteristics, and the calculation range can be prevented from being too large, resulting in low calculation efficiency.

[0127] Assuming that the general overlapping area ranges from at least 1.2 core sizes (1 core + core spacing area), the size of the region of interest of two adjacent images can be set to 0.8 core sizes, and the search range of the region of interest can be set to 0.6 core sizes.

[0128] In practical applications, the preset search range can be flexibly adjusted according to the size of the core particles, and combined with the resolution of the image, the preset search range can be converted into the number of searches.

[0129] For example, when the image resolution is 5120*5120 (pixels), the size of 0.6 core particles is 40 pixels, that is, the number of searches is 40.

[0130] It should be noted that the embodiments of the present application Figure 5 There is no restrictive requirement for the order of the global splicing steps of each component, for example, S1042 can be executed before S1041, or S1041 and S1042 can be executed in parallel.

[0131] During the local stitching process, the regions of interest of two adjacent images are matched pixel by pixel in the four directions of up, down, left, and right to ensure that the regions of interest contain as many identical pixels as possible, thereby reducing the local stitching errors of the two adjacent images in the horizontal and vertical directions. Figure 7 As shown, it mainly includes the following steps:

[0132] S1: Determine a reference image and a search image in two adjacent images according to a target stitching direction corresponding to an image array where the two adjacent images are located.

[0133] like Figure 8As shown, the target stitching direction of subarray 1 in the first reference image array is horizontally to the right. At this time, the left image of the two adjacent images is the reference image, and the right image is the search image, that is, the right image is stitched to the left image; the target stitching direction of subarray 2 in the first reference image array is horizontally to the left. At this time, the right image of the two adjacent images is the reference image, and the left image is the search image, that is, the left image is stitched to the right image; the target stitching direction of the second reference image array is vertically upward. At this time, the lower image of the two adjacent images is the reference image, and the upper image is the search image, that is, the upper image is stitched to the lower image; the target stitching direction of the third reference image array is vertically downward. At this time, the upper image of the two adjacent images is the reference image, and the lower image is the search image, that is, the lower image is stitched to the upper image; the target stitching direction of the quadrant image array is upper right. At this time, the left image of the two adjacent images on the left and right is the reference image, the right image is the search image, and the lower image of the two adjacent images on the top and bottom is stitched to the bottom image. The target splicing direction of the two-quadrant image array is the upper left. At this time, the right image of the two adjacent images is the reference image, the left image is the search image, and the lower image of the two adjacent images is the reference image, and the upper image is the search image, that is, the left image is spliced ​​to the right and the upper image is spliced ​​to the lower image; the target splicing direction of the three-quadrant image array is the lower left. At this time, the right image of the two adjacent images is the reference image, the left image is the search image, and the upper image of the two adjacent images is the reference image, and the lower image is the search image, that is, the left image is spliced ​​to the right and the lower image is spliced ​​to the upper image; the target splicing direction of the four-quadrant image array is the lower right. At this time, the left image of the two adjacent images is the reference image, the right image is the search image, and the upper image of the two adjacent images is the reference image, and the lower image is the search image, that is, the right image is spliced ​​to the left and the lower image is spliced ​​to the upper image.

[0134] S2: Determine the search order in the horizontal and vertical dimensions according to the reference image.

[0135] When the reference image is the left image or the right image, the horizontal search is performed first and then the vertical search; when the reference image is the upper image or the lower image, the vertical search is performed first and then the horizontal search.

[0136] S3: Keeping the reference image stationary, the search image is gradually moved closer to the reference image in both horizontal and vertical dimensions according to the search order with a preset step length until it reaches the preset search range of the corresponding dimension and stops moving.

[0137] The local stitching process of two adjacent images can be divided into four cases: stitching the right image to the left image, stitching the left image to the right image, stitching the upper image to the lower image, and stitching the lower image to the upper image.

[0138] like Fig.9AAs shown in the figure, it is a schematic diagram of stitching the right image onto the left image. The right image and the left image are two adjacent images. The filled area represents the region of interest of the right image and the left image. When stitching locally, the region of interest of the right image is horizontally moved by X toward the left image. However, since the robotic arm will also move vertically when moving horizontally, the regions of interest of the left and right images will also have an offset Y in the vertical direction. To ensure accurate stitching, the right image should be moved up or down by Y to make the regions of interest of the right image overlap with those of the left image.

[0139] like Fig. 9B As shown in the figure, it is a schematic diagram of stitching the left image to the right image. The right image and the left image are two adjacent images. The filled area represents the region of interest of the right image and the left image. When stitching locally, the region of interest of the left image is horizontally moved by X to the right image. However, since the robot arm will also move vertically when moving horizontally, the regions of interest of the left image and the right image will also have an offset Y in the vertical direction. To ensure accurate stitching, the left image should be moved up or down by Y to make the regions of interest of the left image and the right image overlap.

[0140] like Fig. 9C As shown in the figure, it is a schematic diagram of stitching the upper image to the lower image. The lower image and the upper image are two adjacent images. The filled area represents the region of interest of the upper image and the lower image. When stitching locally, the region of interest of the upper image is moved vertically to the lower image by Y. However, since the robot arm will also move horizontally when moving vertically, the regions of interest of the upper image and the lower image will also have an offset X in the horizontal direction. To ensure accurate stitching, the upper image should be moved to the right or left by X to make the regions of interest of the upper image and the lower image overlap.

[0141] like Fig.9D As shown in the figure, it is a schematic diagram of stitching the lower image to the upper image. The lower image and the upper image are two adjacent images. The filled area represents the region of interest of the upper image and the lower image. When stitching locally, the region of interest of the lower image is moved vertically to the upper image by Y. However, since the robotic arm will also move horizontally when moving vertically, the regions of interest of the upper image and the lower image will also have an offset X in the horizontal direction. To ensure accurate stitching, the lower image should be moved left or right by X to make the region of interest of the lower image overlap with that of the upper image.

[0142] Therefore, taking into account the motion error of the motion system, the embodiments of the present application perform local stitching of two adjacent images from both the horizontal and vertical dimensions. During stitching, the reference image is kept stationary, and the search image (i.e., the region of interest of the search image) is gradually approached to the reference image (i.e., the region of interest of the reference image) from the target direction dimension with a preset step length until it reaches the first preset search range and stops moving. Then, the reference image is kept stationary, and the search image (i.e., the region of interest of the search image) is gradually approached to the reference image (i.e., the region of interest of the reference image) from both sides perpendicular to the target direction with a preset step length, respectively, until it reaches the second preset search range and stops moving. Optionally, the preset step length is 1 pixel.

[0143] The preset search range is smaller than the range of the region of interest, and the number of matching pixels in the region of interest of the reference image and the search image is counted during each movement.

[0144] When performing local stitching of two adjacent images, the embodiment of the present application moves one image based on the other image, and performs a pixel-by-pixel matching search process in the horizontal and vertical directions respectively, thereby reducing the cumulative error in a direction perpendicular to the direction when the robotic arm moves in one direction, and improving the local stitching accuracy.

[0145] S4: Determine the movement distance in the corresponding dimension according to the number of matching pixels of the multiple movements corresponding to the horizontal and vertical dimensions respectively.

[0146] During the local stitching process, for any horizontal or vertical dimension, the number of matching pixels obtained by multiple movements in that dimension is sorted, and the position corresponding to the maximum number of matching pixels is determined as the movement distance corresponding to that dimension.

[0147] For example, taking the puzzle of the right image to the left image as an example, the right image is moved horizontally to the left by 1 pixel, and the pixel values ​​at the same position in the region of interest of the left image and the right image are counted to see if they are consistent. If they are consistent, they are recorded as 1, and if they are inconsistent, they are recorded as 0, so as to obtain the number of matching pixels in the region of interest, and the above operation is repeated. When the preset search range corresponding to the horizontal direction is reached, the horizontal search is stopped and the number of matching pixels counted each time is sorted, and the position corresponding to the maximum number of matching pixels is determined as the horizontal moving distance of the right image to the left image and saved. Further, the right image is moved downward by 1 pixel, and the number of matching pixels is also determined once, and the above operation is repeated. When the half of the preset search range corresponding to the vertical direction is reached, the downward search is stopped. Similarly, the right image is moved upward by 1 pixel, and the number of matching pixels is also determined once. The above operation is repeated. When the half of the preset search range corresponding to the vertical direction is reached, the upward search is stopped. Then, the number of matching pixels counted downward and upward is sorted, and the position corresponding to the maximum number of matching pixels is determined as the vertical moving distance of the right image to the left image and saved.

[0148] For another example, take the upper image and the lower image as an example, move the upper image vertically downward by 1 pixel, count the pixel values ​​of the same position in the region of interest of the upper image and the lower image once, if they are consistent, record them as 1, if they are inconsistent, record them as 0, so as to obtain the number of matching pixels in the region of interest, repeat the above operation, when reaching the preset search range corresponding to the vertical direction, stop the vertical search and sort the number of matching pixels counted each time, determine the position corresponding to the maximum number of matching pixels as the vertical moving distance of the upper image to the lower image and save it. Further, move the upper image to the right by 1 pixel, count the number of matching pixels in the region of interest once, repeat the above operation, when reaching half of the preset search range corresponding to the horizontal direction, stop searching to the right, similarly, move the upper image to the left by 1 pixel, also count the number of matching pixels in the region of interest once, repeat the above operation, when reaching half of the preset search range corresponding to the vertical direction, stop searching to the left, then sort the number of matching pixels counted to the left and right, determine the position corresponding to the maximum number of matching pixels as the horizontal moving distance of the upper image to the lower image and save it.

[0149] In the embodiment of the present application, the moving distance is determined by comparing the matching pixel numbers to ensure that two adjacent images have the most identical pixels in the overlapping area in the horizontal and vertical directions, thereby improving the stitching accuracy in the horizontal and vertical directions.

[0150] Considering the influence of the periodic characteristics of the core particles, if the moving distance in the target direction (horizontal or vertical) is calculated only once, there may be pixel matching errors. Therefore, in some embodiments, after determining the moving distance in the target direction, an additional matching verification process is added. The specific process is as follows: Fig.10 As shown, it mainly includes the following steps:

[0151] S401: Keep the reference image stationary and move the search image by a moving distance corresponding to the next direction dimension in the search order.

[0152] Still taking the example of piecing the right image together with the left image, the right image is the search image, the left image is the reference image, and the search order is horizontal search first and then vertical search. Therefore, after determining the vertical moving distance of the right image, move the right image vertically (upward or downward) according to the vertical moving distance.

[0153] S402: The moved search image is gradually moved closer to the reference image according to the previous direction dimension in the search order with a preset step length until the search image reaches the preset search range of the previous direction dimension and stops moving.

[0154] Still taking the example of piecing the right image together with the left image, after the right image is moved vertically, the left image is kept stationary, and the right image is moved horizontally to the left in steps of 1 pixel until it reaches the corresponding preset search range in the horizontal direction.

[0155] S403: Counting the number of new matching pixels in the region of interest of the reference image and the search image again at each movement, and re-determining the movement distance corresponding to the previous direction dimension according to the position corresponding to the maximum number of new matching pixels.

[0156] Still taking the example of piecing together the right image and the left image, each time the right image moves toward the left image, it is re-determined whether the pixel values ​​at the same position in the region of interest of the right and left images are consistent, and a new number of matching pixels is obtained. The position corresponding to the largest one in the new number of matching pixels is re-determined as the horizontal (horizontal to the left) movement distance of the right image toward the left image.

[0157] S404: Replace the moving distance in the corresponding direction dimension with the re-determined moving distance.

[0158] Still taking the puzzle of the right picture to the left picture as an example, the re-determined horizontal leftward movement distance is determined as the final movement distance of the horizontal dimension.

[0159] In the embodiment of the present application, considering the influence of periodic characteristics, there may be pixel matching errors in the search and matching process of local stitching. Therefore, a search check is added to further improve the local stitching accuracy of the wafer image.

[0160] S5: splicing the reference image and the search image according to the corresponding moving distances in the horizontal and vertical dimensions.

[0161] The search image is horizontally moved leftward or rightward according to the lateral movement distance, and the search image is vertically moved upward or downward according to the longitudinal movement distance, so that the regions of interest of two adjacent images overlap.

[0162] like Fig.11A The following is the effect of partial splicing of the left and right pictures. Fig. 11B This is the effect of partial splicing of the above and below pictures. Fig.12 This is a global stitching effect diagram of the reference image array and quadrant image array.

[0163] See also Fig.13 The wafer image stitching device includes an acquisition module 1301, an array selection module 1302, an array division module 1303 and an image stitching module 1304, wherein:

[0164] An acquisition module 1301 is used to acquire a sequence of local wafer images taken by a camera according to a preset moving trajectory;

[0165] An array selection module 1302 is used to select a first reference image array, a second image reference array, and a third image reference array from the local wafer image sequence according to the number of rows and columns of the local wafer image sequence; wherein the first reference image array runs through the local wafer image array with the same row number or column number, and the second reference image array and the third reference image array are vertically distributed on both sides of the first reference image array;

[0166] An array division module 1303 is used to divide the remaining images in the local wafer image sequence into four quadrant image arrays according to the first reference image array, the second image reference array and the third image reference array;

[0167] The image stitching module 1304 is used to stitch two adjacent images in the horizontal and vertical dimensions in a different stitching order away from the center of the local wafer image array for the first reference image array, the second image reference array, the third image reference array and the four quadrant image arrays; wherein the two adjacent images contain overlapping areas.

[0168] Optionally, the image stitching module 1304 is specifically used for:

[0169] For the first reference image array, the center point of the local wafer image array is used as the center image of the first reference image array, and two adjacent images are stitched in sequence in the horizontal and vertical dimensions in a first stitching direction away from the center image based on the center image;

[0170] For any reference image array of the second reference image array and the third reference image array, in a second stitching direction perpendicular to the first stitching direction and away from the central image, stitching two adjacent images in sequence from two dimensions, horizontally and vertically;

[0171] For any one of the four quadrant image arrays, two adjacent images are sequentially spliced ​​in the horizontal and vertical dimensions in a third splicing direction that is perpendicular to the first splicing direction and the second splicing direction and away from the central image.

[0172] Optionally, the image stitching module 1304 is specifically used for:

[0173] Determine a reference image and a search image in two adjacent images according to a target stitching direction corresponding to an image array where the two adjacent images are located;

[0174] According to the reference image, determining the search order in the horizontal and vertical dimensions;

[0175] Keeping the reference image stationary, the search image is gradually moved closer to the reference image in the horizontal and vertical dimensions in a preset step size according to the search order, until the preset search range of the corresponding dimension is reached and the movement is stopped; wherein, during each movement, the number of matching pixels in the region of interest of the reference image and the search image is counted, the range of the region of interest is less than or equal to the range of the overlapping area, and the preset search range is less than the range of the region of interest;

[0176] According to the number of matched pixels of the multiple movements corresponding to the horizontal and vertical dimensions, the movement distances in the corresponding dimensions are determined respectively;

[0177] The reference image and the search image are spliced ​​according to the corresponding moving distances in the horizontal and vertical dimensions.

[0178] Optionally, the image stitching module 1304 is specifically used for:

[0179] Keep the reference image stationary, and move the search image by the moving distance corresponding to the next direction dimension in the search order;

[0180] The search image after movement is gradually moved closer to the reference image according to the previous direction dimension in the search order with a preset step length until it reaches the preset search range of the previous direction dimension and stops moving;

[0181] The number of new matching pixels in the region of interest of the reference image and the search image is counted again at each movement, and the moving distance corresponding to the previous direction dimension is re-determined according to the position corresponding to the maximum number of new matching pixels;

[0182] Replace the moving distance in the corresponding direction dimension with the re-determined moving distance.

[0183] Optionally, the image stitching module 1304 is specifically used for:

[0184] Keeping the reference image stationary, the search image is gradually moved closer to the reference image from the target direction dimension with a preset step length until it reaches a first preset search range and stops moving; wherein the target direction is horizontal or vertical;

[0185] The reference image is kept stationary, and the search image is gradually moved closer to the reference image from both sides perpendicular to the target direction with a preset step length until it reaches the second preset search range and stops moving.

[0186] Optionally, the image stitching module 1304 is specifically used for:

[0187] For any directional dimension of the horizontal and vertical dimensions, the number of matching pixels that have been moved multiple times in the dimension is sorted, and the position corresponding to the maximum number of matching pixels is determined as the moving distance corresponding to the directional dimension.

[0188] Optionally, the image stitching module 1304 is specifically used for:

[0189] For any first partial wafer image in the first reference image array, determine the global coordinates of the first partial wafer image in the partial wafer image array according to the horizontal and vertical movement distances between the first partial wafer image and the first adjacent image, wherein the first adjacent image is an adjacent image of the first partial wafer image in the first stitching direction;

[0190] For any second partial wafer image in the second reference image array and the third reference image array, determine the global coordinates of the second partial wafer image in the partial wafer image array according to the horizontal and vertical movement distances between the second partial wafer image and the second adjacent image, wherein the second adjacent image is an adjacent image of the first partial wafer image in the second stitching direction;

[0191] For any third local wafer image in the four quadrant image arrays, the first global coordinate of the third local wafer image is determined according to the moving distances between the third local wafer image and the third adjacent image in the lateral and longitudinal directions respectively, and the second global coordinate of the third local wafer image is determined according to the moving distances between the third local wafer image and the fourth adjacent image in the lateral and longitudinal directions respectively, and the global coordinates of the third local wafer image in the local wafer image sequence are determined according to the first global coordinates and the second global coordinates; wherein the third adjacent image and the fourth adjacent image are adjacent images of the third local wafer image in the direction close to the center image of the array.

[0192] Optionally, the array selection module 1302 is specifically used for:

[0193] Determine the center point of the local wafer image sequence according to the number of rows and columns of the local wafer image sequence;

[0194] using a plurality of local wafer images having the same row number or column number as the center point as a first reference image array;

[0195] A plurality of local wafer images having the same column number or row number as the center point on both sides perpendicular to the first reference image array are respectively used as the second reference image array and the third reference image array.

[0196] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing this application, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

[0197] After introducing the wafer image stitching method and apparatus according to an exemplary embodiment of the present application, next, a positioning device according to another exemplary embodiment of the present application is introduced.

[0198] Those skilled in the art will appreciate that various aspects of the present application may be implemented as a system, method or program product. Therefore, various aspects of the present application may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to as "circuit", "module" or "system" herein.

[0199] Based on the same inventive concept as the above method embodiment, the electronic device provided in the embodiment of the present application may be a wafer quality inspection device. In this embodiment, the structure of the electronic device may be as follows: Fig.14 As shown, it includes a processor 1401, a memory 1402 and a communication interface 1403;

[0200] The communication interface 1403 is used to send and receive data;

[0201] The memory 1402 stores a computer program, and the processor 1401 executes the steps of any one of the wafer image stitching methods in the above embodiments according to the computer program.

[0202] In the embodiment of the present application, the memory 1402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, and programs required to run the instant messaging function, etc.; the data storage area may store various instant messaging information and operation instruction sets, etc. The memory 1402 may be a volatile memory (volatile memory), such as a random-access memory (RAM); the memory may also be a non-volatile memory (non-volatile memory), such as a read-only memory, a flash memory (flash memory), a hard disk drive (HDD) or a solid-state drive (SSD); or the memory 1402 may be any other medium that can be used to carry or store a desired computer program in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 1402 may be a combination of the above memories.

[0203] The processor 1401 may include one or more central processing units (CPU), GPU or a digital processing unit, etc.

[0204] In the embodiment of the present application, the specific connection medium between the communication interface 1403, the memory 1402 and the processor 1401 is not limited. In the embodiment of the present application, the bus 1404 between the communication interface 1403, the memory 1402 and the processor 1401 is Fig.14The connections between the other components are only for illustration and are not intended to be limiting. The bus 1404 can be divided into an address bus, a data bus, a control bus, etc. For ease of description, Fig.14 The diagram shows that only one thick line is used, but this does not mean that there is only one bus or only one type of bus.

[0205] It should be noted that Fig.14 It is only the necessary equipment for the electronic device to realize the wafer image stitching in the embodiment of the present application. Not shown, the electronic device may also include hardware of conventional electronic devices such as a display screen, a camera, a power supply, and buttons.

[0206] The embodiment of the present application also provides a computer-readable storage medium for storing some instructions, which, when executed, can complete the steps of any one of the wafer image stitching methods in the aforementioned embodiments.

[0207] An embodiment of the present application also provides a computer program product for storing a computer program, wherein the computer program is used to execute the steps of any one of the wafer image stitching methods in the aforementioned embodiments.

[0208] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0209] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0210] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0211] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0212] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A wafer image stitching method, characterized in that: include: Acquire a sequence of local wafer images taken by the camera according to a preset moving trajectory; According to the number of rows and columns of the local wafer image sequence, a first reference image array, a second image reference array and a third image reference array are selected from the local wafer image sequence; wherein the first reference image array runs through the local wafer image array with the same row number or column number, and the second reference image array and the third reference image array are vertically distributed on both sides of the first reference image array; dividing the remaining images in the local wafer image sequence into four quadrant image arrays according to the first reference image array, the second image reference array, and the third image reference array; For the first reference image array, the second image reference array, the third image reference array and the four quadrant image arrays, two adjacent images are stitched in sequence from the horizontal and vertical dimensions in different stitching orders away from the center of the local wafer image array; wherein the two adjacent images contain overlapping areas.

2. The method according to claim 1, characterized in that The first reference image array, the second image reference array, the third image reference array, and the four quadrant image arrays are respectively stitched in different stitching orders away from the center of the local wafer image array, and two adjacent images are stitched in sequence from two dimensions, horizontally and vertically, including: For the first reference image array, taking the center point of the local wafer image array as the center image of the first reference image array, taking the center image as a reference and moving in a first stitching direction away from the center image, stitching two adjacent images in sequence from two dimensions, horizontally and vertically; For any reference image array of the second reference image array and the third reference image array, sequentially stitching two adjacent images in a second stitching direction perpendicular to the first stitching direction and away from the central image in two dimensions, horizontally and vertically; For any one of the four quadrant image arrays, two adjacent images are sequentially stitched in the horizontal and vertical dimensions in a third stitching direction perpendicular to the first stitching direction and the second stitching direction and away from the central image.

3. The method according to claim 2, characterized in that The step of sequentially splicing two adjacent images in horizontal and vertical dimensions includes: Determining a reference image and a search image in the two adjacent images according to a target stitching direction corresponding to the image array where the two adjacent images are located; Determining a search order in both horizontal and vertical dimensions according to the reference image; Keeping the reference image stationary, the search image is gradually moved closer to the reference image in the horizontal and vertical dimensions according to the search order with a preset step length, until the preset search range of the corresponding dimension is reached and the movement is stopped; wherein, during each movement, the number of matching pixels in the region of interest of the reference image and the search image is counted, the range of the region of interest is less than or equal to the range of the overlapping region, and the preset search range is less than the range of the region of interest; According to the number of matched pixels of the multiple movements corresponding to the horizontal and vertical dimensions, the movement distances in the corresponding dimensions are determined respectively; The reference image and the search image are spliced ​​according to the corresponding moving distances in the horizontal and vertical dimensions.

4. The method according to claim 3, characterized in that After determining the movement distances in the corresponding dimensions according to the numbers of matched pixels of the multiple movements corresponding to the horizontal and vertical dimensions, the method further includes: Keeping the reference image stationary, the search image is moved according to the moving distance corresponding to the next direction dimension in the search order; The moved search image is gradually moved closer to the reference image according to the previous direction dimension in the search order with a preset step length until the search image reaches the preset search range of the previous direction dimension and stops moving; Counting again the number of new matching pixels in the region of interest of the reference image and the search image at each movement, and re-determining the moving distance corresponding to the previous direction dimension according to the position corresponding to the maximum number of new matching pixels; Replace the moving distance in the corresponding direction dimension with the re-determined moving distance.

5. The method according to claim 3, characterized in that The step of keeping the reference image stationary and gradually moving the search image closer to the reference image in both horizontal and vertical dimensions according to the search order with a preset step length until reaching a preset search range of the corresponding dimension and stopping the movement includes: Keeping the reference image stationary, the search image is gradually moved closer to the reference image from a target direction dimension with a preset step length until it reaches a first preset search range and stops moving; wherein the target direction is horizontal or vertical; The reference image is kept stationary, and the search image is gradually moved closer to the reference image from both sides perpendicular to the target direction with a preset step length until it reaches a second preset search range and stops moving.

6. The method according to claim 3, characterized in that The step of determining the moving distance in the corresponding dimension according to the number of matched pixels of the multiple movements corresponding to the horizontal and vertical dimensions respectively includes: For any directional dimension of the horizontal and vertical dimensions, the number of matching pixels that have been moved multiple times in the dimension is sorted, and the position corresponding to the maximum number of matching pixels is determined as the moving distance corresponding to the directional dimension.

7. The method according to any one of claims 2 to 6, characterized in that The method of respectively using different stitching orders to stitch each two adjacent images contained in the first reference image array, the second image reference array, the third image reference array and the quadrant image array in two dimensions, horizontally and vertically, includes: For any first partial wafer image in the first reference image array, determine the global coordinates of the first partial wafer image in the partial wafer image array according to the horizontal and vertical movement distances between the first partial wafer image and the first adjacent image, respectively; wherein the first adjacent image is an adjacent image of the first partial wafer image in the first stitching direction; For any second partial wafer image in the second reference image array and the third reference image array, determine the global coordinates of the second partial wafer image in the partial wafer image array according to the horizontal and vertical movement distances between the second partial wafer image and the second adjacent image, respectively; wherein the second adjacent image is an adjacent image of the first partial wafer image in the second stitching direction; For any third local wafer image in the four quadrant image arrays, the first global coordinate of the third local wafer image is determined according to the horizontal and vertical movement distances between the third local wafer image and the third adjacent image respectively, and the second global coordinate of the third local wafer image is determined according to the horizontal and vertical movement distances between the third local wafer image and the fourth adjacent image respectively, and the global coordinates of the third local wafer image in the local wafer image sequence are determined according to the first global coordinates and the second global coordinates; wherein the third adjacent image and the fourth adjacent image are adjacent images of the third local wafer image in the direction close to the center image of the array.

8. The method according to any one of claims 2 to 6, characterized in that: The selecting a first reference image array, a second image reference array, and a third image reference array from the local wafer image sequence according to the number of rows and columns of the local wafer image sequence comprises: Determining a center point of the partial wafer image sequence according to the number of rows and columns of the partial wafer image sequence; Using a plurality of local wafer images having the same row number or column number as the center point as the first reference image array; A plurality of local wafer images on both sides perpendicular to the first reference image array and having the same column number as the center point are respectively used as the second reference image array and the third reference image array.

9. A wafer image stitching device, characterized in that: include: An acquisition module is used to acquire a sequence of local wafer images taken by a camera according to a preset moving trajectory; an array selection module, configured to select a first reference image array, a second image reference array, and a third image reference array from the local wafer image sequence according to the number of rows and columns of the local wafer image sequence; wherein the first reference image array runs through the local wafer image array with the same row number or column number, and the second reference image array and the third reference image array are vertically distributed on both sides of the first reference image array; an array division module, configured to divide the remaining images in the local wafer image sequence into four quadrant image arrays according to the first reference image array, the second image reference array and the third image reference array; An image stitching module is used to stitch two adjacent images in the horizontal and vertical dimensions in sequence in different stitching orders away from the center of the local wafer image array for the first reference image array, the second image reference array, the third image reference array and the four quadrant image arrays; wherein the two adjacent images include overlapping areas.

10. An electronic device, characterized in that: It includes a processor, a memory and a communication interface, wherein the communication interface, the memory and the processor are connected via a bus; The communication interface is used to send and receive data; The memory stores a computer program, and the processor executes the method according to any one of claims 1 to 8.

Images