Image splicing method and device, equipment, storage medium and program product

Abstract

The invention relates to an image splicing method and device, optical microscopic measurement equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring a first image and a second image to be spliced; determining first reference information corresponding to the first image and second reference information corresponding to the second image based on the target matching mode; based on the first reference information and the second reference information, respectively determining a first overlapping region in the first image and a second overlapping region in the second image; based on first height information corresponding to the first overlapping region and second height information corresponding to the second overlapping region, determining a morphology error between the first image and the second image; and based on the first overlapping region, the second overlapping region and the morphology error, splicing the first image and the second image to obtain a spliced image. By adopting the method, the image splicing effect can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image stitching method, apparatus, device, readable storage medium, and program product. Background Technology

[0002] With the development of precision manufacturing, micro-nano fabrication, and materials testing, the demand for high-precision, large-scale measurement of sample surface morphology is constantly increasing. Optical microscopy techniques such as confocal microscopy and white light interferometry are widely used for surface morphology inspection due to their advantages such as non-contact operation and high resolution.

[0003] Limited by the size of the optical field of view, such devices can only acquire a local surface morphology of the sample in a single acquisition, making it difficult to directly complete large-area measurements. To achieve large-scale surface morphology acquisition, existing technologies typically control the lens to acquire multiple local surface morphology images of adjacent locations and then stitch these images together to obtain a complete image.

[0004] Traditional splicing methods typically involve simple splicing based on preset movement parameters, resulting in poor splicing quality. Summary of the Invention

[0005] Therefore, it is necessary to provide an image stitching method, apparatus, device, computer-readable storage medium, and computer program product that can improve the image stitching effect in response to the above-mentioned technical problems.

[0006] Firstly, this application provides an image stitching method, including:

[0007] Acquire a first image and a second image to be stitched together; the first image and the second image are local surface morphology images of the sample acquired at adjacent acquisition positions during the process of the lens moving relative to the sample under test in the target plane according to preset movement parameters.

[0008] Based on the target matching method, the first reference information corresponding to the first image and the second reference information corresponding to the second image are determined;

[0009] Based on the first reference information and the second reference information, the first overlapping region in the first image and the second overlapping region in the second image are determined respectively;

[0010] Based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region, the shape error between the first image and the second image is determined.

[0011] Based on the first overlapping region, the second overlapping region, and the shape error, the first image and the second image are stitched together to obtain the stitched image.

[0012] Secondly, this application also provides an image stitching device, comprising:

[0013] The image acquisition module is used to acquire a first image and a second image to be stitched together; the first image and the second image are local surface morphology images of the sample acquired at adjacent acquisition positions during the process of the lens moving relative to the sample under test in the target plane according to preset movement parameters.

[0014] The reference information determination module is used to determine the first reference information corresponding to the first image and the second reference information corresponding to the second image based on the target matching method.

[0015] The overlapping region determination module is used to determine the first overlapping region in the first image and the second overlapping region in the second image based on the first reference information and the second reference information, respectively.

[0016] The topography error determination module is used to determine the topography error between the first image and the second image based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region.

[0017] The stitching module is used to stitch the first image and the second image based on the first overlapping region, the second overlapping region, and the shape error to obtain a stitched image.

[0018] Thirdly, this application also provides an optical microscopy measuring device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method.

[0019] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0020] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0021] The aforementioned image stitching method, apparatus, device, computer-readable storage medium, and computer program product acquire a first image and a second image to be stitched. The first and second images are local surface topography images of a sample acquired at adjacent acquisition positions during the movement of a lens relative to the sample in a target plane according to preset movement parameters. Based on a target matching method, first reference information corresponding to the first image and second reference information corresponding to the second image are determined. Based on the first and second reference information, a first overlapping region in the first image and a second overlapping region in the second image are determined, respectively. Based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region, the topography error between the first and second images is determined. Based on the first overlapping region, the second overlapping region, and the topography error, the first and second images are stitched together to obtain a stitched image. This eliminates topography errors such as height shift and measurement plane tilt caused by lens movement errors, sample posture changes, and measurement system errors, based on the actual overlap relationship between adjacent images. This ensures that the stitched image maintains height continuity and topography consistency within the overlapping region, avoiding abrupt height changes or topography discontinuities at the stitching boundary, and improving the accuracy and reliability of large-scale surface topography image stitching. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is an application environment diagram of the image stitching method in one embodiment;

[0024] Figure 2 This is a flowchart illustrating an image stitching method in one embodiment;

[0025] Figure 3 This is a schematic diagram of an image stitching scheme in one embodiment;

[0026] Figure 4 This is a schematic diagram of splicing a parameter setting page in one embodiment;

[0027] Figure 5 This is a schematic diagram of splicing the parameter setting page in another embodiment;

[0028] Figure 6 This is a schematic diagram of a navigation graph in one embodiment;

[0029] Figure 7This is a schematic diagram illustrating image overlap in one embodiment;

[0030] Figure 8 This is a schematic diagram illustrating image error conditions in one embodiment;

[0031] Figure 9 This is a schematic diagram of a height compensation scheme in one embodiment;

[0032] Figure 10 This is a structural block diagram of an image stitching device in one embodiment;

[0033] Figure 11 This is an internal structural diagram of an optical microscopy measurement device in one embodiment. Detailed Implementation

[0034] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0035] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0036] The image stitching method provided in this application embodiment can be applied to white light interferometers or confocal microscopes. It can be applied to applications such as... Figure 1 Taking the white light interferometer shown as an example, the white light interferometer acquires a first image and a second image to be stitched together. The first image and the second image are local surface morphology images of the sample acquired at adjacent acquisition positions during the process of the lens of the white light interferometer moving relative to the sample under test in the target plane according to preset movement parameters. Based on the target matching method, the first reference information corresponding to the first image and the second reference information corresponding to the second image are determined. Based on the first reference information and the second reference information, the first overlapping region in the first image and the second overlapping region in the second image are determined respectively. Based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region, the morphology error between the first image and the second image is determined. Based on the first overlapping region, the second overlapping region and the morphology error, the first image and the second image are stitched together to obtain the stitched image.

[0037] In one exemplary embodiment, such as Figure 2As shown, an image stitching method is provided, which can be applied to... Figure 1 Taking optical microscopy equipment (white light interferometer or confocal microscope) as an example, the following steps are included:

[0038] S202, acquire the first image and the second image to be stitched together; the first image and the second image are local surface morphology images of the sample acquired at adjacent acquisition positions during the process of the lens moving relative to the sample under test in the target plane according to preset movement parameters.

[0039] The sample to be measured refers to the object whose surface morphology needs to be measured. The sample to be measured can be a workpiece, material sample, micro / nano structure sample, or other solid object with the surface to be measured. During measurement, the sample to be measured can be placed on the stage of the optical microscopic measuring equipment.

[0040] The first image refers to a local surface morphology image of a sample acquired at a certain acquisition position according to preset movement parameters during the image stitching process. In some embodiments, the first image may also be a stitched image obtained by stitching a local surface morphology image of a sample acquired at at least one acquisition position with at least one other image, and used as a reference image in the subsequent stitching process.

[0041] The second image refers to another local surface morphology image of the sample acquired at a collection position adjacent to the first image under the control of preset movement parameters, relative to the first image. The second image is used for stitching with the first image.

[0042] A lens is an imaging component set in an optical microscopy measuring device. It is used to perform optical imaging and morphological acquisition of the surface of the sample to be measured. The lens can move relative to the sample to be measured in the target plane, or the relative displacement between the sample to be measured and the lens can be achieved by moving the stage.

[0043] The target plane refers to the reference plane, which can be the surface of the stage or a horizontal plane. It is used to locate the relative position of the sample and the lens, ensuring that the acquired images have accurate spatial positioning and surface morphology data.

[0044] Preset movement parameters are a set of parameters used to control the movement of the lens or stage within the target plane. The preset movement parameters include at least one or more of the following: movement direction, movement distance or movement step, and overlap ratio, to ensure that there is an overlapping area between images corresponding to adjacent acquisition positions.

[0045] Adjacent acquisition positions refer to two acquisition positions that, under the control of preset movement parameters, are continuously moved within the target plane by the lens or stage, and whose distance between them is less than the lens field of view and whose corresponding acquired images have overlapping areas on the sample surface.

[0046] A sample local surface morphology image refers to image data obtained by acquiring a local area of ​​the sample surface using an optical microscopy measuring device. The image not only contains two-dimensional positional information of the sample surface, but also height information related to the surface height or morphology, which is used to characterize the three-dimensional morphological features of the sample surface.

[0047] Specifically, when performing the image stitching method, the optical microscopic measuring equipment first places the sample to be tested on the stage and positions it. After initialization, the optical microscopic measuring equipment controls the lens or stage to move relative to the sample in the target plane according to preset movement parameters, and performs imaging acquisition on the surface of the sample at each preset acquisition position. After the lens or stage completes imaging acquisition at adjacent acquisition positions, the local surface morphology image of the sample acquired at the first acquisition position is obtained as the first image to be stitched. At the same time, the local surface morphology image of the sample acquired at the second acquisition position adjacent to the first acquisition position is obtained as the second image to be stitched.

[0048] S204, Based on the target matching method, determine the first reference information corresponding to the first image and the second reference information corresponding to the second image.

[0049] The target matching method refers to a matching strategy used to determine the correspondence between adjacent images during the image stitching process. The target matching method is used to determine reference information for matching for each image to be stitched. In different embodiments, the target matching method may include a coordinate matching method based on physical location information or a feature point matching method based on image content features.

[0050] The first reference information refers to the reference information obtained from the first image when using the target matching method, which is used to match with other images. The first reference information is used to characterize the reference basis of the first image in the image stitching process. The first reference information can be the physical location information corresponding to the first image when it was acquired, or the feature point information extracted from the first image.

[0051] The second reference information refers to the reference information obtained from the second image when using the target matching method, which is used to match the first image. The second reference information is used to characterize the reference basis of the second image in the image stitching process. The second reference information can be the physical location information corresponding to the second image when it was acquired, or the feature point information extracted from the second image.

[0052] Specifically, after acquiring the first and second images to be stitched together, the optical microscopy measuring device processes the first and second images respectively according to a pre-set target matching method to determine the corresponding reference information. When the target matching method is coordinate matching, the optical microscopy measuring device acquires the position information recorded when acquiring the first and second images. The position information is used to characterize the actual acquisition position of the lens relative to the stage during acquisition, and the position information corresponding to the first image is used as the first reference information, and the position information corresponding to the second image is used as the second reference information. When the target matching method is feature point matching, the optical microscopy measuring device performs image feature analysis on the first and second images, extracts feature point information used to characterize the surface morphology of the sample from each image, and uses the feature point information extracted from the first image as the first reference information and the feature point information extracted from the second image as the second reference information.

[0053] S206, based on the first reference information and the second reference information, determine the first overlapping region in the first image and the second overlapping region in the second image, respectively.

[0054] The first overlapping region refers to the region located in the first image, determined based on the first reference information and the second reference information during the image stitching process. The first overlapping region and the corresponding region in the second image represent the same physical surface region of the same sample to be tested, and are used for morphological information comparison and stitching processing with the second overlapping region.

[0055] The second overlapping region refers to the region located in the second image, determined based on the first and second reference information during the image stitching process. The second overlapping region corresponds to the first overlapping region in the first image and represents the same physical surface region of the same sample under test. It is used for morphological information comparison and stitching processing with the first overlapping region.

[0056] Specifically, after obtaining the first reference information corresponding to the first image and the second reference information corresponding to the second image, the optical microscopy measuring device determines the actual overlap relationship between the first and second images based on the first and second reference information. When the first and second reference information are physical position information, the optical microscopy measuring device calculates the relative displacement relationship between the two images in the target plane based on the spatial position information corresponding to the first and second images at the time of acquisition, and determines the area in the first image that covers the same physical area as the second image as the first overlapping area, based on the field of view size of a single image. At the same time, it determines the area in the second image that corresponds to the first overlapping area and represents the same physical surface area of ​​the sample to be measured as the second overlapping area. When the first and second reference information are feature point information, the optical microscopy measuring device analyzes the spatial distribution of matching feature points in their respective images based on the correspondence of feature points extracted from the first and second images, determines the range of areas in both images that simultaneously contain matching feature points, and defines the areas as the first overlapping area in the first image and the second overlapping area in the second image, respectively.

[0057] S208, based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region, determine the shape error between the first image and the second image.

[0058] The first height information refers to the information obtained from the first overlapping region in the first image, which is used to characterize the height features of the physical surface region corresponding to the sample under test. The first height information may include the height value corresponding to each pixel in the first overlapping region, the height distribution formed by the height values, or the statistical feature parameters calculated from the height values.

[0059] The second height information refers to the information obtained from the second overlapping region in the second image, which is used to characterize the height features of the physical surface region of the same test sample corresponding to the first overlapping region. The second height information may include the height value corresponding to each pixel in the second overlapping region, the height distribution formed by the height values, or the statistical feature parameters calculated from the height values.

[0060] Topographic error refers to the inconsistency between the first height information and the second height information caused by measurement deviations during image acquisition when the first overlapping region and the second overlapping region represent the same physical surface area of ​​the sample under test. Topographic error includes at least the height error caused by the overall height offset and the tilt error caused by the change in the posture of the measurement plane, and is used to characterize the topographic differences between the first image and the second image in the overlapping region.

[0061] Specifically, after determining the first overlapping region in the first image and the second overlapping region in the second image, the optical microscopy measuring device obtains the corresponding height information from the first overlapping region and the second overlapping region, respectively, as the first height information and the second height information. The height information is used to characterize the height features of the physical surface region of the same sample under test in different images. Based on this, the optical microscopy measuring device compares and analyzes the first height information and the second height information to determine the morphological error between the first image and the second image.

[0062] In one embodiment, the optical microscopy measuring device can calculate the overall height difference between the two regions based on the height data in the first and second overlapping regions to determine the height error caused by the inconsistency of the height reference. At the same time, the optical microscopy measuring device can perform plane fitting or trend analysis on the height data of the first and second overlapping regions based on the horizontal trend of the height data, and determine the tilt error caused by the change in the posture of the measurement plane based on the difference between the fitting results.

[0063] S210, based on the first overlapping region, the second overlapping region, and the shape error, the first image and the second image are stitched together to obtain the stitched image.

[0064] The stitched image refers to the image formed by combining the fused overlapping area with the non-overlapping area of ​​the original image after morphological correction and fusion processing of the first image and the second image in the corresponding overlapping area. The stitched image is used to characterize the continuous surface morphological features of the sample under test over a larger range, and has a highly continuous and consistent display effect in the original overlapping area.

[0065] Specifically, after determining the first overlapping region in the first image, the second overlapping region in the second image, and the morphological error between the first and second images, the optical microscopic measurement equipment uses the first image as a reference image to perform morphological correction processing on the second image. The optical microscopic measurement equipment then fuses the first image and the morphologically corrected second image within their corresponding overlapping regions, and combines the fused overlapping regions with their respective non-overlapping regions to generate a stitched image.

[0066] In the above image stitching method, a first image and a second image to be stitched are acquired. The first image and the second image are local surface morphology images of the sample acquired at adjacent acquisition positions during the process of the lens moving relative to the sample under test in the target plane according to preset movement parameters. Based on the target matching method, the first reference information corresponding to the first image and the second reference information corresponding to the second image are determined. Based on the first reference information and the second reference information, the first overlapping region in the first image and the second overlapping region in the second image are determined respectively. Based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region, the morphology error between the first image and the second image is determined. Based on the first overlapping region, the second overlapping region, and the morphology error, the first image and the second image are stitched together to obtain the stitched image. Thus, based on determining the actual overlap relationship between adjacent images, the morphology errors such as height offset and measurement plane tilt introduced by lens movement error, sample posture change, and measurement system error are eliminated. This ensures that the stitched image maintains height continuity and morphology consistency in the overlapping region, avoids height abrupt changes or morphology discontinuities at the stitching boundary, and improves the accuracy and reliability of large-area surface morphology image stitching.

[0067] In one embodiment, the target matching method includes a coordinate matching method based on physical location information; the first reference information includes first physical location information; the second reference information includes second physical location information; the process by which the optical microscopic measuring device determines the first reference information corresponding to the first image and the second reference information corresponding to the second image based on the target matching method includes the following steps: acquiring the first physical location information corresponding to the first image at the time of acquisition and the second physical location information corresponding to the second image at the time of acquisition; the first physical location information and the second physical location information are used to characterize the actual acquisition position of the lens relative to the stage at the time of acquisition; the process by which the optical microscopic measuring device determines the first overlapping region in the first image and the second overlapping region in the second image based on the first reference information and the second reference information includes the following steps: determining the relative displacement relationship between the first image and the second image in the target plane based on the first physical location information and the second physical location information; determining the first overlapping region in the first image and the second overlapping region in the second image based on the relative displacement relationship and the field of view of the lens.

[0068] The first physical position information refers to the position information recorded by the optical microscopic measuring device during the acquisition of the first image, which is used to characterize the acquisition position of the lens relative to the stage during acquisition. The first physical position information can be the coordinate value of the lens in the target plane, or other equivalent position information that can reflect the actual acquisition position of the first image.

[0069] The second physical position information refers to the position information recorded by the optical microscopic measuring equipment when acquiring the second image, which is used to characterize the acquisition position of the lens relative to the stage during acquisition; the second physical position information is in the same coordinate system as the first physical position information and is used to determine the spatial positional relationship between the first image and the second image.

[0070] The relative displacement relationship refers to the displacement relationship calculated based on the first physical position information and the second physical position information, which is used to characterize the relative movement of the first image and the second image in the target plane. The relative displacement relationship is used to reflect the relative positional offset of the two images on the sample surface.

[0071] The field of view of a lens refers to the range of sample surface area that an optical microscopy measuring device can cover when acquiring an image in a single session, under the current lens magnification and imaging parameters. The field of view is used to define the spatial coverage size of a single sample local surface morphology image and serves as the basis for determining the range of overlapping areas between adjacent images.

[0072] Specifically, during the acquisition of the first and second images, the optical microscopy measuring device records the first physical position information of the lens relative to the stage in the target plane corresponding to the first image acquisition position, and the second physical position information corresponding to the second image acquisition position. After acquiring the first and second physical position information, the optical microscopy measuring device calculates the relative displacement relationship between the first and second images in the target plane based on the first and second physical position information to determine the relative movement amount and direction between the two image acquisition positions. Furthermore, based on the relative displacement relationship and combined with the current field of view of the lens, the optical microscopy measuring device determines the spatial coverage relationship between the first and second images on the sample surface, thereby determining the area in the first image that covers the same physical surface area as the second image as the first overlapping area, and determining the area in the second image corresponding to the first overlapping area as the second overlapping area.

[0073] In the above embodiments, the optical microscopic measurement device acquires the first physical position information corresponding to the first image at the time of acquisition and the second physical position information corresponding to the second image at the time of acquisition. Based on the first and second physical position information, it determines the relative displacement relationship between the first and second images in the target plane. According to the relative displacement relationship and the field of view of the lens, it determines the first overlapping region in the first image and the second overlapping region in the second image, respectively. This enables the accurate determination of the actual overlapping relationship between adjacent images based on the physical position information at the time of image acquisition, avoiding the problem of inaccurate judgment of overlapping regions due to device movement deviation. This provides a reliable regional basis for the subsequent determination of morphological errors and high-precision image stitching, improving the accuracy and stability of image stitching.

[0074] In one embodiment, the target matching method includes a feature point matching method based on image content features; the first reference information includes first feature point information; the second reference information includes second feature point information; the process by which the optical microscopic measuring device determines the first reference information corresponding to the first image and the second reference information corresponding to the second image based on the target matching method includes the following steps: performing content feature analysis on the first image and the second image respectively to obtain the first feature point information corresponding to the first image and the second feature point information corresponding to the second image; the first feature point information and the second feature point information are used to characterize the significant morphological features in their respective images; the process by which the optical microscopic measuring device determines the first overlapping region in the first image and the second overlapping region in the second image based on the first reference information and the second reference information includes the following steps: determining the matching feature point pairs in the first image and the second image based on the first feature point information and the second feature point information; the feature point pairs include feature points from the first image and feature points from the second image; and determining the first overlapping region in the first image and the second overlapping region in the second image according to the spatial distribution range of the feature point pairs in their respective images.

[0075] Content feature analysis refers to the process of analyzing and processing image content that characterizes changes in the surface morphology of a sample. Content feature analysis is used to identify and extract stable and discriminative local features from an image. Specifically, mathematically, the extracted local features can be defined as locations in the image where the height data changes in two orthogonal directions have significant derivatives. This approach can employ feature point extraction methods from the field of computer vision, such as the SIFT or SURF algorithms, to extract local features.

[0076] The first feature point information refers to the feature point information extracted from the first image by performing content feature analysis on the first image, which is used to characterize the significant morphological features of the sample surface. The first feature point information includes at least the location information of the feature points in the first image, and may further include the descriptive information of the feature points.

[0077] The second feature point information refers to the feature point information extracted from the second image by performing content feature analysis on the second image, which is used to characterize the significant morphological features of the sample surface. The second feature point information includes at least the location information of the feature points in the second image, and may further include the descriptive information of the feature points.

[0078] The mutually matching feature point pairs are a set of feature points determined by feature similarity analysis in the first feature point information and the second feature point information, which are respectively from the first image and the second image and correspond to the same physical surface position of the sample to be tested. The set of feature point pairs includes feature points in the first image and feature points in the second image.

[0079] The spatial distribution range of feature point pairs in their respective images refers to the area covered by the matching feature point pairs in the first and second images, respectively. The spatial distribution range is used to characterize the area in the two images that commonly contains matching feature points, and serves as the basis for determining the first overlapping area in the first image and the second overlapping area in the second image.

[0080] Specifically, after acquiring the first image and the second image, the optical microscopy measuring device performs content feature analysis on the first image and the second image respectively to extract feature point information for characterizing the significant morphological features of the sample surface from each image, thereby obtaining the first feature point information corresponding to the first image and the second feature point information corresponding to the second image. After obtaining the first feature point information and the second feature point information, the optical microscopy measuring device performs matching analysis on the first feature point information and the second feature point information based on feature similarity to determine the matching feature point pairs in the first image and the second image. Each pair of feature points includes feature points from the first image and feature points from the second image. Further, the optical microscopy measuring device determines the area range in the two images that simultaneously contains matching feature points according to the spatial distribution of the feature point pairs in the first image and the second image, and defines the area range as the first overlapping area in the first image and the second overlapping area in the second image, respectively.

[0081] In one embodiment, the optical microscopy measuring device can determine a first ideal overlapping region in a first image and a second ideal overlapping region in a second image based on preset movement parameters, and perform content feature analysis processing on the first and second ideal overlapping regions respectively to extract feature point information for characterizing significant morphological features of the sample surface from their respective images, thereby obtaining first feature point information corresponding to the first image and second feature point information corresponding to the second image.

[0082] The first ideal overlap region refers to the region with the best overlap position between the first image and the second image, determined when the optical microscopic measuring device acquires the first image according to preset movement parameters. This region is selected as the ideal overlap region for subsequent stitching processing.

[0083] The second ideal overlap region refers to the region with the best overlap position between the first image and the second image, determined when the optical microscopic measuring device acquires the second image according to preset movement parameters. This region is selected as the ideal overlap region for subsequent stitching processing.

[0084] In the above embodiments, the optical microscopic measurement device performs content feature analysis on the first image and the second image respectively to obtain the first feature point information corresponding to the first image and the second feature point information corresponding to the second image. Based on the first feature point information and the second feature point information, it determines the matching feature point pairs in the first image and the second image. According to the spatial distribution range of the feature point pairs in their respective images, it determines the first overlapping region in the first image and the second overlapping region in the second image respectively. This enables accurate identification of overlapping regions corresponding to the same physical surface area of ​​the sample under test between adjacent images based on the matching relationship of feature points in the image content. It avoids the deviation in the judgment of overlapping regions caused by equipment movement errors or inaccurate position information, improves the adaptability and accuracy of image stitching in scenarios with complex surface morphology or limited position information acquisition, and provides a reliable basis for the subsequent determination of morphology errors and high-precision image stitching.

[0085] In one embodiment, the topographic error includes a height error. The process by which the optical microscopic measuring device determines the topographic error between a first image and a second image based on first height information corresponding to a first overlapping region and second height information corresponding to a second overlapping region includes the following steps: calculating a first height statistical value corresponding to the first overlapping region based on the first height information in the first overlapping region; calculating a second height statistical value corresponding to the second overlapping region based on the second height information in the second overlapping region; and determining the height error between the first image and the second image based on the difference between the first height statistical value and the second height statistical value.

[0086] Among them, height error refers to the difference between the first height statistical value and the second height statistical value caused by the inconsistency of height reference during the image acquisition process when the first overlapping region and the second overlapping region represent the same physical surface area of ​​the sample to be tested. Height error is used to characterize the overall height offset between the first image and the second image in the overlapping region.

[0087] The first height statistic refers to a statistical quantity used to characterize the overall height characteristics of the first overlapping region, calculated based on the first height information in the first overlapping region. The first height statistic can be the average, median, weighted average, or other statistical characteristic values ​​that can reflect the overall height level in the first overlapping region.

[0088] The second height statistic refers to a statistical quantity used to characterize the overall height characteristics of the second overlapping region, calculated based on the second height information in the second overlapping region. The second height statistic is under the same height measurement system as the first height statistic and is used to compare with the first height statistic to determine the height error.

[0089] Specifically, after determining the first overlapping region in the first image and the second overlapping region in the second image, the optical microscopy measuring device obtains corresponding height information from the first and second overlapping regions, respectively, as the first height information and the second height information. After obtaining the first height information, the optical microscopy measuring device performs statistical analysis on the height data within the first overlapping region to calculate a first height statistical value used to characterize the overall height characteristics of the first overlapping region. Correspondingly, after obtaining the second height information, the optical microscopy measuring device performs statistical analysis on the height data within the second overlapping region to calculate a second height statistical value used to characterize the overall height characteristics of the second overlapping region. Subsequently, based on the difference between the first and second height statistical values, the optical microscopy measuring device determines the overall height offset between the first and second images within the overlapping region, and defines the overall height offset as the height error between the first and second images. This height error is used for subsequent height compensation and stitching processing of the images.

[0090] In the above embodiments, the optical microscopic measurement device calculates the first height statistical value corresponding to the first overlapping region based on the first height information in the first overlapping region, and calculates the second height statistical value corresponding to the second overlapping region based on the second height information in the second overlapping region. According to the difference between the first height statistical value and the second height statistical value, the height error between the first image and the second image is determined. This can accurately depict the overall height offset between the first image and the second image in the overlapping region due to the inconsistency of the height reference, providing a reliable basis for subsequent height compensation of the second image, avoiding abrupt height changes or step effects in the stitching region during image stitching, and improving the continuity of the stitched image in the height direction and the surface morphology measurement accuracy.

[0091] In one embodiment, the topography error includes tilt error. The process by which the optical microscopic measuring device determines the topography error between a first image and a second image based on first height information corresponding to a first overlapping region and second height information corresponding to a second overlapping region includes the following steps: performing plane fitting on the height data corresponding to the first overlapping region based on the first height information in the first overlapping region to obtain a first measurement plane parameter; performing plane fitting on the height data corresponding to the second overlapping region based on the second height information in the second overlapping region to obtain a second measurement plane parameter; and determining the tilt error between the first image and the second image based on the difference between the first measurement plane parameter and the second measurement plane parameter.

[0092] The tilt error refers to the difference between the parameters of the first and second measurement planes caused by the inconsistency of the measurement plane posture during image acquisition when the first and second overlapping regions represent the same physical surface area of ​​the sample under test. The tilt error is used to characterize the degree of inconsistency in the height change trend of the first and second images along the horizontal direction in the overlapping region.

[0093] The first measurement plane parameter refers to the set of parameters used to characterize the attitude characteristics of the measurement plane in the first overlapping region, obtained by performing plane fitting on the height data corresponding to the first overlapping region based on the first height information in the first overlapping region. The first measurement plane parameter includes at least parameters that can reflect the tilt state of the measurement plane.

[0094] The second measurement plane parameters refer to the set of parameters used to characterize the attitude features of the measurement plane in the second overlapping region, obtained by fitting the height data corresponding to the second overlapping region to the second height information in the second overlapping region. The second measurement plane parameters include at least parameters that can reflect the tilt state of the measurement plane. It should be noted that the second measurement plane parameters and the first measurement plane parameters are in the same parameter system.

[0095] Specifically, after determining the first overlapping region in the first image and the second overlapping region in the second image, the optical microscopy measuring device acquires corresponding height data from the first and second overlapping regions, respectively, as first height information and second height information. After acquiring the first height information, the optical microscopy measuring device performs plane fitting processing on the height data corresponding to the first overlapping region based on the height data within the first overlapping region, obtaining a first measurement plane parameter used to characterize the attitude characteristics of the measurement plane of the first overlapping region. Correspondingly, after acquiring the second height information, the optical microscopy measuring device performs plane fitting processing on the height data corresponding to the second overlapping region based on the height data within the second overlapping region, obtaining a second measurement plane parameter used to characterize the attitude characteristics of the measurement plane of the second overlapping region. Subsequently, the optical microscopy measuring device compares and analyzes the first and second measurement plane parameters, and determines the difference in the height change trend along the horizontal direction between the first and second images within the overlapping region based on the parameter differences between the two. This difference is defined as the tilt error between the first and second images, and the tilt error is used for subsequent tilt compensation and stitching processing of the images.

[0096] In the above embodiments, the optical microscopic measurement device performs plane fitting on the height data corresponding to the first overlapping region based on the first height information in the first overlapping region to obtain the first measurement plane parameters. Based on the second height information in the second overlapping region, it performs plane fitting on the height data corresponding to the second overlapping region to obtain the second measurement plane parameters. According to the difference between the first measurement plane parameters and the second measurement plane parameters, the tilt error between the first image and the second image is determined. This can accurately characterize the tilt difference between the first image and the second image in the overlapping region caused by the inconsistency of the measurement plane posture, providing a reliable basis for subsequent tilt compensation of the second image. This avoids surface morphology distortion or discontinuity in the stitched area caused by different measurement plane tilts during image stitching, and improves the morphology consistency and overall measurement accuracy of the stitched image in the horizontal direction.

[0097] In one embodiment, an optical microscopic measurement device stitches together a first image and a second image based on a first overlapping region, a second overlapping region, and a topographic error to obtain a stitched image. This includes: using the first image as a reference image, performing topographic correction on the height data of the second image according to the topographic error to obtain a corrected second image; and stitching together the first image and the corrected second image based on the first overlapping region and the second overlapping region to obtain the stitched image.

[0098] Among them, shape correction refers to the process of adjusting the height data in the second image based on the determined shape error, using the first image as the reference image. Shape correction is used to eliminate or reduce the shape difference between the first image and the second image in the overlapping area due to height error and / or tilt error, so that the corrected second image is consistent with the first image in terms of height reference and shape change trend, thereby providing a consistent shape basis for the subsequent seamless stitching of the two images.

[0099] Specifically, the optical microscopic measurement equipment performs overall height compensation on the height data of the second image based on the height error in the topographic error, and simultaneously performs tilt compensation on the height data of the second image based on the tilt error in the topographic error, to obtain a corrected second image. After obtaining the corrected second image, the optical microscopic measurement equipment aligns the corrected second image with the first image in spatial position based on the regional correspondence determined in the first and second overlapping regions, and performs fusion processing on the height data of the two images in the overlapping region. Finally, the corrected second image and the first image are stitched together to form a stitched image.

[0100] In the above embodiments, the optical microscopic measurement device uses a first image as a reference image, corrects the height data of a second image based on morphological errors to obtain a corrected second image, and stitches the first image and the corrected second image together based on a first overlapping region and a second overlapping region to obtain a stitched image. This eliminates or reduces morphological errors introduced between adjacent images due to inconsistent height references and differences in the tilt of the measurement plane before image stitching, avoids problems such as abrupt height changes, tilt discontinuities, or morphological distortions in the stitching area, and ensures that the stitched image maintains continuous and consistent surface morphological features in both the height and horizontal directions, thereby improving the accuracy and reliability of large-scale surface morphological measurement results.

[0101] In one embodiment, the stitched image obtained by stitching the first image and the second image is a first partial image of the sample to be tested. The image stitching method further includes the following steps: obtaining a second partial image of the sample to be tested; the second partial image is obtained by stitching other images; and stitching the first partial image and the second partial image together to obtain a stitched image of the sample to be tested.

[0102] The first local image refers to the stitched image obtained by stitching the first image and the second image during the image stitching process, which is used to characterize the surface morphology of a certain continuous region of the sample under test. The first local image corresponds to a part of the surface of the sample under test and can be used as a reference image or one of the images to be stitched in the subsequent stitching process.

[0103] The second local image refers to a stitched image obtained by stitching together at least two images that are different from the first and second images during the image stitching process. It is used to characterize the surface morphology of another continuous region of the sample under test. The second local image corresponds to a different region of the sample surface from the first local image and is used to further stitch together with the first local image to form a stitched image with a larger range.

[0104] Other images may include images acquired at different acquisition locations than the first and second images, and these other images may have overlapping areas on the sample surface to support corresponding stitching processes.

[0105] Specifically, after completing the stitching process of the first and second images, the optical microscopy measuring device determines the resulting stitched image as the first local image of the sample under test, used to characterize a continuous region of the sample's surface. Based on this, the optical microscopy measuring device further selects other images from the multiple acquired images of the sample's local surface morphology that did not participate in generating the first local image, and performs stitching processing on these other images based on their overlap relationships, thereby obtaining a second local image used to characterize another continuous region of the sample under test. After obtaining the first and second local images, the optical microscopy measuring device aligns and fuses the first and second local images based on the overlapping area or position information between them, thus stitching the first and second local images together to obtain a stitched image used to characterize a larger area of ​​the sample's surface morphology.

[0106] For example, after completing the acquisition of all images of the sample to be tested, the optical microscopy measuring equipment does not immediately stitch them together according to the preset acquisition order. Instead, it analyzes the adjacent domains between the images based on the data relationships of all the acquired images to determine a more reasonable stitching order.

[0107] Specifically, due to the influence of measurement range, skipped areas, or local non-measurement settings during actual measurement, the obtained images to be stitched may not be completely continuous in spatial distribution. Optical microscopic measuring equipment can divide all images to be stitched into multiple continuous regions based on the spatial adjacency relationship between images; for example... Figure 3 As shown, the optical microscopic measurement device divides all images to be stitched into continuous regions 1, 2, and 3 based on the adjacency domain of the image data, and prioritizes performing stitching processing within each continuous region. Specifically, the optical microscopic measurement device selects one image from each continuous region as the first image for stitching that region, and uses the first image as the reference image to stitch the remaining images in that continuous region sequentially, thereby obtaining local stitched images corresponding to continuous regions 1, 2, and 3, respectively. The first images selected for each continuous region are denoted as image a, image b, and image c, respectively.

[0108] After completing the stitching within each continuous region, the optical microscopy measuring device further stitches together the locally stitched images corresponding to continuous regions 1, 2, and 3 based on the image coordinate relationship between images a, b, and c on the sample surface. Simultaneously, the optical microscopy measuring device uses the starting Z-grating position information recorded when acquiring images a, b, and c to calculate the height difference between each starting Z-grating position, and calibrates the height error between different continuous regions based on the height difference, thereby achieving height consistency between different continuous regions while ensuring the continuity of the morphology within each continuous region.

[0109] In this way, the optical microscopy measurement equipment can first complete high-precision stitching in a local area when there is a discontinuous acquisition area, and then complete the overall stitching between different local areas. It can also calibrate the height error introduced by different scanning start positions, thereby obtaining a high-precision stitched image for characterizing the overall surface morphology of the sample under test.

[0110] In one embodiment, the first image is the first selected image in the target local region to which it belongs; the process of obtaining the first image and the second image includes: selecting a candidate image located at the center position from multiple images in the target local region; selecting a candidate image whose feature richness meets a preset screening condition as the first image from the candidate images; and selecting an image whose overlap with the first image meets a preset overlap condition from multiple images in the target local region as the second image.

[0111] The central position refers to the relative position of the central location of the target local area, determined by the image coordinates or acquisition location information of each image. The image located at the central position is the image whose corresponding acquisition location is closer to the geometric center in the row and column distribution or two-dimensional coordinate distribution of the target local area, which is used to reduce the risk of error accumulation in the subsequent stitching process.

[0112] Feature richness refers to the abundance of effective morphological features in an image that can be used for matching and stitching. Feature information includes at least one or more of the following: the number of feature points extracted from image height data or morphological data, feature point distribution density, feature point response intensity, or feature point stability. The higher the feature richness, the more sufficient the feature information in the image can be reliably matched with other images.

[0113] Preset screening conditions refer to the judgment rules set in advance by the optical microscopy measuring equipment or its measuring software before image stitching, used to determine whether candidate images meet the conditions for being the first image. The preset screening conditions include at least the feature richness threshold condition, which is used to limit the feature information contained in the selected first image to meet the minimum requirements for stable stitching.

[0114] Overlap is a parameter used to characterize the size of the area jointly covered by two images on the sample surface in a local region of a target. Overlap can be expressed as the ratio of the area of ​​the overlapping region of the two images in the horizontal direction to the area of ​​a single image, or as the ratio of the number of matching feature points in the overlapping region of the two images to the total number of feature points.

[0115] The preset overlap conditions are the overlap judgment rules set in advance by the optical microscopy measuring equipment or its measuring software to determine whether the images meet the stitching conditions. The preset overlap conditions include at least an overlap threshold to ensure that there is a sufficient overlap area between the selected second image and the first image to support the reliability of subsequent topography error calculation and stitching fusion processing.

[0116] Specifically, before performing stitching processing on multiple images within a target local area, the optical microscopy measurement equipment first acquires the acquisition position information or image coordinate information corresponding to each image within the target local area, and determines the spatial distribution relationship of each image within the target local area based on the acquisition position information or image coordinate information. On this basis, the optical microscopy measurement equipment determines at least one image located at or near the geometric center of the target local area, and identifies this at least one image as a candidate image. After obtaining the candidate images, the optical microscopy measurement equipment performs feature analysis processing on each candidate image, extracting feature point information used to characterize the surface morphology of the sample, and calculates the feature richness corresponding to each candidate image based on the feature point information. Subsequently, the optical microscopy measurement equipment stitches the candidate images together... The corresponding feature richness is compared with preset screening conditions, and candidate images whose feature richness meets the preset screening conditions are selected. The selected candidate images are determined as the first image. For example, the candidate image with the highest feature richness is selected as the first image. After determining the first image, the optical microscopic measurement device calculates the overlap between the first image and each of the other images based on the spatial position information or feature matching information between the first image and the other images in the target local area. On this basis, the optical microscopic measurement device selects at least one image from multiple images in the target local area whose overlap with the first image meets the preset overlap conditions. For example, the image with the largest overlap with the first image is selected from the adjacent images. The selected image is determined as the second image for subsequent stitching and fusion processing based on the first image.

[0117] For example, after acquiring all images of the sample to be measured, the optical microscopy measuring device can determine the spatial distribution of each image within the target local area to be stitched, based on the acquisition location information or image coordinate information of each image within that local area. Based on this, the optical microscopy measuring device prioritizes selecting the image located at the center of the target local area as the first image for stitching. If multiple images exist at the center, the optical microscopy measuring device can randomly select one as the first image, or further select the image with higher feature richness based on the feature data corresponding to each candidate image, such as selecting the candidate image with the highest feature richness as the first image, to improve the stability and accuracy of subsequent stitching and fusion. After determining the first image, the optical microscopy measuring device continues to select images from the target local area... Among the remaining images, the adjacent image with a high degree of actual overlap with the first image is selected as the second image. For example, the image with the largest overlap with the first image is selected as the second image. Using the first image as a reference, the first image and the second image are stitched together. After the stitching is completed, the optical microscopic measuring device uses the fused image as the new reference image and performs stitching on the remaining images in the order of adjacent images with high overlap with the fused image until all images in the target local area are stitched together to obtain the stitched image corresponding to the target local area. Of course, in some implementations, the optical microscopic measuring device can also adopt a simpler stitching strategy, that is, according to the image acquisition order, multiple images in the target local area are stitched together to obtain the stitched image of the target local area.

[0118] In some embodiments, the stitching and fusion process can also be completed synchronously during image acquisition. Specifically, when the optical microscopic measurement device acquires images to be stitched sequentially according to the set stitching route, each time a new image is acquired, the newly acquired image can be stitched and fused with previously acquired images or existing stitched images, thereby gradually updating the stitching results during image acquisition. In this implementation, the optical microscopic measurement device can use the first acquired image as the initial reference image. When the next adjacent image is acquired, based on the determined overlapping area and the corresponding morphological error, the adjacent image is morphologically corrected and then stitched and fused with the current reference image. Subsequently, the fused image is used as the new reference image and stitched and fused with the subsequently acquired images until all images are acquired and stitched. By performing the stitching and fusion process synchronously during image acquisition, the computational pressure of subsequent centralized processing can be reduced while ensuring stitching accuracy, and the stitching results can be updated in real time or near real time, making it convenient for users to observe the stitching effect of a large area of ​​surface morphology in real time during the measurement process.

[0119] In one embodiment, an application scenario for the image stitching method is also provided. Specifically, this scenario may involve using a white light interferometer to detect the surface morphology of a workpiece, material sample, or micro / nano structure sample. The specific process is as follows:

[0120] S1. Place the sample to be tested on the stage and set the splicing route.

[0121] Specifically, the operator places the sample to be tested on the stage of the optical microscopy measuring equipment, turns on the equipment, and selects a lens magnification that matches the surface characteristics of the sample. In the measurement software, the operator selects the stitching measurement function, enters the stitching parameter configuration interface, and sets the stitching path by configuring the stitching parameters.

[0122] The splicing parameters may include the measurement range, the size of the overlapping area or the step distance, the measurement order, and the splicing matching method. For example... Figure 4 As shown, the measurement range can be set to an X-axis measurement range and a Y-axis measurement range to define the overall scanning area of ​​the lens within the target plane. Depending on the device structure, optical microscopy measuring equipment can employ either a method where the stage moves in the X and Y directions within the target plane while the lens remains stationary, or a method where the lens moves in the X and Y directions within the target plane while the stage remains stationary.

[0123] When setting the size of the overlapping area, an overlap percentage can be used to specify the overlap ratio of adjacent images in the X and Y directions, allowing the system to automatically calculate the movement distance between adjacent acquisition positions. Since different magnification lenses correspond to different actual field-of-view sizes, under the same measurement range and overlap ratio, higher magnification lenses will have more acquisition positions and smaller movement steps. Alternatively, the movement steps in the X and Y directions can be directly set, but it must be ensured that the step size does not exceed the current lens's field-of-view size to guarantee an overlapping area between adjacent images.

[0124] The software offers two stitching matching methods: coordinate matching and feature point matching. Coordinate matching uses the positional information from when the images were captured to determine the actual overlapping area between two adjacent images. Feature point matching uses feature data (such as height data) obtained from the captured images to identify common feature points between adjacent images, and then determines the actual overlapping area based on the location of these specific points. Users can choose the appropriate stitching matching method according to their needs. For example, if a faster matching speed is desired, coordinate matching can be selected; if more precise stitching accuracy is required, feature point matching can be chosen.

[0125] The measurement sequence is used to indicate the movement path of the lens or stage within the measurement range, such as... Figure 5 As shown, the measurement sequence can be set to a serpentine path, an S-shaped path, or a meandering path, etc. After configuring the stitching parameters, the measurement software can generate a navigation map of the stitching measurement in the navigation window, which visually displays the acquisition location and quantity of each image to be stitched, and allows operators to enable or disable certain acquisition locations. Figure 6 As shown, each square frame (referred to as a frame) in the diagram represents an image to be stitched together. The size of the frame corresponds to the field of view of the lens. The magnification of the navigation window can be adjusted according to the number of images to be stitched. By clicking and selecting a frame on the navigation map, you can configure it, such as determining whether to measure that location. Figure 6 In the navigation map, the white area indicates that the area is set to not be measured.

[0126] S2. Take photos according to the stitching route to obtain multiple images to be stitched together.

[0127] After the stitching route is set, the optical micro-measuring device controls the lens or stage to move sequentially relative to the sample in the target plane according to the stitching route and measurement sequence set in step S1, and performs imaging acquisition on the surface of the sample at each adjacent acquisition position, thereby obtaining multiple local surface morphology images of the sample as images to be stitched.

[0128] S3. Based on the selected stitching matching method, identify the actual overlapping area of ​​adjacent images to be stitched.

[0129] After acquiring multiple images to be stitched, the measurement software identifies the actual overlapping areas between adjacent images according to the stitching matching method selected in step S1. The stitching matching method may include coordinate matching based on physical location information or feature point matching based on image content features.

[0130] Taking the movement of the stage as an example, the stage is driven by a guide rail. Considering that the guide rail may bounce during movement, the movement accuracy in the X or Y direction is not high enough. For example, if the original intention was to move along the X direction, but a bounce occurs, there will be an offset in the Y direction, causing the actual moved position to not match the step distance corresponding to the set overlap percentage. Figure 7 As shown, the actual overlapping area is not exactly the same as the set overlapping area. Therefore, when using the coordinate matching method, the measurement software uses the position information recorded by the optical microscopic measurement equipment during image acquisition (e.g., the position of the lens or stage obtained by the grating ruler) to determine the actual acquisition position of each image to be stitched on the sample surface, and calculates the relative displacement relationship between adjacent images in the target plane, thereby determining the actual overlapping area between adjacent images. In this way, even if there is jumping or offset during the movement of the stage or lens, the true overlapping relationship can be accurately identified.

[0131] When using feature point matching, the measurement software analyzes height data or morphological features in adjacent images to extract feature points that stably reflect the sample surface structure. Based on the spatial distribution of matching feature points in their respective images, the overlapping regions corresponding to the same physical surface area in adjacent images are determined. To improve computational efficiency, the feature point extraction process can be performed only within a preset overlapping region.

[0132] S4. Merge the actual overlapping areas to achieve image stitching.

[0133] Considering factors such as guide rail bounce and platform tilt during movement, height (e.g.) will be introduced. Figure 8 As shown in the figure, and topographic errors such as tilt angle, after determining the actual overlapping area of ​​adjacent images, the measurement software analyzes the height data within the overlapping area to eliminate topographic errors introduced by lens or stage movement errors, sample posture changes, and measurement system errors.

[0134] Specifically, the measurement software can calculate the overall height difference between adjacent images based on height data within the overlapping area, and use one image as a reference to perform overall height compensation on the other image (e.g., Figure 9(as shown in the figure). At the same time, plane fitting can be performed on the height data in the overlapping area to analyze the tilt state of the measurement plane corresponding to the adjacent images, and tilt compensation can be performed on another image accordingly, thereby reducing height offset and measurement plane tilt error.

[0135] After completing height and tilt compensation, the measurement software fuses the topography-corrected image with the reference image within the overlapping area. It then combines the fused overlapping area with their respective non-overlapping areas to generate a stitched image. This method enables high-precision stitched measurement of a large area of ​​the sample's surface topography while ensuring continuity in height and consistency in topography between adjacent images.

[0136] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0137] Based on the same inventive concept, this application also provides an image stitching apparatus for implementing the image stitching method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations in one or more image stitching apparatus embodiments provided below can be found in the limitations of the image stitching method described above, and will not be repeated here.

[0138] In one exemplary embodiment, such as Figure 10 As shown, an image stitching device is provided, including: an image acquisition module 1002, a reference information determination module 1004, an overlapping area determination module 1006, a shape error determination module 1008, and a stitching module 1010, wherein:

[0139] The image acquisition module 1002 is used to acquire the first image and the second image to be stitched together. The first image and the second image are local surface morphology images of the sample acquired at adjacent acquisition positions during the process of the lens moving relative to the sample under test in the target plane according to preset movement parameters.

[0140] The reference information determination module 1004 is used to determine the first reference information corresponding to the first image and the second reference information corresponding to the second image based on the target matching method;

[0141] The overlapping region determination module 1006 is used to determine the first overlapping region in the first image and the second overlapping region in the second image based on the first reference information and the second reference information, respectively.

[0142] The topography error determination module 1008 is used to determine the topography error between the first image and the second image based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region.

[0143] The stitching module 1010 is used to stitch the first image and the second image based on the first overlapping region, the second overlapping region and the shape error to obtain the stitched image.

[0144] In the above embodiments, a first image and a second image to be stitched are acquired. The first image and the second image are local surface topography images of the sample acquired at adjacent acquisition positions during the process of the lens moving relative to the sample under test in the target plane according to preset movement parameters. Based on the target matching method, the first reference information corresponding to the first image and the second reference information corresponding to the second image are determined. Based on the first reference information and the second reference information, the first overlapping region in the first image and the second overlapping region in the second image are determined respectively. Based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region, the topography error between the first image and the second image is determined. Based on the first overlapping region, the second overlapping region and the topography error, the first image and the second image are stitched together to obtain the stitched image. Thus, based on determining the actual overlap relationship between adjacent images, the topography errors such as height offset and measurement plane tilt introduced by lens movement error, sample posture change and measurement system error are eliminated. This ensures that the stitched image maintains height continuity and topography consistency in the overlapping region, avoids height abruptness or topography discontinuity at the stitching boundary, and improves the accuracy and reliability of large-area surface topography image stitching.

[0145] In one embodiment, the target matching method includes a coordinate matching method based on physical location information; the first reference information includes first physical location information; the second reference information includes second physical location information; the reference information determination module 1004 is further configured to: acquire the first physical location information corresponding to the first image at the time of acquisition and the second physical location information corresponding to the second image at the time of acquisition; the first physical location information and the second physical location information are used to characterize the actual acquisition position of the lens relative to the stage at the time of acquisition; the overlapping region determination module 1006 is further configured to: determine the relative displacement relationship between the first image and the second image in the target plane based on the first physical location information and the second physical location information; and determine the first overlapping region in the first image and the second overlapping region in the second image according to the relative displacement relationship and the field of view of the lens.

[0146] In one embodiment, the target matching method includes a feature point matching method based on image content features; the first reference information includes first feature point information; the second reference information includes second feature point information; the reference information determination module 1004 is further configured to: perform content feature analysis on the first image and the second image respectively to obtain the first feature point information corresponding to the first image and the second feature point information corresponding to the second image; the first feature point information and the second feature point information are used to characterize the significant morphological features in their respective images; the overlapping region determination module 1006 is further configured to: determine the matching feature point pairs in the first image and the second image based on the first feature point information and the second feature point information; the feature point pair includes a first target feature point from the first image and a feature point from the second image; and determine the first overlapping region in the first image and the second overlapping region in the second image respectively according to the spatial distribution range of the feature point pair in their respective images.

[0147] In one embodiment, the shape error includes a height error. The shape error determination module 1008 is further configured to: calculate a first height statistical value corresponding to the first overlapping region based on the first height information in the first overlapping region; calculate a second height statistical value corresponding to the second overlapping region based on the second height information in the second overlapping region; and determine the height error between the first image and the second image based on the difference between the first height statistical value and the second height statistical value.

[0148] In one embodiment, the topography error includes a tilt error. The topography error determination module 1008 is configured to: perform plane fitting on the height data corresponding to the first overlapping region based on the first height information in the first overlapping region to obtain a first measurement plane parameter; perform plane fitting on the height data corresponding to the second overlapping region based on the second height information in the second overlapping region to obtain a second measurement plane parameter; and determine the tilt error between the first image and the second image based on the difference between the first measurement plane parameter and the second measurement plane parameter.

[0149] In one embodiment, the shape error determination module 1008 is further configured to: use the first image as a reference image, perform shape correction on the height data of the second image according to the shape error, and obtain a corrected second image; and stitch the first image and the corrected second image based on the first overlapping region and the second overlapping region to obtain a stitched image.

[0150] Each module in the aforementioned image stitching device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of the optical microscopy measuring device in hardware form or independent of it, or stored in the memory of the optical microscopy measuring device in software form, so that the processor can call and execute the corresponding operations of each module.

[0151] In one exemplary embodiment, an optical microscopic measurement device is provided, the internal structure of which can be shown in the figure below. Figure 8 As shown, the optical microscopy measurement device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides the environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements an image stitching method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the optical microscopy measuring equipment can be a touch layer covering the display screen, or a button, trackball, or touchpad set on the housing of the optical microscopy measuring equipment, or an external keyboard, touchpad, or mouse, etc.

[0152] Those skilled in the art will understand that Figure 11The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the optical microscopy measurement device to which the present application is applied. A specific optical microscopy measurement device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0153] This application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, which, when executed by one or more processors, cause the processors to perform the steps of the methods described in the above embodiments.

[0154] This application also provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps of the methods described in the above embodiments.

[0155] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0156] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0157] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0158] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An image stitching method, characterized by, The method includes: Acquire a first image and a second image to be stitched together; the first image and the second image are local surface morphology images of the sample acquired at adjacent acquisition positions during the process of the lens moving relative to the sample under test in the target plane according to preset movement parameters. Based on the target matching method, the first reference information corresponding to the first image and the second reference information corresponding to the second image are determined; Based on the first reference information and the second reference information, the first overlapping region in the first image and the second overlapping region in the second image are determined respectively; Based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region, the shape error between the first image and the second image is determined. Based on the first overlapping region, the second overlapping region, and the shape error, the first image and the second image are stitched together to obtain the stitched image.

2. The method of claim 1, wherein, The target matching method includes a coordinate matching method based on physical location information; the first reference information includes first physical location information; the second reference information includes second physical location information; The step of determining the first reference information corresponding to the first image and the second reference information corresponding to the second image based on the target matching method includes: The first physical position information corresponding to the first image at the time of acquisition and the second physical position information corresponding to the second image at the time of acquisition are obtained; the first physical position information and the second physical position information are used to characterize the actual acquisition position of the lens relative to the stage at the time of acquisition; The step of determining the first overlapping region in the first image and the second overlapping region in the second image based on the first reference information and the second reference information includes: Based on the first physical location information and the second physical location information, the relative displacement relationship between the first image and the second image in the target plane is determined; Based on the relative displacement relationship and the field of view of the lens, the first overlapping region in the first image and the second overlapping region in the second image are determined respectively.

3. The method of claim 1, wherein, The target matching method includes a feature point matching method based on image content features; the first reference information includes first feature point information; the second reference information includes second feature point information; determining the first reference information corresponding to the first image and the second reference information corresponding to the second image based on the target matching method includes: Content feature analysis is performed on the first image and the second image respectively to obtain the first feature point information corresponding to the first image and the second feature point information corresponding to the second image; the first feature point information and the second feature point information are used to characterize the significant morphological features in their respective images. The step of determining the first overlapping region in the first image and the second overlapping region in the second image based on the first reference information and the second reference information includes: Based on the first feature point information and the second feature point information, matching feature point pairs are determined in the first image and the second image; the feature point pair includes a first target feature point from the first image and a feature point from the second image. Based on the spatial distribution range of the feature point pairs in their respective images, the first overlapping region in the first image and the second overlapping region in the second image are determined respectively.

4. The method of claim 1, wherein, The shape error includes a height error. Determining the shape error between the first image and the second image based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region includes: Based on the first height information in the first overlapping region, calculate the first height statistical value corresponding to the first overlapping region; Based on the second height information in the second overlapping region, calculate the second height statistical value corresponding to the second overlapping region; The height error between the first image and the second image is determined based on the difference between the first height statistical value and the second height statistical value.

5. The method of claim 1, wherein, The shape error includes tilt error. Determining the shape error between the first image and the second image based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region includes: Based on the first height information in the first overlapping region, plane fitting is performed on the height data corresponding to the first overlapping region to obtain the first measurement plane parameters; Based on the second height information in the second overlapping region, plane fitting is performed on the height data corresponding to the second overlapping region to obtain the second measurement plane parameters; The tilt error between the first image and the second image is determined based on the difference between the first measurement plane parameters and the second measurement plane parameters.

6. The method according to any one of claims 1 to 5, characterized in that, The step of stitching the first image and the second image together based on the first overlapping region, the second overlapping region, and the shape error to obtain a stitched image includes: Using the first image as a reference image, the height data of the second image is corrected according to the shape error to obtain the corrected second image; Based on the first overlapping region and the second overlapping region, the first image and the corrected second image are stitched together to obtain the stitched image.

7. An image stitching apparatus characterized by comprising: The device includes: The image acquisition module is used to acquire a first image and a second image to be stitched together; the first image and the second image are local surface morphology images of the sample acquired at adjacent acquisition positions during the process of the lens moving relative to the sample under test in the target plane according to preset movement parameters. The reference information determination module is used to determine the first reference information corresponding to the first image and the second reference information corresponding to the second image based on the target matching method. The overlapping region determination module is used to determine the first overlapping region in the first image and the second overlapping region in the second image based on the first reference information and the second reference information, respectively. The topography error determination module is used to determine the topography error between the first image and the second image based on the first height information corresponding to the first overlapping region and the second height information corresponding to the second overlapping region. The stitching module is used to stitch the first image and the second image based on the first overlapping region, the second overlapping region, and the shape error to obtain a stitched image.

8. An optical microscopic measuring apparatus comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images