Image matching method and device, medium and product

Abstract

The invention discloses an image matching method and device, a medium and a product, and relates to the technical field of semiconductor integrated circuits. The image matching method comprises the following steps: acquiring a template image and a to-be-compared image; determining a template gradient corresponding to a template image parameter of each template pixel point in the template image, and determining a to-be-compared gradient corresponding to a to-be-compared image parameter of each to-be-compared pixel point in the to-be-compared image; determining correlation parameters between each template gradient and each gradient to be compared, and determining a gradient matching graph between the template image and the image to be compared according to each correlation parameter; and determining an offset between a known feature region in the template image and a to-be-determined feature region in the to-be-compared image according to a peak value in the gradient matching image, and determining the to-be-determined feature region in the to-be-compared image according to the offset and the known feature region.

Description

Technical Field

[0001] This application belongs to the technical field of semiconductor integrated circuits, and particularly relates to an image matching method, device, medium and product. Background Art

[0002] As the foundation of the modern electronics industry, with the continuous development of semiconductor technology, the structure of chips has become increasingly complex, the process line width has been continuously reduced, and the requirements for detection equipment have also become higher and higher. In the semiconductor production process, the accuracy and stability of equipment are crucial for product quality and production efficiency. Therefore, semiconductor detection equipment plays a crucial role in the semiconductor manufacturing process. They can detect whether there are defects on semiconductors such as wafers or chips, such as particle contamination, surface scratches, open circuits and short circuits, etc. These defects may have an adverse impact on the performance of semiconductors.

[0003] With the reduction of the semiconductor process line width, in order to perform more accurate positioning in detection, high-resolution scanning electron microscope imaging technology is used to collect images of semiconductors, so as to find the characteristic regions on the semiconductors through the images for defect detection.

[0004] However, there are some problems with scanning electron microscope images, such as charging effects and edge effects, etc. The charging effect will cause abnormal dark regions to appear in the image, and the edge effect will cause abnormal increase in secondary electron emission in the edge region of the image, affecting the recognition and extraction of feature points in the image, resulting in low recognition accuracy of the characteristic regions on the chip or wafer. Summary of the Invention

[0005] Embodiments of this application provide an image matching method, device, medium and product, which are used to solve the problem of low recognition accuracy of characteristic regions on chips or wafers.

[0006] In the first aspect of the embodiments of this application, an image matching method is provided, including:

[0007] Obtain a template image and an image to be compared;

[0008] Determine the template gradients corresponding to the template image parameters of each template pixel point in the template image, and determine the comparison gradients corresponding to the comparison image parameters of each comparison pixel point in the image to be compared;

[0009] Determine the correlation parameters between each of the template gradients and each of the comparison gradients, and determine the gradient matching map between the template image and the image to be compared according to each of the correlation parameters;

[0010] Determine the offset between the known feature region in the template image and the to-be-determined feature region in the image to be compared according to the peak value in the gradient matching graph, and determine the to-be-determined feature region in the image to be compared according to the offset and the known feature region.

[0011] In a second aspect of the embodiments of the present application, an image matching device is provided, including:

[0012] An acquisition module, configured to acquire a template image and an image to be compared;

[0013] A first determination module, configured to determine the template gradient corresponding to the template image parameters of each template pixel point in the template image, and determine the to-be-compared gradient corresponding to the to-be-compared image parameters of each to-be-compared pixel point in the image to be compared;

[0014] A second determination module, configured to determine the correlation parameter between each template gradient and each to-be-compared gradient, and determine the gradient matching graph between the template image and the image to be compared according to each correlation parameter;

[0015] A third determination module, configured to determine the offset between the known feature region in the template image and the to-be-determined feature region in the image to be compared according to the peak value in the gradient matching graph, and determine the to-be-determined feature region in the image to be compared according to the offset and the known feature region.

[0016] In a third aspect of the embodiments of the present application, an electronic device is provided, and the device includes: a memory and a program or instruction stored on the memory and executable on a processor, and when the program or instruction is executed by the processor, it implements the image matching method provided in any one of the above aspects of the embodiments of the present application.

[0017] In a fourth aspect of the embodiments of the present application, a readable storage medium is provided, and a program or instruction is stored on the readable storage medium, and when the program or instruction is executed by the processor, it implements the image matching method provided in any one of the above aspects of the embodiments of the present application.

[0018] In a fifth aspect of the embodiments of the present application, a computer program product is provided, and when the instructions in the computer program product are executed by the processor of the electronic device, the electronic device is enabled to execute the image matching method provided in any one of the above aspects of the embodiments of the present application.

[0019] In the image matching method provided by the embodiments of the present application, a template gradient is obtained based on the image parameters of the pixel points in the template image, a gradient to be compared is obtained through the image parameters of the pixel points in the image to be compared, and a gradient matching map between the template image and the image to be compared is determined based on the correlation parameter between the template gradient and the gradient to be compared. Moreover, the offset between the known feature region in the template image and the feature region to be determined in the image to be compared is determined according to the peak value in the gradient matching map, so as to locate the feature region to be determined in the image to be compared based on the offset and the known feature region. In the present application, the gradient is determined by the image parameters of the pixel points in the image, that is, the gradient characterizes the change of the image parameters of the pixel points, and the pixel points represent the points on the chip or wafer. Therefore, the gradient characterizes the change of the feature points in the chip or wafer. Even if there are charging effects and edge effects in the image, the change of the feature points on the chip or wafer in the image will not be affected. That is, the change of the feature points in the chip or wafer can be accurately characterized by the gradient, so as to accurately determine the feature region in the image to be compared, and improve the recognition accuracy of the feature region on the chip or wafer. Description of the Drawings

[0020] To more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required to be used in the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0021] Figure 1 is one of the flow diagrams of the image matching method provided by an embodiment of the present application;

[0022] Figure 2 is the schematic diagram of the images corresponding to the qualified object and the object to be detected involved in an embodiment of the present application;

[0023] Figure 3 is the gradient signal diagram involved in an embodiment of the present application;

[0024] Figure 4 is the gradient matching map involved in an embodiment of the present application;

[0025] Figure 5 is the second flow diagram of the image matching method provided by an embodiment of the present application;

[0026] Figure 6 is the third flow diagram of the image matching method provided by an embodiment of the present application;

[0027] Figure 7 is the schematic diagram of the image signal involved in an embodiment of the present application;

[0028] Figure 8It is the fourth schematic flowchart of the image matching method provided by an embodiment of the present application;

[0029] Figure 9 It is the fifth schematic flowchart of the image matching method provided by an embodiment of the present application;

[0030] Figure 10 It is the schematic structural diagram of the image matching device provided by an embodiment of the present application;

[0031] Figure 11 It is the schematic diagram of the electronic device provided by an embodiment of the present application. Detailed implementation manners

[0032] The features and exemplary embodiments of various aspects of the present application will be described in detail below. To make the objectives, technical solutions and advantages of the present application clearer and more understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than limiting the present application. For those skilled in the art, the present application can be implemented without some of these specific details. The following description of the embodiments is only intended to provide a better understanding of the present application by showing examples of the present application.

[0033] It should be noted that, in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "including", "comprising" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, the elements defined by the statement "including..." do not exclude the existence of additional identical elements in the process, method, article or device including the said elements.

[0034] It should be noted that the acquisition, storage, use, processing, etc. of data in the technical solution of the present application all comply with the relevant regulations of national laws and regulations.

[0035] It should be noted that in the embodiments of the present application, some industry-existing solutions such as certain software, components, models, etc. may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility in the implementation of the technical solution of the present application, but it does not mean that the applicant has already or necessarily used this solution.

[0036] As the foundation of modern electronics industry, with the continuous development of semiconductor technology, the structure of chips is becoming increasingly complex, the process line width is constantly shrinking, and the requirements for detection equipment are also getting higher and higher. In the semiconductor production process, the accuracy and stability of equipment are crucial for product quality and production efficiency. Therefore, semiconductor detection equipment plays a vital role in the semiconductor manufacturing process. They can detect whether there are defects on semiconductors such as wafers or chips, such as particle contamination, surface scratches, open circuits and short circuits, etc. These defects may have an adverse impact on the performance of semiconductors.

[0037] With the reduction of the semiconductor process line width, in order to perform more accurate positioning in detection, high-resolution scanning electron microscope imaging technology is used to collect images of semiconductors, so as to find the characteristic areas on the semiconductors through the images for defect detection.

[0038] However, there are some problems with scanning electron microscope images, such as charging effect and edge effect, etc. The charging effect will cause abnormal dark areas to appear in the image, and the edge effect will cause the secondary electron emission in the edge area of the image to increase abnormally, affecting the recognition and extraction of feature points in the image, thus resulting in a relatively low recognition accuracy of the characteristic areas on the chip or wafer.

[0039] In view of this, the present application provides an image matching method, device, medium and product. In the image matching method provided by the embodiments of the present application, there are some problems with scanning electron microscope images, such as charging effect and edge effect, etc. The charging effect will cause abnormal dark areas to appear in the image, and the edge effect will cause the secondary electron emission in the edge area of the image to increase abnormally, affecting the recognition and extraction of feature points in the image, thus resulting in a relatively low recognition accuracy of the characteristic areas on the chip or wafer.

[0040] It should be noted that the application scenarios described in the embodiments of the present application above are for more clearly explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those of ordinary skill in the art know that with the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems. The image matching method provided by the embodiments of the present application can be applied to various application scenarios that require positioning of characteristic areas in semiconductors.

[0041] The following introduces the specific embodiments of the image matching method, device, equipment, medium and product provided by the embodiments of the present application. First, the image matching method is introduced.

[0042] Refer to Figure 1 , Figure 1 which shows one of the flow diagrams of the image matching method of the present application. The image matching method includes the following steps:

[0043] Step S101, obtain a template image and an image to be compared.

[0044] In this embodiment, the execution subject is an image matching device, which can be any electronic device with the function of positioning semiconductor feature regions. For the convenience of description, the device is hereinafter used to refer to the image matching device.

[0045] After a wafer or a chip is generated, it is necessary to detect whether there are defects in the wafer or the chip. The defects are, for example, particle contamination, surface scratches, open / short circuits, etc. The wafers, chips to be detected for defects or the chips obtained by processing the wafers are objects to be detected. A scanning electron microscope acquires an image of the surface of the object to be detected to obtain an image to be compared, and then sends the image to be compared to the device. In addition, the device can also obtain from the database the images acquired by the scanning electron microscope of the surface of qualified objects, where the qualified objects refer to wafers, chips or chips that are detected to have no defects. In addition, the template image can also be the original design drawing of the wafer, chip or chip.

[0046] It should be noted that the coordinate systems of the template image and the image to be compared are the same. In addition, when the object to be detected is a chip, the qualified object is also a chip; when the object to be detected is a wafer, the qualified object is also a wafer. The template image is an image of a qualified object detected to have no defects. Therefore, the feature regions in the template image are known, that is, the device can obtain the coordinates of the feature regions in the template image in the coordinate system. The feature regions in the template image are defined as known feature regions. The device needs to locate the to-be-determined feature regions of the object to be detected in the image to be compared based on the known feature regions in the template image. The to-be-determined feature regions can be regions composed of corner points in the object to be detected, can also be edge regions in the object to be detected, or can also be regions with rich texture in the object to be detected.

[0047] Step S102, determine the template gradients corresponding to the template image parameters of each template pixel point in the template image, and determine the comparison gradients corresponding to the comparison image parameters of each comparison pixel point in the image to be compared.

[0048] Each pixel point in the template image has corresponding image parameters. The pixel points in the template image are defined as template image pixel points, and their image parameters are defined as template image parameters. The template image parameters can be the brightness value, pixel value or gray scale value of the template pixel point; each pixel point in the image to be compared also has corresponding image parameters. Each pixel point in the image to be compared is defined as a comparison pixel point, and the image parameters of the comparison pixel point are defined as comparison image parameters. The comparison image parameters can be the brightness value, pixel value or gray scale value of the pixel point, and the types of the template image parameters and the comparison image parameters are the same.

[0049] Due to the charging efficiency or edge effect of the scanning electron microscope, abnormal bright areas may appear in the image to be compared. Refer to Figure 2 , Figure 2 which includes a template image 210 and an image to be compared 220. Different from the fact that the brightness of each area in the template image 210 is the same, it is obvious that the brightness of a part of the area in the image to be compared 220 is higher than that of other areas. The square in the template image 210 is a known feature area, while the square in the image to be compared 220 is a feature area to be located, and the square in the image to be compared 220 is determined after being located by the device.

[0050] The gradient represents the rate of change or slope of a multivariate function at a certain point. When there is a bright area in the image to be compared and the feature area is located in the bright area, the change amplitude of the image parameters of each pixel point in the feature area is the same. Therefore, the rate of change of the image parameters of each pixel point is constant, and the image parameters of the pixel point can be used as a multivariate function, so that the gradient of the image parameters can represent the rate of change of the image parameters of the pixel point. The image parameters are, for example, pixel values, and the pixel values are (255, 11, 12). Therefore, the image parameters can be used as a multivariate function to calculate the gradient.

[0051] Based on this, the device determines the gradient corresponding to the template image parameters of each template pixel point in the template image, and the gradient corresponding to the template image parameters is defined as the template gradient; the device then determines the gradient corresponding to the image parameters to be compared of each pixel point to be compared in the image to be compared, and the gradient corresponding to the image parameters to be compared is defined as the gradient to be compared.

[0052] Step S103, determine the correlation parameters between each template gradient and each gradient to be compared, and determine the gradient matching map between the template image and the image to be compared according to each correlation parameter.

[0053] There are multiple template image parameters in the template image, so multiple template gradients can be obtained, and there are also multiple image parameters to be compared in the image to be compared, and the device obtains multiple gradients to be compared. After determining each template gradient and each gradient to be compared, the device calculates the correlation parameter between each template gradient and each gradient to be compared.

[0054] Exemplarily, the correlation parameter can be the cross-correlation function between the template gradient and the gradient to be compared. The role of the cross-correlation function is to find at which moment a signal is most similar to another signal. One signal corresponds to the template gradient, and the other signal corresponds to the gradient to be compared. Each template gradient can form a template gradient signal map, and each gradient to be compared can form a gradient signal map to be compared. Refer to Figure 3 , Figure 3 where (a) inFigure 3 In (b), it represents the gradient signal map to be compared composed of each gradient to be compared. By Figure 3 It can be seen that the gradient can enhance the intensity of the signal of the feature, making it easier to find two highly similar signals in (a) and (b). For example, from Figure 3 It can be seen that the similarity between the first peak signal in (a) and the first peak signal in (b) is greater than the preset similarity, that is, the first peak signals in (a) and (b) are highly similar.

[0055] The device can obtain the gradient matching map between the template image and the image to be compared based on the correlation parameter. The gradient matching map is specifically as Figure 4 .

[0056] Step S104: Determine the offset between the known feature region in the template image and the to-be-determined feature region in the image to be compared according to the peak in the gradient matching map, and determine the to-be-determined feature region in the image to be compared according to the offset and the known feature region.

[0057] After the device determines the gradient matching map, it determines the peak in the gradient matching map. The peak can determine the offset of the to-be-determined feature region in the image to be compared relative to the known feature region. In one example, the device determines the x value of the peak in the gradient matching map. The x value is the abscissa in the gradient matching map, and the offset in the x direction in the coordinate system is obtained by converting the x value.

[0058] In another example, the device determines the y value of the peak in the gradient matching map. The y value is the abscissa in the gradient matching map, and the offset in the y direction in the coordinate system is obtained by converting the y value.

[0059] In still another example, the device determines the x value and the y value of the peak in the gradient matching map, and determines the offset through x and y.

[0060] After the device determines the offset, it obtains the coordinates of the known feature region. The to-be-determined feature region in the image to be compared can be located through the coordinates and the offset. In one example, the device can obtain the abscissa of the to-be-determined feature region in the image to be compared in the coordinate system through the offset in the x direction. The device performs feature matching on the region with the abscissa in the image to be compared and the known feature region to determine the to-be-determined feature region.

[0061] In another example, the device can obtain the ordinate of the to-be-determined feature region in the image to be compared in the coordinate system through the offset in the y direction. The device performs feature matching on the region with the ordinate in the image to be compared and the known feature region to determine the to-be-determined feature region.

[0062] In yet another example, the to-be-determined feature region is directly located in the to-be-compared image through offsets x and y.

[0063] After the device determines the to-be-determined feature region, it then uses a semiconductor detection device to detect whether there are defects in the to-be-determined feature region.

[0064] In this embodiment, a template gradient is obtained based on the image parameters of the pixel points in the template image, a to-be-compared gradient is obtained based on the image parameters of the pixel points in the to-be-compared image, and a gradient matching map between the template image and the to-be-compared image is determined based on the correlation parameter between the template gradient and the to-be-compared gradient. Moreover, the offset between the known feature region in the template image and the to-be-determined feature region in the to-be-compared image is determined according to the peak value in the gradient matching map, so as to locate the to-be-determined feature region in the to-be-compared image based on the offset and the known feature region. In this embodiment, the gradient is determined by the image parameters of the pixel points in the image, that is, the gradient represents the change of the image parameters of the pixel points, and the pixel points represent the points on the chip or wafer. Therefore, the gradient represents the change of the feature points in the chip or wafer. Even if there are charging effects and edge effects in the image, the change of the feature points on the chip or wafer in the image will not be affected. That is, the gradient can accurately represent the change of the feature points in the chip or wafer, so as to accurately determine the feature region in the to-be-compared image and improve the recognition accuracy of the feature region on the chip or wafer.

[0065] Refer to Figure 5 , Figure 5 shows the second schematic flowchart of the image matching method of the present application. Based on Figure 1 the embodiment shown, step S102 includes:

[0066] Step S501, determining a plurality of intermediate image parameters corresponding to the template image according to the template image parameters of each pixel point in the template image, and determining a plurality of target image parameters corresponding to the to-be-compared image according to the to-be-compared image parameters of each to-be-compared pixel point in the to-be-compared image. The number of intermediate image parameters is less than the number of template image parameters, and the number of target image parameters is less than the number of to-be-compared image parameters.

[0067] In this embodiment, a plurality of intermediate image parameters corresponding to the template image are determined based on the template image parameters of each template pixel point in the template image, and the number of intermediate image parameters is less than the number of template image parameters. In this way, the calculation amount of the device can be reduced.

[0068] Exemplarily, the device calculates the average value of the template image parameters of two or more adjacent pixel points as the intermediate image parameter. In one example, the average value of the template image parameters of N sequentially adjacent pixel points in the same row is calculated as the intermediate image parameter of these N pixel points. If there are M sequentially adjacent N pixel points in one row of the template image, then there are M intermediate image parameters for one row of the template image. In this way, M intermediate image parameters for each row can be obtained;

[0069] In another example, the average value of the template image parameters of N sequentially adjacent pixel points in the same column is calculated as the intermediate image parameter of these N pixel points. If there are K sequentially adjacent N pixel points in one column of the template image, then there are K intermediate image parameters for one column of the template image. In this way, K intermediate image parameters for each column can be obtained.

[0070] In still another example, the device calculates both the average value of the template image parameters of N sequentially adjacent pixel points in the same row and the average value of the template image parameters of N sequentially adjacent pixel points in the same column. Calculating the average value of the template image parameters of N sequentially adjacent pixel points in the same row can calculate the offset in the x direction in the coordinate system, and calculating the average value of the template image parameters of N sequentially adjacent pixel points in the same column can calculate the offset in the y direction in the coordinate system.

[0071] The device then determines multiple target image parameters corresponding to the image to be compared based on the image parameters to be compared of each pixel point to be compared in the image to be compared, and the number of target image parameters is less than the number of image parameters to be compared.

[0072] Exemplarily, the device calculates the average value of the image parameters to be compared of two or more adjacent pixel points as the intermediate image parameter. In one example, the average value of the image parameters to be compared of N sequentially adjacent pixel points in the same row of the image to be compared is calculated as the target image parameter of these N pixel points. If there are M sequentially adjacent N pixel points in one row of the image to be compared, then there are M target image parameters for one row of the image to be compared. In this way, M target image parameters for each row can be obtained; In another example, the average value of the image parameters to be compared of N sequentially adjacent pixel points in the same column of the image to be compared is calculated as the target image parameter of these N pixel points. If there are K sequentially adjacent N pixel points in one column of the image to be compared, then there are K target image parameters for one column of the image to be compared. In this way, K target image parameters for each column can be obtained.

[0073] It should be noted that when calculating the intermediate image parameters corresponding to N pixel points in the same row in the template image, the target image parameters corresponding to N pixel points in the same row are also calculated for the image to be compared; when calculating the intermediate image parameters corresponding to N pixel points in the same column in the template image, the target image parameters corresponding to N pixel points in the same column are also calculated for the image to be compared.

[0074] Step S502: Determine the gradient of each intermediate image parameter as the template gradient, and determine the gradient of each target image parameter as the gradient to be compared.

[0075] After determining each intermediate image parameter and each target image parameter, the device determines the gradient corresponding to each intermediate image parameter as the template gradient, and determines the gradient of each fourth image as the gradient to be compared.

[0076] In this embodiment, the device partially compresses the image parameters of the pixel points in the template image and the image to be compared, so that the device only needs to locate the feature area to be determined in the image to be compared based on the intermediate image parameters and target image parameters obtained by compression, reducing the computational amount of the device.

[0077] Refer to Figure 6 , Figure 6 shows the third schematic flow chart of the image matching method of the present application. Based on Figure 5 the embodiment shown, step S501 includes:

[0078] Step S601: According to the template image parameters of each row of template pixel points in the template image, determine the intermediate image parameters corresponding to each row in the template image. The number of template image parameters of each template pixel point in a row of the template image is greater than the number of intermediate image parameters.

[0079] In this embodiment, the device determines the image parameters corresponding to each row of the template image based on the template image parameters of each row of template pixel points in the template image.

[0080] In an example, the device determines the average value of the template image parameters of each row of template pixel points in the template image, that is, the sum of the template image parameters of each pixel point belonging to the same row divided by the number of pixel points in that row, to obtain the average value, and the average value is used as the intermediate image parameter corresponding to each row in the template image.

[0081] In another example, the device determines the interval in which the template image parameters of each pixel point in each row are located. If the ratio between the number of template image parameters in the same interval and the number of pixel points in that row is greater than a preset ratio, then the image parameters corresponding to the interval are used as the image parameters corresponding to the pixel points in that row, and the image parameters corresponding to the interval are within that interval. If the above ratio is less than or equal to the preset ratio, then the average value of the template image parameters of each pixel point in each row is used as the intermediate image parameter corresponding to each pixel point in each row.

[0082] As can be seen from the above, the number of template image parameters of each template pixel point in a row of the template image is greater than the number of intermediate image parameters.

[0083] Refer to Figure 7 , Figure 7 is a schematic diagram of the image signal intensity in the compressed template image and the image signal intensity in the compressed image to be compared. As can be seen from Figure 7 , the image signal intensity of the image to be compared ( Figure 7 the dotted line in Figure 7 ) is significantly higher than the image signal intensity of the template image ( Figure 3 the solid line in

[0084] Step S602: Determine the target image parameter corresponding to each row in the image to be compared according to the image parameter to be compared of each pixel point to be compared in each row in the image to be compared. The number of image parameters to be compared of each pixel point to be compared in a row in the image to be compared is greater than the number of target image parameters.

[0085] In addition, the device determines the image parameter corresponding to each row of the image to be compared based on the image parameter to be compared of each pixel point to be compared in each row in the image to be compared.

[0086] In one example, the device determines the average value of the image parameters to be compared of each pixel point to be compared in each row in the image to be compared, that is, the sum of the image parameters to be compared of each pixel point belonging to the same row divided by the number of pixel points in that row, to obtain the average value, and the average value is used as the target image parameter corresponding to each row in the image to be compared.

[0087] In another example, the device determines the interval in which the image parameters to be compared of each pixel point in each row are located. If the ratio between the number of image parameters to be compared in the same interval and the number of pixel points in that row is greater than a preset ratio, the image parameters corresponding to the interval are used as the image parameters corresponding to the pixel points in that row, and the image parameters corresponding to the interval are within that interval. If the above ratio is less than or equal to the preset ratio, the average value of the image parameters to be compared of each pixel point in each row is used as the target image parameter corresponding to each pixel point in each row.

[0088] As can be seen from the above, the number of image parameters to be compared of each pixel point to be compared in a row of the image to be compared is greater than the number of target image parameters.

[0089] In this embodiment, the device reduces the computational amount of the device by compressing the image parameters of each pixel point in the template image and the image to be compared, so that the device locates the feature region to be determined in the image to be compared based on the intermediate image parameters and target image parameters obtained by compression.

[0090] Refer to Figure 8 , Figure 8 FIG. shows the fourth schematic flowchart of the image matching method of the present application. Based on the embodiment shown in Figure 5 or Figure 6 Step S501 includes:

[0091] Step S801, according to the template image parameters of each template pixel point in each column of the template image, determine the intermediate image parameter corresponding to each column in the template image. The number of template image parameters of each template pixel point in a column of the template image is greater than the number of intermediate image parameters.

[0092] In this embodiment, the device determines the image parameter corresponding to each column of the template image based on the template image parameters of each template pixel point in each column of the template image.

[0093] In one example, the device determines the average value of the template image parameters of each template pixel point in each column of the template image, that is, the sum of the template image parameters of each pixel point belonging to the same column divided by the number of pixel points in that column, and the average value can be obtained. The average value is used as the intermediate image parameter corresponding to each column in the template image.

[0094] In another example, the device determines the interval in which the template image parameters of each pixel point in each column are located. If the ratio between the number of template image parameters in the same interval and the number of pixel points in that column is greater than a preset ratio, the image parameters corresponding to the interval are used as the image parameters corresponding to the pixel points in that column, and the image parameters corresponding to the interval are within that interval. If the above ratio is less than or equal to the preset ratio, the average value of the template image parameters of each pixel point in each column is used as the intermediate image parameter corresponding to each pixel point in each column.

[0095] As can be seen from the above, the number of template image parameters of each template pixel in a column of the template image is greater than the number of intermediate image parameters.

[0096] Step S802: Determine the target image parameter corresponding to each column in the image to be compared according to the image parameter to be compared of each pixel to be compared in each column of the image to be compared. The number of image parameters to be compared of each pixel to be compared in a column of the image to be compared is greater than the number of target image parameters.

[0097] In addition, the device determines the target image parameter corresponding to each column of the image to be compared based on the image parameter to be compared of each pixel to be compared in each column of the image to be compared.

[0098] In one example, the device determines the average value of the image parameters to be compared of each pixel to be compared in each column of the image to be compared, that is, the sum of the image parameters to be compared of each pixel belonging to the same column divided by the number of pixels in that column, and the average value can be obtained. The average value is used as the target image parameter corresponding to each column in the image to be compared.

[0099] In another example, the device determines the interval in which the image parameters to be compared of each pixel in each column are located. If the ratio between the number of image parameters to be compared in the same interval and the number of pixels in that column is greater than a preset ratio, the image parameter corresponding to the interval is used as the image parameter corresponding to the pixels in that column, and the image parameter corresponding to the interval is within that interval. If the above ratio is less than or equal to the preset ratio, the average value of the image parameters to be compared of each pixel in each column is used as the target image parameter corresponding to each pixel in each column.

[0100] As can be seen from the above, the number of image parameters to be compared of each pixel to be compared in a column of the image to be compared is greater than the number of target image parameters

[0101] Furthermore, the device passes through Figure 6 The peak value in the gradient matching map obtained by the shown embodiment is used to determine the offset in the x direction between the feature region to be determined and the known feature region. The peak value in the gradient matching map obtained by this embodiment is used to determine the offset in the y direction between the feature region to be determined and the known feature region. Thus, the offset is determined based on the offsets in the x direction and the y direction, so as to quickly determine the feature region to be determined directly in the image to be compared. It can be seen that Figure 6 The shown embodiment and this embodiment can form a new implementation scheme.

[0102] In this embodiment, the device compresses the image parameters of each pixel in the template image and the image to be compared, which can achieve image noise reduction without loss of image signal strength, and can reduce the calculation amount of the device.

[0103] Reference Figure 9 , Figure 9 shows the fifth schematic flowchart of the image matching method of the present application. Based on Figures 1 to 8 any of the illustrated embodiments, step S102 includes:

[0104] Step S901, perform noise reduction processing on the template image and the image to be compared.

[0105] In this embodiment, the template image and the image to be compared are images collected under low signal-to-noise ratio conditions. The device needs to perform image processing on the template image and the image to be compared, so as to reduce the influence of noise in the image on the positioning accuracy of the feature region to be determined. For this, the device performs noise reduction processing on the template image and the image to be compared respectively. The noise reduction processing method can adopt mean filtering, Gaussian filtering and curvature filtering.

[0106] Step S902, determine the template gradient corresponding to the template image parameters of each pixel point in the noise-reduced template image, and determine the comparison gradient corresponding to the comparison image parameters of each comparison pixel point in the noise-reduced image to be compared.

[0107] After performing noise reduction processing on the template image and the image to be compared, the device determines the template gradient corresponding to the template image parameters of each pixel point in the noise-reduced template image, and determines the comparison gradient corresponding to the comparison image parameters of each comparison pixel point in the noise-reduced image to be compared. The determination methods of the template gradient and the comparison gradient are specifically referred to the above embodiments, that is, the template gradient obtained in the above embodiments is determined based on the noise-reduced template image, and the comparison gradient is determined based on the noise-reduced image to be compared.

[0108] In this embodiment, the device performs noise reduction processing on the template image and the image to be compared, so as to reduce the influence of noise in the image on the positioning of the feature region in the image to be compared.

[0109] Based on the image matching method. Correspondingly, the present application also provides a specific embodiment of the image matching device.

[0110] As Figure 10 shown, the image matching device 1000 provided in the embodiment of the present application includes:

[0111] An acquisition module 1010, configured to acquire a template image and an image to be compared;

[0112] A first determination module 1020, configured to determine the template gradient corresponding to the template image parameters of each template pixel point in the template image, and determine the comparison gradient corresponding to the comparison image parameters of each comparison pixel point in the image to be compared;

[0113] A second determination module 1030, configured to determine correlation parameters between each template gradient and each gradient to be compared, and determine a gradient matching map between the template image and the image to be compared according to each correlation parameter;

[0114] A third determination module 1040, configured to determine an offset between a known feature region in the template image and a to-be-determined feature region in the image to be compared according to a peak value in the gradient matching map, and determine the to-be-determined feature region in the image to be compared according to the offset and the known feature region.

[0115] As an optional embodiment, the image matching device 1000 is configured to:

[0116] Determine a plurality of intermediate image parameters corresponding to the template image according to the template image parameters of each pixel point in the template image, and determine a plurality of target image parameters corresponding to the image to be compared according to the to-be-compared image parameters of each to-be-compared pixel point in the image to be compared, where the number of intermediate image parameters is less than the number of template image parameters, and the number of target image parameters is less than the number of to-be-compared image parameters;

[0117] Determine the gradient of each intermediate image parameter as the template gradient, and determine the gradient of each target image parameter as the gradient to be compared.

[0118] As an optional embodiment, the image matching device 1000 is configured to:

[0119] Determine intermediate image parameters corresponding to each row in the template image according to the template image parameters of each template pixel point in each row of the template image, where the number of template image parameters of each template pixel point in one row of the template image is greater than the number of intermediate image parameters;

[0120] Determine the image parameters corresponding to each row in the image to be compared according to the to-be-compared image parameters of each to-be-compared pixel point in each row of the image to be compared, and use them as the target image parameters corresponding to the template image, where the number of to-be-compared image parameters of each to-be-compared pixel point in one row of the image to be compared is greater than the number of target image parameters.

[0121] As an optional embodiment, the image matching device 1000 is configured to:

[0122] Determine the average value of the template image parameters of each pixel point in each row of the template image as the intermediate image parameter corresponding to each row in the template image;

[0123] Determine the target image parameters corresponding to each row of the template image according to the to-be-compared image parameters of each to-be-compared pixel point in each row of the image to be compared, including:

[0124] Determine the average value of the image parameters to be compared for each pixel point in each row of the image to be compared, and use it as the target image parameter corresponding to each row in the image to be compared.

[0125] As an alternative embodiment, the image matching device 1000 is configured to:

[0126] Determine the intermediate image parameter corresponding to each column in the template image according to the template image parameters of each template pixel point in each column of the template image. The number of template image parameters of each template pixel point in a column of the template image is greater than the number of intermediate image parameters.

[0127] Determine the target image parameter corresponding to each column in the image to be compared according to the image parameters to be compared of each pixel point to be compared in each column of the image to be compared. The number of image parameters to be compared of each pixel point to be compared in a column of the image to be compared is greater than the number of target image parameters.

[0128] As an alternative embodiment, the image matching device 1000 is configured to:

[0129] Determine the average value of the template image parameters of each template pixel point in each column of the template image, and use it as the intermediate image parameter corresponding to each row in the template image;

[0130] Determine the target image parameter corresponding to each column in the image to be compared according to the image parameters to be compared of each pixel point to be compared in each column of the image to be compared, including:

[0131] Determine the average value of the image parameters to be compared of each pixel point to be compared in each column of the image to be compared, and use it as the target image parameter corresponding to each column in the image to be compared.

[0132] As an alternative embodiment, the image matching device 1000 is configured to:

[0133] Perform noise reduction processing on the template image and the image to be compared;

[0134] Determine the template gradient corresponding to the template image parameter of each pixel point in the noise-reduced template image, and determine the comparison gradient corresponding to the image parameter to be compared of each pixel point to be compared in the noise-reduced image to be compared.

[0135] Based on the image matching method. Accordingly, the present application also provides a specific embodiment of an electronic device for performing image matching.

[0136] Figure 11 Shows a schematic hardware structure diagram of an electronic device for an image matching method provided by an embodiment of the present application.

[0137] The electronic device may include a processor 1101 and a memory 1102 storing computer program instructions.

[0138] Specifically, the above-mentioned processor 1101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured as one or more integrated circuits for implementing the embodiments of the present application.

[0139] The memory 1102 may include a mass storage for data or instructions. By way of example and not limitation, the memory 1102 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. In a suitable case, the memory 1102 may include a removable or non-removable (or fixed) medium. In a suitable case, the memory 1102 may be internal or external to the integrated gateway disaster recovery device. In a specific embodiment, the memory 1102 is a non-volatile solid state memory.

[0140] The processor 1101 reads and executes the computer program instructions stored in the memory 1102 to implement any one of the image matching methods in the above embodiments.

[0141] In one example, the electronic device may further include a communication interface 1103 and a bus 1110. Among them, as Figure 11 shown, the processor 1101, the memory 1102, and the communication interface 1103 are connected through the bus 1110 and complete communication with each other.

[0142] The communication interface 1103 is mainly used to implement communication between the modules, devices, units, and / or devices in the embodiments of the present application.

[0143] Bus 1110 includes hardware, software, or both, and couples components of the electronic device to each other. By way of example and not limitation, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a MicroChannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable bus or a combination of two or more of these. Where appropriate, bus 1110 may include one or more buses. Although embodiments of the present application describe and illustrate specific buses, the present application contemplates any suitable bus or interconnect.

[0144] In addition, in combination with the image matching method in the above embodiments, an embodiment of the present application can be implemented by providing a computer storage medium. Computer program instructions are stored on the computer storage medium; when the computer program instructions are executed by a processor, any one of the image matching methods in the above embodiments is implemented.

[0145] In addition, in combination with the image matching method in the above embodiments, an embodiment of the present application can be implemented by providing a computer program product. When instructions in the computer program product are executed by a processor of an electronic device, the electronic device is caused to execute the image matching method provided in any aspect of the above embodiments of the present application.

[0146] It should be clear that the present application is not limited to the specific configurations and processes described above and illustrated in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and illustrated as examples. However, the method process of the present application is not limited to the specific steps described and illustrated, and those skilled in the art can make various changes, modifications, and additions, or change the order between steps after understanding the spirit of the present application.

[0147] The functional blocks shown in the above structural block diagram can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a functional card, and so on. When implemented in software, the elements of the present application are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted via a data signal carried in a carrier wave over a transmission medium or a communication link. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical discs, hard disks, fiber optic media, radio frequency (RF) links, and so on. The code segment can be downloaded via a computer network such as the Internet, an intranet, and so on.

[0148] It should also be noted that in the exemplary embodiments mentioned in the present application, some methods or systems are described based on a series of steps or devices. However, the present application is not limited to the order of the above steps, that is, the steps can be executed in the order mentioned in the embodiments, or different from the order in the embodiments, or several steps can be executed simultaneously.

[0149] As described above with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It should be understood that each block in the flowchart and / or block diagram, and the combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, a special purpose computer, or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the functions / actions specified in one or more blocks of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general purpose processor, a special purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, can also be implemented by dedicated hardware that performs the specified functions or actions, or by a combination of dedicated hardware and computer instructions.

[0150] The above is only the specific implementation manner of the present application. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described systems, modules, and units can refer to the corresponding processes in the foregoing method embodiments and will not be elaborated herein. It should be understood that the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily think of various equivalent modifications or substitutions, and these modifications or substitutions should be covered within the protection scope of the present application.

Claims

1. An image matching method, characterized in that, Including: Obtaining a template image and an image to be compared; Determining the template gradients corresponding to the template image parameters of each template pixel point in the template image, and determining the comparison gradients corresponding to the comparison image parameters of each comparison pixel point in the image to be compared; Determining the correlation parameters between each of the template gradients and each of the comparison gradients, and determining a gradient matching map between the template image and the image to be compared according to each of the correlation parameters; Determining the offset between the known feature region in the template image and the to-be-determined feature region in the image to be compared according to the peak value in the gradient matching map, and determining the to-be-determined feature region in the image to be compared according to the offset and the known feature region.

2. The method according to claim 1, characterized in that The determining the template gradients corresponding to the template image parameters of each template pixel point in the template image, and determining the comparison gradients corresponding to the comparison image parameters of each comparison pixel point in the image to be compared includes: Determining a plurality of intermediate image parameters corresponding to the template image according to the template image parameters of each template pixel point in the template image, and determining a plurality of target image parameters corresponding to the image to be compared according to the comparison image parameters of each comparison pixel point in the image to be compared, the number of the intermediate image parameters being less than the number of the template image parameters, and the number of the target image parameters being less than the number of the comparison image parameters; Determining the gradient of each of the intermediate image parameters as the template gradient, and determining the gradient of each of the target image parameters as the comparison gradient.

3. The method according to claim 2, wherein The determining a plurality of intermediate image parameters corresponding to the template image according to the template image parameters of each template pixel point in the template image, and determining a plurality of target image parameters corresponding to the image to be compared according to the comparison image parameters of each comparison pixel point in the image to be compared includes: Determining the intermediate image parameters corresponding to each row in the template image according to the template image parameters of each template pixel point in each row of the template image, the number of the template image parameters of each template pixel point in one row of the template image being greater than the number of the intermediate image parameters; Determining the target image parameters corresponding to each row in the image to be compared according to the comparison image parameters of each comparison pixel point in each row of the image to be compared, the number of the comparison image parameters of each comparison pixel point in one row of the image to be compared being greater than the number of the target image parameters.

4. The method according to claim 3, wherein The determining the intermediate image parameters corresponding to each row in the template image according to the template image parameters of each template pixel point in each row of the template image includes: Determining the average value of the template image parameters of each pixel point in each row of the template image as the intermediate image parameters corresponding to each row in the template image; The determining the target image parameters corresponding to each row in the image to be compared according to the comparison image parameters of each comparison pixel point in each row of the image to be compared includes: Determining the average value of the comparison image parameters of each pixel point in each row of the image to be compared as the target image parameters corresponding to each row in the image to be compared.

5. The method according to claim 2, characterized in that, Determining a plurality of intermediate image parameters corresponding to the template image according to the template image parameters of each template pixel point in the template image, and determining a plurality of target image parameters corresponding to the image to be compared according to the image-to-be-compared parameters of each pixel point to be compared in the image to be compared, includes: Determining the intermediate image parameters corresponding to each column in the template image according to the template image parameters of each column of template pixel points in the template image, where the number of template image parameters of each template pixel point in a column of the template image is greater than the number of the intermediate image parameters; Determining the target image parameters corresponding to each column in the image to be compared according to the image-to-be-compared parameters of each column of pixel points to be compared in the image to be compared, where the number of image-to-be-compared parameters of each pixel point to be compared in a column of the image to be compared is greater than the number of the target image parameters.

6. The method according to claim 5, wherein The determining the intermediate image parameters corresponding to each column in the template image according to the template image parameters of each column of template pixel points in the template image, includes: Determining the average value of the template image parameters of each template pixel point in each column of the template image as the intermediate image parameters corresponding to each row in the template image; The determining the target image parameters corresponding to each column in the image to be compared according to the image-to-be-compared parameters of each column of pixel points to be compared in the image to be compared, includes: Determining the average value of the image-to-be-compared parameters of each pixel point to be compared in each column of the image to be compared as the target image parameters corresponding to each column in the image to be compared.

7. The method according to claim 1, wherein The determining the template gradient corresponding to the template image parameters of each template pixel point in the template image and determining the comparison gradient corresponding to the image-to-be-compared parameters of each pixel point to be compared in the image to be compared, includes: Performing noise reduction processing on the template image and the image to be compared; Determining the template gradient corresponding to the template image parameters of each template pixel point in the template image after noise reduction processing, and determining the comparison gradient corresponding to the image-to-be-compared parameters of each pixel point to be compared in the image to be compared after noise reduction processing.

8. An electronic device, characterized in that, The device includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the image matching method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, Computer program instructions are stored on the computer-readable storage medium, and when the computer program instructions are executed by the processor, they implement the image matching method according to any one of claims 1-7.

10. A computer program product, characterized in that, When the instructions in the computer program product are executed by the processor of the electronic device, the electronic device is caused to execute the image matching method according to any one of claims 1-7.

Images