Image processing method, device and equipment and computer readable storage medium

Abstract

The invention relates to the technical field of image processing, in particular to an image processing method, device and equipment and a computer readable storage medium. Performing template matching on the to-be-detected image and a pre-stored template image, and determining an initial matching area in the to-be-detected image and a corresponding matching association degree; respectively comparing the matching correlation degree with a preset upper end threshold value and a preset lower end threshold value; when the matching correlation degree is between the upper end threshold value and the lower end threshold value, generating a pre-detection area according to the initial matching area, and performing matching processing on the pre-detection area again to obtain center position information of the object to be detected; the pre-detection area comprises the initial matching area and is consistent with the template image in size. And the pre-detection region with high similarity between the pattern and the reference image is obtained by judging the size of the double thresholds, so that the success rate and the universality of image processing are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image processing method, device, equipment and computer-readable storage medium. Background Art

[0002] In photolithography technology, different photolithography layers need to be aligned through overlay marks so that the structures between different layers can match each other and achieve the preset design effect. This alignment process is called image processing.

[0003] Among various overlay marks, due to the strong periodic diffraction characteristics of grating overlay marks, compared with strip overlay marks, the contrast of the mark will be reversed with the change of height, which may cause the brightness of the collected image information to change. Usually, in overlay measurement equipment, it is necessary to compare and identify the image acquired at the measured point with the stored reference image, find the mark, move the mark to the specified position in the field of view (complete alignment) and measure it. Therefore, for some material combinations with fast phase changes, when performing height optimization or overlay measurement, the reversal of the mark contrast makes it impossible for traditional grayscale matching algorithms to recognize and capture the mark image, which leads to image processing failure.

[0004] Therefore, how to overcome the contrast inversion problem of grating-type overlay marks and improve the success rate of image processing is an urgent problem to be solved by those skilled in the art. Summary of the invention

[0005] The purpose of the present invention is to provide an image processing method, device, equipment and computer-readable storage medium to solve the problem in the prior art that contrast inversion of marks causes matching failure during image processing.

[0006] In order to solve the above technical problems, the present invention provides an image processing method, comprising:

[0007] Receiving an image of the object to be tested;

[0008] Performing template matching between the image to be tested and a pre-stored template image to determine an initial matching area in the image to be tested and a corresponding matching correlation degree;

[0009] Compare the matching correlation degree with a preset upper threshold and a preset lower threshold respectively;

[0010] When the matching correlation degree is between the upper threshold and the lower threshold, a pre-detection area is generated based on the initial matching area, and the pre-detection area is re-matched to obtain the center position information of the object to be detected; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image.

[0011] Optionally, in the image processing method, performing a re-matching process on the pre-detection area includes:

[0012] Acquire a reference image, wherein the reference image is centrally symmetrical;

[0013] Allowing the reference image and the pre-detection area to move relative to each other, and obtaining the correlation between the reference image and the pre-detection area at each relative position;

[0014] The center position information of the object to be detected is determined according to the position information of the point corresponding to the center of the reference image in the pre-detection area when the correlation between the reference image and the pre-detection area is the largest.

[0015] Optionally, in the image processing method, the pre-detection area is a centrally symmetrical image, and acquiring a reference image includes:

[0016] Performing centrosymmetric processing on the pre-detection area to obtain a reference image;

[0017] Alternatively, obtaining a reference image includes:

[0018] A standard image of the object to be tested is retrieved from a preset template library as a reference image.

[0019] Optionally, in the image processing method, before generating the pre-detection area according to the initial matching area, the method further includes:

[0020] Obtaining a template center point corresponding to the center of the template image on the initial matching area;

[0021] Determining whether the center point of the template exceeds the boundary of the image to be tested;

[0022] Accordingly, generating a pre-detection area according to the initial matching area includes:

[0023] When the center point of the template does not exceed the boundary of the image to be detected, a pre-detection area is generated according to the initial matching area.

[0024] Optionally, the image processing method further includes: performing a constraint region judgment operation on the initial matching region, the constraint region judgment operation including: obtaining a template center of the template image at the initial matching region; judging whether the template center exceeds a standard constraint region in the image to be tested, the standard constraint region being configured inside a boundary of the image to be tested, and a distance from the boundary of the image to be tested being equal to or greater than half of a corresponding side length of the template image;

[0025] If not, generating the pre-detection area according to the initial matching area includes: using the initial matching area as the pre-detection area;

[0026] If so, generating a pre-detection area based on the initial matching area includes: setting the non-matching part of the template image as the image out-of-bounds area of ​​the image to be tested, and obtaining image content corresponding to the image out-of-bounds area from the image to be tested or an image adjacent to the image to be tested; and obtaining the preset detection area by supplementing the initial matching area with the obtained image content.

[0027] Optionally, in the image processing method, when it is determined that the center of the template does not exceed the boundary of the image to be tested, the constraint area determination operation is performed on the initial matching area.

[0028] Optionally, in the image processing method, obtaining image content corresponding to the image out-of-bounds area from the image to be tested includes:

[0029] The image in the initial matching area is subjected to central symmetry or mirror symmetry to obtain image content corresponding to the out-of-bounds area of ​​the image; wherein the initial matching area does not include the out-of-bounds area of ​​the image.

[0030] Optionally, in the image processing method, moving the reference image relative to the pre-detection area and acquiring the correlation between the reference image and the pre-detection area at each relative position includes:

[0031] Moving the reference image relative to the pre-detection area along a preset first direction, and calculating the correlation of pixels in the first direction at each relative position;

[0032] and / or, moving the reference image relative to the pre-detection area along a preset second direction, and calculating the correlation of pixels in the second direction at each relative position;

[0033] The first direction is perpendicular to the second direction;

[0034] Correspondingly, determining the center position information of the object to be detected according to the position information of the point corresponding to the center of the reference image in the pre-detection area when the correlation between the reference image and the pre-detection area is the largest includes:

[0035] Obtaining a first center position of the reference image when the correlation degree of pixels in the first direction is maximum;

[0036] and / or, obtaining a second center position of the reference image when the correlation of pixels in the second direction is maximum;

[0037] The center position information of the object to be measured is determined according to the first center position and / or the second center position.

[0038] Optionally, in the image processing method, moving the reference image relative to the pre-detection area along a preset first direction and calculating the correlation of pixels in the first direction at each relative position includes:

[0039] Acquire a first region of interest in the pre-detection area and a second region of interest in the reference image, wherein the first region of interest and the second region of interest are both centrally symmetrical and have respective centers of symmetry;

[0040] Based on the pixel grayscale distribution in the first direction, the first region of interest and the second region of interest are relatively moved to obtain correlation at each relative position;

[0041] Accordingly, the determination of the center position information of the object to be measured includes:

[0042] Determine the position information of the center of the object to be detected in the first direction according to the position information of the point corresponding to the symmetry center of the second region of interest in the pre-detection area when the correlation between the first region of interest and the second region of interest in the first direction is the largest regarding the pixel grayscale distribution;

[0043] And / or, moving the reference image relative to the pre-detection area along a preset second direction, and calculating the correlation of pixels in the second direction at each relative position, comprises:

[0044] Acquire a third region of interest in the pre-detection area and a fourth region of interest in the reference image, wherein the third region of interest and the fourth region of interest are both centrally symmetrical and have respective centers of symmetry;

[0045] Based on the pixel grayscale distribution in the second direction, the third region of interest and the fourth region of interest are relatively moved to obtain correlation at each relative position;

[0046] Accordingly, the determination of the center position information of the object to be measured includes:

[0047] According to the position information of the point corresponding to the point in the pre-detection area when the correlation between the third region of interest and the fourth region of interest in the second direction regarding the pixel grayscale distribution is the largest, the position information of the center of the object to be detected in the second direction is determined.

[0048] Optionally, in the image processing method, moving the reference image relative to the pre-detection area and acquiring the correlation between the reference image and the pre-detection area at each relative position includes:

[0049] Acquire relative movement parameters between the reference image and the pre-detection area, wherein the relative movement parameters include at least one of a movement direction, a stepping distance, and a reference image size;

[0050] The reference image is moved relative to the pre-detection area based on the relative movement parameter, and the correlation between the reference image and the pre-detection area at each relative position is obtained during the movement.

[0051] Optionally, in the image processing method, the re-matching processing of the pre-detection area includes at least one of the mean absolute difference correlation calculation method, the absolute error sum correlation calculation method, the error square sum correlation calculation method, the average error square sum correlation calculation method, the normalized cross-correlation correlation calculation method, the sequential similarity detection correlation calculation method or the Hadamard transform correlation calculation method.

[0052] Optionally, in the image processing method, the reference image has a plurality of pattern features of the object to be detected, and are periodically distributed in the first direction and the second direction.

[0053] Optionally, in the image processing method, after comparing the matching association degree with a preset upper threshold and a preset lower threshold respectively, the method further includes:

[0054] When the matching correlation degree is greater than the upper threshold, the center position information of the object to be detected is determined according to the initial matching area.

[0055] Optionally, in the image processing method, determining the center position information of the object to be detected according to the initial matching area includes:

[0056] Acquire the vertex coordinate information of the area according to the initial matching area;

[0057] The center position information of the object to be measured is determined according to the vertex coordinate information.

[0058] Optionally, in the image processing method, performing template matching between the image to be tested and a pre-stored template image to determine an initial matching area in the image to be tested and a corresponding matching correlation degree includes:

[0059] Determining a frame selection window according to a pre-stored template image, wherein a pixel size of the frame selection window is consistent with a pixel size of the template image;

[0060] Using the frame selection window to scan and screenshot the image to be tested to obtain a plurality of areas to be selected;

[0061] The candidate matching association degrees corresponding to the plurality of candidate regions are calculated respectively, and the candidate region with the largest candidate matching association degree is determined as the initial matching region, and the corresponding candidate matching association degree is the corresponding matching association degree.

[0062] Optionally, in the image processing method, after comparing the matching association degree with a preset upper threshold and a preset lower threshold respectively, the method further includes:

[0063] When the matching correlation degree is less than the lower threshold, an alarm message about an error prompt is generated.

[0064] An image processing device, comprising:

[0065] A receiving module, used for receiving an image of the object to be tested;

[0066] A template matching module, used to perform template matching between the image to be tested and a pre-stored template image, and determine an initial matching area in the image to be tested and a corresponding matching correlation degree;

[0067] A comparison module, used to compare the matching correlation degree with a preset upper threshold and a preset lower threshold respectively;

[0068] A re-matching module is used to generate a pre-detection area based on the initial matching area when the matching correlation degree is between the upper threshold and the lower threshold, and to perform re-matching processing on the pre-detection area to obtain the center position information of the object to be detected; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image.

[0069] An image processing device, comprising:

[0070] Memory for storing computer programs;

[0071] A processor is used to implement the steps of any one of the above-mentioned image processing methods when executing the computer program.

[0072] A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned image processing methods.

[0073] The image processing method provided by the present invention receives the image to be tested of the object to be tested; performs template matching between the image to be tested and the pre-stored template image, determines the initial matching area in the image to be tested and the corresponding matching correlation; compares the matching correlation with the preset upper threshold and the preset lower threshold respectively; when the matching correlation is between the upper threshold and the lower threshold, generates a pre-detection area according to the initial matching area, performs matching processing on the pre-detection area again, and obtains the center position information of the object to be tested; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image. The present invention first determines the initial matching area whose image features are relatively close to the template image through template matching, and then screens out the pre-detection area where there may be contrast inversion but the pattern is consistent with the reference image through double threshold size judgment, determines the geometric center of the object to be tested in the pre-detection area, and can still accurately identify the object to be tested when the contrast is reversed due to height changes or process differences, which greatly improves the success rate and versatility of image processing. The present invention also provides an image processing device, equipment and computer-readable storage medium having the above-mentioned beneficial effects. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0075] Figure 1 A flowchart of a specific implementation of the image processing method provided by the present invention;

[0076] Figure 2 A flowchart of another specific implementation of the image processing method provided by the present invention;

[0077] Figure 3 A process flow chart of a specific implementation of the image processing method provided by the present invention;

[0078] Figure 4 A flowchart of another specific implementation of the image processing method provided by the present invention;

[0079] Figure 5A schematic structural diagram of a specific implementation of the image processing device provided by the present invention.

[0080] The figure includes: 100-receiving module, 200-template matching module, 300-comparison module, 400-re-matching module, 01-image out-of-bounds area, 02-initial matching area. DETAILED DESCRIPTION

[0081] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0082] The core of the present invention is to provide an image processing method, a flowchart of a specific implementation method thereof is shown as follows: Figure 1 As shown, it is called specific implementation mode 1, including:

[0083] S101: receiving an image of an object to be tested.

[0084] The image to be tested includes the graphics corresponding to the grating type overlay marks to be found.

[0085] S102: performing template matching between the image to be tested and a pre-stored template image to determine an initial matching area in the image to be tested and a corresponding matching correlation degree.

[0086] The matching correlation (or matching score) reflects the similarity between the matching area and the template. The higher the matching correlation, the higher the matching accuracy.

[0087] It should be noted that the template matching mentioned is a technology for finding the most matching (similar) part of an image with another reference image. For example, when searching for a target in a large image, it is known that the image contains the target to be found, and the target has the same size, direction and image elements as the template. Through a certain algorithm, the target can be found in the image and its coordinate position can be determined.

[0088] In this embodiment, the object to be tested is a wafer or a specific mark on a wafer. In other embodiments, the object to be tested can also be other types of targets to be tested, such as a glass substrate, etc., which is not limited here.

[0089] As a specific implementation, a design image corresponding to the object to be tested may be obtained as a template image, wherein the design image may be a binary image or a design layout.

[0090] In other embodiments, obtaining a template image corresponding to the object to be tested includes: obtaining a photographed image of a standard object as a template image, wherein the standard object is identical to the object to be tested and meets the design requirements (for example, when the object to be tested is an overlay mark, the standard object meets the alignment requirements).

[0091] In other embodiments, obtaining a template image without a corresponding object to be tested includes: providing a theoretical model of the interaction between the sample and light, wherein the theoretical model is used to represent the relationship between the intensity distribution of the outgoing light after the light acts on the sample and the physical parameters of the sample; substituting the physical parameters of the object to be tested into the theoretical model to obtain a simulated image as a template image.

[0092] As a specific implementation, the specific process of performing template matching between the image to be tested and the pre-stored template image in this step includes:

[0093] S1021: Determine a frame selection window according to a pre-stored template image, wherein a pixel size of the frame selection window is consistent with a pixel size of the template image.

[0094] S1022: Using the frame selection window to perform a scanning screenshot of the image to be tested to obtain a plurality of areas to be selected.

[0095] It can be seen from the above two steps that the frame selection window does not specifically refer to a certain area on the image to be tested, but is a tool for assisting in capturing an image, which specifies the size of the selected area to be captured.

[0096] S1023: Calculate the candidate matching association degrees corresponding to the plurality of candidate regions respectively, determine the candidate region with the largest candidate matching association degree as the initial matching region, and the corresponding candidate matching association degree as the corresponding matching association degree.

[0097] In this specific embodiment, a method is adopted in which a selection window is determined according to the template image, and then multiple screenshots are taken on the image to be tested according to the selection window to determine the selected area, and a correlation degree is calculated for the selected area. The process of obtaining the selected area by this method is simple and the calculation efficiency is high. Of course, the image to be tested can also be divided by other methods, and the present invention is not limited here.

[0098] The template matching process can be referred to for further understanding of this specific implementation method. The template matching processing includes: using a matching window of the same size as the template image, traversing each pixel point of the image to be tested, and calculating the correlation score between the area where the matching window is located in the image to be tested and the template image as the similarity. The correlation score is negatively correlated to the variance or standard deviation of the grayscale of each pixel point of the matching window and the template image; obtaining the area where the matching window with the maximum correlation score is located as the matching area, and each pixel point of the template image has a one-to-one correspondence with each pixel point of the matching area.

[0099] Specifically, the matching window is slid in the image to be tested according to a preset sliding direction, and the correlation score between the template image and the area where the current matching window is located is calculated after each sliding, thereby obtaining multiple correlation scores; the area where the matching window corresponding to the maximum correlation score is located is selected as the matching area that matches the template image. For example, the matching window can be slid from the upper left corner of the image to be tested to the right, and each sliding amount is the size of a column of pixels. When it slides to the rightmost side, it slides downward, and the downward sliding amount is the size of a row of pixels, and then starts to slide from the leftmost side of the image to be tested to the left, and so on, until the matching window traverses every pixel point of the image to be tested.

[0100] It should be noted that the greater the similarity, the smaller the variance or standard deviation of the grayscale of each pixel in the matching window area and the template image. Therefore, the correlation score is negatively correlated with the variance and standard deviation of the grayscale of each pixel in the matching window and the template image.

[0101] Methods for calculating the correlation score between the image to be tested and the template image include Mean Absolute Differences (MAD), Sum of Absolute Differences (SAD), Sum of Squared Differences (SSD), Mean Square Differences (MSD), Normalized Cross Correlation (NCC), Sequential Similiarity Detection Algorithm (SSDA) or Hadamard Transform. As an example, the normalized cross correlation algorithm is used to calculate the correlation score.

[0102] Specifically, the formula of the normalized cross-correlation algorithm is:

[0103] or

[0104]

[0105] Where I1 is the template image, I2 is the image to be tested, Wp is the matching window, and the value range of NCC is between [-1,1].

[0106] Among them, NCC(p,d) represents the correlation score, I1(x,y) represents the gray value of the pixel (x,y) in the template image, represents the grayscale mean of the pixels in the template image, I2(x+p,y+d) represents the grayscale value at the pixel point (x+p,y+d) in the image to be tested, represents the grayscale mean of the pixels in the image to be tested, Wp represents the area where the matching window is located in the image to be tested, and · represents the product operation.

[0107] As a preferred implementation, performing template matching between the image to be tested and a pre-stored template image, and determining an initial matching area in the image to be tested and a corresponding matching correlation degree includes:

[0108] Perform template matching between the image to be tested and the pre-stored template image to determine an initial matching area in the image to be tested and a corresponding normalized matching correlation degree.

[0109] In this preferred embodiment, the matching correlation is normalized so that the matching correlations in different evaluation criteria and different systems can be subject to the same set of upper and lower threshold constraints, thereby improving the compatibility and versatility of the system, while reducing the amount of computation required for subsequent calculations and improving computational efficiency.

[0110] S103: Compare the matching correlation degree with a preset upper threshold and a preset lower threshold respectively.

[0111] The upper threshold and the lower threshold are two thresholds of different sizes, and the upper threshold is greater than the lower threshold. As can be seen from the previous text, the higher the matching correlation, the higher the similarity between the obtained matching area and the reference image, that is, the higher the probability that there is a grating-type overlay mark within the initial matching area.

[0112] As a preferred implementation, this step includes:

[0113] A1: Determine whether the matching correlation is greater than a preset upper threshold;

[0114] A2: When the matching correlation degree is not greater than the upper threshold, determine whether the matching correlation degree is less than a preset lower threshold.

[0115] In this preferred embodiment, the relationship between the matching correlation degree and the upper threshold is first judged. In actual production, in most cases the image to be tested and the template image can be successfully matched, that is, the corresponding matching scores are higher than the upper threshold. Therefore, the matching score is first compared with the upper threshold. If it exceeds the upper threshold, it can be directly judged as qualified and the mark center position information can be determined. No other comparison calculations are required, which can greatly save computing power and improve computing efficiency.

[0116] S104: When the matching correlation degree is between the upper threshold and the lower threshold, a pre-detection area is generated according to the initial matching area, and the pre-detection area is re-matched to obtain the center position information of the object to be detected; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image.

[0117] In the prior art, only one threshold is set. If it is higher than the threshold, the match is determined to be successful, and if it is lower than the threshold, the match is determined to be unsuccessful. In the present application, if it is higher than the larger upper threshold, the match is directly determined to be successful, and the subsequent processing can be the same as in the prior art. If it is lower than the smaller lower threshold, the match is determined to be unsuccessful, and the subsequent processing can also be the same as in the prior art. In addition, the present invention also establishes a separately processed matching correlation interval, that is, when the matching correlation is between the upper threshold and the lower threshold, it is determined that there may be grating-type overlay marks in the matching area at this time, and a pre-detection area consistent with the size of the template image is divided to enable a more accurate re-matching to improve the matching success rate.

[0118] Specifically, the rematching process performed on the pre-detection area includes at least one of the mean absolute difference correlation calculation method, the absolute error sum correlation calculation method, the error square sum correlation calculation method, the average error square sum correlation calculation method, the normalized cross-correlation correlation calculation method, the sequential similarity detection correlation calculation method or the Hadamard transform correlation calculation method. A variety of correlation calculation methods are given above, all of which can maintain a high calculation efficiency while ensuring a high correlation calculation accuracy. Of course, other correlation calculation methods can also be used according to actual conditions, and the present invention is not limited here.

[0119] Specifically, the re-matching process in this step includes:

[0120] B1: Acquire a reference image, where the reference image is centrally symmetrical.

[0121] This specific implementation can be divided into two categories: autocorrelation processing and template matching processing. The corresponding acquisition of the reference image includes:

[0122] B1-1: Perform centrosymmetric processing on the pre-detection area to obtain a reference image.

[0123] The center processing is to perform center-symmetric mapping on the pre-detection area. The specific processing method may be to rotate the pre-detection area 180° around the midpoint, and the obtained image is the reference image.

[0124] This step uses the high degree of symmetry of the grating type overlay mark about its own midpoint, and uses autocorrelation matching for further verification to obtain a reference image. Autocorrelation matching can be regarded as a process of finding symmetry in the matching area. By finding symmetry in the matching area, the midpoint of the symmetry is determined, and the position information of the midpoint at this time can be regarded as the central position information.

[0125] It should be noted that the autocorrelation matching algorithm is essentially the matched filtering process of image signal processing, which is manifested as the autocorrelation process of the signal with respect to a certain time shift. Correlation functions and convolution functions are often used to participate in matched filtering to achieve the purpose of autocorrelation matching processing. In addition, the time domain processing of the image signal can be linked with the frequency domain processing through convolution, and FFT can be used as a processing tool for fast calculation.

[0126] or:

[0127] B1-2: Retrieve a standard image of the object to be tested from a preset template library as a reference image.

[0128] There is no order relationship between step B1-1 and step B1-2. They are just two parallel means of obtaining the reference image, which can be selected according to actual conditions. It should be noted that the reference image is a periodic image of the object to be tested. Of course, the reference image should be selected with higher accuracy or similarity than the template image, so as to improve the accuracy of subsequent re-matching.

[0129] B2: Move the reference image and the pre-detection area relative to each other, and obtain the correlation between the reference image and the pre-detection area at each relative position.

[0130] As a specific implementation, this step includes:

[0131] C1-1: Move the reference image relative to the pre-detection area along a preset first direction, and calculate the correlation of pixels in the first direction at each relative position.

[0132] and / or,

[0133] C1-2: Move the reference image relative to the pre-detection area along a preset second direction, and calculate the correlation of pixels in the second direction at each relative position. There is no order relationship between step C1-1 and step C1-2. In actual work, only one of them can be selected for execution. It should be noted that the first direction is perpendicular to the second direction.

[0134] That is, when performing correlation detection on the reference image and the pre-detection area, it is possible to detect the correlation between the reference image and the pre-detection area in only a single direction. Of course, it is also possible to monitor the correlation in two mutually perpendicular directions, thereby achieving a more accurate correlation measurement effect.

[0135] Correspondingly, determining the center position information of the object to be detected according to the position information of the point corresponding to the center of the reference image in the pre-detection area when the correlation between the reference image and the pre-detection area is the largest includes:

[0136] C2-1: Obtain the first center position of the reference image when the correlation of pixels in the first direction is maximum.

[0137] It can be considered that the position of the center of the object to be measured in the first direction is obtained in this step. Of course, step C2-1 can be executed only on the premise of executing step C1-1.

[0138] and / or,

[0139] C2-2: Obtain the second center position of the reference image when the correlation of pixels in the second direction is maximum.

[0140] It can be considered that the position of the center of the object to be measured in the second direction is obtained in this step. Of course, step C2-2 can only be executed on the premise that step C1-2 is executed.

[0141] C3: Determine the center position information of the object to be measured according to the first center position and / or the second center position.

[0142] The first center position can be directly used as the position of the center position in the first direction in the center position information of the object to be measured. Similarly, the second center position can be directly used as the position of the center position in the second direction in the center position information of the object to be measured. Of course, further calculations can be made based on the first center position and the second center position to obtain the center position information of the object to be measured. In addition, it should be noted that only if the corresponding first center position or the second center position is obtained in the aforementioned step can it be used in this step to calculate the center position information of the object to be measured.

[0143] B3: Determine the center position information of the object to be detected according to the position information of the point corresponding to the center of the reference image in the pre-detection area when the correlation between the reference image and the pre-detection area is the largest.

[0144] Furthermore, the aforementioned step C1-1 includes:

[0145] D1-1: Acquire a first region of interest in the pre-detection area and a second region of interest in the reference image, wherein the first region of interest and the second region of interest are both centrally symmetrical and have respective centers of symmetry.

[0146] The region of interest in this step is a centrally symmetric region of interest, which is determined by the properties of the overlay mark itself.

[0147] D1-2: Based on the pixel grayscale distribution in the first direction, the first region of interest and the second region of interest are relatively moved to obtain correlation at each relative position.

[0148] That is, in this step, two corresponding regions of interest are used to record the pixel correlations at different positions in the first direction.

[0149] It should be noted that the correlation calculation between the two regions of interest in this specific implementation is performed based on the grayscale of the pixels, discarding the color data of the pixels themselves and only considering the grayscale, which can greatly reduce the difficulty of calculation and improve the calculation efficiency.

[0150] Accordingly, the determination of the center position information of the object to be measured includes:

[0151] D1-3: Determine the position information of the center of the object to be detected in the first direction based on the position information of the symmetry center of the second region of interest at the point corresponding to the pre-detection area when the correlation between the first region of interest and the second region of interest in the first direction regarding the grayscale distribution of pixels is the largest.

[0152] In this specific implementation, through the region of interest (ROI), template matching of image recognition is used to determine the potential area where the matching object may exist. The pre-detection area and the reference image can be quickly segmented using the region of interest. Each time the correlation is calculated, only the pixels in the corresponding region of interest need to be calculated, which greatly simplifies the calculation complexity and improves the calculation efficiency. Of course, other means can also be used to calculate the correlation in the first direction, which will not be elaborated in this application.

[0153] The above article describes in detail the correlation detection of the pixel grayscale distribution in the first direction in combination with two regions of interest. The following is a similar process for the second direction, that is, the above step C1-2, including:

[0154] D2-1: Acquire a third region of interest in the pre-detection area and a fourth region of interest in the reference image, wherein the third region of interest and the fourth region of interest are both centrally symmetrical and have respective centers of symmetry.

[0155] D2-2: Based on the pixel grayscale distribution in the second direction, the third region of interest and the fourth region of interest are relatively moved to obtain correlations at respective relative positions.

[0156] Accordingly, the determination of the center position information of the object to be measured includes:

[0157] D2-3: Determine the position information of the center of the object to be detected in the second direction based on the position information of the point corresponding to the symmetry center of the fourth region of interest in the pre-detection area when the correlation between the third region of interest and the fourth region of interest in the second direction regarding the pixel grayscale distribution is the largest.

[0158] For the correlation matching based on the region of interest in the second direction, please refer to the correlation matching based on the region of interest in the first direction mentioned above, which will not be elaborated here.

[0159] On the other hand, the step of moving the reference image relative to the pre-detection area and obtaining the correlation between the reference image and the pre-detection area at each relative position includes:

[0160] E1: Acquire relative movement parameters between the reference image and the pre-detection area, where the relative movement parameters include at least one of a movement direction, a step distance, and a reference image size.

[0161] That is, the relative movement parameters in this step are the basis for the relative movement between the reference image and the pre-detection area. Preferably, the relative movement parameters include the moving direction, the stepping distance and the reference image size.

[0162] E2: moving the reference image relative to the pre-detection area based on the relative movement parameter, and obtaining the correlation between the reference image and the pre-detection area at each relative position during the movement.

[0163] This step illustrates the relative movement parameters obtained in the aforementioned step E1. Of course, it can also be combined with other parameters pre-stored or obtained by other means, which is not limited in the present invention. In this preferred embodiment, a processing method of using template matching for the re-matching is provided. Of course, the re-matching can use other matching means, which is not limited in the present invention.

[0164] In addition, the reference image has a plurality of pattern features of the object to be tested, and is periodically distributed in the first direction and the second direction.

[0165] The image processing method provided by the present invention receives the image to be tested of the object to be tested; performs template matching between the image to be tested and the pre-stored template image, determines the initial matching area in the image to be tested and the corresponding matching correlation; compares the matching correlation with the preset upper threshold and the preset lower threshold respectively; when the matching correlation is between the upper threshold and the lower threshold, generates a pre-detection area according to the initial matching area, performs re-matching processing on the pre-detection area, and obtains the center position information of the object to be tested; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image. The present invention first determines the initial matching area whose image morphology is relatively close to the template image through template matching, and then screens out the pre-detection area where there may be contrast inversion, but the pattern is consistent with the reference image through double threshold size judgment, determines the geometric center of the object to be tested in the pre-detection area, and can still accurately identify the object to be tested when the contrast is reversed due to height changes or process differences, which greatly improves the success rate and versatility of image processing.

[0166] On the basis of the first specific implementation mode, the generation process of the pre-detection area is further improved to obtain the second specific implementation mode, the flow chart of which is as follows: Figure 2 As shown, including:

[0167] S201: Receive an image of an object to be tested.

[0168] S202: Perform template matching between the image to be tested and a pre-stored template image to determine an initial matching area in the image to be tested and a corresponding matching correlation degree.

[0169] S203: Compare the matching correlation degree with a preset upper threshold and a preset lower threshold respectively.

[0170] S204: When the matching correlation degree is between the upper threshold and the lower threshold, obtaining a template center point corresponding to the center of the template image on the initial matching area.

[0171] Note that the template center point is a point on the initial matching area, which just corresponds to the center of the template image. The center position of the template image in this step refers to the center position of the position where the template image is located when the template image matches the initial matching area.

[0172] S205: Determine whether the center point of the template exceeds the boundary of the image to be tested.

[0173] S206: When the center point of the template does not exceed the boundary of the image to be tested, a pre-detection area is generated according to the initial matching area, and the pre-detection area is re-matched to obtain the center position information of the object to be tested; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image.

[0174] The difference between this specific implementation and the above specific implementation is that, in this specific implementation, before determining the pre-detection area, the position of the center point of the template is checked, and the remaining steps are the same as the above specific implementation, which will not be elaborated here.

[0175] If the center point of the template exceeds the boundary of the image to be tested, it means that only a small half of the template image is within the image to be tested. At this time, it can be regarded as that most of the features of the template image are not matched. In this case, the initial matching area obtained does not have much reference value, and there is no need to execute subsequent steps. In this case, the matching needs to be terminated. Only when the center point of the template does not exceed the boundary of the image to be tested, the pre-detection area will be generated according to the initial matching area, and the subsequent steps will be performed. This is equivalent to discovering problems in matching early and avoiding waste of computing power.

[0176] Furthermore, this specific implementation further includes:

[0177] F1: Perform a constraint area judgment operation on the initial matching area, the constraint area judgment operation comprising: obtaining the template center of the template image at the initial matching area; judging whether the template center exceeds a standard constraint area in the image to be tested, the standard constraint area being arranged on the inner side of the boundary of the image to be tested, and the distance from the boundary of the image to be tested is equal to or greater than half of the corresponding side length of the template image.

[0178] In this step, the standard constraint area is configured on the inner side of the boundary of the image to be tested, and the distance from the boundary of the image to be tested is equal to or greater than half of the corresponding side length of the template image. This ensures that if the center of the template falls within the standard constraint area, the position of the template image corresponding to the initial matching area does not exceed the image to be tested. In other words, the initial matching area is completely in the image to be tested, and a complete image with the same size as the template image can be obtained.

[0179] F2: If not, generating a pre-detection area according to the initial matching area includes: using the initial matching area as the pre-detection area.

[0180] The "no" situation refers to the situation where the center of the template does not exceed the standard constraint area in the image to be tested, that is, the template image position corresponding to the initial matching area does not exceed the image to be tested, which is the situation that the technicians hope to see. The size of the initial matching area is comparable to that of the template image, and all image content can be obtained in the image to be tested, and can be directly used as the subsequent pre-detection area.

[0181] F3: In the case where the pre-detection area is generated based on the initial matching area, the process includes: setting the non-matching portion of the template image as the image out-of-bounds area of ​​the image to be detected, and obtaining image content corresponding to the image out-of-bounds area from the image to be detected or an image adjacent to the image to be detected; and obtaining the preset detection area by supplementing the initial matching area with the obtained image content.

[0182] The described situation refers to that the center of the template exceeds the standard constraint area in the image to be tested, that is, a part of the template image corresponding to the initial matching area is outside the image to be tested. In this case, the initial matching area needs to be filled. There are two sources for filling the image content of the initial matching area. In one case, the image to be tested is a cutout area of ​​a larger image (called the total image). At this time, the missing image content of the initial matching area (that is, the image content corresponding to the out-of-bounds area of ​​the image) can be obtained from a position adjacent to the image to be tested in the larger image.

[0183] That is, in this preferred embodiment, when the initial matching area is obtained, but the template image does not completely overlap with the image to be tested, and there is an out-of-bounds area outside the image to be tested, the image information of the out-of-bounds area is not discarded, but more image data is further obtained from the overall image of the image to be tested to fill the image information of the out-of-bounds area, and finally the pre-detection area with complete image information is obtained, so as to improve the accuracy and precision of the subsequent re-matching processing and improve the recognition success rate.

[0184] Of course, it may be impossible to obtain the overall image corresponding to the image to be tested. In this case, there is no other source of image content except the image to be tested. At this time, the image to be tested can be processed accordingly to obtain the image content of the missing part.

[0185] Specifically, obtaining the image content corresponding to the out-of-bounds area of ​​the image from the image to be tested includes:

[0186] The image in the initial matching area is subjected to central symmetry or mirror symmetry to obtain image content corresponding to the out-of-bounds area of ​​the image; wherein the initial matching area does not include the out-of-bounds area of ​​the image.

[0187] For reference Figure 3 , Figure 3 The initial matching area is divided into two parts on the left and right by the edge of the image to be tested. The initial matching area on the right is located inside the image to be tested, while the left is the out-of-bounds area of ​​the image. The image inside the out-of-bounds area of ​​the image is obtained by performing central symmetry or mirror symmetry on the image in the initial matching area.

[0188] It should be noted that the image out-of-bounds area is not a part of the template image, but a kind of position area information relative to the image to be tested.

[0189] The operations from F1 to F3 described above specifically describe the step of "generating a pre-detection area based on the initial matching area", that is, under different results of the constraint area judgment, the methods of obtaining the pre-detection area from the initial matching area include the two methods represented by F2 and F3 described above.

[0190] Further on the basis of the above preferred implementation manner, when it is determined that the center of the template does not exceed the boundary of the image to be tested, the constraint area determination operation is performed on the initial matching area.

[0191] When the center of the template exceeds the boundary of the image to be tested, it means that at least half of the template image is not matched. At this time, the matching credibility is too low. In this preferred implementation, it is chosen to be discarded, that is, only when at least half of the template image is matched, the constraint area judgment operation is performed, which eliminates the situation of low credibility and improves calculation efficiency.

[0192] Of course, further, the solutions F1 to F3 can be combined with the technical solution of the specific implementation method 2 to obtain a specific implementation method 3, the flow chart of which is as follows: Figure 4 As shown, including:

[0193] S301: Receive an image of an object to be tested.

[0194] S302: Perform template matching between the image to be tested and a pre-stored template image to determine an initial matching area in the image to be tested and a corresponding matching correlation degree.

[0195] S303: Compare the matching correlation degree with a preset upper threshold and a preset lower threshold respectively.

[0196] S304: When the matching correlation degree is between the upper threshold and the lower threshold, obtaining a template center point corresponding to the center of the template image on the initial matching area.

[0197] S305: Determine whether the center point of the template exceeds the boundary of the image to be tested.

[0198] S306: When the template center point does not exceed the boundary of the image to be tested, a constraint area judgment operation is performed on the initial matching area, and the constraint area judgment operation includes: obtaining the template center of the template image at the initial matching area; judging whether the template center exceeds a standard constraint area in the image to be tested, and the standard constraint area is configured on the inner side of the boundary of the image to be tested, and the distance from the boundary of the image to be tested is equal to or greater than half of the corresponding side length of the template image.

[0199] S307: If not, use the initial matching area as the pre-detection area.

[0200] S308: If yes, set the non-matching part of the template image to the image out-of-bounds area of ​​the image to be tested, and obtain the image content corresponding to the image out-of-bounds area from the image to be tested or an image adjacent to the image to be tested; and obtain the preset detection area by supplementing the initial matching area with the obtained image content.

[0201] S307 and S308 are just two different situations after S306 makes the judgment. There is no order of precedence between them and they can be switched at will.

[0202] S309: performing a re-matching process on the pre-detection area to obtain the center position information of the object to be detected; wherein the pre-detection area includes the initial matching area and has the same size as the template image.

[0203] The difference between this specific implementation and the above specific implementation is that the conditions for generating the pre-detection area are further specified in this specific implementation, and the remaining steps are the same as those in the above specific implementation, which will not be elaborated here.

[0204] This specific implementation combines the two pre-processing methods for the pre-detection area mentioned above. First, the position of the template center point is used to determine whether most of the features of the template image are matched. If the template center point does not exceed the boundary of the image to be tested, it can be regarded as that most of the features are matched. Only then is the processing matching area further processed. This situation is divided into two situations: the position of the template image corresponding to the initial matching area exceeds or does not exceed the image to be tested, which respectively correspond to the two processing methods after the constraint area judgment operation, namely step S307 and step S308. When it exceeds, the image of the out-of-bounds area of ​​the image is further obtained to ensure the success rate and matching accuracy of the subsequent re-matching. Of course, in actual operation, the constraint area judgment operation can also be performed first, and then it is determined whether the template center point exceeds the boundary of the image to be tested. The present invention is not limited here.

[0205] The previous article describes in detail the processing method when the matching correlation degree is between the upper threshold and the lower threshold. The following describes in detail the processing method when the matching correlation degree is greater than the upper threshold or less than the lower threshold, so as to complete all situations after the matching correlation degree is compared with the preset upper threshold and the preset lower threshold respectively.

[0206] First, after comparing the matching relevance with a preset upper threshold and a preset lower threshold respectively, the method further includes:

[0207] When the matching correlation degree is greater than the upper threshold, the center position information of the object to be detected is determined according to the initial matching area.

[0208] This is the case where only one threshold is set in the related technology, and the matching correlation is greater than the unique threshold. In this case, the match is considered successful. In this case, the frame selection area of ​​the matching area can be regarded as very accurate, and the edge of the frame selection area of ​​the matching area can be directly regarded as the edge of the grating-type overlay mark. Therefore, the center position of the matching area is also the center position of the mark.

[0209] As a specific implementation manner, determining the center position information of the object to be detected according to the initial matching area includes:

[0210] G1: Acquire the vertex coordinate information of the area according to the initial matching area.

[0211] G2: Determine the center position information of the object to be measured according to the vertex coordinate information.

[0212] Once the vertex information of the regional polygon is known, the center position information of the initial matching region can be determined by diagonal interconnection. Of course, other methods can also be used to obtain the center position information through the vertex coordinate information, and the present invention is not limited here.

[0213] Secondly, after comparing the matching relevance with the preset upper threshold and the preset lower threshold respectively, the method further includes:

[0214] H1: When the matching correlation degree is less than the lower threshold, an alarm message about an error prompt is generated.

[0215] Continuing from the previous text, this situation is when only one threshold is set in the related technology, and the matching correlation is less than the unique threshold. In this case, it is considered that the match fails, and the algorithm does not find the grating-type overlay mark in the image to be tested. The image to be tested should be ignored or discarded. However, in actual production, non-existent grating-type overlay marks will not be sent to the machine. Therefore, the matching correlation is less than the lower threshold mostly due to other reasons, such as the clarity of the image to be tested is too low. Therefore, this specific implementation method generates an alarm message at this time to remind the staff and record the abnormal situation. It should be noted that the alarm message of the error prompt includes information such as the content of the error prompt and the type of error.

[0216] The image processing device provided by the embodiment of the present invention is introduced below. The image processing device described below and the image processing method described above can be referred to each other.

[0217] Figure 5 The structural block diagram of the image processing device provided by the embodiment of the present invention is shown in FIG. Figure 5 The image processing device may include:

[0218] The receiving module 100 is used to receive the image of the object to be tested;

[0219] The template matching module 200 is used to perform template matching between the image to be tested and a pre-stored template image, and determine an initial matching area in the image to be tested and a corresponding matching correlation degree;

[0220] A comparison module 300 is used to compare the matching correlation degree with a preset upper threshold and a preset lower threshold respectively;

[0221] The re-matching module 400 is used to generate a pre-detection area according to the initial matching area when the matching correlation degree is between the upper threshold and the lower threshold, and to perform re-matching processing on the pre-detection area to obtain the center position information of the object to be detected; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image.

[0222] As a preferred implementation, the re-matching module 400 includes:

[0223] A reference acquisition unit, used for acquiring a reference image, wherein the reference image is centrally symmetrical;

[0224] A correlation recording unit, used for moving the reference image and the pre-detection area relative to each other, and obtaining the correlation between the reference image and the pre-detection area at each relative position;

[0225] The center position determination unit is used to determine the center position information of the object to be detected based on the position information of the point corresponding to the center of the reference image in the pre-detection area when the correlation between the reference image and the pre-detection area is the largest.

[0226] As a preferred implementation, the pre-detection area is a centrally symmetrical image, and the re-matching module 400 includes:

[0227] A centrosymmetric unit, used for performing centrosymmetric processing on the pre-detection area to obtain a reference image;

[0228] Alternatively, the re-matching module 400 includes:

[0229] The template retrieving unit is used to retrieve the standard image of the object to be tested from a preset template library as a reference image.

[0230] As a preferred implementation, the re-matching module 400 further includes:

[0231] A center point corresponding unit, used for obtaining a template center point corresponding to the center of the template image on the initial matching area;

[0232] An out-of-bounds judgment unit, used to judge whether the center point of the template exceeds the boundary of the image to be tested;

[0233] Accordingly, the re-matching module 400 includes:

[0234] The first conditional pre-detection unit is configured to generate a pre-detection area according to the initial matching area when the center point of the template does not exceed the boundary of the image to be detected.

[0235] As a preferred embodiment, it also includes:

[0236] A constraint area unit, used to perform a constraint area judgment operation on the initial matching area, the constraint area judgment operation comprising: obtaining a template center of the template image at the initial matching area; judging whether the template center exceeds a standard constraint area in the image to be tested, the standard constraint area being configured inside the boundary of the image to be tested, and the distance from the boundary of the image to be tested being equal to or greater than half of the corresponding side length of the template image;

[0237] A constraint negation unit, used for, if no, taking the initial matching area as the pre-detection area;

[0238] The constraint affirmation unit is used to set the non-matching part of the template image to the image out-of-bounds area of ​​the image to be tested, and obtain the image content corresponding to the image out-of-bounds area from the image to be tested or an image adjacent to the image to be tested; and obtain the preset detection area by supplementing the initial matching area with the obtained image content.

[0239] As a preferred embodiment, it also includes:

[0240] The more than half judgment unit is used to perform the constraint area judgment operation on the initial matching area when judging that the center of the template does not exceed the boundary of the image to be tested.

[0241] As a preferred implementation, the constraint affirmation unit includes:

[0242] A symmetric image acquisition unit is used to perform central symmetry or mirror symmetry on the image in the initial matching area to obtain image content corresponding to the out-of-bounds area of ​​the image; wherein the initial matching area does not include the out-of-bounds area of ​​the image.

[0243] As a preferred implementation, the re-matching module 400 includes:

[0244] A first direction correlation unit, used to move the reference image relative to the pre-detection area along a preset first direction, and calculate the correlation of pixels in the first direction at each relative position;

[0245] And / or, a second direction correlation unit, used to move the reference image relative to the pre-detection area along a preset second direction, and calculate the correlation of pixels in the second direction at each relative position;

[0246] The first direction is perpendicular to the second direction;

[0247] Accordingly, the re-matching module 400 includes:

[0248] A first direction center determination unit, used for acquiring a first center position of the reference image when the correlation of pixels in the first direction is maximum;

[0249] and / or, a second direction center determination unit, configured to obtain a second center position of the reference image when the correlation of pixels in the second direction is maximum;

[0250] The two-direction center determination unit is used to determine the center position information of the object to be detected according to the first center position and / or the second center position.

[0251] As a preferred implementation, the re-matching module 400 includes:

[0252] A first direction interest unit, used for acquiring a first interest region in the pre-detection area and a second interest region in the reference image, wherein the first interest region and the second interest region are both centrally symmetrical and have respective symmetry centers;

[0253] A first interest correlation unit, configured to relatively move the first interest region and the second interest region based on the pixel grayscale distribution in the first direction, and obtain correlations at respective relative positions;

[0254] Accordingly, the re-matching module 400 includes:

[0255] A first direction interested position unit, used to determine the position information of the center of the object to be detected in the first direction according to the position information of the point corresponding to the symmetry center of the second area of ​​interest in the pre-detection area when the correlation between the first area of ​​interest and the second area of ​​interest in the first direction is the largest regarding the pixel grayscale distribution;

[0256] And / or, the re-matching module 400 includes:

[0257] A second direction interest unit, used for acquiring a third region of interest in the pre-detection area and a fourth region of interest in the reference image, wherein the third region of interest and the fourth region of interest are both centrally symmetrical and have respective centers of symmetry;

[0258] A second interest correlation unit is used to relatively move the third region of interest and the fourth region of interest based on the pixel grayscale distribution in the second direction to obtain correlation at each relative position;

[0259] Accordingly, the re-matching module 400 includes:

[0260] The second direction position of interest unit is used to determine the position information of the center of the object to be detected in the second direction according to the position information of the point corresponding to the symmetry center of the fourth area of ​​interest in the pre-detection area when the correlation between the third area of ​​interest and the fourth area of ​​interest in the second direction with respect to the pixel grayscale distribution is the largest.

[0261] As a preferred implementation, the re-matching module 400 includes:

[0262] A relative movement parameter unit, used to obtain relative movement parameters between the reference image and the pre-detection area, wherein the relative movement parameters include at least one of a movement direction, a step distance, and a reference image size;

[0263] A parameter moving unit is used to move the reference image relative to the pre-detection area based on the relative movement parameter, and obtain the correlation between the reference image and the pre-detection area at each relative position during the movement.

[0264] As a preferred implementation, the comparison module 300 further includes:

[0265] A high-quality matching unit is used to determine the center position information of the object to be detected according to the initial matching area when the matching correlation degree is greater than the upper threshold.

[0266] As a preferred implementation, the comparison module 300 includes:

[0267] A vertex unit, used for acquiring vertex coordinate information of an area according to the initial matching area;

[0268] A vertex-center calculation unit is used to determine the center position information of the object to be measured according to the vertex coordinate information.

[0269] As a preferred implementation, the template matching module 200 includes:

[0270] A frame selection window unit, used to determine a frame selection window according to a pre-stored template image, wherein the pixel size of the frame selection window is consistent with the pixel size of the template image;

[0271] A screenshot scanning unit, used to scan and screenshot the image to be tested using the frame selection window to obtain a plurality of areas to be selected;

[0272] The screenshot correlation calculation unit is used to calculate the candidate matching correlations corresponding to the plurality of candidate areas, determine the candidate area with the largest candidate matching correlation as the initial matching area, and the corresponding candidate matching correlation as the corresponding matching correlation.

[0273] As a preferred implementation, the comparison module 300 further includes:

[0274] The matching failure unit is used to generate an alarm message about an error prompt when the matching correlation degree is less than the lower threshold.

[0275] The image processing device provided by the present invention includes a receiving module 100, which is used to receive a test image of an object to be tested; a template matching module 200, which is used to perform template matching between the test image and a pre-stored template image to determine an initial matching area in the test image and a corresponding matching correlation degree; a comparison module 300, which is used to compare the matching correlation degree with a preset upper threshold and a preset lower threshold respectively; a re-matching module 400, which is used to generate a pre-detection area according to the initial matching area when the matching correlation degree is between the upper threshold and the lower threshold, and to perform re-matching processing on the pre-detection area to obtain the center position information of the object to be tested; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image. The present invention first determines the initial matching area whose image morphology is relatively close to the template image through template matching, and then screens out the pre-detection area where contrast inversion may exist but the pattern is consistent with the reference image through double threshold size judgment, and determines the geometric center of the object to be detected in the pre-detection area. When the object to be detected has contrast inversion due to height change or process difference, it can still be accurately identified, which greatly improves the success rate and versatility of image processing.

[0276] The image processing device of this embodiment is used to implement the aforementioned image processing method, so the specific implementation method of the image processing device can be seen in the embodiment part of the image processing method in the previous text. For example, the receiving module 100, the template matching module 200, the comparison module 300, and the re-matching module 400 are respectively used to implement steps S101, S102, S103 and S104 in the aforementioned image processing method. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part, which will not be repeated here.

[0277] The present invention also provides an image processing device, comprising:

[0278] Memory for storing computer programs;

[0279] A processor is used to implement the steps of any of the above-mentioned image processing methods when executing a computer program. The image processing method provided by the present invention receives an image of an object to be tested; performs template matching on the image to be tested and a pre-stored template image to determine the initial matching area and the corresponding matching correlation degree in the image to be tested; compares the matching correlation degree with a preset upper threshold and a preset lower threshold respectively; when the matching correlation degree is between the upper threshold and the lower threshold, generates a pre-detection area according to the initial matching area, performs a re-matching process on the pre-detection area, and obtains the center position information of the object to be tested; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image. The present invention first determines the initial matching area whose image morphology is relatively close to the template image through template matching, and then screens out the pre-detection area where there may be contrast inversion, but the pattern is consistent with the reference image through double threshold size judgment, determines the geometric center of the object to be tested in the pre-detection area, and can still accurately identify the object to be tested when the contrast is reversed due to height changes or process differences, which greatly improves the success rate and versatility of image processing.

[0280] The present invention also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of any of the above-mentioned image processing methods are implemented. The image processing method provided by the present invention receives an image to be tested of the object to be tested; performs template matching between the image to be tested and a pre-stored template image to determine an initial matching area in the image to be tested and a corresponding matching correlation; compares the matching correlation with a preset upper threshold and a preset lower threshold respectively; when the matching correlation is between the upper threshold and the lower threshold, generates a pre-detection area according to the initial matching area, performs a re-matching process on the pre-detection area, and obtains the center position information of the object to be tested; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image. The present invention first determines the initial matching area whose image morphology is relatively close to the template image through template matching, and then screens out the pre-detection area where contrast inversion may exist but the pattern is consistent with the reference image through double threshold size judgment, and determines the geometric center of the object to be detected in the pre-detection area. When the object to be detected has contrast inversion due to height change or process difference, it can still be accurately identified, which greatly improves the success rate and versatility of image processing.

[0281] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0282] It should be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprises a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0283] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0284] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0285] The image processing method, device, equipment and computer-readable storage medium provided by the present invention are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. An image processing method, characterized in that: include: Receiving an image of the object to be tested; Performing template matching between the image to be tested and a pre-stored template image to determine an initial matching area in the image to be tested and a corresponding matching correlation degree; Compare the matching correlation degree with a preset upper threshold and a preset lower threshold respectively; When the matching correlation degree is between the upper threshold and the lower threshold, a pre-detection area is generated based on the initial matching area, and the pre-detection area is re-matched to obtain the center position information of the object to be detected; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image.

2. The image processing method according to claim 1, characterized in that: Performing a rematching process on the pre-detection area includes: Acquire a reference image, wherein the reference image is centrally symmetrical; Allowing the reference image and the pre-detection area to move relative to each other, and obtaining the correlation between the reference image and the pre-detection area at each relative position; The center position information of the object to be detected is determined according to the position information of the point corresponding to the center of the reference image in the pre-detection area when the correlation between the reference image and the pre-detection area is the largest.

3. The image processing method according to claim 2, characterized in that: The pre-detection area is a centrally symmetrical image, and the obtaining of a reference image includes: Performing centrosymmetric processing on the pre-detection area to obtain a reference image; Alternatively, obtaining a reference image includes: A standard image of the object to be tested is retrieved from a preset template library as a reference image.

4. The image processing method according to claim 1, wherein: Before generating the pre-detection area according to the initial matching area, the method further includes: Obtaining a template center point corresponding to the center of the template image on the initial matching area; Determining whether the center point of the template exceeds the boundary of the image to be tested; Accordingly, generating a pre-detection area according to the initial matching area includes: When the center point of the template does not exceed the boundary of the image to be detected, a pre-detection area is generated according to the initial matching area.

5. The image processing method according to claim 1 or 4, characterized in that: Also includes: Performing a constraint region judgment operation on the initial matching region, the constraint region judgment operation comprising: obtaining a template center of the template image at the initial matching region; judging whether the template center exceeds a standard constraint region in the image to be tested, the standard constraint region being arranged inside a boundary of the image to be tested, and having a distance from the boundary of the image to be tested equal to or greater than half of the corresponding side length of the template image; If not, generating the pre-detection area according to the initial matching area includes: using the initial matching area as the pre-detection area; If so, generating a pre-detection area based on the initial matching area includes: setting the non-matching part of the template image as the image out-of-bounds area of ​​the image to be tested, and obtaining image content corresponding to the image out-of-bounds area from the image to be tested or an image adjacent to the image to be tested; and obtaining the preset detection area by supplementing the initial matching area with the obtained image content.

6. The image processing method according to claim 5, characterized in that: When it is determined that the center of the template does not exceed the boundary of the image to be tested, the constraint area determination operation is performed on the initial matching area.

7. The image processing method according to claim 5, characterized in that: Acquiring image content corresponding to the out-of-bounds area of ​​the image from the image to be tested, including: The image in the initial matching area is subjected to central symmetry or mirror symmetry to obtain image content corresponding to the out-of-bounds area of ​​the image; wherein the initial matching area does not include the out-of-bounds area of ​​the image.

8. The image processing method according to claim 2, characterized in that: The step of moving the reference image relative to the pre-detection area and obtaining the correlation between the reference image and the pre-detection area at each relative position includes: Moving the reference image relative to the pre-detection area along a preset first direction, and calculating the correlation of pixels in the first direction at each relative position; and / or, moving the reference image relative to the pre-detection area along a preset second direction, and calculating the correlation of pixels in the second direction at each relative position; The first direction is perpendicular to the second direction; Correspondingly, determining the center position information of the object to be detected according to the position information of the point corresponding to the center of the reference image in the pre-detection area when the correlation between the reference image and the pre-detection area is the largest includes: Obtaining a first center position of the reference image when the correlation degree of pixels in the first direction is maximum; and / or, obtaining a second center position of the reference image when the correlation of pixels in the second direction is maximum; The center position information of the object to be measured is determined according to the first center position and / or the second center position.

9. The image processing method according to claim 8, characterized in that: The step of moving the reference image relative to the pre-detection area along a preset first direction and calculating the correlation of pixels in the first direction at each relative position includes: Acquire a first region of interest in the pre-detection area and a second region of interest in the reference image, wherein the first region of interest and the second region of interest are both centrally symmetrical and have respective centers of symmetry; Based on the pixel grayscale distribution in the first direction, the first region of interest and the second region of interest are relatively moved to obtain correlation at each relative position; Accordingly, the determination of the center position information of the object to be measured includes: Determine the position information of the center of the object to be detected in the first direction according to the position information of the point corresponding to the symmetry center of the second region of interest in the pre-detection area when the correlation between the first region of interest and the second region of interest in the first direction is the largest regarding the pixel grayscale distribution; And / or, moving the reference image relative to the pre-detection area along a preset second direction, and calculating the correlation of pixels in the second direction at each relative position, comprises: Acquire a third region of interest in the pre-detection area and a fourth region of interest in the reference image, wherein the third region of interest and the fourth region of interest are both centrally symmetrical and have respective centers of symmetry; Based on the pixel grayscale distribution in the second direction, the third region of interest and the fourth region of interest are relatively moved to obtain correlation at each relative position; Accordingly, the determination of the center position information of the object to be measured includes: According to the position information of the point corresponding to the point in the pre-detection area when the correlation between the third region of interest and the fourth region of interest in the second direction regarding the pixel grayscale distribution is the largest, the position information of the center of the object to be detected in the second direction is determined.

10. The image processing method according to claim 8, characterized in that: The reference image has a plurality of pattern features of the object to be tested and is periodically distributed in the first direction and the second direction.

11. The image processing method according to claim 2, characterized in that: The step of moving the reference image relative to the pre-detection area and obtaining the correlation between the reference image and the pre-detection area at each relative position includes: Acquire relative movement parameters between the reference image and the pre-detection area, wherein the relative movement parameters include at least one of a movement direction, a stepping distance, and a reference image size; The reference image is moved relative to the pre-detection area based on the relative movement parameter, and the correlation between the reference image and the pre-detection area at each relative position is obtained during the movement.

12. The image processing method according to claim 1, characterized in that: The re-matching processing of the pre-detection area includes at least one of the mean absolute difference correlation calculation method, the absolute error sum correlation calculation method, the error square sum correlation calculation method, the average error square sum correlation calculation method, the normalized cross-correlation correlation calculation method, the sequential similarity detection correlation calculation method or the Hadamard transform correlation calculation method.

13. The image processing method according to claim 1, characterized in that: After comparing the matching relevance with the preset upper threshold and the preset lower threshold respectively, the method further includes: When the matching correlation degree is greater than the upper threshold, the center position information of the object to be detected is determined according to the initial matching area.

14. The image processing method according to claim 13, characterized in that: Determining the center position information of the object to be detected according to the initial matching area includes: Acquire the vertex coordinate information of the area according to the initial matching area; The center position information of the object to be measured is determined according to the vertex coordinate information.

15. The image processing method according to claim 1, characterized in that: The step of performing template matching between the image to be tested and a pre-stored template image to determine an initial matching area in the image to be tested and a corresponding matching correlation degree includes: Determining a frame selection window according to a pre-stored template image, wherein a pixel size of the frame selection window is consistent with a pixel size of the template image; Using the frame selection window to scan and screenshot the image to be tested to obtain a plurality of areas to be selected; The candidate matching association degrees corresponding to the plurality of candidate regions are calculated respectively, and the candidate region with the largest candidate matching association degree is determined as the initial matching region, and the corresponding candidate matching association degree is the corresponding matching association degree.

16. The image processing method according to claim 1, characterized in that: After comparing the matching relevance with the preset upper threshold and the preset lower threshold respectively, the method further includes: When the matching correlation degree is less than the lower threshold, an alarm message about an error prompt is generated.

17. An image processing device, characterized in that: include: A receiving module, used for receiving an image of the object to be tested; A template matching module, used to perform template matching between the image to be tested and a pre-stored template image, and determine an initial matching area in the image to be tested and a corresponding matching correlation degree; A comparison module, used to compare the matching correlation degree with a preset upper threshold and a preset lower threshold respectively; A re-matching module is used to generate a pre-detection area based on the initial matching area when the matching correlation degree is between the upper threshold and the lower threshold, and to perform re-matching processing on the pre-detection area to obtain the center position information of the object to be detected; wherein the pre-detection area includes the initial matching area and is consistent with the size of the template image.

18. An image processing device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the image processing method according to any one of claims 1 to 16 when executing the computer program.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the image processing method according to any one of claims 1 to 16 are implemented.

Images