Image distortion restoration method and device, detection method and device, terminal device and medium

Abstract

The invention discloses an image distortion restoration method, a detection method and device, a terminal device and a medium, and the image distortion restoration method comprises the steps: obtaining a reference image and a to-be-restored image; calculating gradient maps respectively corresponding to the reference image and the image to be restored in the horizontal direction and the vertical direction; calculating a pixel gray scale difference image of the reference image and the to-be-restored image; calculating the offset of each pixel point of the to-be-restored image relative to the pixel point at the corresponding position in the reference image in the horizontal direction and / or the vertical direction according to each gradient map and the pixel gray difference image; and according to the offset corresponding to each pixel point, remapping the position of each pixel point in the to-be-restored image, and obtaining a distorted restored image by using image resampling. According to the invention, the distortion of the to-be-repaired image can be eliminated, so that the detection precision during subsequent defect detection is improved.

Description

Technical Field

[0001] The present invention relates to the field of visual detection technology, and in particular, to an image distortion repair method, a detection method and device, a terminal device, and a medium. Background Art

[0002] In the visual detection industry, the difference method is a commonly used technical means in the defect detection process. Generally, a detection image of a to-be-detected part is obtained through an image detector, and differential calculation is performed with a preset comparison image to obtain defect information.

[0003] However, due to factors such as the camera installation method, lens distortion, and the jitter of the motion mechanism itself, the detection image and the comparison image are often not exactly the same. Compared with the detection image, there are some distortions in the comparison image, which has a great impact on defect detection. Summary of the Invention

[0004] The main technical problem to be solved by the present invention is that the existing comparison image is distorted relative to the detection image, affecting the detection accuracy.

[0005] According to a first aspect, an embodiment provides an image distortion repair method, including:

[0006] Obtain a reference image and an image to be repaired;

[0007] Calculate the gradient maps corresponding to the reference image and the image to be repaired in the horizontal and vertical directions respectively;

[0008] Calculate the pixel gray difference map between the reference image and the image to be repaired;

[0009] According to each gradient map and the pixel gray difference map, calculate the offset of each pixel point in the image to be repaired relative to the corresponding pixel point in the reference image in the horizontal direction and / or the vertical direction;

[0010] According to the offset corresponding to each pixel point, remap the positions of each pixel point in the image to be repaired, and obtain a distortion repair image by using image resampling.

[0011] According to a second aspect, an embodiment provides a detection method, including:

[0012] Obtain a reference image corresponding to a to-be-detected part, where the reference image is obtained by scanning the to-be-detected part through an image detector, and the scanning direction is the horizontal direction and / or the vertical direction;

[0013] Obtain a corresponding image to be repaired according to the reference image;

[0014] Obtain a distortion repair image according to the image distortion repair method described in the first aspect;

[0015] Based on the distorted repaired image and the reference image, perform preset feature detection on the reference image to obtain a detection result.

[0016] According to a third aspect, in one embodiment, a terminal device is provided, including:

[0017] A memory for storing programs;

[0018] A processor for implementing the method described in the first aspect by executing the programs stored in the memory.

[0019] According to a fourth aspect, in one embodiment, a detection device is provided, including:

[0020] An image detector configured to detect a workpiece to be measured to obtain a reference image;

[0021] A first storage module configured to store an image to be repaired;

[0022] And the terminal device described in the third aspect, configured to obtain the reference image acquired by the image detector, perform preset feature detection on the reference image according to the reference image and the image to be repaired, and output a detection result.

[0023] According to a fifth aspect, in one embodiment, a computer-readable storage medium is provided, on which a program is stored, and the program can be executed by a processor to implement the method described in the first aspect.

[0024] According to the image distortion repair method, detection method and device, terminal device and medium of the above embodiments, by calculating the gradient information of the reference image and the image to be repaired and the gray difference information between the two, the offset of each pixel point in the image to be repaired relative to the same position of the reference image can be calculated, and each pixel in the image to be repaired can be resampled according to the offset to regenerate a distorted repaired image after eliminating distortion, so as to ensure the detection accuracy during subsequent defect detection. Description of the Drawings

[0025] Figure 1 It is a flowchart of an image distortion repair method provided by an embodiment of the present application;

[0026] Figure 2 It is a schematic diagram of gradient maps corresponding to a reference image and an image to be repaired provided by an embodiment of the present application in the horizontal and vertical directions respectively;

[0027] Figure 3 It is a schematic diagram of a pixel gray difference map of a reference image and an image to be repaired provided by an embodiment of the present application;

[0028] Figure 4Schematic diagram of the offset of the image to be repaired relative to the reference image in the horizontal direction and / or vertical direction provided by an embodiment of the present application;

[0029] Figure 5 Schematic diagram of the distorted repaired image provided by an embodiment of the present application;

[0030] Figure 6 Schematic diagram of the reference image, the image to be repaired, and the distorted repaired image provided by an embodiment of the present application;

[0031] Figure 7 Schematic diagram of the gray difference map between the reference image and the image to be repaired, and the gray difference map between the reference image and the distorted repaired image provided by an embodiment of the present application;

[0032] Figure 8 For Figure 7 Gray histograms of the two corresponding gray difference maps in

[0033] Figure 9 Schematic diagram of the distortion distribution vector map of the image to be repaired and the distorted repaired image provided by an embodiment of the present application;

[0034] Figure 10 Flowchart of the detection method provided by an embodiment of the present application;

[0035] Figure 11 Schematic diagram of the reference image provided by an embodiment of the present application;

[0036] Figure 12 Schematic diagram of the image to be repaired provided by an embodiment of the present application;

[0037] Figure 13 Schematic diagram of the structure of the detection device provided by an embodiment of the present application.

[0038] Reference numerals: 1 - image detector; 2 - terminal device; 3 - first storage module. Detailed implementation manners

[0039] The present invention will be further described in detail below in conjunction with the accompanying drawings through specific embodiments. Similar elements in different embodiments are labeled with related similar element numbers. In the following embodiments, many detailed descriptions are provided to enable a better understanding of the present application. However, those skilled in the art can easily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, or methods. In some cases, some operations related to the present application are not shown or described in the specification to avoid overwhelming the core part of the present application with excessive descriptions. For those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations based on the descriptions in the specification and the general technical knowledge in the art.

[0040] In addition, the features, operations, or characteristics described in the specification can be combined in any appropriate manner to form various embodiments. At the same time, the steps or actions in the method description can also be reordered or adjusted in a manner obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for clearly describing a certain embodiment and do not mean that they are the necessary sequences, unless it is stated otherwise that a certain sequence must be followed.

[0041] The serial numbers assigned to the components in this article, such as "first", "second", etc., are only used to distinguish the described objects and do not have any sequential or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, both include direct and indirect connection (coupling).

[0042] In the field of visual detection technology, generally, according to the detection items of the test piece (such as bare wafers, processed wafers, display panels, etc.), corresponding comparison images are configured, and the differential method can be used for detection in combination with the actual detection images. The above visual detection can detect the preset features of the test piece. The preset features can be defects, positioning marks, etc. This application takes the detection of defects as an example for illustration, such as problems like scratches, impurities, breakages, and feature size deviations.

[0043] Among them, the comparison image can be understood as an image of a defect-free test piece, which can be drawn by software, or obtained by taking an image of a standard piece, or obtained by modifying a defective image. When performing defect detection, it is necessary to take a picture of the test area through an image detector to obtain the detection image of the test area. Generally, due to the resolution and detection accuracy of the image detector, etc., it is impossible to cover all the test areas of the test piece by taking a single detection image. Therefore, the test piece is generally divided into multiple test areas, and each test area needs to be subjected to an image acquisition to obtain the detection image corresponding to each test area, and a corresponding comparison image is also configured for each test area.

[0044] The research of this application finds that, since the detection images are obtained for the actual components to be measured, due to the influence of various factors such as the imaging conditions in each imaging process, the control accuracy of the motion mechanism, the installation accuracy between the detector and the motion mechanism, and the vibration during the motion process, there will be a certain degree of distortion in the multiple detection images corresponding to the same measurement area of multiple components to be measured. And the comparison image corresponding to this measurement area is generally the only one, and there will be a relative distortion between any one of the detection images and the comparison image. The above-mentioned distortion will affect the accuracy of defect detection using the differential method.

[0045] The research of this application finds that the influence brought by the distortion between the two images can be reduced by performing image distortion repair on the above-mentioned comparison image and detection images.

[0046] In the embodiment of this application, by using the detection image as the reference image and the comparison image as the image to be repaired, by calculating the gradient information of the reference image and the image to be repaired and the gray difference information between the two, the offset of each pixel point in the image to be repaired relative to the same position in the reference image can be calculated. According to the offset, each pixel in the image to be repaired can be resampled to regenerate a distortion repair image after eliminating the distortion, so as to ensure the detection accuracy during subsequent defect detection.

[0047] This application uses the detection image as the reference image and does not perform distortion repair on the detection image to avoid modifying the defects in the detection image, resulting in inaccurate results of subsequent defect detection.

[0048] As Figure 1 shown, the embodiment of this application provides an image distortion repair method, which may include:

[0049] Step 1: Obtain the reference image and the image to be repaired.

[0050] For example, the reference image (i.e., the detection image) can be collected in real time by an image detector or imported from external storage; the image to be repaired (i.e., the comparison image) can be pre-stored or imported from outside.

[0051] Step 2: Calculate the gradient maps corresponding to the reference image and the image to be repaired in the horizontal direction (such as the X direction) and the vertical direction (such as the Y direction) respectively.

[0052] It should be noted that the horizontal direction and the vertical direction mentioned here are the direction settings relative to the coordinate system of the image, rather than the horizontal and vertical concepts in the physical space; for example, the horizontal direction corresponds to the X axis of the image coordinate system, and the vertical direction corresponds to the Y axis of the image coordinate system.

[0053] In some embodiments, an edge detection algorithm can be used to calculate the gradient maps corresponding to the reference image and the image to be restored in the horizontal and vertical directions respectively. The gradient value represents the direction with the largest rate of change and the magnitude of the rate of change of a function at a certain point in mathematics. For an image, the places where the pixel values change the most are the edges of the image. Calculating the gradient of the image can detect the edge positions of the image.

[0054] In these above embodiments, the edge detection algorithm can be the Prewitt algorithm, the Sobel algorithm, the Laplacian algorithm, or the Roberts cross algorithm.

[0055] For example, when calculating the gradient values using the Sobel algorithm, the convolution kernels dx and dy corresponding to the X direction and the Y direction can be:

[0056]

[0057] As Figure 2 shown, the gradient maps corresponding to the reference image and the image to be restored in the horizontal and vertical directions respectively can be calculated.

[0058] Step 3: Calculate the pixel gray-level difference map between the reference image and the image to be restored.

[0059] As Figure 3 shown, the pixel gray-level maps of the reference image and the image to be restored are obtained respectively and the difference between the corresponding pixel points is performed to obtain the above pixel gray-level difference map. It can be seen that there are many white areas in the above pixel gray-level difference map, indicating that there is a large distortion between the reference image and the image to be restored.

[0060] Step 4: Calculate the offset of each pixel point of the image to be restored relative to the corresponding pixel point in the reference image in the horizontal direction and / or the vertical direction according to each gradient map and the pixel gray-level difference map.

[0061] In some embodiments, calculating the offset of each pixel point of the image to be restored relative to the corresponding pixel point in the reference image in the horizontal direction and / or the vertical direction according to each gradient map and the pixel gray-level difference map may include:

[0062] Step 410: For each pixel point in the image to be restored, obtain the gradient values corresponding to the pixel point in the horizontal and vertical directions in the image to be restored according to each gradient map, and obtain the gradient values corresponding to the corresponding pixel point in the reference image in the horizontal and vertical directions.

[0063] Step 420: Obtain the pixel gray-level difference corresponding to the same pixel point in the image to be restored according to the pixel gray-level difference map.

[0064] Step 430: Configure the execution function of the non-rigid image registration algorithm.

[0065] Step 440: Input the obtained gradient values and gray difference values into the execution function to obtain the offset of each pixel point in the image to be repaired relative to the corresponding pixel point in the reference image in the horizontal direction and / or vertical direction.

[0066] In some detection items, only the information of the defect in one direction is concerned. When calculating the offset, the offset in the horizontal direction or only the vertical direction can be calculated.

[0067] In some embodiments, configuring the execution function of the non-rigid image registration algorithm may include:

[0068] Set Ux as the offset in the horizontal direction and Uy as the offset in the vertical direction. Then the execution function of the non-rigid image registration algorithm (active demons) is expressed by the formula:

[0069]

[0070]

[0071] Where Dx and Dy are the gradient values in the horizontal and vertical directions at the point (x, y) on the reference image, Gx and Gy are the gradient values in the horizontal and vertical directions at the point (x, y) on the image to be repaired, ▽f is the pixel gray difference between the reference image and the image to be repaired at the corresponding point (x, y), and α is the diffusion speed. The diffusion speed coefficient can be used to control the magnitude of the offset.

[0072] In the above embodiments, configuring the execution function of the non-rigid image registration algorithm may further include:

[0073] Obtain the iterative calculation condition for outputting the offset in the horizontal direction and / or the vertical direction when the iterative calculation terminates; where the iterative calculation condition is expressed by the formula

[0074]

[0075]

[0076] Where k is the current iteration number; β is the iteration coefficient, and β can be a coefficient from 0 to 1. To further improve the convergence speed and registration accuracy, the offset obtained from the previous iterative calculation is added to the offset calculation of the current layer iteration.

[0077] Considering that the effect after one-time image distortion repair is not sufficient to meet the similarity requirements of subsequent steps, the gradient value of the image to be repaired can be calculated iteratively so that the similarity between the distorted repair image after image resampling based on the iterated gradient value and the reference image meets the requirements.

[0078] As Figure 4 shown, in some embodiments, the gradient values corresponding to the horizontal direction and the vertical direction can be different. In actual detection projects, the image distortion direction is generally not uniform. For example, when scanning the image of the workpiece to be measured, it is scanned from the X direction. The influence of factors such as the jitter of the moving mechanism itself is greater in the X direction, and the deformation in the X direction is larger than that in the Y direction. At this time, the distortion in the comparison image is mainly concentrated in the X direction.

[0079] Then, if the same diffusion parameter is used to calculate the offset in the horizontal direction and the vertical direction, it will result in a better repair effect in one direction and a worse repair effect in the other direction; or the repair effects in both directions are not good.

[0080] For the above embodiments, configuring the execution function of the non-rigid image registration algorithm may further include:

[0081] Obtain the diffusion speeds of different parameters in the horizontal direction and / or the vertical direction, which are used to update the iterative calculation conditions through variable parameter expansion calculation speed. The updated iterative calculation conditions are expressed by the formula:

[0082]

[0083]

[0084] where α x is the diffusion speed in the horizontal direction, α y is the diffusion speed in the vertical direction, β x is the iteration coefficient in the horizontal direction, and β y is the iteration coefficient in the vertical direction.

[0085] By updating the iterative calculation conditions through variable parameter expansion calculation speed and using different diffusion parameters and iteration parameters in the horizontal direction and the vertical direction to calculate the offset, the distortions in different degrees in the X and Y directions can be repaired flexibly.

[0086] The distortion repair method provided by the embodiments of the present application combines the specific detection environment of the workpiece to be measured in actual detection, and specifically adopts a variable parameter non-rigid image registration algorithm to calculate the offset in the horizontal direction and the vertical direction of the image to be repaired, which is more in line with the actual scenario and can achieve a better distortion repair effect.

[0087] In some embodiments, after calculating the offsets corresponding to each pixel, the method may further include:

[0088] Step 450: Smooth the offsets corresponding to each pixel in the image to be repaired, and update the offset of the target pixel according to the offsets corresponding to the pixels around the target pixel.

[0089] After calculating the offsets corresponding to each pixel, the change of the above offsets between each pixel may not be continuous. To make the offsets smooth and continuous globally, the coordinate offsets of the entire image are also smoothed during each iteration, which can effectively avoid the burr phenomenon that appears in the image after resampling. In these embodiments, Gaussian filtering is used to smooth the offsets corresponding to each pixel in the image to be repaired.

[0090] The principle of Gaussian filtering is to use the Gaussian function to perform a convolution operation on the image to achieve image smoothing and noise reduction. The convolution kernel G(x, y) of Gaussian filtering can be expressed by the formula:

[0091]

[0092] where (x, y) are the coordinates of the pixel, and σ is the standard deviation. When performing Gaussian filtering, it is first necessary to select a suitable Gaussian kernel size and standard deviation. Then, the Gaussian kernel is convolved with the image to calculate the weighted average of the pixels around each pixel, and this value is used to replace the gray value of the current pixel.

[0093] Step 5: Remap the positions of each pixel in the image to be repaired according to the offsets corresponding to each pixel, and obtain the warped repaired image by using image resampling.

[0094] In some embodiments, remapping the positions of each pixel in the image to be repaired according to the offsets corresponding to each pixel, and obtaining the warped repaired image by using image resampling may include:

[0095] Configure the pixel mapping function, which is expressed by the formula

[0096] Dst(x,y) = Src(x + x', y + y');

[0097] where Src is the image to be repaired, Dst is the warped repaired image, (x, y) are the positions of each pixel, x` is the offset of each pixel in the image to be repaired relative to the reference image in the horizontal direction, and y` is the offset of each pixel in the image to be repaired relative to the reference image in the vertical direction;

[0098] As Figure 5As shown, the image to be repaired is processed based on the pixel mapping function to obtain the distorted repaired image.

[0099] In some embodiments, after obtaining the distorted repaired image to be repaired, the method may further include:

[0100] Step 6: Calculate the similarity between the reference image and the distorted repaired image, and determine whether the similarity is greater than or equal to a preset threshold.

[0101] If so, output the current distorted repaired image.

[0102] If not, use the current distorted repaired image as the new image to be repaired, repeat the above steps 2-5, calculate the offset of each pixel point of the new image to be repaired relative to the corresponding pixel point in the reference image in the horizontal direction and / or vertical direction, and obtain a new distorted repaired image until the similarity between the new distorted repaired image and the reference image is greater than or equal to the preset threshold.

[0103] In the above embodiments, the mean square error algorithm or the structural similarity algorithm or the gray histogram can be used to calculate the similarity between the reference image and the distorted repaired image.

[0104] MSE (mean square error algorithm) evaluates the similarity between two images by calculating the average of the sum of the squares of the differences between the corresponding pixels of the two images; SSIM (structural similarity) measures the image similarity from three aspects: brightness, contrast, and structure; histogram similarity calculates the statistical histogram of the gray values of different pixels in the image, then normalizes it to obtain the probability distribution histogram, and calculates the Euclidean distance between the two images in turn to judge the similarity between the two images.

[0105] Such as Figure 6 And Figure 7 As shown, through the image distortion repair method provided by the present application, the image to be repaired (comparison image) can be distorted and repaired, so that the similarity between the obtained distorted repaired image and the reference image (detection image) is improved, and it can be seen that the gray difference becomes smaller, and the detection error caused by distortion can be avoided.

[0106] Such as Figure 7 As shown, the white area in the difference map is the area with a large gray difference, and the black area is the area with a small gray difference. It can be clearly seen from the figure that the white area in the repaired difference map is significantly reduced, indicating that the gray difference between the image to be repaired and the reference image has been significantly reduced after repair.

[0107] At the same time, the gray difference map between the reference image and the image to be repaired (defined as difference Figure 1 ) and the gray difference map between the reference image and the distorted repaired image (defined as difference Figure 2The mean square error and difference Figure 1 The average gray value is 2.07, and the variance is 1.36. The difference Figure 2 The average gray value is 0.1, and the variance is 0.06. Judging from the data, the image restoration effect is obvious.

[0108] See Figure 8 , in which (A) shows the difference Figure 1 The gray distribution histogram, and in which (B) shows the difference Figure 2 The gray distribution histogram. In these two gray distribution histograms, the middle vertical white line is the position where the gray difference is 0. The statistical results of pixels with negative gray differences are on the left side of the white line, and the statistical results of pixels with positive gray differences are on the right side of the white line. The difference Figure 2 The gray distribution histogram of Figure 1 Compared with the gray distribution histogram of

[0109] As Figure 9 shown, the distortion distribution vector diagrams of the image to be restored and the distortion-restored image can also be obtained respectively. As Figure 9 shown in (A) of Figure 9 before restoration, the distortion distribution presents a disordered state; as

[0110] In summary, through the above distortion restoration method, the interior of the image to be restored can be distorted and restored, and the similarity between the obtained distortion-restored image and the reference image is high, that is, there is no relative distortion deformation. Based on this, the accuracy of defect detection is higher.

[0111] The above is the description of the image distortion restoration method provided by the embodiment of the present application. This image distortion restoration method can be applied to the detection method of defect detection. The present application also provides a detection method, which will be specifically described below.

[0112] As Figure 10 shown, the detection method provided by the embodiment of the present application may include:

[0113] Step 10: Obtain a reference image corresponding to the part to be measured. The reference image is obtained by scanning the part to be measured with an image detector, and the scanning direction is the horizontal direction and / or the vertical direction.

[0114] By scanning with an image detector, reference images corresponding to multiple parts to be measured can be obtained, such as Figure 11 "a", "b", and "c" in

[0115] Step 20: Obtain the corresponding image to be restored according to the reference image.

[0116] In some embodiments, step 20 may include:

[0117] In a preset image database, using a template matching algorithm, obtain a to-be-repaired image that matches the reference image, where the image database is configured to store multiple different to-be-repaired images.

[0118] For example, multiple to-be-modified images can be pre-stored through a storage module such as a memory to form an image database, such as Figure 12 "A", "B", and "C" in. Among them, if the image to be detected currently is the reference image a, the to-be-repaired image A can be obtained by template matching from the image database.

[0119] Since existing detection devices generally perform defect detection on images while scanning to ensure detection efficiency. At this time, all the detection images of the to-be-detected areas are not obtained at once, that is, the reference image is to obtain the reference image a first, and then scan to obtain the reference image b. Then, when scanning the reference image b, the reference image a can be detected. At this time, the corresponding to-be-repaired image A can be quickly obtained by template matching and image distortion repair and defect detection can be performed.

[0120] Step 30: Obtain a distortion-repaired image according to the image distortion repair method described in the above embodiments.

[0121] Step 40: Perform preset feature detection on the reference image according to the distortion-repaired image and the reference image to obtain a detection result.

[0122] Based on the image obtained after distortion repair, performing differential detection with the detection image can improve detection accuracy and reduce the influence brought by the relative distortion between the two images.

[0123] The image distortion repair method and detection method provided in this application can be implemented by the terminal device 2, and the terminal device 2 may include a memory and a processor. For example, the terminal device 2 can be a device with computing and data processing capabilities such as a computer or a server.

[0124] The memory is used to store programs. The processor is used to implement the image distortion repair method and detection method as described in the above embodiments by executing the programs stored in the memory.

[0125] As Figure 13 shown, an embodiment of this application also provides a detection device, which may include: an image detector 1, a first storage module 3, and the terminal device 2 described in the above embodiments.

[0126] The image detector 1 is configured to detect a to-be-tested part to obtain a reference image.

[0127] The first storage module 3 is configured to store the image to be repaired. The first storage module 3 can be a local storage unit or a cloud storage.

[0128] The terminal device 2 is configured to obtain a reference image acquired by the image detector 1, perform a preset feature detection on the reference image according to the reference image and the image to be repaired, and output a detection result. Specifically, it is based on the detection method described in the above embodiments.

[0129] In some embodiments, the detection device may further include a motion mechanism, which is configured to drive the image detector or the object to be measured to move, so that a relative motion occurs between the image detector and the object to be measured, and then the image detector can collect images of the area to be detected.

[0130] For example, when the object to be measured is a wafer, the motion mechanism can drive the wafer to move in the horizontal direction and / or the vertical direction corresponding to the imaging of the image detector.

[0131] The detection device provided in this application can implement the detection method of the above embodiments, can reduce the influence brought by the distortion and deformation inside the image, and improve the detection efficiency.

[0132] Those skilled in the art can understand that all or part of the functions of the above methods can be implemented in a hardware manner or in a computer program manner. When all or part of the functions in the above embodiments are implemented in a computer program manner, the program can be stored in a computer-readable storage medium, and the storage medium can include: read-only memory, random access memory, magnetic disk, optical disk, hard disk, etc. The above functions are realized by a computer executing the program. For example, the program is stored in the memory of the device, and when the processor executes the program in the memory, the above all or part of the functions can be realized. In addition, when all or part of the functions in the above embodiments are implemented in a computer program manner, the program can also be stored in a storage medium such as a server, another computer, magnetic disk, optical disk, flash drive or mobile hard disk, downloaded or copied and saved to the memory of the local device, or the system of the local device is updated, and when the processor executes the program in the memory, the above all or part of the functions in the above embodiments can be realized.

[0133] This article is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications can be made to the exemplary embodiments without departing from the scope of this article. For example, various operation steps and the components used to perform the operation steps can be implemented in different ways according to a specific application or considering any number of cost functions associated with the operation of the system (for example, one or more steps can be deleted, modified or combined into other steps).

[0134] Although the principles herein have been shown in various embodiments, many modifications of structure, arrangement, proportions, elements, materials, and components that are particularly adapted to specific environments and operational requirements may be used without departing from the principles and scope of this disclosure. The above modifications and other changes or alterations will be included within the scope herein.

[0135] The foregoing detailed description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of this disclosure. Accordingly, the contemplation of this disclosure is illustrative in nature and not restrictive in sense, and all such modifications will be included within its scope. Also, the advantages of the various embodiments, other advantages, and solutions to problems have been described above. However, benefits, advantages, solutions to problems, and any elements that produce these, or that make them more explicit, should not be construed as critical, required, or essential. As used herein, the term "comprising" and any other variants thereof are non-exclusive inclusions such that a process, method, article, or apparatus that includes a list of elements does not include only those elements but also other elements not expressly listed or inherent to the process, method, system, article, or apparatus. Further, as used herein, the term "coupled" and any other variants thereof refer to physical connection, electrical connection, magnetic connection, optical connection, communication connection, functional connection, and / or any other connection.

[0136] Those having skill in the art will recognize that many changes may be made in the details of the above-described embodiments without departing from the basic principles of the invention. Accordingly, the scope of the invention should be determined solely by the claims.

Claims

1. An image distortion repair method, characterized in that, Including: Obtain a reference image and an image to be repaired; Calculate the gradient maps corresponding to the reference image and the image to be repaired in the horizontal and vertical directions respectively; Calculate the pixel gray difference map between the reference image and the image to be repaired; According to each of the gradient maps and the pixel gray difference map, calculate the offset of each pixel point in the image to be repaired relative to the pixel point at the corresponding position in the reference image in the horizontal direction and / or vertical direction; According to the offset corresponding to each pixel point, remap the positions of each pixel point in the image to be repaired, and obtain a warped repaired image by using image resampling.

2. The method according to claim 1, characterized in that, According to each of the gradient maps and the pixel gray difference map, calculating the offset of each pixel point in the image to be repaired relative to the pixel point at the corresponding position in the reference image in the horizontal direction and / or vertical direction includes: For each pixel point in the image to be repaired, obtain the gradient values corresponding to the pixel point in the horizontal and vertical directions in the image to be repaired according to each of the gradient maps, and obtain the gradient values corresponding to the pixel point at the corresponding position in the reference image in the horizontal and vertical directions respectively; Obtain the gray difference corresponding to the same pixel point in the image to be repaired according to the pixel gray difference map; Configure the execution function of the non-rigid image registration algorithm; Input the obtained gradient values and gray differences into the execution function to obtain the offset of each pixel point in the image to be repaired relative to the pixel point at the corresponding position in the reference image in the horizontal direction and / or vertical direction.

3. The method according to claim 2, characterized in that, Configuring the execution function of the non-rigid image registration algorithm includes: Set Ux as the offset in the horizontal direction and Uy as the offset in the vertical direction. Then, the execution function of the non-rigid image registration algorithm is expressed by the formula: where Dx and Dy are the gradient values in the horizontal and vertical directions at the point (x, y) on the reference image, respectively, and Gx and Gy are the gradient values in the horizontal and vertical directions at the point (x, y) on the image to be restored, respectively. is the pixel gray-level difference between the reference image and the image to be restored at the corresponding point (x, y), and α is the diffusion rate.

4. The method according to claim 3, characterized in that, Configuring the execution function of the non-rigid image registration algorithm further includes: Obtain the iterative calculation condition for outputting the offset in the horizontal direction and / or the offset in the vertical direction when the iterative calculation terminates; where the iterative calculation condition is expressed by the formula where k is the current iteration number; β is the iteration coefficient.

5. The method according to claim 4, characterized in that, Configuring the execution function of the non-rigid image registration algorithm further includes: Obtain the diffusion speed of different parameters in the horizontal direction and / or vertical direction for updating the iterative calculation condition by the variable parameter diffusion speed. The updated iterative calculation condition is expressed by the formula: Among them, α x is the diffusion speed in the horizontal direction, and α y is the diffusion speed in the vertical direction. β x is the iteration coefficient in the horizontal direction, and β y is the iteration coefficient in the vertical direction.

6. The method according to claim 1, characterized in that, After calculating the offset corresponding to each pixel point, the method further includes: Smooth the offset corresponding to each pixel point in the image to be repaired to update the offset of the target pixel point according to the offset corresponding to the pixel points around the target pixel point.

7. The method according to claim 6, characterized in that, Use Gaussian filtering to smooth the offset corresponding to each pixel point in the image to be repaired.

8. The method according to claim 1, characterized in that, According to the offset corresponding to each pixel point, remap the positions of each pixel point in the image to be repaired, and obtain a warped repaired image by using image resampling, including: Configure the pixel mapping function, and express it by the formula Dst(x,y) = Src(x + x', y + y'); Among them, Src is the image to be repaired, Dst is the distorted repaired image, (x, y) is the position of each pixel point, x` is the horizontal offset of each pixel point in the image to be repaired relative to the reference image, and y` is the vertical offset of each pixel point in the image to be repaired relative to the reference image; Process the image to be repaired based on the pixel mapping function to obtain the distorted repaired image.

9. The method according to claim 1, characterized in that, After obtaining the image to be repaired after distortion repair, the method further includes: Calculate the similarity between the reference image and the distorted repaired image, and determine whether the similarity is greater than or equal to a preset threshold; If so, output the current distorted repaired image; If not, use the current distorted repaired image as the new image to be repaired, calculate the horizontal and / or vertical offsets of each pixel point of the new image to be repaired relative to the corresponding pixel point in the reference image to obtain a new distorted repaired image until the similarity between the new distorted repaired image and the reference image is greater than or equal to the preset threshold.

10. The method according to claim 9, characterized in that, Use the mean square error algorithm or the structural similarity algorithm to calculate the similarity between the reference image and the distorted repaired image.

11. The method according to claim 1, characterized in that, Use the edge detection algorithm to calculate the gradient maps corresponding to the reference image and the image to be repaired in the horizontal and vertical directions respectively.

12. The method according to claim 11, characterized in that, The edge detection algorithm is the Prewitt algorithm, the Sobel algorithm, the Laplace algorithm or the Robert cross algorithm.

13. A detection method, characterized in that, Includes: Obtain a reference image corresponding to the test piece, where the reference image is obtained by scanning the test piece with an image detector, and the scanning direction is the horizontal direction and / or the vertical direction; Obtain the corresponding image to be repaired according to the reference image; Obtain the distorted repaired image according to the image distortion repair method according to any one of claims 1-12; Perform a preset feature detection on the reference image according to the distorted repaired image and the reference image to obtain a detection result.

14. The method according to claim 13, characterized in that, Obtaining the corresponding image to be repaired according to the reference image includes: In a preset image database, use the template matching algorithm to obtain an image to be repaired that matches the reference image, and the image database is configured to store multiple different images to be repaired.

15. A terminal device, characterized in that, Includes: A memory for storing programs; A processor for implementing the method according to any one of claims 1-14 by executing the program stored in the memory.

16. A detection device, characterized in that, Includes: An image detector configured to detect a test piece to obtain a reference image; A first storage module configured to store the image to be repaired; And the terminal device according to claim 15, configured to obtain the reference image obtained by the image detector, perform a preset feature detection on the reference image according to the reference image and the image to be repaired, and output a detection result.

17. A computer-readable storage medium, characterized in that, A program is stored on the medium, and the program can be executed by a processor to implement the method according to any one of claims 1-14.

Images