Image correction method, system, computer-readable recording medium storing program therein, and program product

Abstract

The present application provides an image correction method, system, computer readable recording medium and program product. The image correction method comprises: obtaining and recording a plurality of real pattern coordinates of a pattern array on a template in advance; capturing the pattern array by a camera and recording a plurality of pattern pixel coordinates of the pattern array; capturing an object to be measured by the camera to obtain a target image, and mapping the pattern array to the target image according to the pattern pixel coordinates; setting at least one target pixel coordinate on the target image, and obtaining pattern pixel coordinates corresponding to the nearest four corner patterns around the at least one target pixel coordinate; obtaining a conversion matrix from the four pattern pixel coordinates and corresponding four real pattern coordinates; and substituting the at least one target pixel coordinate into the conversion matrix to obtain at least one correction coordinate. The present application effectively obtains the actual size in space from the image by a simple algorithm, and solves the problem that the lengths of the shooting ranges of the pixel points are not the same.

Description

TECHNICAL FIELD

[0001] The present application relates to an image correction method, system, computer readable recording medium storing a program and program product, and in particular to an image correction method, system, computer readable recording medium storing a program and program product that can help an image processing device obtain real distances in an image. BACKGROUND

[0002] In the technical field of automatic optical inspection (AOI), size measurement using machine vision is a commonly used technique, which is mainly based on pixel size, focal length, and object distance to deduce the actual distance between points in real space. However, in practice, the image range displayed by each pixel is limited by environmental conditions (such as the optical properties of the lens, the precision of the mechanism, etc.), and it is difficult to standardize each pixel, so it is not reliable to deduce the distance on the image only by pixel size. In the case of today's highly integrated technology, conventional techniques are difficult to meet the demand for precision.

[0003] As shown in Figure 1 , in the case of shooting the object to be measured A1 by the camera, the range captured by each pixel point is not consistent. Specifically, the shooting range of each pixel point is not the same, which can be imagined as a ruler R to measure the length of the object, but each scale R1 on the ruler R is of different length, so the measurement result will be incorrect in this case. Similarly, the pixel corresponds to the scale R1 on the ruler R, and since the image range obtained by each pixel is not consistent, the result of deducing the actual distance according to the magnification and pixel size will be inaccurate. Taking a grid as an example, as shown in Figure 2 , if the length of each pixel is not the same, the actual object width of the left object to be measured A2 is equivalent to 3 units of distance, but the actual image A3 captured only includes 2 units of distance. The general purpose of correction is to correct each scale of this ruler with pixels as the smallest unit to be equal in length. SUMMARY

[0004] The main object of the present application is to provide a method for image correction, comprising: obtaining and recording a plurality of real coordinates of a pattern array on a template in advance; capturing the pattern array by a camera and recording a plurality of pixel coordinates of the pattern array; capturing an object to be measured by the camera to obtain a target image, and mapping the pattern array to the target image according to the pixel coordinates; setting at least one target pixel coordinate on the target image, and obtaining the pixel coordinates of the four corner patterns closest to the target pixel coordinate, obtaining a conversion matrix from the four pixel coordinates and the corresponding four real coordinates; and substituting the at least one target pixel coordinate into the conversion matrix to obtain at least one corrected coordinate.

[0005] Another object of the present application is to provide an image correction system, comprising a camera and an image processing device connected to the camera. The camera is used to capture an object to be measured to obtain a target image. The image processing device stores a pattern array captured by the camera in advance and records a plurality of pixel coordinates and a plurality of real coordinates of the pattern array. The image processing device obtains the target image from the camera and maps the pattern array to the target image according to the pixel coordinates. At least one target pixel coordinate is set on the target image, and the pixel coordinates of the four corner patterns closest to the target pixel coordinate are obtained. A conversion matrix is obtained from the four pixel coordinates and the corresponding four real coordinates. The at least one target pixel coordinate is substituted into the conversion matrix to obtain at least one corrected coordinate.

[0006] Still another object of the present application is to provide a computer-readable recording medium storing a program, which, when loaded and executed by a computer, can complete the method described above.

[0007] Still another object of the present application is to provide a computer program product storing a program, which, when loaded and executed by a computer, can complete the method described above.

[0008] Therefore, the present application can effectively obtain the actual size in space from the image by a simple algorithm, and solve the problem of different lengths of the pixel shooting range. In addition, the correction method of the present application directly corrects the results of image shooting, so that all problems of different lengths of the pixel can be corrected at one time, and more computational burden can be saved compared with the prior art. BRIEF DESCRIPTION OF DRAWINGS

[0009] Figure 1 Reference diagram (I) for prior art;

[0010] Figure 2 Reference diagram (II) for prior art;

[0011] Figure 3A block diagram of an image correction system according to the present application;

[0012] Figure 4 A schematic diagram of the appearance of a pattern array according to the present application;

[0013] Figure 5 A schematic diagram of a partial enlargement of a pattern array according to the present application;

[0014] Figure 6 A schematic diagram of the mapping of a pattern array onto a target image according to the present application;

[0015] Figure 7 A schematic diagram of the marking of a correction coordinate on a target image according to the present application;

[0016] Figure 8 A flowchart of an image correction method according to the present application.

[0017] Explanation of reference numerals:

[0018] Prior art:

[0019] A1 object to be measured

[0020] R scale

[0021] R1 scale

[0022] A2 object to be measured

[0023] A3 actual image

[0024] Embodiments of the present application:

[0025] 100 image correction system

[0026] 10 camera

[0027] 20 image processing device

[0028] A4 object to be measured

[0029] 21 processor

[0030] 22 storage unit

[0031] 30 template

[0032] 31 pattern array

[0033] 32 template carrier

[0034] K target pattern unit

[0035] P origin pattern unit

[0036] G1 target pixel coordinate

[0037] K1 corner pattern

[0038] K2 corner pattern

[0039] K3 corner pattern

[0040] K4 corner pattern

[0041] G2 target pixel coordinate

[0042] K5 corner pattern

[0043] K6 corner pattern

[0044] K7 corner pattern

[0045] K8 corner pattern

[0046] MA4 target image

[0047] coordinate distance

[0048] A(x A ,y A ) first corrected coordinate

[0049] B(x B ,y B ) second corrected coordinate

[0050] Steps S01-S06. DETAILED DESCRIPTION

[0051] The detailed description and technical content of the present application will be described below with reference to a specific embodiment and the accompanying drawings. For the convenience of description, some of the drawings in the present application are not necessarily drawn according to the actual proportions, and the sizes and proportions of the drawings are not intended to limit the scope of the present application. Hereinafter, the drawings are described first.

[0052] Please refer to Figure 3 , which is a block diagram of an image correction system in the present application. The image correction system 100 in the present application includes a camera 10 and an image processing device 20 connected to the camera 10.

[0053] The camera 10 is used to capture a target object A4 to obtain a target image. In an embodiment, the camera 10 can be, but is not limited to, an area scan camera or a line scan camera, which is not limited in the present application. The target object A4 can be any object, such as a panel, a circuit board, or any workpiece, which is not limited in the present application.

[0054] The image processing device 20 is connected or coupled to the camera 10 to obtain the target image and perform the correction procedure based on the target image. In one embodiment, the image processing device 20 mainly comprises a processor 21 and a storage unit 22 connected to the processor 21. The processor 21 can be used to load and execute the program stored in the storage unit 22. In another embodiment, the processor 21 and the storage unit 22 can jointly constitute a computer or processor, such as a personal computer, a workstation, a mainframe computer or other types of computers or processors, which are not limited in the present application. In one embodiment, the processor 21 can be coupled to the storage unit 22. The processor 21 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessors (MP), digital signal processors (DSP), programmable controllers, application specific integrated circuits (ASIC), programmable logic devices (PLD) or other similar devices or combinations of these devices.

[0055] The way the image processing device 20 obtains the target image is not necessarily directly from the camera 10 as described above. In one embodiment, the camera 10 can first transmit the target image to an image storage device specially used for storing images, and then the image processing device 20 obtains the target image captured by the camera 10 through the image storage device. Variations of the foregoing embodiment are not intended to limit the scope of the present application.

[0056] Regarding the image correction method in the present application, a specific embodiment is described below with reference to the accompanying drawings. Please refer to Figure 4 、 Figure 5 、 Figure 6 、 Figure 7 and Figure 8 for the appearance schematic diagram of the pattern array, the partial enlarged schematic diagram, the mapping schematic diagram of the pattern array on the target image, the marking schematic diagram of the correction coordinates on the target image and the flow schematic diagram of the image correction method in the present application.

[0057] First, as Figure 4As shown, a template 30 for calibration is prepared in advance (step S01). The template 30 has a template carrier 32 and a pattern array 31 disposed on the surface of the template carrier 32, which can be captured by the camera 10 and the image can be acquired. The pattern array 31 can be a dot array, a grid array, or any other pattern that can clearly indicate the scale, which is not limited in the present application. In an embodiment, the distance between two adjacent pattern units (e.g. grid intersection, dot, etc.) of the pattern array 31 can be equal; in another embodiment, the distance between two adjacent pattern units can be unequal when each pattern real coordinate is marked and recorded in advance, which is not limited in the present application. The template carrier 32 can be glass, wood, paper, plastic sheet, or any other material, preferably a material that does not easily expand or contract with temperature change in general working environment, which is not limited in the present application.

[0058] Next, a plurality of pattern real coordinates of the pattern array 31 of the template 30 are acquired and recorded in advance (step S02). The coordinates of each pattern unit of the pattern array 31 can be stored in the storage unit 22 in advance, which can be accessed by the processor 21 when needed. In another embodiment, the coordinates of the pattern units can be obtained by conversion from a reference point (e.g. origin) when the distance between each pattern unit is fixed, as shown in the following table. Figure 5 For example, when the distance between two adjacent pattern units is 1 mm, 1 mm is taken as the basic unit of the coordinate, when the distance between the target pattern unit K and the origin pattern unit P is 2 units in the x direction and 5 units in the y direction, the coordinate of the target pattern unit K can be set as (2, 5), and the distance between the target pattern unit K and the origin pattern unit P can be confirmed as 2 mm in the x direction and 5 mm in the y direction, which is converted in this embodiment. As long as the position of the origin pattern unit P is confirmed, the coordinates of other pattern units can be obtained, which is not limited in the present application.

[0059] Then, the pattern array 31 is captured by the camera 10 and a plurality of pattern pixel coordinates of the pattern array 31 are recorded (step S03). In this step, the purpose of capturing the pattern array 31 is to obtain the conversion matrix between the actual distance in space and the pixel distance in the subsequent steps, so as to calculate the real distance in space by using the obtained conversion matrix.

[0060] In a large number of detections, the steps S01 to S03 described above can be performed only once, and the pattern pixel coordinates and the pattern real coordinates of the pattern array 31 obtained are stored in the storage unit 22 until the lens, the lens adjustment data, or the environmental conditions are changed (e.g. object distance, height of the object to be detected, etc.).

[0061] Then, the test object A4 is photographed by camera 10 to obtain a target image MA4, and the pattern array 31 is mapped onto the target image MA4 according to the pattern pixel coordinates (step S04). In one embodiment, the mapping is based on pixel coordinates, directly marking the pattern pixel coordinates on the target image MA4 one-to-one to establish the relative positional relationship between each position on the target image MA4 and the pattern array 31; in another embodiment, the mapping can be the direct overlay of the pattern array 31 onto the target image MA4 to generate a pattern array MA4. Figure 6 The images shown are not intended to limit the variations of the embodiments described herein.

[0062] Next, at least one target pixel coordinate G1 is set on the target image MA4, and the pattern pixel coordinates corresponding to the four nearest corner patterns K1, K2, K3, and K4 around the at least one target pixel coordinate G1 are obtained. A transformation matrix is ​​obtained from the four pattern pixel coordinates of the four corner patterns K1, K2, K3, and K4 and the corresponding four pattern real coordinates (step S05). In one embodiment, the above transformation matrix is ​​calculated via a perspective transformation matrix, but this is not a limitation in the present invention.

[0063] Specifically, the transformation matrix mentioned in step S05 is obtained according to the following formula:

[0064]

[0065] in, t1, t2, t3, t4 are the transformation matrix; (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the true coordinates of the patterns corresponding to the four nearest corner patterns K1, K2, K3, K4 around at least one target pixel coordinate G1; (u1, v1), (u2, v2), (u3, v3), (u4, v4) are the pattern pixel coordinates corresponding to the four nearest corner patterns K1, K2, K3, K4 around at least one target pixel coordinate G1; t1, t2, t3, t4 are the elimination parameters.

[0066] Then, at least one target pixel coordinate G1 is substituted into the transformation matrix to obtain a corrected coordinate (step S06).

[0067] Specifically, the correction coordinates mentioned in step S06 are obtained based on the following formula:

[0068]

[0069] Where (u,v) represents the coordinates G1 of at least one target pixel before transformation; (x,y) represents the corrected coordinates after transformation; and t is the elimination parameter. The final obtained matrix contains... The first element, the second element and the third element of the matrix are divided by the elimination parameter t, and the obtained coordinate A(x, y) is the corrected coordinate. The corrected coordinate can be understood as the relative position of the at least one target pixel coordinate G1 in the real space.

[0070] In the evaluation of the real distance between the corrected coordinate (hereinafter referred to as the first corrected coordinate A(x A ,y A )), another position of interest can be marked in the image, and the other position of interest is set as a target pixel coordinate G2. The step S05 is further performed to obtain the pattern pixel coordinates of the nearest four corner patterns K5, K6, K7 and K8 around the target pixel coordinate G2. The conversion matrix is obtained from the four pattern pixel coordinates of the four corner patterns K5, K6, K7 and K8 and the corresponding four pattern real coordinates. The step S06 is then performed to calculate and obtain the second corrected coordinate B(x B ,y B ) of the target pixel coordinate G2 using the conversion matrix obtained from the four corner patterns K5, K6, K7 and K8. The real distance between two points is further calculated by the coordinate distance formula.

[0071]

[0072] wherein A(x A ,y A ) is the first corrected coordinate, B(x B ,y B ) is the second corrected coordinate, is the coordinate distance. The real distance between the two corrected coordinates is calculated accordingly.

[0073] The method steps described above can be implemented by a non-transitory computer readable recording medium, such as a read-only memory, a flash memory, a floppy disk, a hard disk, an optical disc, a USB flash drive, a magnetic tape, a database accessible via a network or a storage medium with the same function that can be easily thought of by those skilled in the art. When the computer loads the program stored in the non-transitory computer readable recording medium and executes it, the image position alignment method described above can be completed.

[0074] In addition to the computer readable recording medium, the method steps described above can also be implemented as a computer program product for storage in a hard disk, a memory device or an application online distribution platform of a network server, and can be implemented by uploading the computer program product to the server for users to download by paying.

[0075] In summary, the present application can effectively obtain the actual size in space from the image by a simple algorithm, and solve the problem of different lengths of pixel shooting range. In addition, the correction method of the present application directly corrects the results of image shooting, so it can correct all problems of different lengths of pixel once, and can save more operation burden compared with the prior art.

[0076] The present application has been described in detail above, but the above is only a preferred embodiment of the present application, and cannot limit the scope of the present application. Any equivalent changes and modifications made within the scope of the present application are still within the scope of the present application.

Claims

1. An image correction method characterized by, The method comprises: pre-acquiring and recording a plurality of real coordinates of a pattern array on a template; capturing the pattern array by a camera and recording a plurality of pixel coordinates of the pattern array; capturing an object to be measured by the camera to obtain a target image, and mapping the pattern array to the target image according to the pixel coordinates; setting at least one target pixel coordinate on the target image, and obtaining the pixel coordinates corresponding to the four corner patterns around the target pixel coordinate, obtaining a conversion matrix from the four pixel coordinates and the corresponding four real coordinates; and substituting the target pixel coordinate into the conversion matrix to obtain a corrected coordinate.

2. The image correction method of claim 1, wherein The image correction method further comprises obtaining a plurality of corrected coordinates, and calculating the real distance between at least two target pixel coordinates according to the corrected coordinates.

3. The image correction method of claim 1, wherein The conversion matrix is obtained according to the following formula: wherein, is the conversion matrix; (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the real pattern coordinates of the nearest four corner patterns around the target pixel coordinate; (u1, v1), (u2, v2), (u3, v3), (u4, v4) are the pattern pixel coordinates corresponding to the nearest four corner patterns around the target pixel coordinate; t1, t2, t3, t4 are elimination parameters.

4. The image correction method of claim 3, wherein The corrected coordinate is obtained according to the following formula: wherein (u, v) is the target pixel coordinate before conversion; (x, y) is the corrected coordinate after conversion, and t is a cancellation parameter.

5. An image correction system characterized by comprising: The method comprises: a camera for capturing an object to be measured to obtain a target image; and an image processing device connected to the camera, the image processing device pre-storing a pattern array captured by the camera and recording a plurality of pixel coordinates and a plurality of real coordinates of the pattern array, the image processing device obtaining the target image from the camera, mapping the pattern array to the target image according to the pixel coordinates, setting at least one target pixel coordinate on the target image, and obtaining the pixel coordinates corresponding to the four corner patterns around the target pixel coordinate, obtaining a conversion matrix from the four pixel coordinates and the corresponding four real coordinates, and substituting the target pixel coordinate into the conversion matrix to obtain a corrected coordinate.

6. The image correction system of claim 5, wherein, The image processing device can further obtain a plurality of corrected coordinates, and calculate the real distance between at least two target pixel coordinates according to the corrected coordinates.

7. The image correction system of claim 5, wherein The conversion matrix is obtained according to the following formula: wherein, is the conversion matrix; (x1, y1), (x2, y2), (x3, y3), (x4, y4) are the real pattern coordinates of the nearest four corner patterns around the target pixel coordinate; (u1, v1), (u2, v2), (u3, v3), (u4, v4) are the pattern pixel coordinates corresponding to the nearest four corner patterns around the target pixel coordinate; t1, t2, t3, t4 are elimination parameters.

8. The image correction system of claim 7, wherein, The corrected coordinate is obtained according to the following formula: wherein (u, v) is the target pixel coordinate before conversion; (x, y) is the corrected coordinate after conversion, and t is a cancellation parameter.

9. A computer-readable recording medium storing a program, wherein the program is loaded into a computer and executed to perform the method of any one of claims 1 to 4.

10. A computer program product storing a program, wherein the program is loaded into a computer and executed to perform the method of any one of claims 1 to 4.

Images