Matching compensation method, device and storage medium for focusing sequence images

Abstract

The present invention discloses a matching compensation method, device and storage medium for focusing sequence images. The method comprises: selecting an original image whose clarity meets preset requirements in the original focusing sequence images as a template image; establishing an ROI area containing a target object in the template image, and recording the image coordinates (xr, yr) of the ROI area; performing template matching on the original image using the template image as a search template, marking the most similar area in the original image to the ROI area as a corresponding search area, and recording the image coordinates (xi, yi) of the search area; calculating the coordinate difference (xi-xr, yi-yr) between the search area and the ROI area; compensating the original image according to the coordinate difference (xi-xr, yi-yr) to obtain a compensated focusing sequence image; the present invention solves the mismatching problem caused by the non-vertical installation and non-vertical displacement of a focusing platform camera by a matching compensation method, and effectively improves the accuracy of the focusing recovery depth technology.

Description

Technical Field

[0001] The present invention relates to the technical field of machine vision algorithms, and in particular to a matching compensation method, device and storage medium for focusing sequence images. Background Art

[0002] In recent years, as industrial control has become more and more demanding in terms of precision and automation, 3D machine vision will play an increasingly important role in industrial visual inspection. As a branch of 3D machine vision methods, the focus recovery depth technology has broad application prospects in the field of industrial visual inspection. In the focus recovery depth, an object of unknown height needs to be moved relative to the imaging system to obtain a series of images corresponding to different levels of target focus. In the image sequence, the position where each pixel is clearly focused needs to be determined, and the depth value of the pixel can be obtained based on this position. Therefore, this method requires that the same pixel position in the focus sequence image corresponds to the same point on the object.

[0003] However, in the process of focusing and restoring depth, due to the installation error of the visual platform, the same pixel position in the focused sequence image may correspond to different points on the object surface, affecting the accuracy of 3D reconstruction. Conventional focus restoration depth algorithms rarely consider the matching problem of sequence image pixels. Therefore, matching and compensation for focused sequence images are innovative and important. Summary of the invention

[0004] The purpose of the present invention is to provide a matching compensation method, device and storage medium for focusing sequence images, which can solve the mismatching problem caused by non-vertical installation and displacement of the focusing platform camera by matching compensation, and effectively improve the accuracy of the focus recovery depth technology.

[0005] In order to achieve the above object, the present invention discloses a matching compensation method for focusing sequence images, wherein the original focusing sequence images include n original images obtained by continuous shooting, and the matching compensation method includes the following steps:

[0006] S1, selecting an original image whose clarity meets a preset requirement from the original focus sequence images as a template image;

[0007] S2, establishing a ROI region containing the target object in the template image, and recording the image coordinates (xr, yr) of the ROI region;

[0008] S3, performing template matching on each original image using the template image as a search template to obtain the most similar region in each original image to the ROI region, marking the most similar region in each original image to the ROI region as a corresponding search region, and recording the image coordinates (xi, yi) of the search region, where i=1, 2, ..., n;

[0009] S4, calculating the coordinate difference (xi-xr, yi-yr) between each search area and the ROI area;

[0010] S5. Compensate each original image according to the coordinate difference (xi-xr, yi-yr) between all search areas and the ROI area to obtain compensated focus sequence images.

[0011] Compared with the prior art, the present invention uses a template image as a search template to perform template matching on each original image to obtain a search area in each original image that is most similar to the ROI area, and obtains a compensated focus sequence image by compensating each original image using the coordinate difference between all search areas and the ROI area. The present invention can match and compensate the focus sequence images in an image processing manner, and can effectively eliminate the problem that the same pixel position of the focus image cannot correspond to the same point on the object surface due to the installation error of the focus platform, thereby greatly improving the accuracy of the focus recovery depth technology.

[0012] Preferably, the number of tangency between the edge of the ROI area and the edge of the target object is set to be maximized.

[0013] Preferably, the image coordinates (xr, yr) of the ROI area are the coordinates of the upper left corner of the ROI area, and the image coordinates (xi, yi) of the search area are the coordinates of the upper left corner of the search area.

[0014] Specifically, assuming that the resolution of all original images of the original focus sequence image is Width×Height, the image resolutions of all search areas and the ROI area are (Width-2*Xmax)×(Height-2*Ymax), respectively. Step S5 specifically includes the following steps:

[0015] S51, respectively calculating the maximum value Xmax of the coordinate difference in the X direction and the maximum value Ymax of the coordinate difference in the Y direction between all search areas and the ROI area (xi-xr, yi-yr);

[0016] S52. If the coordinate difference (xi-xr, yi-yr) between the current search area and the ROI area is (0, 0), the image coordinates of the current search area obtained by compensation are (Xmax, Ymax); otherwise, the image coordinates of the current search area obtained by compensation are (Xmax+xi-xr, Ymax+yi-yr);

[0017] S53, after respectively calculating the image coordinates of all search areas, a compensated focused sequence image is obtained.

[0018] Preferably, the maximum value Xmax of the coordinate difference (xi-xr, yi-yr) in the X direction between all search areas and the ROI area is calculated according to the formula Xmax=max{abs(x1-xr),abs(x2-xr),abs(x3-xr),...,abs(xn-xr)}.

[0019] Preferably, the maximum value Ymax of the coordinate difference (xi-xr, yi-yr) in the Y direction between all search areas and the ROI area is calculated according to the formula Ymax=max{abs(y1-yr),abs(y2-yr),abs(y3-yr),...,abs(yn-yr)}.

[0020] Preferably, the step S1 specifically includes the following steps:

[0021] S11, respectively calculating the clarity of all original images;

[0022] S12, sorting all original images according to their clarity;

[0023] S13, selecting the original image with the maximum definition as the template image.

[0024] Preferably, before step S1, the step further includes:

[0025] S101, reading the original focus sequence images.

[0026] Correspondingly, the present invention further discloses a matching compensation device for focusing sequence images, wherein the original focusing sequence images include n original images obtained by continuous shooting, and the matching compensation device for focusing sequence images includes:

[0027] A selection unit is configured to select an original image whose clarity meets a preset requirement from the original focus sequence images as a template image;

[0028] An execution unit is configured to establish a ROI region containing a target object in the template image and record image coordinates (xr, yr) of the ROI region;

[0029] a matching unit configured to perform template matching on each original image using the template image as a search template to obtain a region in each original image that is most similar to the ROI region, mark the region in each original image that is most similar to the ROI region as a corresponding search region, and record image coordinates (xi, yi) of the search region, where i=1, 2, ..., n;

[0030] A calculation unit, configured to calculate a coordinate difference (xi-xr, yi-yr) between each search area and the ROI area;

[0031] The compensation unit is configured to compensate each original image according to the coordinate difference (xi-xr, yi-yr) between all search areas and the ROI area to obtain a compensated focus sequence image.

[0032] Correspondingly, the present invention also discloses a storage medium for storing a computer program, wherein when the program is executed by a processor, the matching compensation method for focusing sequence images as described above is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flowchart of a matching compensation method for focusing sequence images of the present invention;

[0034] Figure 2 is a schematic diagram of the ROI region of the present invention in the template image;

[0035] Figure 3 is a schematic diagram of the case where the image coordinates (xi, yi) of the search area in any original image of the present invention are (0, 0);

[0036] Figure 4 is a schematic diagram of the present invention when the image coordinates (xi, yi) of the search area in any original image are not (0, 0);

[0037] Figure 5 It is a structural block diagram of the matching compensation device for focusing sequence images of the present invention. DETAILED DESCRIPTION

[0038] In order to explain the technical content, structural features, achieved objectives and effects of the present invention in detail, the following is a detailed description in conjunction with the implementation methods and the accompanying drawings.

[0039] See also Figure 1-Figure 4 In this embodiment, a matching compensation method for focus sequence images is used to convert the original focus sequence images into compensated focus sequence images by image matching and compensation, so as to provide for subsequent focus recovery depth processing. The original focus sequence images include n original images obtained by continuous shooting. The method includes the following steps:

[0040] S1. Selecting an original image whose clarity meets a preset requirement from among the original focus sequence images as a template image.

[0041] It is understandable that if Figure 2As shown, since the surface of the object generally undergoes a process of "defocusing-focusing-defocusing" during shooting, the clearer image is generally located in the middle of the original focused sequence image in the focused image sequence. The selection method here can be selected manually and intuitively, or it can be selected by an algorithm. The clarity here meets the preset requirements, which can be understood as the selected template image being the clearer original image in the original focused sequence image. The clearer original image can be the clearest original image, or it can be close to the clearest original image.

[0042] S2. Establish a ROI region containing the target object in the template image, and record the image coordinates (xr, yr) of the ROI region. Preferably, the number of tangents between the edge of the ROI region and the edge of the target object is set to be maximized, so that the boundary of the intercepted ROI region and the boundary of the target object are as follows: Figure 2 The images shown are as consistent or similar as possible to reduce matching errors and improve subsequent image accuracy.

[0043] S3. Perform template matching on each original image using the template image as a search template to obtain the area in each original image that is most similar to the ROI area, mark the area in each original image that is most similar to the ROI area as the corresponding search area, and record the image coordinates (xi, yi) of the search area, where i = 1, 2, ..., n.

[0044] S4. Calculate the coordinate difference (xi-xr, yi-yr) between each search area and the ROI area.

[0045] S5. Compensate each original image according to the coordinate difference (xi-xr, yi-yr) between all search areas and the ROI area to obtain compensated focus sequence images.

[0046] Preferably, the image coordinates (xr, yr) of the ROI area are the coordinates of the upper left corner of the ROI area, and the image coordinates (xi, yi) of the search area are the coordinates of the upper left corner of the search area.

[0047] It is understandable that since the shape of the target object is determined, that is, it has a certain boundary, by using a specific position point as the coordinate point of the ROI area, subsequent matching and compensation operations can be facilitated. In this embodiment, the coordinate point of the upper left corner of the ROI area is used as the image coordinate (xr, yr) of the ROI area. Of course, in other embodiments, other positions such as the lower left corner, the upper right corner, or the lower right corner can be used as the image coordinate (xr, yr) of the ROI area, which is not limited here.

[0048] Specifically, assuming that the resolution of all original images of the original focus sequence image is Width×Height, the image resolutions of all search areas and the ROI area are (Width-2*Xmax)×(Height-2*Ymax), respectively. Step S5 specifically includes the following steps:

[0049] S51, respectively calculating the maximum value Xmax of the coordinate difference in the X direction and the maximum value Ymax of the coordinate difference in the Y direction between the coordinate differences (xi-xr, yi-yr) of all search areas and the ROI area.

[0050] S52, if the coordinate difference (xi-xr, yi-yr) between the current search area and the ROI area is (0, 0), then the image coordinates of the current search area obtained by compensation are (Xmax, Ymax), as shown in detail Figure 3 As shown, otherwise the image coordinates of the current search area obtained by compensation are (Xmax+xi-xr, Ymax+yi-yr), as shown in Figure 4 As shown. So far, the present invention has completed the matching and compensation processing of the current original image, which can effectively eliminate the problem that the same pixel position of the focused image cannot correspond to the same point on the object surface due to the installation error of the focusing platform, and greatly improve the accuracy of the focus recovery depth technology. In addition, the image resolution of all search areas and the ROI area can be calculated respectively by the formula (Width-2*Xmax)×(Height-2*Ymax).

[0051] S53, after calculating the image coordinates of all search areas respectively, obtain the compensated focus sequence images. After calculating the image coordinates of all search areas, each original image of the original focus sequence images is compensated, and each compensated original image together constitutes the compensated focus sequence image.

[0052] Preferably, the maximum value Xmax of the coordinate difference (xi-xr, yi-yr) in the X direction between all search areas and the ROI area is calculated according to the formula Xmax=max{abs(x1-xr),abs(x2-xr),abs(x3-xr),...,abs(xn-xr)}.

[0053] Preferably, the maximum value Ymax of the coordinate difference (xi-xr, yi-yr) in the Y direction between all search areas and the ROI area is calculated according to the formula Ymax=max{abs(y1-yr),abs(y2-yr),abs(y3-yr),...,abs(yn-yr)}.

[0054] Preferably, the step S1 specifically includes the following steps:

[0055] S11, respectively calculating the clarity of all original images.

[0056] S12. Sort all original images according to their clarity.

[0057] S13, selecting the original image with the maximum definition as the template image.

[0058] It is understandable that in other implementations, other parameters that can characterize image clarity may be used as a basis for selecting a template image, which is not limited here.

[0059] Preferably, before step S1, the step further includes:

[0060] S101, reading the original focus sequence images.

[0061] See also Figure 5 The present invention also discloses a matching compensation device for focusing sequence images, wherein the original focusing sequence images include n original images obtained by continuous shooting, and the matching compensation device for focusing sequence images includes:

[0062] A selection unit 10 is configured to select an original image whose clarity meets a preset requirement from the original focus sequence images as a template image;

[0063] The execution unit 20 is configured to establish a ROI region containing a target object in the template image and record the image coordinates (xr, yr) of the ROI region;

[0064] The matching unit 30 is configured to perform template matching on each original image using the template image as a search template to obtain the most similar region to the ROI region in each original image, mark the most similar region to the ROI region in each original image as a corresponding search region, and record the image coordinates (xi, yi) of the search region, where i=1, 2, ..., n;

[0065] A calculation unit 40 is configured to calculate a coordinate difference (xi-xr, yi-yr) between each search area and the ROI area;

[0066] The compensation unit 50 is configured to compensate each original image according to the coordinate difference (xi-xr, yi-yr) between all search areas and the ROI area to obtain a compensated focused sequence image.

[0067] Correspondingly, the present invention also discloses a storage medium for storing a computer program, wherein when the program is executed by a processor, the matching compensation method for focusing sequence images as described above is implemented.

[0068] Combination Figure 1-Figure 5 The present invention uses the template image as the search template to perform template matching on each original image to obtain the search area in each original image that is most similar to the ROI area, and obtains the compensated focus sequence images by compensating each original image with the coordinate difference between all the search areas and the ROI area. The present invention can match and compensate the focus sequence images in an image processing manner, and can effectively eliminate the problem that the same pixel position of the focus image cannot correspond to the same point on the object surface due to the installation error of the focus platform, thereby greatly improving the accuracy of the focus recovery depth technology.

[0069] The above disclosure is only the preferred embodiment of the present invention, which certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the scope of the patent application of the present invention are still within the scope covered by the present invention.

Claims

1. A matching compensation method for a focus sequence image, wherein the original focus sequence image includes n original images obtained by continuous shooting, characterized in that: The steps include: Selecting an original image whose clarity meets a preset requirement from the original focus sequence images as a template image; Establishing a ROI region containing the target object in the template image, and recording the image coordinates (xr, yr) of the ROI region; Performing template matching on each original image using the template image as a search template to obtain the most similar region in each original image to the ROI region, marking the most similar region in each original image to the ROI region as a corresponding search region, and recording the image coordinates (xi, yi) of the search region, where i=1, 2, ..., n; Calculate the coordinate difference (xi-xr, yi-yr) between each search area and the ROI area; Each original image is compensated according to the coordinate difference (xi-xr, yi-yr) between all search areas and the ROI area to obtain a compensated focused sequence image.

2. The matching compensation method for focusing sequence images according to claim 1, characterized in that: The number of tangency between the edge of the ROI area and the edge of the target object is set to be maximized.

3. The matching compensation method for focusing sequence images according to claim 1, characterized in that: The image coordinates (xr, yr) of the ROI region are the coordinates of the upper left corner of the ROI region, and the image coordinates (xi, yi) of the search region are the coordinates of the upper left corner of the search region.

4. The matching compensation method for focusing sequence images according to claim 3, characterized in that: The resolution of all original images of the original focus sequence image is Width×Height, and the image resolutions of all search areas and the ROI area are (Width-2*Xmax)×(Height-2*Ymax) respectively. The compensation of each original image according to the coordinate difference (xi-xr, yi-yr) of all search areas and the ROI area to obtain the compensated focus sequence image specifically includes the following steps: Calculate the maximum value Xmax of the coordinate difference in the X direction and the maximum value Ymax of the coordinate difference in the Y direction (xi-xr, yi-yr) of all search areas and the ROI area respectively; If the coordinate difference (xi-xr, yi-yr) between the current search area and the ROI area is (0, 0), the compensated image coordinates of the current search area are (Xmax, Ymax), otherwise the compensated image coordinates of the current search area are (Xmax+xi-xr, Ymax+yi-yr); After the image coordinates of all search areas are calculated respectively, the compensated focusing sequence images are obtained.

5. The matching compensation method for focusing sequence images according to claim 4, characterized in that: According to the formula Xmax=max{abs(x1-xr),abs(x2-xr),abs(x3-xr),...,abs(xn-xr)}, the maximum value Xmax of the coordinate difference (xi-xr, yi-yr) in the X direction between all search areas and the ROI area is calculated.

6. The matching compensation method for focusing sequence images according to claim 4, characterized in that: According to the formula Ymax=max{abs(y1-yr),abs(y2-yr),abs(y3-yr),...,abs(yn-yr)}, the maximum value Ymax of the coordinate difference (xi-xr, yi-yr) in the Y direction between all search areas and the ROI area is calculated.

7. The matching compensation method for focusing sequence images according to claim 1, characterized in that: The selecting of the original image whose clarity meets the preset requirement in the original focus sequence image as the template image specifically includes the following steps: Calculate the clarity of all original images separately; Sort all original images according to their clarity; Select the original image with the greatest clarity as the template image.

8. The matching compensation method for focusing sequence images according to claim 1, characterized in that: The selecting a relatively clear original image in the focusing sequence images as a template image also includes: The original focus sequence images are read.

9. A matching compensation device for focusing sequence images, characterized in that: The original focused sequence images include n original images obtained by continuous shooting, and the matching compensation device for the focused sequence images includes: A selection unit is configured to select an original image whose clarity meets a preset requirement from the original focus sequence images as a template image; An execution unit is configured to establish a ROI region containing a target object in the template image and record image coordinates (xr, yr) of the ROI region; a matching unit configured to perform template matching on each original image using the template image as a search template to obtain a region in each original image that is most similar to the ROI region, mark the region in each original image that is most similar to the ROI region as a corresponding search region, and record image coordinates (xi, yi) of the search region, where i=1, 2, ..., n; A calculation unit, configured to calculate a coordinate difference (xi-xr, yi-yr) between each search area and the ROI area; The compensation unit is configured to compensate each original image according to the coordinate difference (xi-xr, yi-yr) between all search areas and the ROI area to obtain a compensated focused sequence image.

10. A storage medium for storing a computer program, characterized in that: When the program is executed by a processor, the matching compensation method for focusing sequence images according to any one of claims 1 to 8 is implemented.

Images