A fisheye image correction method and system based on a geometric model and an effective area extraction algorithm

Abstract

The application discloses a fisheye image correction method and system based on a geometric model and an effective area extraction algorithm, and relates to the field of image processing.The method comprises the following steps: scanning pixel points from the center of a picture outward, judging the edge condition of a fisheye image where the current row and column are located according to the scanning result, adaptively adjusting the scanning step length, and further shortening the scanning time; and the ratio of the width and height of the fisheye image to the whole picture can be used to distinguish the light spot and the fisheye image, so that the accurate fisheye image radius can be obtained in real time in different devices and various measured pictures, and the image correction based on the geometric model is carried out by using the parameter, so that the accuracy of the extracted fisheye image is ensured, and the fisheye effective area can be extracted in different devices and various measured pictures.

Description

Technical Field

[0001] This invention relates to the field of image processing, and in particular to a fisheye image correction method and system based on a geometric model and an effective region extraction algorithm. Background Technology

[0002] In intelligent transportation, to expand the camera's field of view and adapt to the image target detection algorithm model, a trinocular camera is typically used for target detection: the forward and backward cameras are ordinary cameras; the downward-facing camera, used to expand the field of view, is a fisheye camera. However, this introduces severe distortion. To improve the accuracy of trinocular camera image stitching, distortion correction must first be performed on the downward-facing fisheye camera to ensure the image information is at the correct pixel coordinates. Then, using this corrected image as the intermediate field of view, overlapping fields of view with the forward and backward cameras are found and stitched together. The basis for fisheye distortion correction is extracting the effective region of the fisheye image. Generally, fisheye camera images are standard circular images, with all pixels except the fisheye image having a grayscale value of 0. However, in reality, due to device aperture limitations, the obtained camera fisheye image is only partially circular, with the upper and lower arcs truncated. Most importantly, the device also transmits light, and pixels other than the fisheye image have a certain grayscale value. This causes the fast scanning algorithm to stop prematurely when extracting the effective fisheye region, resulting in an incorrect radius and thus errors in the image acquired by the fisheye camera.

[0003] Currently, to extract the effective region of distorted images, a fast scanning algorithm based on an ideal grayscale fisheye image is used. By setting an image grayscale threshold, it is assumed that the grayscale value of pixels other than the fisheye image is below the threshold, and the scan line will stop at the edge of the fisheye image, thus obtaining the fisheye image radius. However, in reality, due to the transmission phenomenon of devices, the area outside the fisheye image in the image may have long strips or irregular slightly bright areas. If the scan line stops in a bright area, it will be considered that the pixel is the edge of the fisheye image, resulting in an incorrect fisheye image radius, thus leading to errors in the extracted fisheye image. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a fisheye image correction method and system based on a geometric model and an effective region extraction algorithm, which can solve the problems of large errors in the acquisition of fisheye image radius and errors in the extracted fisheye image.

[0005] To achieve the above objectives, on the one hand, the present invention provides a fisheye image correction method based on a geometric model and an effective region extraction algorithm, the method comprising:

[0006] Obtain the three primary color information of each pixel in the fisheye image captured by the camera, and convert the three primary color information into grayscale information corresponding to each pixel;

[0007] Based on the grayscale information of each pixel in the fisheye image, scanning is performed from a preset column in the fisheye image to the left and right directions respectively to determine the column number of the left cutting point and the column number of the right cutting point of the fisheye image;

[0008] Based on the grayscale information of each pixel in the fisheye image, scanning is performed in both upward and downward directions starting from a preset row in the fisheye image to determine the row number of the upper cut point and the row number of the lower cut point in the fisheye image;

[0009] Based on the column numbers of the left and right cut points of the fisheye image, and the row numbers of the top and bottom cut points of the fisheye image, obtain the radius of the fisheye image and the size information of the cropped fisheye image.

[0010] Geometric model image correction is performed on the fisheye image based on the radius of the fisheye image and the size information of the cropped fisheye image.

[0011] Further, the step of determining the column number of the left cutting point and the column number of the right cutting point of the fisheye image by scanning from a preset column in the fisheye image to the left and right respectively based on the grayscale information of each pixel in the fisheye image includes:

[0012] The highest and lowest row numbers of the fisheye image in the preset column are obtained based on the grayscale information of each pixel in the preset column and a preset threshold, and the height of the fisheye image in the preset column is obtained based on the highest and lowest row numbers of the fisheye image in the preset column.

[0013] The column number of the left cutting point of the fisheye image is determined based on the fisheye image height in the preset column, the fisheye image height of the current column after moving to the left by a preset step size, the preset grayscale threshold, and the column number.

[0014] The column number of the right cutting point of the fisheye image is determined based on the fisheye image height in the preset column, the fisheye image height of the current column after moving to the right by a preset step size, the preset grayscale threshold, and the column number.

[0015] Further, the step of determining the row number of the upper tangent point and the row number of the lower tangent point of the fisheye image by scanning upwards and downwards respectively from a preset row in the fisheye image based on the grayscale information of each pixel in the fisheye image includes:

[0016] The row number of the leftmost point and the row number of the rightmost point in the fisheye image of the preset row are obtained based on the grayscale information of each pixel in the preset row and the preset threshold. The width of the fisheye image in the preset row is obtained based on the row number of the leftmost point and the row number of the rightmost point in the fisheye image of the preset column.

[0017] The row number of the upper tangent point of the fisheye image is determined based on the fisheye image width in the preset row, the fisheye image width of the current row after moving upward by a preset step, the preset grayscale threshold, and the row number of the current row.

[0018] The row number of the lower cut point of the fisheye image is determined based on the fisheye image width in the preset row, the fisheye image width of the current row after moving down a preset step, the preset grayscale threshold, and the row number.

[0019] Further, the step of obtaining the fisheye image radius and the cropped fisheye image size information based on the column numbers of the left and right cutting points of the fisheye image, and the row numbers of the top and bottom cutting points of the fisheye image, includes:

[0020] Based on the column numbers of the left and right cut points of the fisheye image, the row numbers of the top and bottom cut points of the fisheye image, the difference between the column numbers of the left and right cut points of the fisheye image, and the difference between the row numbers of the top and bottom cut points of the fisheye image, the radius of the fisheye image and the size information of the cropped fisheye image are obtained.

[0021] Further, the step of performing geometric model image correction on the fisheye image based on the fisheye image radius and the cropped fisheye image size information includes:

[0022] The corrected image height of the fisheye image is obtained based on the corrected vertical pixel coordinates, horizontal pixel coordinates, vertical pixel coordinates of the image center, and horizontal pixel coordinates of the image center. The center of the cropped fisheye image is the corrected image center.

[0023] Based on the image height of the corrected fisheye image, obtain the pixel coordinates of each corrected pixel in the distorted image;

[0024] The image information primary color values ​​corresponding to the pixel coordinates in the distorted image are assigned to the image information primary color values ​​at the pixel coordinates in the corrected image.

[0025] On the other hand, the present invention provides a fisheye image correction system based on a geometric model and an effective region extraction algorithm, the system comprising:

[0026] The acquisition unit is used to acquire the three primary color information of each pixel in the fisheye image captured by the camera, and convert the three primary color information into grayscale information corresponding to each pixel.

[0027] The determining unit is used to scan from a preset column in the fisheye image to the left and right directions respectively, based on the grayscale information of each pixel in the fisheye image, to determine the column number of the left cutting point and the column number of the right cutting point of the fisheye image;

[0028] The determining unit is further configured to scan upwards and downwards respectively from a preset row in the fisheye image based on the grayscale information of each pixel in the fisheye image, and determine the row number of the upper cutting point and the row number of the lower cutting point of the fisheye image;

[0029] The acquisition unit is further configured to acquire the fisheye image radius and the cropped fisheye image size information based on the column number of the left cut point and the column number of the right cut point of the fisheye image, the row number of the upper cut point and the row number of the lower cut point of the fisheye image.

[0030] The correction unit is used to perform geometric model image correction on the fisheye image based on the radius of the fisheye image and the size information of the cropped fisheye image.

[0031] Further, the determining unit is specifically used to obtain the row number of the highest point and the row number of the lowest point of the fisheye image in the preset column based on the grayscale information of each pixel in the preset column and a preset threshold, and to obtain the height of the fisheye image in the preset column based on the row number of the highest point and the row number of the lowest point of the fisheye image in the preset column; to determine the column number of the left cutting point of the fisheye image based on the height of the fisheye image in the preset column, the height of the fisheye image in the current column after moving to the left by a preset step, a preset grayscale threshold, and the column number; and to determine the column number of the right cutting point of the fisheye image based on the height of the fisheye image in the preset column, the height of the fisheye image in the current column after moving to the right by a preset step, a preset grayscale threshold, and the column number.

[0032] Further, the determining unit is specifically configured to obtain the row number of the leftmost point and the row number of the rightmost point of the fisheye image in the preset row based on the grayscale information of each pixel in the preset row and a preset threshold, and to obtain the width of the fisheye image in the preset row based on the row number of the leftmost point and the row number of the rightmost point of the fisheye image in the preset column; to determine the row number of the upper tangent point of the fisheye image based on the width of the fisheye image in the preset row, the width of the fisheye image in the current row after moving upward by a preset step, a preset grayscale threshold, and the row number of the current row; and to determine the row number of the lower tangent point of the fisheye image based on the width of the fisheye image in the preset row, the width of the fisheye image in the current row after moving downward by a preset step, a preset grayscale threshold, and the row number of the current row.

[0033] Furthermore, the acquisition unit is specifically used to acquire the fisheye image radius and the cropped fisheye image size information based on the column number of the left cut point and the column number of the right cut point of the fisheye image, the row number of the top cut point and the row number of the bottom cut point of the fisheye image, the difference between the column number of the left cut point and the column number of the right cut point of the fisheye image, and the difference between the row number of the top cut point and the row number of the bottom cut point of the fisheye image.

[0034] Further, the correction unit is specifically used to obtain the corrected image height of the fisheye image based on the corrected vertical pixel coordinates, horizontal pixel coordinates, vertical pixel coordinates of the image center, and horizontal pixel coordinates of the image center, wherein the center of the cropped fisheye image is the corrected image center; to obtain the pixel coordinates of each corrected pixel point in the distorted image based on the corrected image height of the fisheye image; and to assign the image information primary color values ​​corresponding to the pixel coordinates in the distorted image to the image information primary color values ​​at the pixel coordinates in the corrected image.

[0035] This invention provides a fisheye image correction method and system based on a geometric model and an effective region extraction algorithm. It scans pixels outward from the image center, determines the edge condition of the fisheye image in the current row / column based on the scan results, and adaptively adjusts the scan step size to further shorten the scan time. Furthermore, it distinguishes between light spots and fisheye images by using empirical values ​​of the fisheye image's proportion of the width and height of the entire image. This allows for the real-time acquisition of accurate fisheye image radii from various measured images on different devices. Using these parameters, image correction based on a geometric model is performed, ensuring the accuracy of the extracted fisheye image. The method is also applicable to extracting the effective fisheye region from various measured images on different devices. Attached Figure Description

[0036] Figure 1 This is a flowchart of a fisheye image correction method based on a geometric model and an effective region extraction algorithm provided by the present invention;

[0037] Figure 2 This is a schematic diagram of the structure of a fisheye image correction system based on a geometric model and an effective region extraction algorithm provided by the present invention. Detailed Implementation

[0038] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0039] like Figure 1 As shown in the figure, an embodiment of the present invention provides a fisheye image correction method based on a geometric model and an effective region extraction algorithm, which includes the following steps:

[0040] 101. Obtain the three primary color information of each pixel in the fisheye image captured by the camera, and convert the three primary color information into grayscale information corresponding to each pixel.

[0041] Specifically, for example, i ranges from 1 to the number of rows M in the image, and j ranges from 1 to the number of columns N in the image. The three primary color information of the pixel in the i-th row and j-th column is A(i,j,:) = {R(i,j), G(i,j), B(i,j)}, and the corresponding grayscale information is I(i,j) = 0.59*R(i,j) + 0.11*G(i,j) + 0.3*B(i,j). By performing the above transformation on each pixel, the grayscale value of the entire image can be obtained.

[0042] 102. Based on the grayscale information of each pixel in the fisheye image, scan from the preset column in the fisheye image to the left and right respectively to determine the column number of the left cutting point and the column number of the right cutting point of the fisheye image.

[0043] Specifically, the row number of the highest point and the row number of the lowest point of the fisheye image in the preset column are obtained based on the grayscale information of each pixel in the preset column and a preset threshold. The height of the fisheye image in the preset column is obtained based on the row number of the highest point and the row number of the lowest point of the fisheye image in the preset column. The column number of the left cutting point of the fisheye image is determined based on the height of the fisheye image in the preset column, the height of the fisheye image in the current column after moving to the left by a preset step, the preset grayscale threshold, and the column number. The column number of the right cutting point of the fisheye image is determined based on the height of the fisheye image in the preset column, the height of the fisheye image in the current column after moving to the right by a preset step, the preset grayscale threshold, and the column number.

[0044] For example, scan from the N / 2 column in the middle column of the picture to the left and right directions respectively. (2a) First, calculate the row number k_min of the highest point and the row number k_max of the lowest point of the fisheye image in the middle column: Screen the pixel points in this column whose gray value is greater than the threshold T: First, screen the gray values of pixel points one by one from top to bottom until a pixel point with a gray value greater than the threshold T is encountered, and assign the row number of this pixel point to k_min; continue to scan downwards. Once a pixel point with a gray value less than the threshold T is encountered, it is considered that the bottom point of this column of the fisheye image has been reached, and the row number of this pixel point - 1 is assigned to k_max. Calculate the height h of the fisheye image in this column as h = k_max - k_min. If h < M / 2, it is considered that the image height is too small, and the previously scanned pixel points are light spots; rescan, start from row k_max + 1 until a pixel point with a gray value greater than the threshold is found, assign the row number of this pixel point to k_min, and the following method is the same as above until the last pixel point in this column is scanned, calculate h at this time, until k_max and k_min with h > M / 2 are found, and record the height h of the fisheye image scanned in the middle column as h0. (2b) Taking the left scan as an example, by default, the next scan step uj = the current scan step ui / 2, similar to the dichotomy method. The first left scan step u1 = N / 4, that is, move u1 columns to the left. The processing of the pixel points in this column is the same as (2a), and directly record h as h1 without judging h > M / 2: If both h0 and h1 are not equal to 0 and h1 = h0, it means that the fisheye images of the current column and the middle column are symmetrically distributed, and let the next scan step u2 = u1 / 2; if h1 > h0, it means that the image edge is higher and higher as it moves to the left, and the middle column is on the right side of the image. At this time, increase the step size, and let the next scan step u2 = u1 / 3 * 2; if h1 < h0, it means that the image edge is lower and lower as it moves to the left, and the current column is on the left side of the image. At this time, reduce the step size, and let the next scan step u2 = u1 / 3; After determining the second left scan step u2, continue to execute (2c) If h0 = 0 and h1 is not equal to 0, it means that the fisheye image is completely on the left side of the picture, which does not conform to the actual situation; if h1 = 0 and h0 = 0, it means that the fisheye image is completely on the right side of the picture, which does not conform to the actual situation; if h0 is not equal to 0 and h1 = 0, it means that the first left scan step size is too large. At this time, the scan position is reset to the middle column, and the first left scan starts again. The new step size u1' is reduced to 1 / 2 of the previous first left scan step size u1, and (2b) is executed again until the second left scan starts, and jump to step (2c). (2c) From the second left scan step u2 obtained in (2b), move the current column u2 columns again. The processing method for this column is the same as (2b). Just assign the left first scan height h1 to h0 and the left second scan height h2 to h1, and the subsequent scans are the same.Through multiple scans with the step size reduced until the step size = 1 (if the step size is less than 1, it is taken as 1), and when the gray value of the current column in the latest scan is lower than the threshold, it indicates reaching the left tangent point of the fish-eye image. The column number of this tangent point is j_min = the current column number + 1; or when the current column reaches the image boundary, j_min = the current column number.

[0045] (2d) Scanning to the right is the same. By default, the next scan step size uj = the current scan step size ui / 2, similar to the dichotomy method. The first scan step size to the right u1 = N / 4, that is, moving u1 columns to the right. The processing of the pixel points in this column is the same as (2a). Directly record h as h1 without judging whether h > M / 2: If both h0 and h1 are not equal to 0 and if h1 = h0, it means that the fish-eye images of the current column and the middle column are symmetrically distributed. Let the next scan step size u2 = u1 / 2; if h1 > h0, it means that the image edge of the fish-eye image is higher as it goes further to the right, and the middle column is on the left side of the fish-eye image. At this time, increase the step size, and let the next scan step size u2 = u1 / 3 * 2; if h1 < h0, it means that the image edge of the fish-eye image is lower as it goes further to the right, and the current column is on the right side of the fish-eye image. At this time, reduce the step size, and let the next scan step size u2 = u1 / 3; after determining the second scan step size u2 to the right, continue to execute (2e). If h0 = 0 and h1 is not equal to 0, it means that the fish-eye image is completely on the right side of the picture, which does not conform to the actual situation; if h1 = 0 and h0 = 0, it means that the fish-eye image is completely on the left side of the picture, which does not conform to the actual situation; if h0 is not equal to 0 and h1 = 0, it means that the first scan step size to the right is too large. At this time, the scanning position is reset to the middle column, and the first scan to the right starts again. The new step size u1' is reduced to 1 / 2 of the previous first scan step size u1 to the right, and (2d) is executed again until the second scan to the right starts, and then jump to step (2e). (2e) With the second scan step size u2 to the right obtained from (2d), move the current column u2 columns again. The processing method for this column is the same as (2d). Just assign the height h1 of the first scan to the right to h0 and the height h2 of the second scan to the right to h1. After that, the scanning is the same. Through multiple scans with the step size reduced until the step size = 1 (if the step size is less than 1, it is taken as 1), and when the gray value of the current column in the latest scan is lower than the threshold, it indicates reaching the right tangent point of the fish-eye image. The column number of this tangent point is j_max = the current column number - 1; or when the current column reaches the image boundary, j_max = the current column number.

[0046] 103. According to the gray information of each pixel point of the fish-eye image, start scanning in the upward and downward directions respectively from the preset row in the fish-eye image to determine the row number of the upper tangent point and the row number of the lower tangent point of the fish-eye image.

[0047] Specifically, the row number of the leftmost point and the row number of the rightmost point in the fisheye image of the preset row are obtained based on the grayscale information of each pixel in the preset row and a preset threshold. The width of the fisheye image in the preset row is obtained based on the row number of the leftmost point and the row number of the rightmost point in the fisheye image of the preset column. The row number of the upper tangent point of the fisheye image is determined based on the width of the fisheye image in the preset row, the width of the fisheye image in the current row after moving upward by a preset step, the preset grayscale threshold, and the row number of the current row. The row number of the lower tangent point of the fisheye image is determined based on the width of the fisheye image in the preset row, the width of the fisheye image in the current row after moving downward by a preset step, the preset grayscale threshold, and the row number of the current row.

[0048] For example, from the obtained image grayscale information I, scan upward and downward from the middle row (the M / 2-th row) of the picture. (3a) First, calculate the leftmost column sequence number k_min and the rightmost column sequence number k_max of the fisheye image in the middle row: Screen the pixel points on this row whose grayscale values are greater than the threshold T: First, screen the grayscale values of the pixel points one by one from left to right until a pixel point with a grayscale value greater than the threshold T is encountered, and assign the column sequence number of this pixel point to k_min; continue to scan to the right. Once a pixel point with a grayscale value less than the threshold T is encountered, it is considered that it has reached the rightmost point of the fisheye image on this row, and assign the column sequence number -1 of this pixel point to k_max. Calculate the width w of the fisheye image in this column as w = k_max - k_min. If w < N / 2, it is considered that the image width is too small, and the previously scanned pixel points are light spots; rescan, start from the (k_max + 1)-th row until a pixel point greater than the threshold is found, assign the column sequence number of this pixel point to k_min, and the following method is the same as above until the last pixel point in this column is scanned, calculate w at this time, until k_max and k_min with w > N / 2 are found, and record the width w of the fisheye image scanned in the middle row as w0. (3b) Taking the upward scan as an example, by default, the next scan step size uj = the current scan step size ui / 2. Similar to the dichotomy method, the first upward scan step size u1 = M / 4, that is, move up u1 rows. The processing of the pixel points on this row is the same as that in (3a), and directly record w as w1 without judging whether w > N / 2: If both w0 and w1 are not equal to 0 and if w1 = w0, it means that the fisheye images of the current row and the middle row are symmetrically distributed, and let the next scan step size w2 = w1 / 2; if w1 > w0, it means that the image edge of the fisheye image is higher as it goes up, and the middle column is on the lower side of the image. At this time, increase the step size, and let the next scan step size w2 = w1 / 3 * 2; if w1 < w0, it means that the image edge of the fisheye image is lower as it goes up, and the current column is on the upper side of the image. At this time, reduce the step size, and let the next scan step size w2 = w1 / 3; After determining the second upward scan step size w2, continue to execute (3c). If w0 = 0 and w1 is not equal to 0, it means that the fisheye image is completely on the upper side of the picture, which does not conform to the actual situation; if w1 = 0 and w0 = 0, it means that the fisheye image is completely on the lower side of the picture, which does not conform to the actual situation; if w0 is not equal to 0 and w1 = 0, it means that the first upward scan step size is too large. At this time, reset the scan position to the middle row and start the first upward scan again. The new step size w1' is reduced to 1 / 2 of the previous first upward scan step size w1, and re-execute (3b) until the second upward scan starts, and jump to step (3c). (3c) From the second upward scan step size u2 obtained in (3b), move the current row u2 rows again. The processing method for this row is the same as that in (3b). Just assign the first upward scan height w1 to w0 and the second upward scan height w2 to w1, and the subsequent scans are the same.Through multiple scans with the step size reduced until the step size = 1, if the step size is less than 1, it is taken as 1, and the grayscale value of the current line in the latest scan is lower than the threshold, indicating that it reaches the upper tangent point of the fish-eye image. The line number of this tangent point is i_min = the current line number + 1; or when the current line reaches the image boundary, i_min = the current line number.

[0049] Further, for the downward scan in (3d), by the same token, the default step size uj for the next scan is ui / 2 of the step size ui for the current scan, similar to the dichotomy method. The step size u1 for the first downward scan is 4 / N, that is, moving down u1 lines. The processing of the pixel points in this line is the same as in (3a). Directly record w as w1 without judging whether w > N / 2: If both w0 and w1 are not equal to 0 and w1 = w0, it indicates that the fish-eye images of the current line and the middle line are symmetrically distributed. Let the step size w2 for the next scan be w1 / 2; if w1 > w0, it means that the image edge is higher as it goes down in the fish-eye image, and the middle line is on the upper side of the fish-eye image. At this time, increase the step size, and let the step size w2 for the next scan be w1 / 3*2; if w1 < w0, it means that the image edge is lower as it goes down in the fish-eye image, and the current column is on the lower side of the fish-eye image. At this time, reduce the step size, and let the step size w2 for the next scan be w1 / 3; after determining the step size w2 for the second leftward scan, continue to execute (3e). If w0 = 0 and w1 is not equal to 0, it means that the fish-eye image is completely on the lower side of the picture, which does not conform to the actual situation; if w1 = 0 and w0 = 0, it means that the fish-eye image is completely on the upper side of the picture, which does not conform to the actual situation; if w0 is not equal to 0 and w1 = 0, it means that the step size for the first downward scan is too large. At this time, reset the scan position to the middle line and start the first downward scan again. The new step size w1' is reduced to 1 / 2 of the step size w1 for the previous first downward scan, and (3d) is executed again until the second downward scan starts, and then jump to step (3e). (3e) With the step size u2 for the second downward scan obtained from (3d), move the current line u2 lines again. The processing method for this column is the same as in (3d). Just assign the downward first scan height w1 to w0 and the downward second scan height w2 to w1, and the subsequent scans are the same. Through multiple scans with the step size reduced until the step size = 1, if the step size is less than 1, it is taken as 1, and when the grayscale value of the current line in the latest scan is lower than the threshold, it indicates that it reaches the lower tangent point of the fish-eye image. The line number of this tangent point is i_max = the current line number - 1; or when the current line reaches the image boundary, i_max = the current line number.

[0050] 104. Obtain the radius of the fish-eye image and the size information of the cropped fish-eye image according to the column numbers of the left tangent point and the right tangent point of the fish-eye image, and the row numbers of the upper tangent point and the lower tangent point of the fish-eye image.

[0051] Specifically, the fisheye image radius and cropped fisheye image size information are obtained based on the column numbers of the left and right cut points of the fisheye image, the row numbers of the top and bottom cut points of the fisheye image, the difference between the column numbers of the left and right cut points of the fisheye image, and the difference between the row numbers of the top and bottom cut points of the fisheye image.

[0052] For example, the column number j_mi n of the left cut point, the column number j_max of the right cut point, the row number i_mi n of the top and bottom cut points, and the row number i_max of the bottom cut point of the fisheye image can be used to obtain the fisheye image radius R = max(i_max - i_mi n, j_max - j_mi n) / 2. In addition, the original fisheye image is cropped based on i_max, i_mi n, j_max, and j_mi n: with the pixel (i_mi n, j_mi n) as the top left pixel, the width as j_max - j_mi n, and the height as i_max - i_mi n, to obtain the cropped fisheye image.

[0053] 105. Perform geometric model image correction on the fisheye image based on the radius of the fisheye image and the size information of the cropped fisheye image.

[0054] Specifically, based on the corrected vertical pixel coordinates, horizontal pixel coordinates, vertical pixel coordinates of the image center, and horizontal pixel coordinates of the image center, the corrected image height of the fisheye image is obtained, where the center of the cropped fisheye image is the corrected image center; based on the corrected image height of the fisheye image, the pixel coordinates of each corrected pixel point in the distorted image are obtained; and the image information primary color values ​​corresponding to the pixel coordinates in the distorted image are assigned to the image information primary color values ​​at the pixel coordinates in the corrected image.

[0055] For example, first calculate the image height of the fisheye image after correction. u represents the vertical pixel coordinates after distortion correction, v represents the horizontal pixel coordinates after distortion correction, x0 represents the vertical pixel coordinates of the corrected image center, and y0 represents the horizontal pixel coordinates of the corrected image center. The center of the cropped fisheye image is the center of the corrected image. The fisheye lens long-distance imaging model is then used as the isometric projection imaging model in the dissimilar imaging model. The imaging formula is as follows: the image height h of the fisheye distortion image = f * ω, where the object-side focal length of the lens is... Object half field of view Next, using the image height h of the fisheye distortion image, the pixel coordinates (x', y') of the corrected pixel in the distortion image are calculated: To round up

[0056] Finally, the RGB values ​​of the image information at pixel coordinates (x', y') in the distorted image are assigned to the RGB values ​​of the image information at pixel coordinates (u, v) in the corrected image, thus completing the fisheye image distortion correction.

[0057] This invention provides a fisheye image correction method based on a geometric model and an effective region extraction algorithm. It scans pixels outward from the image center, determines the edge condition of the fisheye image in the current row / column based on the scan results, and adaptively adjusts the scan step size to further shorten the scan time. Furthermore, it distinguishes between light spots and fisheye images by using empirical values ​​of the fisheye image's proportion of the width and height of the entire image. This allows for the real-time acquisition of accurate fisheye image radii from various measured images on different devices. Using these parameters, image correction based on a geometric model is performed, ensuring the accuracy of the extracted fisheye image. This method is also applicable to extracting the effective fisheye region from various measured images on different devices.

[0058] To implement the method provided in the embodiments of the present invention, the embodiments of the present invention provide a fisheye image correction system based on a geometric model and an effective region extraction algorithm, such as... Figure 2 As shown, the system includes: an acquisition unit 21, a determination unit 22, and a correction unit 23;

[0059] The acquisition unit 21 is used to acquire the three primary color information of each pixel in the fisheye image captured by the camera, and convert the three primary color information into grayscale information corresponding to each pixel.

[0060] The determining unit 22 is used to scan from a preset column in the fisheye image to the left and right directions respectively, based on the grayscale information of each pixel in the fisheye image, to determine the column number of the left cutting point and the column number of the right cutting point of the fisheye image.

[0061] The determining unit 22 is further configured to scan upwards and downwards from a preset row in the fisheye image based on the grayscale information of each pixel in the fisheye image, and determine the row number of the upper cutting point and the row number of the lower cutting point of the fisheye image.

[0062] The acquisition unit 21 is further configured to acquire the fisheye image radius and the cropped fisheye image size information based on the column number of the left cut point and the column number of the right cut point of the fisheye image, the row number of the upper cut point and the row number of the lower cut point of the fisheye image.

[0063] The correction unit 23 is used to perform geometric model image correction on the fisheye image based on the radius of the fisheye image and the size information of the cropped fisheye image.

[0064] Further, the determining unit 22 is specifically used to obtain the row number of the highest point and the row number of the lowest point of the fisheye image in the preset column based on the grayscale information of each pixel in the preset column and a preset threshold, and to obtain the height of the fisheye image in the preset column based on the row number of the highest point and the row number of the lowest point of the fisheye image in the preset column; to determine the column number of the left cutting point of the fisheye image based on the height of the fisheye image in the preset column, the height of the fisheye image in the current column after moving to the left by a preset step, a preset grayscale threshold, and the column number; and to determine the column number of the right cutting point of the fisheye image based on the height of the fisheye image in the preset column, the height of the fisheye image in the current column after moving to the right by a preset step, a preset grayscale threshold, and the column number.

[0065] Further, the determining unit 22 is specifically used to obtain the row number of the leftmost point and the row number of the rightmost point of the fisheye image in the preset row based on the grayscale information of each pixel in the preset row and a preset threshold, and to obtain the width of the fisheye image in the preset row based on the row number of the leftmost point and the row number of the rightmost point of the fisheye image in the preset column; to determine the row number of the upper tangent point of the fisheye image based on the width of the fisheye image in the preset row, the width of the fisheye image in the current row after moving upward by a preset step, a preset grayscale threshold, and the row number of the current row; and to determine the row number of the lower tangent point of the fisheye image based on the width of the fisheye image in the preset row, the width of the fisheye image in the current row after moving downward by a preset step, a preset grayscale threshold, and the row number of the current row.

[0066] Further, the acquisition unit 21 is specifically used to acquire the fisheye image radius and the cropped fisheye image size information based on the column number of the left cut point and the column number of the right cut point of the fisheye image, the row number of the top cut point and the row number of the bottom cut point of the fisheye image, the difference between the column number of the left cut point and the column number of the right cut point of the fisheye image, and the difference between the row number of the top cut point and the row number of the bottom cut point of the fisheye image.

[0067] Further, the correction unit 23 is specifically used to obtain the corrected image height of the fisheye image based on the corrected vertical pixel coordinates, horizontal pixel coordinates, vertical pixel coordinates of the image center, and horizontal pixel coordinates of the image center, wherein the center of the cropped fisheye image is the corrected image center; to obtain the pixel coordinates of each corrected pixel point in the distorted image based on the corrected image height of the fisheye image; and to assign the image information primary color values ​​corresponding to the pixel coordinates in the distorted image to the image information primary color values ​​at the pixel coordinates in the corrected image.

[0068] This invention provides a fisheye image correction system based on a geometric model and an effective region extraction algorithm. It scans pixels outward from the image center, determines the edge condition of the fisheye image in the current row / column based on the scan results, and adaptively adjusts the scan step size to further shorten the scan time. Furthermore, it distinguishes between light spots and fisheye images by using empirical values ​​of the fisheye image's proportion of the width and height of the entire image. This allows for the real-time acquisition of accurate fisheye image radii from various measured images on different devices. Using these parameters, image correction based on a geometric model is performed, ensuring the accuracy of the extracted fisheye image. The system is also applicable to extracting the effective fisheye region from various measured images on different devices.

[0069] It should be understood that the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the scope to the specific order or hierarchy described.

[0070] In the above detailed description, various features are combined together in a single embodiment to simplify this disclosure. This approach to disclosure should not be construed as reflecting an intention that embodiments of the claimed subject matter require more features than are explicitly stated in each claim. Rather, as reflected in the appended claims, the invention is presented with fewer features than all of the features of the single disclosed embodiment. Therefore, the appended claims are hereby explicitly incorporated into the detailed description, wherein each claim stands alone as a preferred embodiment of the invention.

[0071] The disclosed embodiments have been described above to enable any person skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the spirit and scope of this disclosure. Therefore, this disclosure is not limited to the embodiments given herein, but is consistent with the broadest scope of the principles and novel features disclosed in this application.

[0072] The foregoing description includes examples of one or more embodiments. It is certainly impossible to describe all possible combinations of components or methods in order to describe the above embodiments, but those skilled in the art will recognize that further combinations and arrangements of the various embodiments are possible. Therefore, the embodiments described herein are intended to cover all such changes, modifications, and variations that fall within the scope of the appended claims. Furthermore, the term "comprising" as used in the specification or claims is interpreted in a manner similar to the term "including," as interpreted when used as a conjunction in the claims. Additionally, the use of any term "or" in the specification of the claims is intended to mean "non-exclusive or."

[0073] Those skilled in the art will also understand that the various illustrative logical blocks, units, and steps listed in the embodiments of the present invention can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, units, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of the present invention.

[0074] The various illustrative logic blocks or units described in the embodiments of this invention can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic system, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing systems, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0075] The steps of the methods or algorithms described in the embodiments of this invention can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0076] In one or more exemplary designs, the functions described in the embodiments of the present invention can be implemented in hardware, software, firmware, or any combination of these three. If implemented in software, these functions can be stored on a computer-readable medium or transmitted on a computer-readable medium in the form of one or more instructions or code. Computer-readable media include computer storage media and communication media that facilitate the transfer of computer programs from one place to another. Storage media can be any available media that can be accessed by a general-purpose or special-purpose computer. For example, such computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage systems, or any other medium that can be used to carry or store program code in the form of instructions or data structures and other forms that can be read by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Furthermore, any connection can be suitably defined as a computer-readable medium, for example, if the software is transmitted from a website, server, or other remote resource via a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wirelessly, such as infrared, wireless, and microwave, it is also included in the defined computer-readable medium. The disks and discs mentioned include compressed disks, laser discs, optical discs, DVDs, floppy disks, and Blu-ray discs. Disks typically copy data magnetically, while discs typically copy data optically using lasers. Combinations of the above can also be contained in computer-readable media.

[0077] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A fisheye image correction method based on a geometric model and an effective region extraction algorithm, characterized in that, The method includes: Obtain the three primary color information of each pixel in the fisheye image captured by the camera, and convert the three primary color information into grayscale information corresponding to each pixel; Based on the grayscale information of each pixel in the fisheye image, scanning is performed from a preset column in the fisheye image to the left and right directions respectively to determine the column number of the left cutting point and the column number of the right cutting point of the fisheye image; Based on the grayscale information of each pixel in the fisheye image, scanning is performed in both upward and downward directions starting from a preset row in the fisheye image to determine the row number of the upper cut point and the row number of the lower cut point in the fisheye image; Based on the column numbers of the left and right cut points of the fisheye image, and the row numbers of the top and bottom cut points of the fisheye image, obtain the radius of the fisheye image and the size information of the cropped fisheye image. Geometric model image correction is performed on the fisheye image based on the radius of the fisheye image and the size information of the cropped fisheye image. The step of determining the column number of the left and right cut points of the fisheye image by scanning from a preset column in the fisheye image to the left and right based on the grayscale information of each pixel in the fisheye image includes: The highest and lowest row numbers of the fisheye image in the preset column are obtained based on the grayscale information of each pixel in the preset column and a preset threshold, and the height of the fisheye image in the preset column is obtained based on the highest and lowest row numbers of the fisheye image in the preset column. The column number of the left cutting point of the fisheye image is determined based on the fisheye image height in the preset column, the fisheye image height of the current column after moving to the left by a preset step size, the preset grayscale threshold, and the column number. The column number of the right cutting point of the fisheye image is determined based on the fisheye image height in the preset column, the fisheye image height of the current column after moving to the right by a preset step size, the preset grayscale threshold, and the column number.

2. The fisheye image correction method based on a geometric model and an effective region extraction algorithm according to claim 1, characterized in that, The step of determining the row number of the upper cut point and the row number of the lower cut point of the fisheye image by scanning upwards and downwards respectively from a preset row in the fisheye image based on the grayscale information of each pixel in the fisheye image includes: The row number of the leftmost point and the row number of the rightmost point in the fisheye image of the preset row are obtained based on the grayscale information of each pixel in the preset row and the preset threshold. The width of the fisheye image in the preset row is obtained based on the row number of the leftmost point and the row number of the rightmost point in the fisheye image of the preset column. The row number of the upper tangent point of the fisheye image is determined based on the fisheye image width in the preset row, the fisheye image width of the current row after moving upward by a preset step, the preset grayscale threshold, and the row number of the current row. The row number of the lower cut point of the fisheye image is determined based on the fisheye image width in the preset row, the fisheye image width of the current row after moving down a preset step, the preset grayscale threshold, and the row number.

3. The fisheye image correction method based on a geometric model and an effective region extraction algorithm according to claim 1, characterized in that, The step of obtaining the fisheye image radius and cropped fisheye image size information based on the column numbers of the left and right cut points, the row numbers of the top and bottom cut points of the fisheye image includes: Based on the column numbers of the left and right cut points of the fisheye image, the row numbers of the top and bottom cut points of the fisheye image, the difference between the column numbers of the left and right cut points of the fisheye image, and the difference between the row numbers of the top and bottom cut points of the fisheye image, the radius of the fisheye image and the size information of the cropped fisheye image are obtained.

4. The fisheye image correction method based on a geometric model and an effective region extraction algorithm according to claim 1, characterized in that, The step of performing geometric model image correction on the fisheye image based on the fisheye image radius and the cropped fisheye image size information includes: The corrected image height of the fisheye image is obtained based on the corrected vertical pixel coordinates, horizontal pixel coordinates, vertical pixel coordinates of the image center, and horizontal pixel coordinates of the image center. The center of the cropped fisheye image is the corrected image center. Based on the image height of the corrected fisheye image, obtain the pixel coordinates of each corrected pixel in the distorted image; The image information primary color values ​​corresponding to the pixel coordinates in the distorted image are assigned to the image information primary color values ​​at the pixel coordinates in the corrected image.

5. A fisheye image correction system based on a geometric model and an effective region extraction algorithm, characterized in that, The system includes: The acquisition unit is used to acquire the three primary color information of each pixel in the fisheye image captured by the camera, and convert the three primary color information into grayscale information corresponding to each pixel. The determining unit is used to scan from a preset column in the fisheye image to the left and right directions respectively, based on the grayscale information of each pixel in the fisheye image, to determine the column number of the left cutting point and the column number of the right cutting point of the fisheye image; The determining unit is further configured to scan upwards and downwards respectively from a preset row in the fisheye image based on the grayscale information of each pixel in the fisheye image, and determine the row number of the upper cutting point and the row number of the lower cutting point of the fisheye image; The acquisition unit is further configured to acquire the fisheye image radius and the cropped fisheye image size information based on the column number of the left cut point and the column number of the right cut point of the fisheye image, the row number of the upper cut point and the row number of the lower cut point of the fisheye image. The correction unit is used to perform geometric model image correction on the fisheye image based on the radius of the fisheye image and the size information of the cropped fisheye image; The determining unit is specifically configured to obtain the row number of the highest point and the row number of the lowest point of the fisheye image in the preset column based on the grayscale information of each pixel in the preset column and a preset threshold, and to obtain the height of the fisheye image in the preset column based on the row number of the highest point and the row number of the lowest point of the fisheye image in the preset column; to determine the column number of the left cutting point of the fisheye image based on the height of the fisheye image in the preset column, the height of the fisheye image in the current column after moving to the left by a preset step, a preset grayscale threshold, and the column number; and to determine the column number of the right cutting point of the fisheye image based on the height of the fisheye image in the preset column, the height of the fisheye image in the current column after moving to the right by a preset step, a preset grayscale threshold, and the column number.

6. The fisheye image correction system based on a geometric model and an effective region extraction algorithm according to claim 5, characterized in that, The determining unit is further configured to obtain the row number of the leftmost point and the row number of the rightmost point of the fisheye image in the preset row based on the grayscale information of each pixel in the preset row and a preset threshold, and to obtain the width of the fisheye image in the preset row based on the row number of the leftmost point and the row number of the rightmost point of the fisheye image in the preset column; to determine the row number of the upper tangent point of the fisheye image based on the width of the fisheye image in the preset row, the width of the fisheye image in the current row after moving upward by a preset step, a preset grayscale threshold, and the row number of the current row; and to determine the row number of the lower tangent point of the fisheye image based on the width of the fisheye image in the preset row, the width of the fisheye image in the current row after moving downward by a preset step, a preset grayscale threshold, and the row number of the current row.

7. The fisheye image correction system based on a geometric model and an effective region extraction algorithm according to claim 5, characterized in that, The acquisition unit is specifically used to acquire the fisheye image radius and the cropped fisheye image size information based on the column number of the left and right cut points of the fisheye image, the row number of the top and bottom cut points of the fisheye image, the difference between the column number of the left and right cut points of the fisheye image, and the difference between the row number of the top and bottom cut points of the fisheye image.

8. A fisheye image correction system based on a geometric model and an effective region extraction algorithm according to claim 5, characterized in that, The correction unit is specifically used to obtain the corrected image height of the fisheye image based on the corrected vertical pixel coordinates, horizontal pixel coordinates, vertical pixel coordinates of the image center, and horizontal pixel coordinates of the image center, wherein the center of the cropped fisheye image is the corrected image center; obtain the pixel coordinates of each corrected pixel point in the distorted image based on the corrected image height of the fisheye image; and assign the image information primary color values ​​corresponding to the pixel coordinates in the distorted image to the image information primary color values ​​at the pixel coordinates in the corrected image.

Images