A fisheye image correction method and system based on adaptive threshold of fisheye image

Abstract

The application discloses a fish-eye image correction method and system based on a self-adaptive threshold of a fish-eye image, relates to the field of image processing, and comprises the following steps: obtaining the pixel gray scale ratio in the fish-eye image, the pixel gray scale ratio between the inside and the outside of the fish-eye image, setting double thresholds to obtain the edge contour points of the fish-eye image, improving the confidence of the pixels being the fish-eye edge, then based on the edge contour point set of the fish-eye image, obtaining the fish-eye image radius and the size information of the cropped fish-eye image, and correcting the fish-eye image, thereby ensuring the accuracy of the extracted fish-eye image, and the method can also be applied to the extraction of the effective fish-eye area in various actually measured pictures of different devices.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image processing, in particular to a fisheye image correction method and system based on adaptive threshold of fisheye image. BACKGROUND

[0002] In intelligent transportation, in order to expand the camera field of view and adapt to the model of image target detection algorithm, a three-camera is usually used to detect targets: the cameras for collecting forward and backward are ordinary cameras; the camera for collecting downward is a fisheye camera for expanding the field of view, but this will bring serious distortion. In order to improve the correctness of image stitching of the three-camera, the fisheye camera for looking downward needs to be corrected first, so that the image information is on the correct pixel coordinates, and then the corrected image of this camera is used as the intermediate field of view to find the overlapping field of view of the forward and backward cameras respectively, and then the image stitching is performed. The basis for fisheye distortion correction needs to extract the effective area of the fisheye image first. Generally, the fisheye camera image is a standard circular image, and the gray value of other pixel points except the fisheye image is 0. However, in the actual situation, due to the opening limitation of the device, only part of the circular fisheye image is obtained, and the upper and lower arcs are cut off. Most importantly, the device will also transmit light, and other pixels except the fisheye image have a certain gray value, which will make the fast scanning algorithm stop too early when extracting the effective area of the fisheye, resulting in an error in the fisheye image collected by the fisheye camera.

[0003] At present, in order to extract the effective area of the distorted image, a fast scanning algorithm based on ideal gray fisheye image is used. By setting the image gray threshold, the gray value of the pixel points except the fisheye image is lower than the threshold by default, and the scanning line will stop at the edge of the fisheye image, so as to obtain the fisheye image radius. However, in the actual situation, due to the light transmission phenomenon of the device, there may be long strip or irregular slightly bright areas in the area except the fisheye image in the picture. If the scanning line stops in the bright area and considers that the pixel point is the edge of the fisheye image, an error fisheye image radius will be obtained, resulting in an error in the extracted fisheye image. SUMMARY

[0004] To solve the above technical problems, the present application provides a fisheye image correction method and system based on adaptive threshold of fisheye image, which can solve the problem of large error in the existing fisheye image radius acquisition and the error in the extracted fisheye image.

[0005] To achieve the above purpose, on the one hand, the present application provides a fisheye image correction method based on adaptive threshold of fisheye image, which comprises:

[0006] obtaining the three-primary-color information of each pixel point in the fisheye image collected by the camera, and converting the three-primary-color information into gray information corresponding to each pixel point;

[0007] According to the gray information of the respective pixel points, a threshold of pixel gray ratio inside the fisheye image, a pixel gray outside the fisheye image and a threshold of pixel gray ratio inside the fisheye image are obtained;

[0008] According to the threshold of pixel gray ratio inside the fisheye image, the pixel gray outside the fisheye image and the threshold of pixel gray ratio inside the fisheye image, and the gray information of the respective pixel points, a set of sampling points of the left and right circular contours of the fisheye image is obtained;

[0009] According to the set of sampling points of the left and right circular contours of the fisheye image, the fisheye image radius and the size information of the cropped fisheye image are obtained;

[0010] The fisheye image is corrected according to the fisheye image radius and the size information of the cropped fisheye image.

[0011] Further, the step of obtaining the threshold of pixel gray ratio inside the fisheye image according to the gray information of the respective pixel points comprises:

[0012] The pixel points at the center of the image are scanned in a cross shape to obtain the threshold of pixel gray ratio in each direction;

[0013] The threshold of pixel gray ratio inside the fisheye image is obtained according to the threshold of pixel gray ratio in each direction.

[0014] Further, the step of obtaining the threshold of pixel gray ratio inside the fisheye image and the pixel gray outside the fisheye image according to the gray information of the respective pixel points comprises:

[0015] The pixel points at the center of the image are scanned in an X shape to obtain the threshold of pixel gray ratio inside the fisheye image and the pixel gray outside the fisheye image in each direction;

[0016] The threshold of pixel gray ratio inside the fisheye image and the pixel gray outside the fisheye image are obtained according to the threshold of pixel gray ratio inside the fisheye image and the pixel gray outside the fisheye image in each direction.

[0017] Further, the step of obtaining the fisheye image radius and the size information of the cropped fisheye image according to the set of sampling points of the left and right circular contours of the fisheye image comprises:

[0018] The fisheye image radius and the center of the circle are obtained according to a preset linear least square method and the set of sampling points of the left and right circular contours of the fisheye image;

[0019] The fisheye image is cropped according to the fisheye image radius and the center of the circle.

[0020] Further, the step of correcting the fisheye image according to the fisheye image radius and the size information of the cropped fisheye image comprises:

[0021] According to the corrected longitudinal pixel coordinate, the transverse pixel coordinate, the longitudinal pixel coordinate of the image center, and the transverse pixel coordinate of the image center, the corrected image height of the fisheye image based on the adaptive threshold of the fisheye image is obtained, and the center of the cropped fisheye image is the corrected image center.

[0022] According to the corrected image height of the fisheye image based on the adaptive threshold of the fisheye image, the pixel coordinates of each corrected pixel point in the distortion image are obtained.

[0023] The image information tricolor values corresponding to the pixel coordinates in the distortion image are assigned to the image information tricolor values at the pixel coordinates in the corrected image.

[0024] On the other hand, the present application provides a fisheye image correction system based on an adaptive threshold of a fisheye image, which comprises:

[0025] An acquisition unit is configured to acquire the tricolor information of each pixel point in a fisheye image collected by a camera, and convert the tricolor information into the gray scale information corresponding to each pixel point.

[0026] The acquisition unit is further configured to acquire the threshold of the pixel gray scale ratio in the fisheye image, the pixel gray scale outside the fisheye image, and the threshold of the pixel gray scale ratio in the fisheye image according to the gray scale information of each pixel point.

[0027] The acquisition unit is further configured to acquire the sampling point set of the left and right circular contours of the fisheye image according to the threshold of the pixel gray scale ratio in the fisheye image, the pixel gray scale outside the fisheye image, the threshold of the pixel gray scale ratio in the fisheye image, and the gray scale information of each pixel point.

[0028] The acquisition unit is further configured to acquire the fisheye image radius and the size information of the cropped fisheye image according to the sampling point set of the left and right circular contours of the fisheye image.

[0029] A correction unit is configured to correct the fisheye image according to the fisheye image radius and the size information of the cropped fisheye image.

[0030] Further, the acquisition unit is specifically configured to scan in a cross shape with the pixel point of the image center as the center, acquire the threshold of the pixel gray scale ratio in each direction, and acquire the threshold of the pixel gray scale ratio in the fisheye image according to the threshold of the pixel gray scale ratio in each direction.

[0031] Further, the acquisition unit is specifically further configured to scan and acquire the threshold of the fish-eye image outer pixel gray scale and the fish-eye image inner pixel gray scale in an X-shaped manner with the pixel point at the image center as the center; and acquire the threshold of the fish-eye image outer pixel gray scale and the fish-eye image inner pixel gray scale according to the threshold of the fish-eye image outer pixel gray scale and the fish-eye image inner pixel gray scale in each direction.

[0032] Further, the acquisition unit is specifically further configured to acquire the fish-eye image radius and the circular center according to the preset linear least square method and the sample point set of the left and right circular contours of the fish-eye image; and crop the fish-eye image according to the fish-eye image radius and the circular center.

[0033] Further, the correction unit is specifically configured to acquire the corrected image height of the fish-eye image based on the adaptive threshold of the fish-eye image according to the corrected longitudinal pixel coordinate, the corrected transverse pixel coordinate, the longitudinal pixel coordinate of the image center and the transverse pixel coordinate of the image center, the cropped fish-eye image center being the corrected image center; acquire the pixel coordinate of each corrected pixel point in the distortion image according to the corrected image height of the fish-eye image based on the adaptive threshold of the fish-eye image; and assign the image information three primary color values corresponding to the pixel coordinate in the distortion image to the image information three primary color values at the pixel coordinate in the corrected image.

[0034] The fish-eye image correction method and system based on the adaptive threshold of the fish-eye image provided by the application can improve the confidence of the fish-eye edge pixel by acquiring the pixel gray scale ratio inside the fish-eye image and the pixel gray scale ratio between the inside and the outside of the fish-eye image, setting a double threshold to acquire the edge contour points of the fish-eye image, acquiring the fish-eye image radius and the size information of the cropped fish-eye image based on the edge contour point set of the fish-eye image, and correcting the fish-eye image, thereby ensuring the accuracy of the extracted fish-eye image and making the fish-eye effective area extraction applicable to different devices and various actually measured images. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 is a flowchart of the fish-eye image correction method based on the adaptive threshold of the fish-eye image provided by the application;

[0036] Figure 2 is a structural schematic diagram of the fish-eye image correction system based on the adaptive threshold of the fish-eye image provided by the application. DETAILED DESCRIPTION

[0037] The technical solutions of the application will be further described in detail below with reference to the drawings and embodiments.

[0038] As Figure 1As shown, the fish-eye image correction method based on adaptive threshold of fish-eye image provided by the embodiment of the present application comprises the following steps:

[0039] 101. Obtain the three primary color information of each pixel point in the fish-eye image collected by the camera, and convert the three primary color information into the gray scale information corresponding to each pixel point.

[0040] Specifically, for example, i is from 1 to the number of image rows M, j is from 1 to the number of image columns N, the three primary color information A(i,j, :) of the pixel point in the i-th row and the j-th column is {R(i,j), G(i,j), B(i,j)}, and the corresponding gray scale information I(i,j) is 0.59*R(i,j)+0.11*G(i,j)+0.3*B(i,j). The conversion is performed on each pixel point, and the gray scale value of the whole image can be obtained.

[0041] 102. Obtain the threshold of the pixel gray scale ratio in the fish-eye image, the pixel gray scale outside the fish-eye image, and the threshold of the pixel gray scale ratio in the fish-eye image according to the gray scale information of each pixel point.

[0042] Specifically, the pixel point at the center of the image is taken as the center to perform cross-type scanning, and the threshold of the pixel gray scale ratio in each direction is obtained. The threshold of the pixel gray scale ratio in the fish-eye image is obtained according to the threshold of the pixel gray scale ratio in each direction. The pixel point at the center of the image is taken as the center to perform X-type scanning, and the pixel gray scale outside the fish-eye image and the threshold of the pixel gray scale ratio in the fish-eye image in each direction are obtained. The threshold of the pixel gray scale outside the fish-eye image and the threshold of the pixel gray scale ratio in the fish-eye image are obtained according to the pixel gray scale outside the fish-eye image and the threshold of the pixel gray scale ratio in the fish-eye image in each direction.

[0043] For example, taking the pixel coordinate (M / 2, N / 2) of the picture center as the center, the pixel point of the picture center is generally large in the fisheye image and spreads outward step by step, and the threshold value Tn of the pixel gray scale ratio in the fisheye image, the pixel gray scale outside the fisheye image and the threshold value Tw of the pixel gray scale ratio in the fisheye image can be obtained from the image gray scale information. Taking the pixel point of the coordinate (M / 2, N / 2) as the center, scanning left, right, up and down in a cross shape, the horizontal scanning step is N / 4, and the vertical scanning step is M / 4: that is, the pixel point coordinate of the left scanning is (M / 2, N / 4), and the gray scale ratio Kleft at this time is recorded as Kleft=I(M / 2, N / 4) / I(M / 2, N / 2); the pixel point coordinate of the right scanning is (M / 2, 3*N / 4), and the gray scale ratio KRi ght at this time is recorded as KRi ght=I(M / 2, 3*N / 4) / I(M / 2, N / 2); the pixel point coordinate of the up scanning is (M / 4, N / 2), and the gray scale ratio KUp at this time is recorded as KUp=I(M / 4, N / 2) / I(M / 2, N / 2); the pixel point coordinate of the down scanning is (3*M / 4, N / 2), and the gray scale ratio KDown at this time is recorded as KDown=I(3*M / 4, N / 2) / I(M / 2, N / 2). Then Tn=(KLeft+KRi ght+KUp+KDown) / 4.

[0044] Likewise, taking the pixel point of the coordinate (M / 2, N / 2) as the center, scanning left-up, right-up, left-down and right-down in an X shape, only the scanning points directly reach the picture edge, and the four corner points of the image edge are considered to be outside the fisheye image: that is, the pixel point coordinate of the left-up corner is (1, 1), and the gray scale ratio KLeftUp at this time is recorded as KLeftUp=I(1, 1) / I(M / 2, N / 2); the pixel point coordinate of the right-up corner is (1, N), and the gray scale ratio KRi ghtUp at this time is recorded as KRi ghtUp=I(1, N) / I(M / 2, N / 2); the pixel point coordinate of the left-down corner is (M, 1), and the gray scale ratio KLeftDown at this time is recorded as KLeftDown=I(M, 1) / I(M / 2, N / 2); the pixel point coordinate of the right-down corner is (M, N), and the gray scale ratio KRi ghtDown at this time is recorded as KRi ghtDown=I(M, N) / I(M / 2, N / 2). Then Tw=(KLeftUp+KRi ghtUp+KLeftDown+KRi ghtDown) / 4.

[0045] 103、According to the threshold value of the pixel gray scale ratio in the fisheye image, the threshold value of the pixel gray scale ratio in the fisheye image, the gray scale information of each pixel point, the sampling point set of the left and right circular contours of the fisheye image is obtained.

[0046] Specifically, for example, scanning from the left and right edges of the picture with a certain step u, a set of sample points A = {a1, a2...aK} of the circular profile of the left and right edges of the fisheye image is obtained, where a i is the pixel point coordinate (i, j). The selection rule of the row number of the scanning point is as follows: first, starting from the first row i = 1, scanning at an interval of step u1 = M / NSamp l e, NSamp l e can be selected according to the accuracy requirement: the value is small for low accuracy, and the value is large for high accuracy. Here, NSamp l e = 20 is selected, i.e., the row number of the second row is i = 1 + u1, the row number of the third row is i = 1 + 2*u1, and so on until the bottom of the picture, so that the row number of each scanning is obtained, a total of NSamp l e times. The selection rule of the column number of the scanning point is as follows: left side: first, starting from the first column j = 1, scanning at an interval of step u2 = N / NSamp l e to the right, i.e., the column number of the second column is 1 + u2, the column number of the third column is 1 + 2*u2, and so on until the right side. Right side: first, starting from the last column j = N, scanning at an interval of step u2 = N / NSamp l e to the left, i.e., the column number of the second column is N-u2, the column number of the third column is N-2*u2, and so on until the column number of the left side >= the column number of the right side, so that the column number of each scanning is obtained, which is the union of the column numbers of the left and right sides, a total of NSampl e+2 times. Therefore, the total number of scans is NSamp l e*(NSamp l e+2) times: the pixel point coordinate of the first scan is (1, 1), and the gray value is processed as follows:

[0047] If condition 1: I(1, 1)<I(M / 2,N / 2)*1.2Tw is met, it is considered to be an edge region, and the pixel point coordinate is not added to the edge point set A of the fisheye image; if the above condition is not met, it is further judged whether condition 2: I(M / 2,N / 2)*0.8Tn<I(1, 1)<I(M / 2,N / 2)*1.2Tn is met, if met, it is considered to be a highlight region, and it is further judged whether condition 3 is met; if not met, the pixel point coordinate is not added to the edge point set A of the fisheye image; if it is an irregular light spot, the light spot width is generally small, condition 3: it is judged whether I(M / 2,N / 2)*0.8Tn<I(1, 1+N / 4)<I(M / 2,N / 2)*1.2Tn is met, i.e., a new pixel point is taken N / 4 columns to the right to judge, if met, the pixel point coordinate is added to the edge point set A of the fisheye image; if not met, it is not added. The pixel point coordinates of the subsequent scans are (1, 1+u2), …, (1, 1+(NSamp l e / 2)*u2), (1, N), (1, N-u2), …, (1, N-(NSamp l e / 2)*u2); the row number of the pixel point coordinate of the second scan is 2; …, the row number of the pixel point coordinate of the NSamp l e-th scan is M: the processing process of each point is the same as above.

[0048] 104. Obtaining the fish-eye image radius and the size information of the cropped fish-eye image according to the sample point set of the left and right circular contours of the fish-eye image.

[0049] Specifically, obtaining the fish-eye image radius and the center of the circle according to the preset linear least square method and the sample point set of the left and right circular contours of the fish-eye image; and cropping the fish-eye image according to the fish-eye image radius and the center of the circle.

[0050] For example, a set of fish-eye contour points A = {a1, a2…aK} is obtained, where a i is a pixel point coordinate (i, j), the center of the circle x and the radius r can be obtained by using the linear least square method. That is, the obtained x and r can make the distance between each sample point and the center of the circle closest to r, and the optimal equation is: After expanding the above equation, let The equation can be obtained as Let B T = {b1, b2,..., b K} again, The final optimal equation can be obtained as y = (B T B) + B T d can be solved, so that x and r can be obtained:

[0051] 105. Correcting the fish-eye image according to the fish-eye image radius and the size information of the cropped fish-eye image.

[0052] Specifically, obtaining the corrected image height of the fish-eye image based on the adaptive threshold of the fish-eye image according to the corrected longitudinal pixel coordinate, the corrected transverse pixel coordinate, the longitudinal pixel coordinate of the image center, and the transverse pixel coordinate of the image center, the center of the cropped fish-eye image being the corrected image center; obtaining the pixel coordinates of each corrected pixel point in the distortion image according to the corrected image height of the fish-eye image based on the adaptive threshold of the fish-eye image; and assigning the image information three primary color values corresponding to the pixel coordinates in the distortion image to the image information three primary color values at the pixel coordinates in the corrected image.

[0053] For example, the corrected image height of the fish-eye image based on the adaptive threshold of the fish-eye image is first obtained as u represents the vertical pixel coordinates after distortion correction, v represents the horizontal pixel coordinates after distortion correction, x0 represents the vertical pixel coordinates of the corrected image center, and y0 represents the horizontal pixel coordinates of the corrected image center. The center of the cropped fisheye image is the center of the corrected image. The fisheye lens long-distance imaging model is then used as the isometric projection imaging model in the dissimilar imaging model. The imaging formula is as follows: the image height h of the fisheye distortion image = f * ω, where the object-side focal length of the lens is... Object half field of view Next, using the image height h of the fisheye distortion image, the pixel coordinates (x', y') of the corrected pixel in the distortion image are calculated: To round up

[0054] Finally, the RGB values ​​of the image information at pixel coordinates (x', y') in the distorted image are assigned to the RGB values ​​of the image information at pixel coordinates (u, v) in the corrected image, thus completing the fisheye image distortion correction.

[0055] This invention provides a fisheye image correction method based on adaptive threshold thresholds. By obtaining the pixel grayscale ratio inside and outside the fisheye image, and setting dual thresholds to obtain the edge contour points of the fisheye image, the confidence of pixels being fisheye edges is improved. Then, based on the set of edge contour points of the fisheye image, the radius of the fisheye image and the size information of the cropped fisheye image are obtained and the fisheye image is corrected, ensuring the accuracy of the extracted fisheye image. At the same time, it can also be applied to extract the effective fisheye region from various test images on different devices.

[0056] To implement the method provided in the embodiments of the present invention, the embodiments of the present invention provide a fisheye image correction system based on an adaptive threshold threshold for fisheye images, such as... Figure 2 As shown, the system includes: an acquisition unit 21 and a correction unit 22;

[0057] The acquisition unit 21 is used to acquire the three primary color information of each pixel in the fisheye image captured by the camera, and convert the three primary color information into grayscale information corresponding to each pixel.

[0058] The acquisition unit 21 is further configured to acquire, based on the grayscale information of each pixel, a threshold for the grayscale ratio of pixels within the fisheye image, a threshold for the grayscale of pixels outside the fisheye image, and a threshold for the grayscale ratio of pixels within the fisheye image.

[0059] The acquisition unit 21 is further configured to acquire a set of sampling points for the left and right circular contours of the fisheye image based on the threshold of the pixel grayscale ratio within the fisheye image, the threshold of the pixel grayscale outside the fisheye image and the pixel grayscale ratio within the fisheye image, and the grayscale information of each pixel.

[0060] The acquisition unit 21 is further configured to acquire the fish-eye image radius and the size information of the cropped fish-eye image according to the sample point set of the left and right circular contours of the fish-eye image.

[0061] The correction unit 22 is configured to correct the fish-eye image according to the fish-eye image radius and the size information of the cropped fish-eye image.

[0062] Further, the acquisition unit 21 is specifically configured to scan in a cross shape with the pixel point at the center of the image as the center to acquire the threshold value of the pixel gray scale ratio in each direction, and acquire the threshold value of the pixel gray scale ratio in the fish-eye image according to the threshold value of the pixel gray scale ratio in each direction.

[0063] Further, the acquisition unit 21 is specifically configured to scan in an X shape with the pixel point at the center of the image as the center to acquire the threshold value of the fish-eye image outer pixel gray scale and the fish-eye image inner pixel gray scale ratio in each direction, and acquire the threshold value of the fish-eye image outer pixel gray scale and the fish-eye image inner pixel gray scale ratio according to the threshold value of the fish-eye image outer pixel gray scale and the fish-eye image inner pixel gray scale ratio in each direction.

[0064] Further, the acquisition unit 21 is specifically configured to acquire the fish-eye image radius and the center of the circle according to the preset linear least square method and the sample point set of the left and right circular contours of the fish-eye image, and crop the fish-eye image according to the fish-eye image radius and the center of the circle.

[0065] Further, the correction unit 22 is specifically configured to acquire the corrected image height of the fish-eye image based on the adaptive threshold value threshold according to the corrected longitudinal pixel coordinate, the transverse pixel coordinate, the longitudinal pixel coordinate of the center of the image, the transverse pixel coordinate of the center of the image, the center of the cropped fish-eye image being the corrected center of the image, acquire the pixel coordinate of each corrected pixel point in the distortion image according to the corrected image height of the fish-eye image based on the adaptive threshold value threshold, and assign the image information three primary color values corresponding to the pixel coordinate in the distortion image to the image information three primary color values at the pixel coordinate in the corrected image.

[0066] The fish-eye image correction system based on the adaptive threshold value threshold of the fish-eye image provided by the application acquires the pixel gray scale ratio inside the fish-eye image and the pixel gray scale ratio inside and outside the fish-eye image, sets the double threshold value to acquire the edge contour point of the fish-eye image, improves the confidence of the pixel being the fish-eye edge, acquires the fish-eye image radius and the size information of the cropped fish-eye image based on the edge contour point set of the fish-eye image, and corrects the fish-eye image, thereby ensuring the accuracy of the extracted fish-eye image, and the fish-eye effective area can also be extracted in different devices and various measured images.

[0067] It should be understood that the particular order or hierarchy of steps in the processes disclosed is an example that can be re-arranged as desired. The particular order or hierarchy of steps in the processes disclosed should not be construed as limitations on the claimed subject matter. The accompanying method claims present elements of the various steps in a sample order, and are not meant to be limited to the specific order or hierarchy presented.

[0068] In the above detailed description, various features are grouped together in single embodiments for the purpose of streamlining the disclosure. This disclosure is not to be interpreted in determining the scope of the claimed subject matter such that the embodiments require a greater scope than the claims allow. Rather, the inventive subject matter is to be determined with the claims and their equivalents, where the claimed subject matter is broader than the disclosed embodiments.

[0069] The disclosed embodiments herein can be described with respect to the following example embodiments. These embodiments are described in order to provide general

[0070] The above description includes example embodiments of one or more embodiments. Of course, not all possible combinations of components or methods described above will be described as embodiments. A person of ordinary skill in the art will recognize that further combinations and permutations of the described embodiments are possible. Therefore, the embodiments described herein are intended to cover all such modifications and permutations of the disclosure that are within the scope of the appended claims. Additionally, the description above is intended to cover all possible combinations of the following terms, as well as other terms, with respect to the description or claims. The term "comprising" is intended to be similar to the term "including" as the term "including" is interpreted in the context of the claims. Additionally, the term "or" as used in the specification and claims is intended to mean "nonexclusive or".

[0071] Those of skill would further appreciate that the various illustrative logical blocks, modules, and steps described in connection with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or combinations of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Skilled artisans can implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present embodiments.

[0072] The various illustrative logical blocks and modules described in connection with the embodiments disclosed herein can be implemented or performed by a general purpose processor, a digital signal processor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general purpose processor can be a microprocessor, but in the alternative, the general purpose processor can be any conventional processor, controller, microcontroller, or state machine. A processor can also be implemented as a combination of computing devices, e.g., a combination of a digital signal processor and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a digital signal processor core, or any other such configuration.

[0073] The steps of a method or algorithm described in connection with the embodiments disclosed herein can be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. A software module can reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is tangible. A storage medium can be coupled to the processor such that the processor can read information from, and write information to, the storage medium. In the alternative, the storage medium can be integral to the processor. The processor and the storage medium can reside in an ASIC. The ASIC can reside in a user terminal. In the alternative, the processor and the storage medium can reside as discrete components in a user terminal.

[0074] In one or more exemplary designs, the functions described can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one place to another. Storage media can be any available media that can be accessed by a computer. By way of example, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a general-purpose or special-purpose computer, or a general-purpose or special-purpose processor. Also, any connection is properly termed a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or other wire-based, fiber- optic based, or wireless based communications, then the coaxial cable, fiber optic cable, twisted pair, DSL, or other wire-based, fiber-optic based, or wireless based communications are included in the definition of medium. Disk and disc, as used herein, includes compact disc (CD), laser disc, optical disc, DVD, floppy disk, and Blu-ray® disc where disks usually reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0075] The specific implementation described above is illustrative for purposes of teaching the present application. The application should not be limited, however, to these specific implementation. Any modifications, equivalents, or improvements, as long as they are within the spirit and scope of the present application are included in the scope of the present application.

Claims

1. A fisheye image correction method based on adaptive threshold for fisheye images, characterized in that, The method includes: Obtain the three primary color information of each pixel in the fisheye image captured by the camera, and convert the three primary color information into grayscale information corresponding to each pixel; Based on the grayscale information of each pixel, thresholds for the pixel grayscale ratio within the fisheye image, the pixel grayscale outside the fisheye image, and the pixel grayscale ratio within the fisheye image are obtained; the step of obtaining the threshold for the pixel grayscale ratio within the fisheye image based on the grayscale information of each pixel includes: Scan in a cross shape with the pixel at the center of the image as the center, and obtain the threshold of the pixel grayscale ratio in each direction; The threshold for pixel grayscale ratio within the fisheye image is obtained based on the threshold for pixel grayscale ratio in each direction; The step of obtaining the threshold for the ratio of pixel grayscale outside the fisheye image to pixel grayscale inside the fisheye image based on the grayscale information of each pixel includes: Scan in an X-shape with the pixel at the center of the image as the center to obtain the threshold of the gray level ratio of pixels outside the fisheye image and pixels inside the fisheye image in each direction; The thresholds for the grayscale values ​​of pixels outside and inside the fisheye image are obtained based on the thresholds for the grayscale values ​​of pixels outside and inside the fisheye image from various directions. Based on the threshold of the pixel grayscale ratio within the fisheye image, the threshold of the pixel grayscale outside the fisheye image and the pixel grayscale ratio within the fisheye image, and the grayscale information of each pixel, obtain the sampling point set of the circular outline on the left and right sides of the fisheye image; Based on the sampling point set of the left and right circular contours of the fisheye image, obtain the radius of the fisheye image and the size information of the cropped fisheye image; The fisheye image is corrected based on the radius of the fisheye image and the size information of the cropped fisheye image.

2. The fisheye image correction method based on adaptive threshold of fisheye image according to claim 1, characterized in that, The step of obtaining the radius of the fisheye image and the size information of the cropped fisheye image based on the sampling point set of the left and right circular contours of the fisheye image includes: Based on the preset linear least squares method and the sampling point set of the left and right circular contours of the fisheye image, the radius and center of the fisheye image are obtained; The fisheye image is cropped based on its radius and center.

3. The fisheye image correction method based on adaptive threshold of fisheye image according to claim 1, characterized in that, The step of correcting the fisheye image based on the fisheye image radius and the cropped fisheye image size information includes: Based on the corrected vertical pixel coordinates, horizontal pixel coordinates, vertical pixel coordinates of the image center, and horizontal pixel coordinates of the image center, the corrected image height of the fisheye image based on the adaptive threshold threshold of the fisheye image is obtained, and the center of the cropped fisheye image is the corrected image center. Based on the image height of the fisheye image after correction using the adaptive threshold threshold for fisheye images, obtain the pixel coordinates of each corrected pixel in the distorted image. The image information primary color values ​​corresponding to the pixel coordinates in the distorted image are assigned to the image information primary color values ​​at the pixel coordinates in the corrected image.

4. A fisheye image correction system based on adaptive threshold for fisheye images, characterized in that, The system includes: The acquisition unit is used to acquire the three primary color information of each pixel in the fisheye image captured by the camera, and convert the three primary color information into grayscale information corresponding to each pixel. The acquisition unit is further configured to acquire, based on the grayscale information of each pixel, a threshold for the grayscale ratio of pixels within the fisheye image, a threshold for the grayscale of pixels outside the fisheye image, and a threshold for the grayscale ratio of pixels within the fisheye image. The acquisition unit is further configured to acquire a set of sampling points for the left and right circular contours of the fisheye image based on the threshold of the pixel grayscale ratio within the fisheye image, the threshold of the pixel grayscale outside the fisheye image and the pixel grayscale ratio within the fisheye image, and the grayscale information of each pixel. The acquisition unit is specifically used to scan in a cross shape with the pixel at the center of the image as the center, and acquire the threshold of the pixel grayscale ratio in each direction; and acquire the threshold of the pixel grayscale ratio in the fisheye image based on the threshold of the pixel grayscale ratio in each direction. The acquisition unit is further configured to scan in an X-shape with the pixel at the center of the image as the center to acquire the threshold of the grayscale ratio of the pixels outside the fisheye image and the pixels inside the fisheye image in each direction; and to acquire the threshold of the grayscale ratio of the pixels outside the fisheye image and the pixels inside the fisheye image based on the threshold of the grayscale ratio of the pixels outside the fisheye image and the pixels inside the fisheye image in each direction. The acquisition unit is further configured to acquire the radius of the fisheye image and the size information of the cropped fisheye image based on the set of sampling points of the circular outlines on the left and right sides of the fisheye image; The correction unit is used to correct the fisheye image based on the radius of the fisheye image and the size information of the cropped fisheye image.

5. A fisheye image correction system based on adaptive threshold for fisheye images according to claim 4, characterized in that, The acquisition unit is further configured to acquire the radius and center of the fisheye image based on a preset linear least squares method and the sampling point set of the left and right circular contours of the fisheye image; and to crop the fisheye image based on the radius and center of the fisheye image.

6. The fisheye image correction system based on adaptive threshold of fisheye images according to claim 4, characterized in that, The correction unit is specifically used to obtain the corrected image height of the fisheye image based on the fisheye image adaptive threshold, according to the corrected vertical pixel coordinates, horizontal pixel coordinates, vertical pixel coordinates of the image center, and horizontal pixel coordinates of the image center, wherein the center of the cropped fisheye image is the corrected image center; to obtain the pixel coordinates of each corrected pixel in the distorted image based on the corrected image height of the fisheye image based on the fisheye image adaptive threshold; and to assign the image information primary color values ​​corresponding to the pixel coordinates in the distorted image to the image information primary color values ​​at the pixel coordinates in the corrected image.

Images