Fish-eye image processing method and device, electronic equipment and storage medium

Abstract

The application discloses a fisheye image processing method, a fisheye image processing device, an electronic device and a nonvolatile computer readable storage medium. The method comprises the following steps: performing color space conversion on a fisheye image of a fisheye camera to generate a grayscale image; calculating the derivative of each pixel in the grayscale image in the horizontal direction and the vertical direction respectively to generate a first gradient image and a second gradient image respectively; calculating the azimuth angle corresponding to each first pixel in the first gradient image according to each first pixel in the first gradient image and a second pixel in the second gradient image with the same position as the first pixel, and performing normalization processing on the azimuth angle to generate an azimuth angle image; and determining the effective imaging area of the fisheye camera according to the azimuth angle image. Since the grayscale variation trend of the original image collected under different shooting scenes is unchanged, the azimuth angle image determined based on the gradient image can adapt to different shooting scenes, thereby ensuring the accuracy of the effective imaging area determined based on the azimuth angle image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fisheye image processing, and more particularly, to a fisheye image processing method, a fisheye image processing device, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] When determining the effective imaging area of a fisheye image, due to the difference between different shooting scenes (light conditions and camera exposure parameters), the fisheye image has great differences. However, the current fisheye image preprocessing algorithm is difficult to adapt to different scenes and different exposure parameters, resulting in unstable accuracy of subsequent determination of the effective imaging area based on the preprocessed image. SUMMARY

[0003] The present application provides a fisheye image processing method, a fisheye image processing device, an electronic device, and a computer readable storage medium.

[0004] The fisheye image processing method of the present application includes obtaining a fisheye image captured by a fisheye camera; performing color space transformation on the fisheye image to generate a grayscale image; calculating the derivative of each pixel in the grayscale image in the horizontal direction to generate a first gradient image, and calculating the derivative of each pixel in the grayscale image in the vertical direction to generate a second gradient image; calculating the azimuth angle corresponding to each first pixel in the first gradient image according to each first pixel in the first gradient image and a second pixel in the second gradient image having the same position as the first pixel, and performing normalization processing on the azimuth angle to generate an azimuth angle image according to the normalized azimuth angle; and determining the effective imaging area of the fisheye camera according to the azimuth angle image.

[0005] In some embodiments, the determining the effective imaging area of the fisheye camera according to the azimuth angle image includes determining a plurality of angle offsets in a preset interval based on a preset step size; adding the angle offset to each azimuth angle in the azimuth angle image to generate an offset image corresponding to each angle offset; performing binaryzation processing on each offset image to generate a binaryzation image corresponding to each offset image; and determining the effective imaging area according to a plurality of binaryzation images.

[0006] In some embodiments, the adding the angle offset to each of the azimuth angles in the azimuth angle image respectively to generate an offset image corresponding to each of the angle offsets comprises: adding the angle offset to each of the azimuth angles in the azimuth angle image respectively; and in case that a value after the azimuth angle is added with the angle offset is greater than or equal to a preset threshold, reducing the value by the preset threshold; and generating the offset image corresponding to each of the angle offsets according to the azimuth angle image after the azimuth angle is added with the angle offset.

[0007] In some embodiments, before the effective imaging area is determined according to the plurality of binary images, the fisheye image processing method comprises: performing median filtering on each of the binary images respectively; and the determining the effective imaging area according to the plurality of binary images comprises: determining the effective imaging area according to the plurality of binary images after the median filtering.

[0008] In some embodiments, the determining the effective imaging area according to the plurality of binary images comprises: accumulating pixel values of pixels at the same position in the plurality of binary images to generate a statistical image; performing normalization processing on each pixel in the statistical image; performing binary processing on the statistical image after the normalization processing; performing median filtering on the statistical image after the binary processing; and determining the effective imaging area according to the statistical image after the median filtering.

[0009] In some embodiments, the accumulating pixel values of pixels at the same position in the plurality of binary images to generate a statistical image comprises: accumulating pixel values of pixels at the same position in the plurality of binary images respectively to generate a plurality of accumulated pixel values; determining an accumulated number of pixels of a preset pixel value corresponding to the accumulated pixel value according to a quotient of the accumulated pixel value and the preset pixel value; and generating the statistical image according to the accumulated number.

[0010] In some embodiments, the determining the effective imaging area according to the statistical image after the median filtering comprises: performing line-by-line scanning on the statistical image after the median filtering to obtain boundary pixels with a preset pixel value in each row; and determining the effective imaging area according to image coordinates of all the boundary pixels.

[0011] The fisheye image processing apparatus of the embodiments of the present application comprises an acquisition module, a color transform module, a first generation module, a second generation module and a determination module. The acquisition module is configured to acquire a fisheye image captured by a fisheye camera; the color transform module is configured to perform color space transformation on the fisheye image to generate a grayscale image; the first generation module is configured to calculate a derivative of each pixel in the grayscale image in a horizontal direction to generate a first gradient image, and calculate a derivative of each pixel in the grayscale image in a vertical direction to generate a second gradient image; the second generation module is configured to calculate a corresponding azimuth angle of each first pixel in the first gradient image according to each first pixel and a second pixel in the second gradient image having the same position as the first pixel, and perform normalization processing on the azimuth angle to generate an azimuth angle image according to the normalized azimuth angle; and the determination module is configured to determine an effective imaging area of the fisheye camera according to the azimuth angle image.

[0012] The electronic device of the embodiments of the present application comprises one or more processors, a memory and one or more programs, wherein the one or more programs are stored in the memory and executed by the one or more processors, and the programs comprise instructions for executing the fisheye image processing method of any of the above embodiments.

[0013] The computer readable storage medium of the embodiments of the present application comprises a computer program, which, when executed by a processor, causes the processor to execute the fisheye image processing method of any of the above embodiments.

[0014] The fisheye image processing method, fisheye image processing apparatus, electronic device and computer readable storage medium of the embodiments of the present application acquire a single-channel grayscale image of a fisheye image after color space conversion, then generate a first gradient image and a second gradient image based on the derivatives of the grayscale image in the horizontal direction and the vertical direction, respectively, to determine the corresponding azimuth angle of each first pixel according to any first pixel in the gradient image and a second pixel in the second gradient image having the same position as the first pixel, generate an azimuth angle image according to the corresponding azimuth angle of each first pixel, and finally determine the effective imaging area based on the azimuth angle image. For the fisheye camera, the grayscale change trend between different image regions of the fisheye image captured in different shooting scenes is unchanged, such as the grayscale change trend from the four corners to the center of the fisheye camera. Therefore, the azimuth angle image determined based on the first gradient image and the second gradient image can adapt to different shooting scenes, thereby ensuring the accuracy of the effective imaging area determined based on the azimuth angle image.

[0015] Additional aspects and advantages of the embodiments of the present application will be set forth in part in the description that follows, and in part will be obvious from the description, or can be learned by practice of the embodiments of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0016] The above and / or additional aspects and advantages of the present application will become apparent and be readily understood from the following description, taken in conjunction with the accompanying drawings, in which:

[0017] Figure 1 is a flowchart of a fisheye image processing method according to some embodiments of the present application;

[0018] Figure 2 is a scene diagram of a fisheye image processing method according to some embodiments of the present application;

[0019] Figure 3 is an azimuth image diagram of a fisheye image processing method according to some embodiments of the present application;

[0020] Figure 4 is a flowchart of a fisheye image processing method according to some embodiments of the present application;

[0021] Figure 5 is a binary image diagram of a fisheye image processing method according to some embodiments of the present application, with an angle offset of 0;

[0022] Figure 6 is a binary image diagram of a fisheye image processing method according to some embodiments of the present application, with an angle offset of 60;

[0023] Figure 7 is a binary image diagram of a fisheye image processing method according to some embodiments of the present application, with an angle offset of 120;

[0024] Figure 8 is a binary image diagram of a fisheye image processing method according to some embodiments of the present application, with an angle offset of 180;

[0025] Figure 9 is a flowchart of a fisheye image processing method according to some embodiments of the present application;

[0026] Figure 10 is a statistical image diagram of a fisheye image processing method according to some embodiments of the present application;

[0027] Figure 11 is a flowchart of a fisheye image processing method according to some embodiments of the present application;

[0028] Figure 12 is a module diagram of a fisheye image processing apparatus according to some embodiments of the present application;

[0029] Figure 13 is a plan view of an electronic device according to some embodiments of the present application;

[0030] Figure 14 is a connection state view of a non-volatile computer readable storage medium and a processor according to some embodiments of the present application. DETAILED DESCRIPTION

[0031] Embodiments of the present application are described in detail below with reference to the attached drawing figures, wherein the same or like reference numerals are used throughout the drawing figures to refer to the same or like elements or elements having the same or similar functionality. The embodiments described below are exemplary and are merely intended to explain the present application, and are not to be understood as limiting the present application.

[0032] Fish-eye cameras have been widely used due to their large wide-angle advantage, but in many image analysis scenarios such as automatic driving, intelligent monitoring, etc., there are strict requirements for the distortion of the input image, and the original photo must be restored through a specific anti-distortion correction algorithm, and the effective imaging area will be accurately calibrated before leaving the factory. However, in application scenarios such as automatic driving and three-dimensional reconstruction, there are a large number of post-factory calibration scenarios: such as vehicle collision accidents resulting in camera damage and replacement, unmanned delivery vehicles after long-distance transportation, jostling and displacement, etc. Without a standard test environment after leaving the factory, if the effective imaging area is to be determined, it can only be based on the images collected by the fish-eye camera in the actual use scenario. Due to the change of the shooting scene, the fish-eye images collected have large differences, and the current pre-processing algorithm for fish-eye images is difficult to adapt to different scenes and different exposure parameters, resulting in unstable accuracy of determining the effective imaging area based on the pre-processed images.

[0033] Referring to Figure 1 , the embodiments of the present application provide a fish-eye image processing method, the fish-eye image processing method comprising:

[0034] Step 011: acquiring a fish-eye image photographed by a fish-eye camera;

[0035] Wherein, the fish-eye camera can be a fish-eye camera, it can be understood that the fish-eye camera can also be other types of cameras, such as wide-angle cameras, long-focus cameras, etc., and the present application takes the fish-eye camera as an example for description. As shown in Figure 2 , the effective imaging area of the fish-eye camera is generally circular, and the final output fish-eye image is generally rectangular, and the fish-eye image includes not only the effective imaging area, but also the on-screen display (OSD) content (such as Figure 2"20XX-XX-XX") is related to the actual application scene of the on-screen display content and the fisheye camera.

[0036] Since the four corners of the fisheye image are regions without scene information, the four corners of the fisheye image are always black, and the image of the effective imaging region contains the brightness information of the actual scene, therefore, no matter which scene is photographed, the gray level change trend from the four corners to the effective imaging region is unchanged, that is, the gray level from the four corners to the effective imaging region is increased.

[0037] The fisheye image is an image captured by the fisheye camera without processing, and the fisheye image can be an image captured by any shooting scene or any shooting parameter (such as exposure parameter), which is not limited by the present application.

[0038] In order to realize the calibration of the effective normal region of the fisheye camera, the fisheye image photographed by the fisheye camera needs to be obtained first to preprocess the fisheye image.

[0039] Step 012: performing color space transformation on the fisheye image to generate a gray scale image;

[0040] The fisheye image is generally a visible light image (such as a red-green-blue (RGB) three-channel image), and the color space transformation can be color space transformation of the fisheye image through the HSV (Hue, Saturation, Value) color model, so as to calculate the hue (H), saturation (S) and brightness (V) of each pixel according to the three-channel pixel value (i.e. R channel pixel value, G channel pixel value and B channel pixel value) of each pixel, and then generate a gray scale image according to the brightness of each pixel.

[0041] Step 013: calculating the derivative of each pixel in the gray scale image in the horizontal direction to generate a first gradient image, and calculating the derivative of each pixel in the gray scale image in the vertical direction to generate a second gradient image;

[0042] Specifically, after obtaining the gray scale image, according to the foregoing discussion, the gray level change trend from the four corners of the fisheye camera to the effective imaging region is unchanged, that is, the gradient change of the gray level is unchanged, therefore, the gray level in different directions of the gray scale image can be calculated to obtain a plurality of gradient images. For example, the gradient image can be obtained by a canny operator.

[0043] Generally, according to the arrangement of the pixels, the gradient in the horizontal direction and the gradient in the vertical direction of the fisheye image are calculated respectively to obtain a first gradient image and a second gradient image with the gradient direction being perpendicular.

[0044] Optionally, when the first gradient image is determined, the derivative of each pixel of the gray image in the horizontal direction is calculated respectively, so as to obtain the gradient of the gray image in the horizontal direction, thereby generating the first gradient image; when the second gradient image is determined, the derivative of each pixel of the gray image in the vertical direction is calculated respectively, so as to obtain the gradient of the gray image in the vertical direction, thereby generating the second gradient image.

[0045] Step 014: calculating the azimuth angle corresponding to each first pixel according to each first pixel of the first gradient image and the second pixel with the same position as the first pixel in the second gradient image respectively, and performing normalization processing on the azimuth angle, so as to generate the azimuth angle image according to the normalized azimuth angle;

[0046] Specifically, after the first gradient image and the second gradient image are determined, the azimuth angle image can be generated according to the first gradient image and the second gradient image.

[0047] The first gradient image includes a plurality of first pixels, the second gradient image includes a plurality of second pixels, and the azimuth angle image includes a plurality of third pixels, the pixel value of the third pixel being the azimuth angle, the first pixel, the second pixel and the third pixel corresponding one by one, and the positions of the one-to-one corresponding first pixel, the second pixel and the third pixel being the same.

[0048] The position of the third pixel in the azimuth angle image can be determined according to the position of the first pixel and the second pixel. For example, the pixel value of the first pixel and the pixel value of the second pixel are used to calculate the azimuth angle corresponding to the first pixel according to the arctangent function, that is, the pixel value of the third pixel corresponding to each first pixel, so as to generate the azimuth angle image according to the pixel value of each third pixel. The azimuth angle is located in the interval [0, 360 degrees], and the image processing is generally based on 8-bit data structure, so it is necessary to perform normalization processing on the azimuth angle image, so as to normalize all the azimuth angles to a preset azimuth angle interval, such as [0, 255]. It can be understood that the preset azimuth angle interval can also be other ranges, and the preset azimuth angle interval can be determined according to the number of bits of image processing, which is not limited here. The azimuth angle is used to represent the gray gradient change direction of each third pixel.

[0049] In this way, the azimuth angle image is generated by the first gradient image and the second gradient image to represent the change trend of the pixel, so as to facilitate subsequent determination of the effective imaging area of the fisheye camera according to the azimuth angle image.

[0050] Please refer to Figure 3 The azimuth angle image generated according to the first gradient image and the second gradient image, wherein the pixel value of different third pixels represents the azimuth angle of the pixel.

[0051] Step 015: determining the effective imaging area of the fisheye camera according to the azimuth angle image.

[0052] Specifically, after the azimuth angle image is determined, the azimuth angle image can be detected to determine the effective imaging area. For example, edge detection can be performed on the azimuth angle image to determine the effective imaging area. Figure 3 In this way, the outline of the circular effective imaging area can be clearly seen, and thus the effective imaging area is obtained. Since the gray level change trend from the four corners to the center of the fisheye camera is unchanged in different shooting scenes, the azimuth angle image determined based on the gradient image can adapt to different shooting scenes, thereby ensuring the accuracy of the effective imaging area determined based on the azimuth angle image.

[0053] The fisheye image processing method of the embodiments of the present application obtains a single-channel gray level image of the fisheye image after color space conversion, and then generates a first gradient image and a second gradient image based on the derivatives of the gray level image in the horizontal direction and the vertical direction, respectively, so as to determine the azimuth angle corresponding to each first pixel according to any first pixel in the gradient image and a second pixel in the second gradient image having the same position as the first pixel, generate an azimuth angle image according to the azimuth angle corresponding to each first pixel, and finally determine the effective imaging area based on the azimuth angle image. For the fisheye camera, since the gray level change trend between different image regions of the fisheye image shot in different shooting scenes is unchanged, such as the gray level change trend from the four corners to the center of the fisheye camera, the azimuth angle image determined based on the first gradient image and the second gradient image can adapt to different shooting scenes, thereby ensuring the accuracy of the effective imaging area determined based on the azimuth angle image.

[0054] Please refer to Figure 4 In some embodiments, step 015: determining the effective imaging area of the fisheye camera according to the azimuth angle image, comprises:

[0055] Step 0151: determining a plurality of angle offsets in a preset interval based on a preset step size;

[0056] Step 0152: respectively adding an angle offset to each azimuth angle in the azimuth angle image to generate an offset image corresponding to each angle offset;

[0057] Step 0153: respectively performing binaryzation processing on each offset image to generate a binaryzation image corresponding to each offset image;

[0058] Step 0154: determining the effective imaging area according to the plurality of binaryzation images.

[0059] Specifically, after the azimuth angle image is determined, the azimuth angle image needs to be offset based on different angle offsets, so as to highlight the third pixels located near the edge of the effective imaging area under different angle offsets.

[0060] Therefore, a plurality of angle offsets can be determined within a preset interval (for example, the preset interval is [0, 360]), such as 0, 30, 50, 80, 110, etc. to make the plurality of angle offsets be distributed in the entire preset interval as much as possible. Further, the plurality of angle offsets can also be determined within the preset interval based on a preset step, such as the preset interval is [0, 360], and the preset step is 10, 30, 50, 60, etc. Then, 13 angle offsets can be obtained, such as 0, 30, 60, 90, …, 360, so as to cover the entire preset interval. At this time, the plurality of angle offsets are arranged in ascending order, and the difference between any two adjacent angle offsets is the same, and is the preset step, so that the plurality of angle offsets are uniformly distributed in the entire preset interval, which is beneficial to improve the accuracy of the offset image processed based on the angle offset subsequently.

[0061] Then, the azimuth angle image is processed according to each angle offset respectively, so as to generate an offset image corresponding to each angle offset. For example, the offset image can be determined according to the following formula: I2 (x，y) = I1 (x，y) + α, where I2 is the offset image, I1 is the azimuth angle image, (x, y) is the image coordinate, (x, y) is located in the image coordinate range of the azimuth angle image, I2 (x，y) represents the fourth pixel with image coordinate (x, y), I1 (x，y) represents the third pixel with image coordinate (x, y), and α is the angle offset. That is to say, the pixel value of each third pixel in the azimuth angle image can be added by the angle offset, so as to obtain the pixel value of the fourth pixel corresponding to each third pixel, and the offset image corresponding to the angle offset can be generated according to the pixel value of the plurality of fourth pixels. In this way, the offset image corresponding to each angle offset can be obtained, that is, a total of 13 offset images.

[0062] In addition, the pixel value of the fourth pixel obtained by adding the angle offset to the pixel value of the third pixel can be greater than or equal to a preset threshold (such as 256), which does not meet the data requirements of image processing, and thus needs to be processed. For the fourth pixel with a pixel value greater than 255, the pixel value of the fourth pixel is replaced by the difference between the pixel value of the fourth pixel and the preset threshold, so as to ensure that the pixel value of all fourth pixels is less than or equal to 255. Therefore, for the fourth pixel with a pixel value greater than 255, the pixel value of the fourth pixel is subtracted by the preset threshold (such as 256). It can be understood that for an angle greater than 360 degrees, the angle obtained by subtracting 360 degrees is actually the same, such as 30 degrees and 390 degrees. Therefore, after the pixel value of the fourth pixel with a pixel value greater than 255 is subtracted by 256, the corresponding azimuth angle is actually unchanged, which does not affect the accuracy of the azimuth angle.

[0063] In order to facilitate data processing, improve data processing efficiency, and make the features of the effective imaging area more obvious, each offset image can be subjected to binarization processing to obtain a plurality of binarization images. In the case that the pixel value of the fourth pixel of the offset image is in the interval [0, 255], the threshold can be 200, so that the pixels with a pixel value greater than 200 are all assigned a value of 255, and the pixels with a pixel value less than or equal to 200 are all assigned a value of 0, thereby forming a black-and-white binarization image. After the binarization of the 13 offset images, 13 binarization images can be obtained.

[0064] Please refer to Figures 5 to 8 , which are binarization images with angle offsets of 0, 60, 120, and 180 degrees. For brevity, only some of the binarization images with angle offsets are displayed here. It can be understood that since the offset images are subjected to angle offset from the azimuth angle image, the white areas after binarization are different. For example, for the azimuth angle images corresponding to different shooting scenes, the edge part of the effective imaging area at the lower left corner is an azimuth angle close to 360 degrees. Therefore, after binarization of the offset image with an angle offset of 0 degrees, only the edge part of the effective imaging area at the lower left corner in the offset image presents white, forming a white arc. Similarly, after angle offset, the images of the edge parts of the effective imaging area corresponding to different angle offsets can be obtained, so that the images of the edge parts of all effective imaging areas in the preset interval (i.e. 0 to 360 degrees) can be obtained, and the effective imaging area can be determined according to the plurality of binarization images corresponding to the plurality of offset images.

[0065] Optionally, each binarization image can be subjected to median filtering to obtain a filtered binarization image, thereby reducing the influence of discrete noise pixels on the subsequent determination of the effective imaging area. Then, the effective imaging area can be accurately determined according to the plurality of median filtered binarization images.

[0066] Referring to Figure 9 In some embodiments, the step 0154 of determining the effective imaging area according to the plurality of binarized images comprises:

[0067] The step 01541 of accumulating pixel values of pixels at the same position in the plurality of binarized images to generate a statistical image;

[0068] The step 01542 of normalizing each pixel in the statistical image;

[0069] The step 01543 of binarizing the normalized statistical image;

[0070] The step 01544 of median filtering the binarized statistical image; and

[0071] The step 01545 of determining the effective imaging area according to the median filtered statistical image.

[0072] Specifically, referring to Figure 10 After obtaining the plurality of binarized images corresponding to different angle offsets, it can be seen that the corresponding white pixel region in each binarized image is part of the effective imaging area, and thus, the statistical image can be generated according to the pixel values of the pixels at the same position in the plurality of binarized images (or the plurality of median filtered binarized images), so as to form the complete effective imaging area.

[0073] For example, the pixel values of the pixels at the same position in the plurality of binarized images are accumulated respectively to determine a plurality of accumulated pixel values, and then the statistical image is generated according to the accumulated pixel values. It can be understood that, in order to facilitate subsequent image processing, the statistical image can be normalized to make the pixel values of the pixels in the statistical image in a preset interval (such as [0, 255]). Alternatively, according to the quotient of the accumulated pixel value and the pixel value of the white pixel in the binarized image (i.e., the preset pixel value), the number of white pixels whose pixel values are accumulated to form each accumulated pixel value is determined, i.e., the number of accumulated white pixels corresponding to each accumulated pixel value is determined, and then the statistical image is generated according to the number of accumulated white pixels corresponding to each accumulated pixel value. At this time, the pixel value of each pixel in the statistical image is the number of white pixels at the same position in the plurality of binarized images. In order to facilitate subsequent image processing, the statistical image can also be normalized to make the pixel values of the pixels in the statistical image in a preset interval (such as [0, 255]).

[0074] After the statistical image is normalized, a threshold value can be used to binarize the normalized statistical image again, so as to obtain a binarized image corresponding to the statistical image, so that the effective imaging area is presented in the binarized image corresponding to the statistical image, as shown in Figure 10 As can be clearly seen, the outline of the effective imaging area in the binarized image corresponding to the statistical image is already relatively obvious, and therefore, the effective imaging area can be quickly and accurately determined based on the statistical image subsequently.

[0075] Optionally, the statistical image or the binarized image corresponding to the statistical image can also be subjected to median filtering to eliminate discrete noise pixels in the image and improve the accuracy of subsequent determination of the effective imaging area.

[0076] Referring to Figure 11 In some embodiments, the step 01545 of determining the effective imaging area according to the statistical image subjected to median filtering comprises:

[0077] The step 01546 of performing line-by-line scanning on the statistical image subjected to median filtering to obtain boundary pixels with a preset pixel value in each row.

[0078] The step 01547 of determining the effective imaging area according to the coordinates of all the boundary pixels.

[0079] Specifically, after the statistical image subjected to binarization processing (or the statistical image subjected to median filtering) is determined, the effective imaging area can be determined according to the statistical image subjected to binarization processing (or the statistical image subjected to median filtering).

[0080] For example, the statistical image subjected to median filtering is subjected to line-by-line scanning, so as to obtain boundary pixels with a preset pixel value (such as 255) in each row; for example, the boundary pixels are the first and last pixels with the preset pixel value (such as 255) in each row, i.e., the first white pixel and the last white pixel in each row of pixels, so as to obtain all the boundary pixels in the statistical image subjected to binarization processing (or the statistical image subjected to median filtering), and the image coordinates of all the boundary pixels can be used to determine the effective imaging area.

[0081] For example, a nonlinear optimization error equation can be constructed according to the image coordinates of all the boundary pixels and preset parameters to be solved, the parameters to be solved including the image coordinates of the center of the effective imaging area and the radius. The image coordinates of the center of the effective imaging area and the radius are obtained by solving the optimal solution of the nonlinear optimization error equation, so as to determine the effective imaging area.

[0082] Referring to Figure 12For better implementing the fisheye image processing method of the embodiments of the present application, the embodiments of the present application further provide a fisheye image processing device 10. The fisheye image processing device 10 comprises an acquisition module 11, a color transform module 12, a first generation module 13, a second generation module 14 and a determination module 15. The acquisition module 11 is configured to acquire a fisheye image captured by a fisheye camera; the color transform module 12 is configured to perform color space transformation on the fisheye image to generate a grayscale image; the first generation module 13 is configured to calculate a derivative of each pixel in the grayscale image in a horizontal direction to generate a first gradient image, and calculate a derivative of each pixel in the grayscale image in a vertical direction to generate a second gradient image; the second generation module 14 is configured to calculate a corresponding azimuth angle of each first pixel in the first gradient image according to each first pixel and a second pixel in the second gradient image having a same position as the first pixel, and perform normalization processing on the azimuth angle to generate an azimuth angle image according to the normalized azimuth angle; and the determination module 15 is configured to determine an effective imaging area of the fisheye camera according to the azimuth angle image.

[0083] The determination module 15 is specifically further configured to determine a plurality of angle offsets in a preset interval based on a preset step size; increase each azimuth angle in the azimuth angle image by an angle offset to generate an offset image corresponding to each angle offset; perform binarization processing on each offset image to generate a binarization image corresponding to each offset image; and determine the effective imaging area according to the plurality of binarization images.

[0084] The determination module 15 is specifically further configured to increase each azimuth angle in the azimuth angle image by an angle offset; and in a case where a value of the azimuth angle after the increase of the angle offset is greater than or equal to a preset threshold value, reduce the value by the preset threshold value.

[0085] The fisheye image processing device 10 further comprises a filtering module 16. The filtering module 16 performs median filtering on each binarization image. The determination module 15 is specifically further configured to determine the effective imaging area according to a plurality of median filtered binarization images.

[0086] The determination module 15 is specifically configured to accumulate pixel values of pixels having a same position in the plurality of binarization images to generate a statistical image; perform normalization processing on each pixel in the statistical image; perform binarization processing on the normalized statistical image; perform median filtering on the binarization processed statistical image; and determine the effective imaging area according to the median filtered statistical image.

[0087] The determination module 15 is specifically further configured to scan each row of the median filtered statistical image row by row to find boundary pixels having a preset pixel value; and determine the effective imaging area according to image coordinates of all the boundary pixels.

[0088] Please refer to Figure 13The electronic device 100 of the embodiments of the present application includes one or more processors 20, a memory 30, and one or more programs 40, wherein the one or more programs 40 are stored in the memory 30 and executed by the one or more processors 20, and the program 40 includes instructions for executing the fisheye image processing method of any of the above embodiments.

[0089] The electronic device 100 can be a fisheye camera or a background server device, a mobile phone, a tablet computer, a notebook computer, a smart watch, etc. in communication with the fisheye camera. As shown in Figure 13 The embodiments of the present application take the fisheye camera as an example for illustration, and it can be understood that the specific form of the electronic device 100 is not limited to the fisheye camera. The fisheye camera of the present application can perform image processing based on the fisheye image after obtaining the fisheye image to obtain the azimuth angle image, thereby facilitating the correction of the fisheye camera at any time during use.

[0090] Please refer to Figure 14 The embodiments of the present application also provide a non-volatile computer readable storage medium 300 having a computer program 310 stored thereon, and the computer program 310, when executed by a processor 320, implements the steps of the fisheye image processing method of any of the above embodiments. For brevity, details are not repeated here.

[0091] In the description of the present specification, the description of the terms "certain embodiments", "in one example", "exemplarily", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0092] Any process or method descriptions in flow charts or otherwise described herein can be understood as representing code modules, segments, or portions of code that include one or more executable instructions for performing specific logic functions or steps in the process, and the various embodiments of the present application include additional implementations in which the functions described with reference to the figures are implemented with different sequences of steps, combinations of steps, and / or additional or different steps. The various embodiments of the present application can be implemented with other hardware or software modules that are functionally or logically equivalent to those described herein.

[0093] Although the embodiments of the present application have been shown and described above, it is understood that the above-described embodiments are exemplary and are not to be construed as limiting the present application, and that changes, modifications, substitutions and variations can be made by those skilled in the art without departing from the scope of the present application.

Claims

1. A fisheye image processing method, characterized by, The fisheye image processing method comprises: obtaining a fisheye image captured by a fisheye camera; performing color space transformation on the fisheye image to generate a grayscale image; calculating the derivative of each pixel in the grayscale image in the horizontal direction to generate a first gradient image, and calculating the derivative of each pixel in the grayscale image in the vertical direction to generate a second gradient image; calculating the azimuth angle corresponding to each first pixel in the first gradient image according to each first pixel in the first gradient image and a second pixel in the second gradient image having the same position as the first pixel, and performing normalization processing on the azimuth angle to generate an azimuth angle image according to the normalized azimuth angle; determining an effective imaging area of the fisheye camera according to the azimuth angle image, which comprises: determining a plurality of angle offsets in a preset interval based on a preset step size; increasing each azimuth angle in the azimuth angle image by the angle offset to generate an offset image corresponding to each angle offset; performing binarization processing on each offset image to generate a binarization image corresponding to each offset image; determining the effective imaging area according to a plurality of binarization images.

2. The fisheye image processing method of claim 1, wherein, The method comprises: increasing each azimuth angle in the azimuth angle image by the angle offset; and in the case where the value of the azimuth angle after the increase of the angle offset is greater than or equal to a preset threshold, reducing the value by the preset threshold; generating the offset image corresponding to the angle offset according to the azimuth angle image after the increase of the angle offset.

3. The fisheye image processing method of claim 1, wherein, Before determining the effective imaging area according to a plurality of binarization images, the fisheye image processing method comprises: performing median filtering on each binarization image; determining the effective imaging area according to a plurality of median filtered binarization images. The method comprises:

4. The fisheye image processing method of claim 1, wherein, accumulating the pixel values of the pixels having the same position in a plurality of binarization images to generate a statistical image; performing normalization processing on each pixel in the statistical image; performing binarization processing on the normalized statistical image; performing median filtering on the binarization processed statistical image; and determining the effective imaging area according to the median filtered statistical image. The method comprises:

5. The fish-eye image processing method according to claim 4, characterized in that, accumulating the pixel values of the pixels having the same position in a plurality of binarization images to generate a statistical image; accumulating the pixel values of the pixels having the same position in a plurality of binarization images to generate a plurality of accumulated pixel values; determining the accumulated number of pixels of a preset pixel value corresponding to the accumulated pixel value according to the quotient of the accumulated pixel value and the preset pixel value; generating the statistical image according to the accumulated number.

6. The fish-eye image processing method according to claim 4, characterized in that, The effective imaging area is determined according to the statistical image after the median filtering, and the method comprises the steps of: performing line-by-line scanning on the statistical image after the median filtering to obtain boundary pixels with a preset pixel value in each line; determining the effective imaging area according to image coordinates of all the boundary pixels.

7. A fisheye image processing apparatus characterized by comprising: The method comprises the steps of: an acquisition module, configured to acquire a fisheye image captured by a fisheye camera; a color transformation module, configured to perform color space transformation on the fisheye image to generate a grayscale image; a first generation module, configured to calculate a derivative of each pixel in the grayscale image in a horizontal direction to generate a first gradient image, and calculate a derivative of each pixel in the grayscale image in a vertical direction to generate a second gradient image; a second generation module, configured to calculate a corresponding azimuth angle of each first pixel in the first gradient image according to a second pixel in the second gradient image with the same position as the first pixel, and perform normalization processing on the azimuth angle to generate an azimuth angle image according to the normalized azimuth angle; a determination module, configured to determine an effective imaging area of the fisheye camera according to the azimuth angle image, and the method comprises the steps of: determining a plurality of angle offsets in a preset interval based on a preset step length; adding the angle offset to each azimuth angle in the azimuth angle image to generate an offset image corresponding to each angle offset; performing binarization processing on each offset image to generate a binarization image corresponding to each offset image; and determining the effective imaging area according to a plurality of binarization images.

8. An electronic device, comprising: The method comprises the steps of: one or more processors, a memory; and one or more programs, wherein one or more programs are stored in the memory and executed by one or more processors, and the programs comprise instructions for executing the fisheye image processing method of any one of claims 1-6. 9.A non-volatile computer readable storage medium containing a computer program, when the computer program is executed by a processor, the processor executes the fisheye image processing method of any one of claims 1-6.

Images