A pixel-by-pixel fisheye camera calibration method, system, medium and device

Abstract

The application discloses a kind of fish-eye camera calibration method, system, medium and equipment of pixel by pixel, make an internal color continuous uniform change half-shell, with component representation different color on half-shell, establish the mapping relationship of position coordinate and color (H, S);Fish-eye camera is placed at the ball center of half-shell and takes picture, measures the (H', S') of corresponding point color, obtains the mapping relationship of position coordinate and color (H, S) by corresponding half-shell, actual point coordinate pair on the imaging plane is measured to multiple points, obtains the look-up table of target object image point coordinate (x, y) and space elevation and azimuth, according to look-up table, determine the actual orientation of target object relative to fish-eye camera, according to the actual space orientation angle of pixel corresponding, reposition each pixel, complete calibration after correcting distorted image, the application can establish the imaging model of camera without correcting distorted image, simple and effective, greatly reduce data operation amount, improve real-time performance.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of fisheye cameras, and particularly relates to a pixel-by-pixel fisheye camera calibration method, system, medium and equipment. BACKGROUND

[0002] In order to obtain a wider field of view, the fisheye camera gives up the requirement of similarity between traditional camera imaging and the real object, which makes the image captured by the fisheye camera have a large distortion compared with the actual view of the human eye. Therefore, in order to determine the relative orientation of a target point in the image and the camera, it is usually necessary to establish an imaging model and fit the imaging function through feature point interpolation to remap the distorted image and repair the distortion.

[0003] At present, the commonly used methods are as follows:

[0004] 1) Chessboard calibration method - a chessboard is used as a calibration object, the corners of the chessboard are extracted, and the camera parameters are estimated to fit the imaging function and correct the distorted image; however, this method can only correct the central region of the image, and the edge region cannot be used due to severe stretching, so this method cannot fully utilize the wide view angle of the fisheye camera.

[0005] 2) Transverse expansion method - mainly uses the large field of view and the feature of shooting from a bird's eye view to perform deformation. However, this method will split the parts that are connected in the real world, resulting in unnatural visual effects.

[0006] 3) Latitude and longitude method - the image is corrected for distortion along the latitude direction or the longitude direction; this method does not cause pixel loss and has good correction effect on the correction direction (latitude direction or longitude direction), but has almost no correction effect on the other direction (longitude direction or latitude direction).

[0007] The above correction methods only consider the imaging correction of the camera in theory, the correction is globally averaged, the system error of the camera itself is not considered, and usually accompanied by large calculation amount, slow program running efficiency, and cannot guarantee real-time performance; the measurement error of each pixel after correction is not further explained.

[0008] Therefore, there is still a lack of a good model for determining the position and orientation of a target point relative to the camera and the corresponding error. SUMMARY

[0009] The technical problem to be solved by the present application is to provide a pixel-by-pixel fisheye camera calibration method, system, medium and equipment to quickly determine the position and orientation of a target point relative to the camera and the corresponding error without correcting the distorted image, and to calculate the distortion degree of each point of the camera.

[0010] The application adopts the following technical solutions:

[0011] A pixel-by-pixel fish-eye camera calibration method, comprising the following steps:

[0012] S1, a half-sphere cover with continuously and uniformly changed internal color is made, different colors on the half-sphere cover are represented by components (H, S), and a mapping relationship between position coordinates and colors (H, S) is established;

[0013] S2, the fish-eye camera is placed at the center of the half-sphere cover obtained in step S1, a picture is taken, (H', S') of the color of the corresponding point is measured, the mapping relationship between position coordinates and colors (H, S) established in step S1 is used, and actual point coordinates on the corresponding half-sphere cover are obtained; A plurality of points on the imaging plane are sampled and measured, a lookup table of target object image point coordinates (x, y) on the imaging plane and elevation and azimuth in space is obtained, the actual orientation of the target object relative to the fish-eye camera is determined according to the lookup table, each pixel is repositioned according to the actual space orientation angle corresponding to the pixel, and the calibration is completed after the distorted image is corrected.

[0014] Specifically, in step S1, the position of any point on the half-sphere cover is represented by components , the lightness component V of the color on the entire half-sphere cover is kept unchanged, and different colors on the half-sphere cover are represented by components (H, S).

[0015] Specifically, in step S1, the HSV model in the color space is used, three components (H, S, V) of hue, saturation and lightness are used to represent a color, two angles corresponding to the actual space orientation in the three components of the color are used, and a mapping relationship between position coordinates and colors (H, S) is established.

[0016] Specifically, in step S1, on the same latitude, the saturation component S is kept unchanged, and the hue component H is uniformly changed from 0° to 360°; on the same longitude, the hue component H is kept unchanged, and the saturation component S is uniformly changed from 0 to 100%, and a mapping relationship between position coordinates and colors (H, S) is established.

[0017] Specifically, in step S2, the hue and saturation (H0, S0) of any point (x0, y0) on the imaging plane are measured, and actual points on the corresponding half-sphere cover corresponding to the corresponding point are obtained.

[0018] Specifically, in step S2, the functional relationship between the target object image point coordinates (x, y) on the imaging plane and the elevation and azimuth in space is as follows: ​​​​​​

[0019] θ = f(x, y)

[0020]

[0021] wherein f(x, y) and g(x, y) are both discrete functions.

[0022] 7. The pixel-by-pixel fish-eye camera calibration method according to claim 1, wherein in step S2, a gradient operator is used to obtain the gradient of the hue H or the saturation S on the original image and the corrected image, the hue H representing the tangential distortion, the saturation S representing the radial distortion, the modulus of the gradient representing the distortion degree, and the direction of the gradient representing the distortion direction, and the distortion degree of the fish-eye camera at each position is obtained by comparing the modulus of the gradient and the direction of the gradient.

[0023] In a second aspect, an embodiment of the present application provides a pixel-by-pixel fish-eye camera calibration system, comprising:

[0024] a mapping module configured to make a half-sphere cover with internal color continuously and uniformly changing, to represent different colors on the half-sphere cover by components (H, S), and to establish a mapping relationship between the position coordinates and the color (H, S) ;

[0025] a calibration module configured to place the fish-eye camera at the center of the half-sphere cover obtained by the mapping module, to obtain the actual point on the half-sphere cover corresponding to the corresponding point by shooting a picture to sample and measure a plurality of points on the imaging plane to obtain a lookup table of the target object image point coordinates (x, y) on the imaging plane and the elevation angle and the azimuth angle in the space, to determine the actual orientation of the target object relative to the fish-eye camera according to the lookup table, and to complete the calibration by repositioning each pixel according to the actual space orientation angle corresponding to the pixel after correcting the distorted image.

[0026] In a third aspect, a computer device comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the pixel-by-pixel fish-eye camera calibration method when executing the computer program.

[0027] In a fourth aspect, an embodiment of the present application provides a computer readable storage medium comprising a computer program, and the computer program implements the steps of the pixel-by-pixel fish-eye camera calibration method when executed by a processor.

[0028] Compared with the prior art, the present application has at least the following beneficial effects:

[0029] The present application discloses a pixel-by-pixel fish-eye camera calibration method, which comprises the following steps: making a special half-sphere cover, letting every point on the cover have different colors, establishing a color-angle mapping, and then measuring the color of every pixel on the photographed picture to establish a pixel-angle mapping.

[0030] Further, in the position representation, the azimuth angle theta is in the range of 0°-360°, and the elevation angle phi is in the range of 0°-90°; while in the HSV color model, the hue H is measured by angle and is in the range of 0°-360°, and the saturation S is usually in the range of 0%-100%. The hue H and the azimuth angle theta have the same range of values, and thus the hue H corresponds to the azimuth angle theta without additional conversion operation. The value of the lightness V is related to the light intensity, and thus it is not used as the mapping variable of the spatial angle, and is kept unchanged, so as to avoid the interference of the environmental light intensity. Further, any color can be regarded as the red-yellow-blue three primary colors mixed in different proportions, which is the RGB color model. Compared with the RGB color model, the HSV color model is an intuitive color model for users, and the two models can be converted into each other. The HSV model is smoother and will not have a sudden change from white to black at the color boundary, and is more suitable for representing the gradient color on the half-sphere cover.

[0031] Further, on the same latitude, the saturation component S is kept unchanged, and the hue component H changes uniformly from 0° to 360°; on the same longitude, the hue component H is kept unchanged, and the saturation component S changes uniformly from 0 to 100%. Therefore, since the color (H, S) of every point on the half-sphere cover is different, the mapping relationship between the position coordinates (x, y) and the color (H, S) is established, so as to convert the difficult spatial angle measurement into the simple pixel color measurement.

[0032] Further, the hue and the saturation (H0, S0) of the imaging plane (x0, y0) are measured, and the actual point on the half-sphere cover corresponding to the corresponding point is obtained.

[0033] Further, the spatial angle of every pixel point on the image is measured, and the look-up table of (x, y) and is obtained. Further, according to the pixel coordinates (x, y) and the spatial elevation angle and azimuth angle , the pixel coordinates (x, y) and the spatial angle are obtained.

[0034] ​​The look-up table of the fisheye camera and the actual orientation of the target object relative to the fisheye camera can be determined. According to the actual spatial orientation angle corresponding to the pixels, each pixel is repositioned, the distorted image is corrected, the camera shooting picture is similar to the picture shot by a normal camera, and the calibration is completed.

[0035] Further, the gradient of a point is a vector, the direction of which is consistent with the direction in which the maximum directional derivative is obtained, and the module is the maximum value of the directional derivative. It can be understood that the directional derivative of the gradient is the maximum at the point, and the change rate is the maximum along the gradient direction. The gradient of the hue and saturation of the original image and the corrected image is calculated by using the gradient operator and compared, so that the distortion degree of the fisheye camera can be evaluated.

[0036] It can be understood that the beneficial effects of the above-mentioned second aspect to the fourth aspect can be referred to the related description in the above-mentioned first aspect, which will not be repeated here.

[0037] In summary, the present application can establish the imaging model of the camera without correcting the distorted image, which is simple and effective, greatly reduces the data operation amount, and improves the real-time performance. At the same time, unlike the pure theoretical modeling method, the model established by the experimental method also considers the system error of the camera itself, and the establishment result is more accurate. The model can also obtain the size and direction of the distortion degree of different points on the imaging plane through gradient operation, and quantitatively analyze the performance of the camera.

[0038] The technical solutions of the present application will be further described in detail below with reference to the drawings and embodiments. BRIEF DESCRIPTION OF DRAWINGS

[0039] Figure 1 The figure shows the coordinates of the imaging point and the actual point;

[0040] Figure 2 The figure shows the color component on the half-shell cover. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0042] In the description of the present application, it should be understood that the terms “include” and “contain” indicate the existence of the described features, whole, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, whole, steps, operations, elements, components and / or sets thereof.

[0043] It should also be understood that the words used in the specification are words of description rather than limitation, and it is not intended to limit the application in any way except as it may be limited by the attached claims. As used herein, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.

[0044] It should also be further understood that the term "and / or" used in the specification and the appended claims, means any one of the associated listed items, or any combination of the associated listed items, and includes all possible combinations, for example, A and / or B can mean only A, or only B, or both A and B. In addition, the character " / " in this text, generally represents the relationship between the front and rear associated objects is "or".

[0045] It will be understood that the word "if" as used herein, meaning "when" or "upon" or "in response to the determination" or "in response to the detection," can be construed to mean "when," or "upon," or "in response to the determination," or "in response to the detection," depending on the context. Similarly, the phrase "if it is determined" or "if [a stated condition or event] is detected" can be construed to mean "upon determining" or "in response to determining," or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]," depending on the context.

[0046] Various structural diagrams according to the disclosed embodiments of the present application are shown in the accompanying drawings. These diagrams are not drawn to scale, in which certain details are exaggerated for clarity and others omitted. The shapes and relative sizes of the various regions, layers, and their relative positions shown in the drawings are merely illustrative and can deviate in actuality due to manufacturing tolerances or technical limitations, and regions / layers with different shapes, sizes, and relative positions can be additionally designed by those skilled in the art according to actual needs.

[0047] The present application provides a pixel-by-pixel fisheye camera calibration method, which can obtain the angle between the target object in the picture and the coordinate axis of the camera coordinate system, quickly determine the relative direction of the target object and the camera optical center, and obtain the pixel-by-pixel angle measurement and corresponding measurement error of the fisheye camera in the field of view, without distortion correction of the fisheye camera picture and greatly reducing the amount of mathematical calculation.

[0048] The present application provides a pixel-by-pixel fisheye camera calibration method, which can obtain the angle between the target object in the picture and the coordinate axis of the camera coordinate system, quickly determine the relative direction of the target object and the camera optical center, and obtain the pixel-by-pixel angle measurement and corresponding measurement error of the fisheye camera in the field of view, without distortion correction of the fisheye camera picture and greatly reducing the amount of mathematical calculation.

[0049] S1, making a half-sphere cover with internal color continuously and uniformly changing, representing different colors on the half-sphere cover by components (H, S), keeping the saturation component S unchanged on the same latitude, and the hue component H uniformly changing from 0° to 360°, keeping the hue component H unchanged on the same longitude, and the saturation component S uniformly changing from 0 to 100%, so as to ensure that the color of each point on the half-sphere cover is unique, as shown in FIG. 1, thereby establishing a mapping relationship between the position coordinates Figure 2 and the color (H, S);

[0050] S2, placing the fisheye camera at the center of the half-sphere cover obtained in step S1, taking a picture, measuring the color (H', S') of the corresponding point, and obtaining the actual point on the half-sphere cover corresponding to the actual point on the imaging plane through the mapping relationship between (H, S) established in step S1; sampling and measuring a plurality of points on the imaging plane of the actual point on the half-sphere cover, to obtain a lookup table of the image point coordinates (x, y) on the imaging plane and the elevation angle and azimuth angle in the space, so as to determine the actual orientation of the target object relative to the fisheye camera according to the lookup table, and reposition each pixel according to the actual spatial orientation angle corresponding to the pixel, correct the distorted image, and make the picture taken by the camera similar to the picture taken by an ordinary camera, that is, the calibration is completed.

[0051] wherein the gradient calculation operator (such as Sobel operator, Laplace operator, etc.) is used to calculate the gradient of the hue H or the saturation S on the original image and the corrected image, the hue H represents the distortion in the tangential direction, the saturation S represents the distortion in the radial direction, the modulus value of the gradient represents the degree of distortion, and the greater the modulus value of the gradient, the greater the degree of distortion, which is used as the measurement error after discretization, and the direction of the gradient represents the direction of the distortion, and the two are compared to obtain the distortion degree of each part of the fisheye camera.

[0052] In space, three independent coordinates are needed to determine a point. In the Cartesian rectangular coordinate system, in addition to using the classic (x, y, z) to represent, the most commonly used representation method is spherical coordinates. The spherical coordinates are represented by , wherein r represents the distance from the target point to the origin, θ is the elevation angle, and φ is the azimuth angle, that is, the angle between the straight line formed by the target point and the origin and the positive direction of the z axis and the positive direction of the x axis. A single camera can measure the azimuth angle and the elevation angle of the target object in the camera coordinate system.

[0053] To determine the orientation of the target point relative to the camera optical center in the actual space, the functional relationship between the image point coordinates (x, y) on the imaging plane and the elevation angle and azimuth angle in the space needs to be obtained:

[0054] θ = f (x, y) ​

[0055]

[0056] To determine the function relationship, a half-sphere cover with internal color continuously and uniformly changing is made.

[0057] The HSV model in color space is adopted, i.e. using three components of hue, saturation and lightness (H, S, V) to represent a color.

[0058] It is considered that the position of any point on the half-sphere cover represents the direction of the target object relative to the camera. The distance from any point on the half-sphere cover to the center of the sphere is the radius, i.e. r' = r0. Therefore, the position of any point on the half-sphere cover can be degenerated to be represented by two components. Thus, the representation of color is also degenerated, keeping the lightness component V = V0 unchanged on the whole half-sphere cover, and only using two components (H, S) to represent different colors on the half-sphere cover.

[0059] Please refer to Figure 2 On the same latitude, the saturation component S is kept unchanged, and the hue component H uniformly changes from 0° to 360°.

[0060] On the same longitude, the hue component H is kept unchanged, and the saturation component S uniformly changes from 0 to 100%.

[0061] Thus, on the half-sphere cover, the position coordinates and the color (H, S) establish a one-to-one mapping relationship, changing the traditional continuous function mode to a table lookup mode more suitable for the characteristics of a discrete camera.

[0062] The fisheye camera is placed at the center of the half-sphere cover, and by shooting a picture, i.e. measuring the hue and saturation (H0, S0) of any point (x0, y0) on the imaging plane, the actual point

[0063] By sampling and measuring multiple points on the image, a corresponding table of (x, y) and can be obtained, which is equivalent to a measurement feature map for a specific fisheye camera, and in actual use, the actual orientation of the target object relative to the camera can be determined by table lookup.

[0064] Therefore, the technical scheme avoids the quantization error between the continuous objective environment-discrete camera pixels-fitted continuous imaging function-discrete corrected image pixels, directly uses the feature image pixel-discrete collected image to replace the idea, grasps the characteristics of the camera pixelization and discretization, avoids the error between the AD / DA conversion, reduces the error after correction to the quantization error of the self imaging, and cancels the error in the fitting process. In actual use, the actual position and direction of the target object in the space can be obtained through the color value in the feature image corresponding to the coordinate on the imaging plane.

[0065] On the half-sphere cover, the hue and saturation of the color are uniformly distributed, and on the imaging plane, the uniformity is destroyed due to the camera distortion. The gradient of the hue or saturation on the image is obtained through the Sobel, Laplace and other operators, the greater the modulus value of the gradient is, the greater the distortion degree is, and the gradient direction represents the distortion direction, so that the performance of the fisheye camera can be quantitatively analyzed.

[0066] In another embodiment of the present application, a pixel-by-pixel fisheye camera calibration system is provided, which can be used to realize the pixel-by-pixel fisheye camera calibration method, and specifically, the pixel-by-pixel fisheye camera calibration system comprises a mapping module and a calibration module.

[0067] The mapping module is used to make an internal color continuously and uniformly changing half-sphere cover, to represent different colors on the half-sphere cover by components (H, S), and to establish the mapping relationship between the position coordinates and the color (H, S).

[0068] The calibration module is used to place the fisheye camera at the center of the half-sphere cover obtained by the mapping module, to obtain the actual point on the half-sphere cover corresponding to the corresponding point through the photographed picture The imaging plane is sampled and measured, to obtain the look-up table of the imaging plane target image point coordinates (x, y) and the space elevation angle and azimuth angle According to the look-up table, the actual direction of the target object relative to the fisheye camera is determined, each pixel is repositioned according to the actual space direction angle corresponding to the pixel, and the calibration is completed after the distorted image is corrected.

[0069] ​In another embodiment of the present invention, a terminal device is provided, comprising a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions to implement a corresponding method flow or corresponding function. The processor described in this embodiment of the present invention can be used for the operation of a pixel-by-pixel fisheye camera calibration method, including:

[0070] Construct a hemispherical dome with a continuously and uniformly varying internal color. Use components (H, S) to represent the different colors on the hemispherical dome and establish positional coordinates. The mapping relationship between color (H, S); place the fisheye camera at the center of the hemispherical dome, take pictures, measure the color (H′, S′) of the corresponding point, and use the position coordinates... The mapping relationship between color (H, S) is used to obtain the actual coordinates of the corresponding point on the hemispherical dome. By sampling and measuring multiple points on the imaging plane, the coordinates (x, y) of the target image point on the imaging plane and its elevation and azimuth angles in space are obtained. The lookup table is used to determine the actual orientation of the target object relative to the fisheye camera. Based on the actual spatial orientation angle corresponding to the pixel, each pixel is repositioned, and the calibration is completed after correcting the distorted image.

[0071] In another embodiment of the present application, the present application also provides a storage medium, specifically a computer readable storage medium (Memory), which is a memory device in the terminal device, used for storing programs and data. It can be understood that the computer readable storage medium herein can include the built-in storage medium in the terminal device, and of course can also include the expansion storage medium supported by the terminal device. The computer readable storage medium provides a storage space, which stores the operating system of the terminal. Moreover, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space, and the instructions can be one or more computer programs (including program codes). It should be noted that the computer readable storage medium herein can be a high-speed RAM memory, or a non-volatile memory such as at least one disk memory.

[0072] The one or more instructions stored in the computer readable storage medium can be loaded and executed by the processor to realize the corresponding steps of the pixel-by-pixel fisheye camera calibration method in the above embodiments; the one or more instructions in the computer readable storage medium are loaded and executed by the processor to perform the following steps:

[0073] A half-sphere cover with a color continuously and uniformly changing inside is made, different colors on the half-sphere cover are represented by components (H, S), a mapping relationship between the position coordinates and the color (H, S) is established; the fisheye camera is placed at the center of the half-sphere cover, a picture is taken, and the color (H', S') of the corresponding point is measured, through the mapping relationship between the position coordinates and the color (H, S), the actual point coordinates on the half-sphere cover are obtained A plurality of points on the imaging plane are sampled and measured, a lookup table of the image point coordinates (x, y) on the imaging plane and the elevation angle and azimuth angle in the space is obtained, the actual orientation of the target object relative to the fisheye camera is determined according to the lookup table, each pixel is repositioned according to the actual space orientation angle corresponding to the pixel, and the calibration is completed after the distorted image is corrected.

[0074] The present application obtains the coordinates (x, y) of the imaging point in the image coordinate system and the orientation The function relationship and the error size of the corresponding measurement are determined, and the imaging model of the camera is established without correcting the distorted image, so that the method is simple and effective, the data operation amount is greatly reduced, and the real-time performance is improved.

[0075] In summary, the fish-eye camera calibration method and system according to the present application can establish a lookup table of the coordinates of the imaging points in the image coordinate system and the orientation of the corresponding target objects relative to the camera optical center in space by using a special hemisphere cover, so that the fish-eye camera can be calibrated.

[0076] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can be in the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware aspects. Moreover, the present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program codes.

[0077] The present application is described with reference to flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of the flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device that implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more blocks.

[0078] These computer program instructions can also be stored in a computer readable storage medium that can guide the computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer readable storage medium produce a product including instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that carries out the functions specified in one or more blocks.

[0079] These computer program instructions can also be loaded into a computer or other programmable data processing devices, so that a series of operational steps are performed on the computer or other programmable data processing devices to generate a computer implemented process, so that the instructions executed on the computer or other programmable data processing devices provide a process for implementing the flowchart Figure 1 one flow or a plurality of flows and / or the functions specified in a block Figure 1 one block or a plurality of blocks.

[0080] The above is only to illustrate the technical idea of the present application, and cannot limit the protection scope of the present application. Any modification made according to the technical idea of the present application on the basis of the technical scheme falls within the protection scope of the claims of the present application.

Claims

1. A pixel-by-pixel fisheye camera calibration method, characterized in that, comprising the steps of: S1, make a half-sphere cover with internal color changing continuously and uniformly, to component characterize different colors on the half-sphere cover, and establish mapping relationship between position coordinates and color ; S2, place the fisheye camera at the center of the semi-dome obtained in step S1, take a picture, measure the color of the corresponding points , the mapping relationship between the position coordinates and the color established by step S1, obtain the actual point coordinates on the corresponding semi-dome , sample and measure a plurality of points on the imaging plane to obtain the image point coordinates of the target object on the imaging plane and the lookup table of the elevation angle and the azimuth angle in space, determine the actual orientation of the target object relative to the fisheye camera according to the lookup table, reposition each pixel according to the actual spatial orientation angle corresponding to the pixel, correct the distorted image, and complete the calibration, use the gradient operator to obtain the gradient of the hue or the saturation on the original image and the corrected image, the hue characterizes the distortion in the tangential direction, the saturation characterizes the distortion in the radial direction, the modulus value of the gradient represents the degree of distortion, and the direction of the gradient represents the direction of the distortion, and the modulus value of the gradient and the direction of the gradient are compared to obtain the distortion degree of the fisheye camera at each position.

2. The pixel-wise fisheye camera calibration method according to claim 1, wherein, In step S1, the position of any point on the hemisphere is represented by the components , keeping the luminance component constant for all colors on the hemisphere, characterized by the components .

3. The pixel-wise fisheye camera calibration method according to claim 1, wherein, In step S1, a color space in which a model is used to represent a color using hue, saturation and lightness three components, and a mapping relationship between position coordinates and color is established using two angles corresponding to two actual spatial orientations in two of the color three components. ​ 4. The pixel-wise fisheye camera calibration method of claim 1, wherein, In step S1, on the same latitude, keep the saturation component unchanged In step S2, on the same longitude, keep the hue component unchanged In step S3, on the same latitude, change the saturation component uniformly from 0% to 100% In step S4, on the same longitude, change the hue component uniformly from 0° to 360° In step S5, establish the mapping relationship between the position coordinates and the color In step S6, change the position coordinates uniformly from 0 to 100% In step S7, change the color uniformly from 0 to 100% 5. The pixel-wise fisheye camera calibration method according to claim 1, wherein, In step S2, the hue and saturation of any point on the imaging plane is measured , obtaining the actual point on the corresponding hemisphere corresponding to the point .

6. The pixel-wise fisheye camera calibration method according to claim 1, wherein, In step S2, the image point coordinates of the target object on the imaging plane and the azimuth angle in space are related by the function ​ wherein and are both discrete functions.

7. A pixel-wise fisheye camera calibration system, characterized in that, comprising: a mapping module for making an internal color continuously and uniformly varying half-shell to component characterizing different colors on the half-shell, establishing a mapping relationship between position coordinates and color ; The calibration module is used to place the fisheye camera at the center of the hemispherical dome obtained by the mapping module, and obtain the actual point on the hemispherical dome corresponding to the corresponding point by taking pictures. Sample measurements are performed on multiple points on the imaging plane to obtain the coordinates of the target image points on the imaging plane. With elevation and azimuth in space A lookup table is used to determine the actual orientation of the target object relative to the fisheye camera. Based on the actual spatial orientation angle corresponding to each pixel, each pixel is repositioned. After correcting the distorted image, calibration is completed. The gradient operator is then used to obtain the tonal values ​​of the original and corrected images. or saturation gradient, hue Characterizing distortion and saturation in the tangential direction The gradient magnitude represents the degree of distortion, and the gradient direction represents the direction of distortion. By comparing the magnitude and direction of the gradient, the degree of distortion at various points on the fisheye camera can be obtained.

8. The pixel-by-pixel fisheye camera calibration system of claim 7, wherein, In the mapping module, the position of any point on the hemispherical dome is represented by components. This means maintaining the lightness component of the color across the entire hemisphere. Unchanged, in terms of quantity The different colors on the hemispherical cover are used to represent the different colors. Using a color space model, a color is represented using hue, saturation and lightness three components Two of the three color components correspond to two angles of the actual space orientation, and a mapping relationship between the position coordinates and the color is established ​ On the same latitude, keep the saturation component unchanged, the hue component changes uniformly from 0° to 360°; on the same longitude, keep the hue component unchanged, the saturation component changes uniformly from 0 to 100%, establishing the position coordinates and the mapping relationship of colors .

9. A computer-readable storage medium storing one or more programs, the one or more programs comprising instructions for: The one or more programs include instructions that when executed by the computing device, cause the computing device to perform any of the methods of claims 1-6.

10. A computing device, comprising: comprising: One or more processors, memory, and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, the one or more programs including instructions for performing any of the methods of claims 1-6.

Images