Method and Device for Determining Intrinsic Parameters of Fish-Eye Module

Abstract

The method and device for determining the internal parameters of a fish-eye module provided by an embodiment of the present invention obtain a plurality of checkerboards; divide the checkerboards according to the marker coordinates corresponding to the identifiers of each checkerboard to obtain an edge area and a central area; extract corner points from the edge area and the central area to obtain original image corner points; calculate the corner point coordinates after distortion correction; calculate corresponding undistorted points according to the corner point coordinates after distortion correction, and determine the corner point coordinates of interest according to the undistorted points; process the corner point coordinates to obtain the original image corner point coordinates corresponding to the corner point coordinates; if the original image corner point coordinates meet the preset rules, determine the distortion point sorting sequence, and calibrate the internal parameters of the fish-eye module according to the distortion point sorting sequence and the physical space coordinates to obtain the target internal parameters and target distortion coefficients corresponding to the fish-eye module. By setting a plurality of identifiers as targets, the internal parameters of the fish-eye module are quickly calculated, improving the calculation efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and particularly to a method and device for determining internal parameters of a fish-eye module. Background Art

[0002] With the development of automotive autonomous driving and intelligence, the requirements for vehicle-mounted cameras are getting higher and higher. In addition to basic performance requirements such as resolution, signal-to-noise ratio, and white balance, it is also necessary to know the internal parameters and distortion coefficients of the module. Therefore, when a large number of modules need to be calibrated, it is very important to quickly and accurately obtain the internal parameters. In the prior art, manual calibration is usually adopted. Most manual calibrations use checkerboards as calibration patterns, such as Zhang's calibration method. The manual calibration method has the advantages of high accuracy and good robustness, but manual point selection takes a long time and has low efficiency, and is not suitable for application scenarios with large batch requirements such as production line calibration. How to quickly obtain the internal parameters of the fish-eye module is an urgent problem to be solved currently. Summary of the Invention

[0003] In view of the above problems, embodiments of the present invention are proposed to provide a method, device, terminal device, and readable storage medium for determining internal parameters of a fish-eye module that overcome the above problems or at least partially solve the above problems.

[0004] In a first aspect, an embodiment of the present invention provides a method for determining internal parameters of a fish-eye module, the method including:

[0005] Obtain a plurality of checkerboards, and respectively mark the plurality of checkerboards with preset identifiers;

[0006] According to the flag coordinates corresponding to the identifiers of each checkerboard, divide the checkerboard to obtain an edge region and a central region;

[0007] Use a corner extraction algorithm to extract corners from the edge region and the central region to obtain original image corners corresponding to the edge region and the central region;

[0008] According to preset internal parameters, preset distortion coefficients, and original image corners, obtain corner coordinates after distortion correction;

[0009] According to the corner coordinates after distortion correction and the identifiers, determine corresponding undistorted points according to a preset rule, and determine corner coordinates of interest according to the undistorted points, and sort the corner coordinates;

[0010] According to the preset internal parameters and preset distortion coefficients, process the corner coordinates to obtain original image corner coordinates corresponding to the corner coordinates;

[0011] If the corner coordinates of the original image satisfy a preset rule, determine the sorting sequence of the distorted points, and calibrate the internal parameters of the fisheye module according to the sorting sequence of the distorted points and the physical space coordinates, so as to obtain the target internal parameters and target distortion coefficients corresponding to the fisheye module.

[0012] Optionally, marking the checkerboard with a preset identifier includes:

[0013] Set identifiers on each boundary of the checkerboard, and the identifiers are located at the outermost periphery of the corner points of the checkerboard.

[0014] Optionally, the identifier at least includes a square pattern or a circular pattern with a square pattern inside.

[0015] Optionally, processing the corner coordinates according to the preset internal parameters and preset distortion coefficients to obtain the original image corner coordinates corresponding to the corner coordinates includes:

[0016] Obtain the corner coordinates of the calibration corner points without distortion correction;

[0017] Process the corner coordinates of the calibration corner points without distortion correction according to the preset distortion parameters of the fisheye module and the focal length ratio of the fisheye module to obtain the corner coordinates of the calibration corner points.

[0018] Optionally, the method further includes:

[0019] Perform distortion reduction on the target internal parameters and target distortion coefficients of the fisheye module to obtain a restored image;

[0020] Detect the restored image through the distortion correction image and the linearity of the outermost quadrilateral of each region.

[0021] In a second aspect, an embodiment of the present invention provides an internal parameter determination device for a fisheye module, and the device includes:

[0022] An acquisition module, configured to acquire a plurality of checkerboards and mark the plurality of checkerboards with preset identifiers respectively;

[0023] A segmentation module, configured to segment the checkerboard according to the flag coordinates corresponding to the identifiers of each checkerboard to obtain an edge region and a central region;

[0024] An extraction module, configured to extract corner points from the edge region and the central region by using a corner point extraction algorithm to obtain original image corner points corresponding to the edge region and the central region;

[0025] A first determination module, configured to obtain the corner coordinates after distortion correction according to preset internal parameters, preset distortion coefficients and original image corner points;

[0026] A calibration module, configured to determine corresponding undistorted points according to the corner coordinates and identifiers after distortion correction according to a preset rule, and determine corner coordinates of interest according to the undistorted points, and sort the corner coordinates.

[0027] A second determination module, configured to process the corner coordinates according to the preset internal parameters and preset distortion coefficients to obtain the original image corner coordinates corresponding to the corner coordinates.

[0028] A calibration module, configured to determine a distortion point sorting sequence if the original image corner coordinates meet a preset rule, and calibrate the internal parameters of the fish-eye module according to the distortion point sorting sequence and physical space coordinates to obtain target internal parameters and target distortion coefficients corresponding to the fish-eye module.

[0029] Optionally, the obtaining module is configured to:

[0030] Set identifiers on each boundary of the checkerboard, and the identifiers are located at the outermost periphery of the corner points of the checkerboard.

[0031] Optionally, the identifier at least includes a square or a circle with a square pattern arranged therein.

[0032] Optionally, the extraction module is configured to:

[0033] Obtain the corner coordinates of the calibration corner points without distortion correction;

[0034] Process the corner coordinates of the calibration corner points without distortion correction according to the preset distortion parameters and focal length ratio of the fish-eye module to obtain the corner coordinates of the calibration corner points.

[0035] Optionally, the calibration module is further configured to:

[0036] Perform distortion reduction on the target internal parameters and target distortion coefficients of the fish-eye module to obtain a restored image;

[0037] Detect the restored image through the distortion correction image and the linearity of the outermost quadrilateral of each region.

[0038] In a third aspect, an embodiment of the present invention provides a terminal device, including: at least one processor and a memory;

[0039] The memory stores a computer program; the at least one processor executes the computer program stored in the memory to implement the method for determining the internal parameters of the fish-eye module provided in the first aspect.

[0040] Fourthly, an embodiment of the present invention provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed, the method for determining the internal parameters of the fisheye module provided in the first aspect is implemented.

[0041] The embodiments of the present invention include the following advantages:

[0042] For the method and device for determining the internal parameters of the fisheye module provided by the embodiments of the present invention, a plurality of checkerboards are obtained, and the plurality of checkerboards are respectively marked with preset identifiers; according to the marked coordinates corresponding to the identifiers of each checkerboard, the checkerboard is segmented to obtain an edge area and a central area; a corner extraction algorithm is used to extract corners from the edge area and the central area to obtain original image corners corresponding to the edge area and the central area; according to the preset internal parameters, preset distortion coefficients and original image corners, the corner coordinates after distortion correction are obtained; according to the corner coordinates after distortion correction and the identifiers, corresponding undistorted points are determined according to preset rules, and according to the undistorted points, the corner coordinates of interest are determined, and the corner coordinates are sorted; according to the preset internal parameters and preset distortion coefficients, the corner coordinates are processed to obtain the original image corner coordinates corresponding to the corner coordinates; if the original image corner coordinates meet the preset rules, a distortion point sorting sequence is determined, and according to the distortion point sorting sequence and physical space coordinates, the internal parameters of the fisheye module are calibrated to obtain the target internal parameters and target distortion coefficients corresponding to the fisheye module. Through the embodiments of the present application, a plurality of identifiers are set as targets to quickly calculate the internal parameters of the fisheye module and improve the calculation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 is a flowchart of the steps of an embodiment of the method for determining the internal parameters of a fisheye module of the present invention;

[0044] Figure 2 is a schematic diagram of the setting of the identifier of the present invention;

[0045] Figure 3 is another schematic diagram of the display of the identifier of the present invention;

[0046] Figure 4 is a display diagram of the identifier of the present invention;

[0047] Figure 5 is a schematic diagram of the regional segmentation of the present invention;

[0048] Figure 6 is a schematic diagram of the display of the region of interest of the present invention;

[0049] Figure 7 is another schematic diagram of the display of the region of interest of the present invention;

[0050] Figure 8It is a schematic diagram showing another region of interest of the present invention;

[0051] Figure 9 It is a schematic diagram of the calibration target of the present invention;

[0052] Figure 10 It is a schematic diagram for determining a distorted corner point of the present invention;

[0053] Figure 11 It is a schematic diagram for determining another distorted corner point of the present invention;

[0054] Figure 12 It is a schematic diagram for determining another distorted corner point of the present invention;

[0055] Figure 13 It is a schematic diagram of an original image corner point of the present invention;

[0056] Figure 14 It is a schematic diagram of another original image corner point of the present invention;

[0057] Figure 14a It is a schematic diagram of the process of finding corner points and arranging them in the present invention;

[0058] Figure 15 It is a schematic diagram of sorting the original image corner points in the present invention;

[0059] Figure 16 It is a structural block diagram of an embodiment of an internal parameter determination device for a fish-eye module according to the present invention;

[0060] Figure 17 It is a structural schematic diagram of a terminal device according to the present invention. Detailed implementation manners

[0061] To make the above objects, features, and advantages of the present invention more obvious and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0062] An embodiment of the present invention provides a method for determining internal parameters of a fish-eye module, which is used to calculate the internal parameters of the fish-eye module. The execution subject of this embodiment is an internal parameter determination device of the fish-eye module, which is set on a terminal device. Among them, the terminal device at least includes a computer, a tablet terminal, etc.

[0063] See Figure 1 、 Figure 9 It shows a flowchart of the steps of an embodiment of a method for determining internal parameters of a fish-eye module according to the present invention. The method may specifically include the following steps:

[0064] S101. Obtain a plurality of checkerboards and mark the plurality of checkerboards respectively with preset identifiers;

[0065] S102. Segment the checkerboard according to the flag coordinates corresponding to the identifiers of each checkerboard grid to obtain an edge region and a central region;

[0066] S103. Use a corner point extraction algorithm to extract corner points from the edge region and the central region to obtain the original image corner points corresponding to the edge region and the central region;

[0067] S104. Obtain the corner point coordinates after distortion correction based on the preset internal parameters, preset distortion coefficients, and original image corner points.

[0068] The specific process is to find the undistorted points from the distorted points.

[0069] S105. According to the corner point coordinates and identifiers after the distortion correction, determine the corresponding undistorted points according to the preset rules, and determine the corner point coordinates of interest according to the undistorted points, and sort the corner point coordinates.

[0070] Specifically, refer to Figure 9 、 10 、As shown in Fig. 12, according to the undistorted points and identifiers, find the corresponding undistorted points according to the rules, obtain the corner point coordinates of interest p1, and sort them; according to the original image corner points, determine the region of interest corresponding to the original image corner points; according to the preset internal parameters and preset distortion coefficients, process the corrected corner points to obtain the original image corresponding to the corrected corner points, that is, find the corner point coordinates p2 of the distorted points from the undistorted points; when implementing the code, only need to find the undistorted points from the distorted points in the previous step.

[0071] S106. According to the preset internal parameters and preset distortion coefficients, process the corner point coordinates to obtain the original image corner point coordinates corresponding to the corner point coordinates;

[0072] S107. If the original image corner point coordinates meet the preset rules, determine the distortion point sorting sequence, and calibrate the internal parameters of the fish-eye module according to the distortion point sorting sequence and the physical space coordinates to obtain the target internal parameters and target distortion coefficients corresponding to the fish-eye module;

[0073] Refer to Figures 13 - 14 As shown in Fig. 13, if the corner point coordinates p2 meet the preset rules, determine the distortion point sorting sequence, obtain p3 and the physical space coordinates according to the distortion point sorting sequence, and calibrate the internal parameters of the fish-eye module to obtain the target internal parameters and target distortion coefficients corresponding to the fish-eye module.

[0074] Refer to Figures 10 - 12 As shown in Fig. 14, in the embodiments of the present application, steps S105 - S107 can be summarized as:

[0075] Perform distortion correction on the original corner point coordinates to obtain the coordinates of the undistorted points on the image;

[0076] According to the identifier (the identifier forms a straight-line equation), find the coordinates p1 of the undistorted points that meet the conditions and sort them;

[0077] Perform distortion processing on the distorted image (opposite to the first point) to obtain the distorted points p2.

[0078] See Figure 13 、 14 As shown, construct a straight-line equation for each column of the distorted points p2, find the corner points that meet the conditions, and sort them to obtain the final distorted points p3. According to the distorted point sorting sequence and the physical space coordinates, calibrate the internal parameters of the fish-eye module to obtain the target internal parameters and target distortion coefficients corresponding to the fish-eye module.

[0079] The method for determining the internal parameters of the fish-eye module provided by the embodiment of the present invention includes obtaining multiple checkerboards and respectively marking the multiple checkerboards with preset identifiers; dividing the checkerboards according to the marked coordinates corresponding to the identifiers of each checkerboard to obtain an edge area and a central area; using a corner point extraction algorithm to extract corner points from the edge area and the central area to obtain the original image corner points corresponding to the edge area and the central area; performing distortion correction on the original image corner points according to the preset internal parameters and preset distortion coefficients to obtain the corresponding undistorted points; determining the corrected corner points after distortion correction of the region of interest according to the distortion correction image and the identifier, and sorting them to obtain p1; performing inverse distortion correction on the corner point P1 to obtain the original corner point coordinates p2 corresponding to p1; if the corner point coordinates of the distorted points meet the preset rules, determine the distorted point sorting sequence, and calibrate the internal parameters of the fish-eye module according to the distorted point sorting sequence and the physical space coordinates to obtain the target internal parameters and target distortion coefficients corresponding to the fish-eye module. By setting multiple identifiers as targets in the embodiments of the present application, the internal parameters of the fish-eye module can be quickly calculated, improving the calculation efficiency.

[0080] Optionally, marking the checkerboard with a preset identifier includes:

[0081] Set identifiers on each boundary of the checkerboard, and the identifiers are located at the outermost periphery of the corner points of the checkerboard.

[0082] Optionally, the identifier at least includes a square or a circle with a square pattern inside.

[0083] See Figure 9 As shown, add two marks to one side of the ordinary checkerboard and marks to both sides respectively for calibration targets. The marked points are set at the outermost periphery of the corner points of the checkerboard. The shape can be a square inside a circle, that is, two squares. The four edge plates can be composed differently for easy distinction. The middle checkerboard does not need to be marked.

[0084] See Figure 3As shown, five checkerboard grids are combined and placed. The 5 calibration targets below and on the edges are combined into an "uncovered square box", and the upper camera module takes pictures inside.

[0085] See Figure 4 As shown, the markings, i.e., identifiers, include but are not limited to the following graphics. In the embodiments of the present invention, the position of the circle is first identified and located, and then the centroid of the small square inside the circle is identified.

[0086] Optionally, according to the preset internal parameters and preset distortion coefficients, the diagonal point coordinates are processed to obtain the original image corner point coordinates corresponding to the corner point coordinates, including:

[0087] Obtain the corner point coordinates of the calibration corner points without distortion correction;

[0088] Among them, the corner point coordinates of the calibration corner points without distortion correction are the coordinates of the points without distortion.

[0089] According to the preset distortion parameters of the fisheye module and the focal length ratio of the fisheye module, the corner point coordinates of the calibration corner points without distortion correction are processed to obtain the corner point coordinates of the calibration corner points.

[0090] Optionally, the method further includes:

[0091] Perform distortion reduction on the target internal parameters and target distortion coefficients of the fisheye module to obtain the restored image;

[0092] Detect the restored image through the linearity of the distortion correction image and the outermost quadrilateral of each region.

[0093] In the embodiments of the present application, the method for determining the internal parameters of the fisheye module includes:

[0094] For the four edge regions, capture the marker coordinates of the checkerboard grids in each region position, and perform region segmentation according to the coordinates. The method of region segmentation is shown (see in detail Figure 5 )

[0095] Steps of region segmentation: For the two regions in the front and the back:

[0096] The captured points are: p1, p2, p3, p4;

[0097] Connect p1 and p2, and draw a straight line from p1 to p4 until it intersects with y = 0 to obtain a new point p4';

[0098] Draw a straight line from p2 to p3 until it intersects with y = 0 to obtain a new point p3';

[0099] p1, p2, p3', p4' form a new roi 1, i.e., region of interest;

[0100] Get the schematic diagram as follows: Figure 6 shown in the figure:

[0101] For the left and right regions, the captured points are: p5, p6, p7, p8;

[0102] Connect p5 and p6 into a straight line. Let x = width (width refers to the long side of the image), when y < 0 or y is greater than height (height refers to the height of the image), at this time, let y = 0 or y = height to find the edge points (p6', p8'). Next, find the points p11 (width, 0) and p11 (width, height). In this way, p5, p6', p11, p11, p8', p7 form a new roi region. The left roi region follows the same principle (see Figure 7 ).

[0103] For the middle region, since the row and column corner points are fixed and will not change, the simplest checkerboard corner point extraction algorithm can be used. For other regions, corner point extraction algorithms are adopted, including non-maximum suppression and sub-pixel extraction, etc., which can accurately extract the corner point algorithm.

[0104] The distortion coefficients of the same batch of lenses are almost the same. The same batch of lenses is of the same model. Therefore, this module can use the internal parameters and distortion coefficients obtained from the previous normal calibration to perform distortion correction and obtain the distortion correction map.

[0105] According to the distortion correction map, capture two marks of the calibration target, connect them into a straight line, and obtain the straight line equation (see Figure 11 ):

[0106] 1) Calculate the minimum distance mind between all corner points and the straight line equation, and then calculate the distance from all corner points to the straight line that is less than the sum of mind and a certain threshold d to obtain the corner points in the first row.

[0107] 2) Calculate the maximum distance maxd between all corner points and the straight line equation, and then calculate the distance from all corner points to the straight line that is greater than the difference between maxd and a certain threshold d to obtain the corner points in the last row.

[0108] Then sort the corner points in each row respectively to obtain the order from left to right.

[0109] Obtain the points after distortion correction. Through the unified internal parameters and distortion coefficients, obtain the corner point coordinates when there is distortion, that is, deduce the points coordinates when there is distortion from the undistorted coordinates. The formula is as follows: The embodiment of the present invention is an equidistant projection model;

[0110] xd and yd are the distorted points to be obtained, xu and yu are the undistorted points. cx and cy are the distortion centers of the unified module, fx and fy are the focal length ratios of the unified module, and k[0]...k[3] are the distortion coefficients of the unified module;

[0111] First, transform the xu, yu image coordinates to the module coordinates xu’, yu’;

[0112] xu’ = ((xu - cx) / fx); yu’ = ((yu - cy) / fy);

[0113] double r = sqrt(xu’ * xu’ + yu’ * yu’);

[0114] Theta = atan(r);

[0115] double theta2 = theta * theta, theta3 = theta2 * theta, theta4 = theta2 * theta2, theta5 = theta4 * theta,

[0116] theta6 = theta3 * theta3, theta7 = theta6 * theta, theta8 = theta4 * theta4, theta9 = theta8 * theta;

[0117] double theta_d = theta + k[0] * theta3 + k[1] * theta5 + k[2] * theta7 + k[3] * theta9;

[0118] double scale = (r == 0)? 1.0 : theta_d / r;

[0119] xd = cy + fy * yu’ * scale; / / x image coordinate of the distorted point;

[0120] yd = cx + fx * xu’ * scale; / / y image coordinate of the distorted point;

[0121] Draw the obtained points onto the original image.

[0122] A certain column is connected into a straight line, and the same sorting coordinates in each row respectively form a straight line. Points whose distances from the straight line on both sides to the straight line are less than a certain threshold are searched. In this way, a series of corner point coordinates can be obtained for each column. Then, the points in a certain column are sorted in the order from bottom to top to obtain the corner point coordinates of each column. However, the number of corner points in each column may be different. In this way, the coordinates of the most corner points can be obtained to the greatest extent. The spatial coordinate points are also set according to these corner point coordinates. During the internal parameter calibration, one-to-one matching is required, and logs are marked in the figure for sorting. For details, see Figure 13 , 14 as shown.

[0123] The method for extracting the corner point coordinates on the left and right sides is as in steps 3-6.

[0124] The corner point coordinates in the central area are extracted by using a common method.

[0125] Finally, the image corner point coordinates are brought in one-to-one correspondence with the physical space coordinates for calibration to obtain the internal parameters and distortion coefficients of this module. In the embodiments of the present application, distortion reduction can be performed according to the internal parameters and distortion coefficients, and then through the corrected image, the linearity of the outermost quadrilateral in each area is used to detect its calibration effect.

[0126] The embodiments of the present invention can quickly extract corner point coordinates, sort them, and the coordinate points can be non-symmetrically distributed and automatically adjusted. It will not be unable to extract due to the module being slightly placed off or the optical center having a large eccentricity, and can extract corner points to the greatest extent, improving the coverage rate.

[0127] Because the calculation speed of the embodiments of the present invention is fast, the module AA can also be rotated by a certain angle, and the SFR resolution is calculated, such as the slant block deflected by 4 degrees to 8 degrees, and then the optical center can be obtained. First, do through focus to obtain the best image distance, adjust tilt, and then calculate the distortion center and adjust the distortion center.

[0128] It should be noted that for the method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the embodiments of the present invention are not limited by the described action sequence, because according to the embodiments of the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily essential for the embodiments of the present invention.

[0129] The method for determining the internal parameters of a fisheye module provided by an embodiment of the present invention includes obtaining a plurality of checkerboards and respectively marking the plurality of checkerboards with preset identifiers; dividing the checkerboards according to the marker coordinates corresponding to the identifiers of each checkerboard to obtain an edge region and a central region; using a corner extraction algorithm to extract corners from the edge region and the central region to obtain original image corners corresponding to the edge region and the central region; performing distortion correction on the original image corners according to preset internal parameters and preset distortion coefficients to obtain corresponding undistorted points; according to the undistorted points and the identifiers, obtaining corresponding undistorted points according to rules and sorting them to obtain corner p1; performing inverse distortion correction on corner P1 to obtain the original corner coordinates p2 corresponding to p1; establishing a straight line equation for each column of p2 and obtaining the corner coordinates of each column to determine the final distortion point arrangement sequence, and calibrating the internal parameters of the fisheye module according to the distortion point coordinates and the physical space coordinates to obtain the target internal parameters and target distortion coefficients corresponding to the fisheye module. Through the embodiments of the present application, by setting a plurality of identifiers as targets, the internal parameters of the fisheye module can be quickly calculated, improving the calculation efficiency.

[0130] Another embodiment of the present invention provides an apparatus for determining the internal parameters of a fisheye module, which is used to execute the method for determining the internal parameters of the fisheye module provided in the above embodiment.

[0131] See Figure 15 , which shows a structural block diagram of an embodiment of an apparatus for determining the internal parameters of a fisheye module of the present invention. The apparatus may specifically include the following modules: an acquisition module 1601, a segmentation module 1602, an extraction module 1603, a first determination module 1604, a correction module 1605, a second determination module 1606, and a calibration module 1607, where:

[0132] The acquisition module 1601 is used to obtain a plurality of checkerboards and respectively mark the plurality of checkerboards with preset identifiers;

[0133] The segmentation module 1602 is used to divide the checkerboards according to the marker coordinates corresponding to the identifiers of each checkerboard to obtain an edge region and a central region;

[0134] The extraction module 1603 is used to use a corner extraction algorithm to extract corners from the edge region and the central region to obtain original image corners corresponding to the edge region and the central region;

[0135] The first determination module 1604 is used to obtain the corner coordinates after distortion correction according to the preset internal parameters, preset distortion coefficients, and original image corners;

[0136] The correction module 1605 is used to determine corresponding undistorted points according to the corner coordinates after distortion correction and the identifiers according to preset rules, and determine the corner coordinates of interest according to the undistorted points, and sort the corner coordinates;

[0137] The second determination module 1606 is configured to process the corner coordinates according to the preset internal parameters and preset distortion coefficients to obtain the original image corner coordinates corresponding to the corner coordinates.

[0138] The calibration module 1607 is configured to determine a distortion point sorting sequence if the original image corner coordinates satisfy a preset rule, and calibrate the internal parameters of the fisheye module according to the distortion point sorting sequence and the physical space coordinates to obtain target internal parameters and target distortion coefficients corresponding to the fisheye module.

[0139] The internal parameter determination device of the fisheye module provided by the embodiment of the present invention obtains multiple checkerboards and uses preset identifiers to mark the multiple checkerboards respectively; divides the checkerboards according to the marked coordinates corresponding to the identifiers of each checkerboard to obtain an edge area and a central area; uses a corner extraction algorithm to extract corners from the edge area and the central area to obtain original image corners corresponding to the edge area and the central area; performs distortion correction on the original image corners according to the preset internal parameters and preset distortion coefficients to obtain corresponding undistorted points; obtains corresponding undistorted points according to the undistorted points and the identifiers according to the rules, and sorts them to obtain corner p1; performs inverse distortion correction on corner P1 to obtain the original corner coordinate p2 corresponding to p1; establishes a straight line equation for each column of p2 and obtains the corner coordinates of each column to determine the final distortion point arrangement sequence, and calibrates the internal parameters of the fisheye module according to the distortion point coordinates and the physical space coordinates to obtain target internal parameters and target distortion coefficients corresponding to the fisheye module. Through the embodiment of the present application, multiple identifiers are set as targets to quickly calculate the internal parameters of the fisheye module and improve the calculation efficiency.

[0140] Another embodiment of the present invention further supplements the internal parameter determination device of the fisheye module provided in the above embodiment.

[0141] Optionally, the acquisition module is configured to:

[0142] Set identifiers on each boundary of the checkerboard, and the identifiers are located at the outermost periphery of the corners of the checkerboard.

[0143] Optionally, the identifier at least includes a square pattern or a circular pattern with a square pattern inside.

[0144] Optionally, the extraction module is configured to:

[0145] Obtain the corner coordinates of the corners without distortion correction.

[0146] Process the corner coordinates of the corners without distortion correction according to the preset distortion parameters of the fisheye module and the focal length ratio of the fisheye module to obtain the corner coordinates of the corrected corners.

[0147] Optionally, the calibration module is further configured to:

[0148] Perform distortion reduction on the target internal parameters and target distortion coefficients of the fisheye module to obtain a restored image;

[0149] Detect the restored image based on the distortion correction image and the linearity of the outermost quadrilateral of each region.

[0150] It should be noted that each implementable manner in this embodiment can be implemented independently or, without conflict, in any combined manner. The present application does not make any limitations.

[0151] For the apparatus embodiment, since it is basically similar to the method embodiment, the description is relatively simple. For related parts, refer to the partial description of the method embodiment.

[0152] The internal parameter determination apparatus of the fisheye module provided by the embodiment of the present invention obtains multiple checkerboards and uses preset identifiers to mark the multiple checkerboards respectively; divides the checkerboards according to the marked coordinates corresponding to the identifiers of each checkerboard to obtain an edge region and a central region; uses a corner point extraction algorithm to extract corner points from the edge region and the central region to obtain original image corner points corresponding to the edge region and the central region; performs distortion correction on the original image corner points according to preset internal parameters and preset distortion coefficients to obtain corresponding undistorted points; according to the undistorted points and identifiers, obtains corresponding undistorted points according to rules and sorts them to obtain corner point p1; performs inverse distortion correction on corner point P1 to obtain the original corner point coordinates p2 corresponding to p1; establishes a straight line equation for each column of p2 and obtains the corner point coordinates of each column to determine the final distortion point arrangement sequence, and calibrates the internal parameters of the fisheye module according to the distortion point coordinates and physical space coordinates to obtain the target internal parameters and target distortion coefficients corresponding to the fisheye module. Through the embodiment of the present application, multiple identifiers are set as targets to quickly calculate the internal parameters of the fisheye module and improve the calculation efficiency.

[0153] Another embodiment of the present invention provides a terminal device for executing the internal parameter determination method of the fisheye module provided by the above embodiment.

[0154] See Figure 16 As shown, the terminal device includes: at least one processor 1701 and a memory 1702;

[0155] The memory stores a computer program; at least one processor executes the computer program stored in the memory to implement the internal parameter determination method of the fisheye module provided by the above embodiment.

[0156] The terminal device provided in this embodiment obtains multiple checkerboards and uses preset identifiers to mark the multiple checkerboards respectively; divides the checkerboards according to the flag coordinates corresponding to the identifiers of each checkerboard to obtain an edge area and a central area; uses a corner extraction algorithm to extract corners from the edge area and the central area to obtain original image corners corresponding to the edge area and the central area; performs distortion correction on the original image corners according to a preset internal parameter and a preset distortion coefficient to obtain corresponding undistorted points; obtains corresponding undistorted points according to the undistorted points and the identifiers, sorts them to obtain corner p1; performs inverse distortion correction on corner P1 to obtain the original corner coordinates p2 corresponding to p1; establishes a straight line equation for each column of p2 and obtains the corner coordinates of each column to determine the final arrangement sequence of distorted points, and calibrates the internal parameter of the fisheye module according to the distorted point coordinates and the physical space coordinates to obtain the target internal parameter and the target distortion coefficient corresponding to the fisheye module. Through the embodiments of the present application, multiple identifiers are set as targets to quickly calculate the internal parameter of the fisheye module and improve the calculation efficiency.

[0157] Another embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed, it implements the method for determining the internal parameter of the fisheye module provided in any of the above embodiments.

[0158] According to the computer-readable storage medium of this embodiment, multiple checkerboards are obtained, and the multiple checkerboards are marked respectively with preset identifiers; the checkerboards are divided according to the flag coordinates corresponding to the identifiers of each checkerboard to obtain an edge area and a central area; a corner extraction algorithm is used to extract corners from the edge area and the central area to obtain original image corners corresponding to the edge area and the central area; distortion correction is performed on the original image corners according to a preset internal parameter and a preset distortion coefficient to obtain corresponding undistorted points; corresponding undistorted points are obtained according to the undistorted points and the identifiers, and are sorted to obtain corner p1; inverse distortion correction is performed on corner P1 to obtain the original corner coordinates p2 corresponding to p1; a straight line equation is established for each column of p2 and the corner coordinates of each column are obtained to determine the final arrangement sequence of distorted points, and the internal parameter of the fisheye module is calibrated according to the distorted point coordinates and the physical space coordinates to obtain the target internal parameter and the target distortion coefficient corresponding to the fisheye module. Through the embodiments of the present application, multiple identifiers are set as targets to quickly calculate the internal parameter of the fisheye module and improve the calculation efficiency.

[0159] The various embodiments in this specification are all described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the various embodiments can be referred to each other.

[0160] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, an apparatus, or a computer program product. Therefore, the embodiments of the present invention can take the form of an all-hardware embodiment, an all-software embodiment, or an embodiment combining software and hardware aspects. Moreover, the embodiments of the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0161] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, electronic devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing electronic devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing electronic devices generate a device for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0162] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing electronic devices to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0163] These computer program instructions can also be loaded onto a computer or other programmable data processing electronic devices, such that a series of operation steps are executed on the computer or other programmable electronic devices to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable electronic devices provide steps for implementing the functions specified in Figure 1 one flow or multiple flows and / or blocks Figure 1 one block or multiple blocks.

[0164] Although the preferred embodiments of the embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0165] Finally, it should also be noted that in this article, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or electronic device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or electronic device. Without further limitation, an element defined by the statement "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or electronic device comprising the element.

[0166] The above has introduced in detail a method for determining internal parameters of a fish-eye module and a device for determining internal parameters of a fish-eye module provided by the present invention. Specific examples are used in this article to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention; at the same time, for those of ordinary skill in the art, according to the idea of the present invention, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for determining the internal parameters of a fish-eye module, characterized in that, the method includes: Obtain a plurality of checkerboards, and use a preset identifier to mark each of the plurality of checkerboards respectively; According to the flag coordinates corresponding to the identifiers of each checkerboard, divide the checkerboard to obtain an edge area and a central area; Use a corner point extraction algorithm to extract corner points from the edge area and the central area, and obtain the original image corner points corresponding to the edge area and the central area; According to the preset internal parameters, preset distortion coefficients, and original image corner points, obtain the corner point coordinates after distortion correction; According to the corner point coordinates after distortion correction and the identifier, determine the corresponding undistorted points according to a preset rule, and according to the undistorted points, determine the corner point coordinates of interest, and sort the corner point coordinates of interest; According to the preset internal parameters and preset distortion coefficients, process the sorted corner point coordinates of interest to obtain the original image distorted points corresponding to the sorted corner point coordinates after distortion correction, so as to obtain an original image distorted point sorting sequence; Calibrate the internal parameters of the fish-eye module according to the distorted point sorting sequence and the physical space coordinates to obtain the target internal parameters and target distortion coefficients corresponding to the fish-eye module; Calibrate the internal parameters of the fish-eye module according to the distorted point sorting sequence and the physical space coordinates to obtain the target internal parameters and target distortion coefficients corresponding to the fish-eye module, including: perform distortion correction on the original image corner points according to the preset internal parameters and preset distortion coefficients to obtain the corresponding undistorted points; determine the corrected corner points after distortion correction of the area of interest according to the distorted correction image and the identifier, and sort them to obtain corner point P1; perform inverse distortion correction on corner point P1 to obtain the original corner point coordinates P2 corresponding to P1, establish a straight line equation for each column of P2, and obtain the corner point coordinates of each column, determine the distorted point sorting sequence, and calibrate the internal parameters of the fish-eye module according to the distorted point sorting sequence and the physical space coordinates to obtain the target internal parameters and target distortion coefficients corresponding to the fish-eye module.

2. The method according to claim 1, characterized in that, the step of using a preset identifier to mark the checkerboard includes: Set an identifier on each boundary of the checkerboard, and the identifier is located at the outermost periphery of the corner points of the checkerboard.

3. The method according to claim 2, characterized in that, the identifier at least includes a square or a circle with a square pattern inside.

4. The method according to claim 1, characterized in that, the step of processing the corner point coordinates after distortion correction according to the preset internal parameters and preset distortion coefficients to obtain the original image corner point coordinates corresponding to the corner point coordinates includes: Obtain the corner point coordinates of the corrected corner points without distortion correction; Process the corner point coordinates of the corrected corner points without distortion correction according to the preset distortion parameters of the fish-eye module and the focal length ratio of the fish-eye module to obtain the corner point coordinates of the corrected corner points.

5. The method according to claim 1, characterized in that, the method further includes: Perform distortion reduction on the target internal parameters and target distortion coefficients of the fish-eye module to obtain a restored image; Detect the restored image based on the distortion correction image and the linearity of the outermost quadrilateral of each region.

6. An internal parameter determination device for a fish-eye module, characterized in that the device includes: an acquisition module for acquiring a plurality of checkerboards and respectively marking the plurality of checkerboards with preset identifiers; a segmentation module for segmenting the checkerboards according to the flag coordinates corresponding to the identifiers of each checkerboard to obtain an edge region and a central region; an extraction module for extracting corner points from the edge region and the central region by using a corner point extraction algorithm to obtain original image corner points corresponding to the edge region and the central region; a first determination module for obtaining the corner point coordinates after distortion correction according to preset internal parameters, preset distortion coefficients, and original image corner points; a correction module for determining corresponding undistorted points according to the corner point coordinates after distortion correction and the identifiers according to preset rules, determining the corner point coordinates of the region of interest according to the undistorted points, and sorting the corner point coordinates of the region of interest; a second determination module for processing the corner point coordinates after distortion correction according to the preset internal parameters and preset distortion coefficients to obtain the original image corner point coordinates corresponding to the corner point coordinates; a calibration module for determining a distortion point sorting sequence if the original image corner point coordinates meet the preset rules, and calibrating the internal parameters of the fish-eye module according to the distortion point sorting sequence and the physical space coordinates to obtain target internal parameters and target distortion coefficients corresponding to the fish-eye module; By acquiring a plurality of checkerboards and respectively marking the plurality of checkerboards with preset identifiers; segmenting the checkerboards according to the flag coordinates corresponding to the identifiers of each checkerboard to obtain an edge region and a central region; extracting corner points from the edge region and the central region by using a corner point extraction algorithm to obtain original image corner points corresponding to the edge region and the central region; performing distortion correction on the original image corner points according to the preset internal parameters and preset distortion coefficients to obtain corresponding undistorted points; determining the corrected corner points after distortion correction of the region of interest according to the distortion correction image and the identifiers, and sorting to obtain corner point P1; performing inverse distortion correction on corner point P1 to obtain the original corner point coordinates P2 corresponding to P1, establishing a straight line equation for each column of P2, and obtaining the corner point coordinates of each column, determining the distortion point sorting sequence, and calibrating the internal parameters of the fish-eye module according to the distortion point sorting sequence and the physical space coordinates to obtain target internal parameters and target distortion coefficients corresponding to the fish-eye module.

7. The device according to claim 6, characterized in that the extraction module is used for: acquiring the corner point coordinates of the corrected corner points without distortion correction; processing the corner point coordinates of the corrected corner points without distortion correction according to the preset distortion parameters of the fish-eye module and the focal length ratio of the fish-eye module to obtain the corner point coordinates of the corrected corner points.

8. The device according to claim 6, characterized in that the calibration module is further used for: performing distortion restoration on the target internal parameters and target distortion coefficients of the fish-eye module to obtain a restored image; Detect the restored image based on the distortion correction image and the linearity of the outermost quadrilateral of each region.

9. A terminal device, characterized in that it includes: at least one processor and a memory; the memory stores a computer program; the at least one processor executes the computer program stored in the memory to implement the method for determining the internal parameters of the fisheye module according to any one of claims 1-5.

10. A computer-readable storage medium, characterized in that a computer program is stored in the computer-readable storage medium, and when the computer program is executed, the method for determining the internal parameters of the fisheye module according to any one of claims 1-5 is implemented.

Images