Camera calibration method, camera calibration system and computer readable storage medium

Abstract

The application discloses a camera calibration method, a camera calibration system and a computer readable storage medium, and relates to the field of image processing. The calibration method comprises the following steps: acquiring first image information, wherein the first image information is used for representing an image after coarse screening of connected domains of a preset calibration template image; performing connected domain screening according to an overlap ratio and an overlapping area of a minimum circumscribed rectangle and a positive circumscribed rectangle of each first reserved connected domain in the first image information, to obtain a plurality of second reserved connected domains; classifying and sorting the second reserved connected domains; determining corner point information according to a classification and sorting result; and obtaining a conversion relationship between a pixel coordinate system and a world coordinate system according to the corner point information. The camera calibration method improves the accuracy and efficiency of corner point detection through twice different connected domain screening, and optimizes the effect of camera calibration.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to a camera calibration method, a camera calibration system, and a computer-readable storage medium. Background Technology

[0002] One of the fundamental tasks of computer vision is to calculate the geometric information of objects in three-dimensional space from image information acquired by a camera, and thereby reconstruct and identify the objects. The relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image is determined by the geometric model of the camera imaging; these geometric model parameters are the camera parameters. By photographing a flat plate with a fixed-spacing pattern array and performing calibration algorithms, the camera's geometric model can be obtained, leading to high-precision measurement and reconstruction results. Whether in image measurement or machine vision applications, camera parameter calibration is a crucial step; the accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the camera's output. Therefore, proper camera calibration is a prerequisite for successful subsequent work.

[0003] Traditional camera calibration methods require a calibration template of known size. Calibration is achieved by establishing a correspondence between the coordinates of corner points on the calibration template and their coordinates in the camera-captured image of the template. Existing camera calibration methods often involve first performing morphological processing on the acquired calibration template image, and then performing corner detection on the result to obtain the coordinates of the corner points in the calibration template image. However, the morphologically processed image still contains a lot of interference information, which affects the efficiency and accuracy of corner detection, thus impacting the effectiveness of camera calibration. Summary of the Invention

[0004] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a camera calibration method, a camera calibration system, and a computer-readable storage medium, which can reduce interference information, thereby simplifying the calibration work and optimizing the camera calibration effect.

[0005] The camera calibration method according to the first aspect of this application includes the following steps:

[0006] Obtain first image information, which is used to characterize the image after coarse screening of connected components on a preset calibration template image;

[0007] Based on the overlap ratio and overlapping area of ​​the minimum bounding rectangle and the positive bounding rectangle of each first reserved connected component in the first image information, connected components are filtered to obtain several second reserved connected components.

[0008] The aforementioned second reserved connected components are classified and sorted.

[0009] The corner information of the several second reserved connected components is determined based on the classification and sorting results;

[0010] Based on the corner point information, the transformation relationship between the pixel coordinate system and the world coordinate system is obtained.

[0011] The camera calibration method according to the first aspect of this application has at least the following beneficial effects: after performing coarse connected component screening on the preset calibration template image, the interference of invalid connected components on corner detection can be reduced, thereby shortening the calculation time; then, the first reserved connected components are screened a second time based on the overlap ratio and overlapping area of ​​the minimum bounding rectangle and the positive bounding rectangle of each first reserved connected component, thereby further reducing the connected components that interfere with corner detection; wherein, the second reserved connected component is determined based on geometric methods, and the corner recognition accuracy is higher. Therefore, the camera calibration method of this application improves the accuracy and efficiency of corner detection through two different connected component screenings, thereby optimizing the effect of camera calibration.

[0012] According to some embodiments of the first aspect of this application, the step of filtering connected components based on the overlap ratio and overlapping area of ​​the minimum bounding rectangle and the positive bounding rectangle of each first reserved connected component in the first image information to obtain a plurality of second reserved connected components includes the following steps:

[0013] Calculate the overlap ratio of the minimum bounding rectangle and the positive bounding rectangle of each of the first reserved connected components;

[0014] The first reserved connected components with an overlap ratio within a preset range are retained and used as the third reserved connected components respectively;

[0015] Calculate the overlapping area of ​​the minimum bounding rectangle and the positive bounding rectangle of each of the third reserved connected components;

[0016] The third reserved connected components with overlapping areas greater than or equal to a preset value are retained and used as the second reserved connected components respectively.

[0017] According to some embodiments of the first aspect of this application, the preset range is 0.6-1.

[0018] According to some embodiments of the first aspect of this application, classifying and sorting the second reserved connected components includes the following steps:

[0019] Obtain the orientation angle of the calibration template image;

[0020] The second reserved connected component is divided into a set of boundary connected components and a set of internal connected components;

[0021] The boundary connected domain set and the internal connected domain set are sorted according to the direction angle.

[0022] According to some embodiments of the first aspect of this application, the orientation angle is determined by the following steps:

[0023] Based on the first image information, calculate the orientation angle of the nearest neighbor connected region of the feature pattern on the calibration template image;

[0024] The direction angle is taken as the direction angle of the nearest connected region of the feature pattern.

[0025] According to some embodiments of the first aspect of this application, obtaining the first image information includes the following steps:

[0026] Obtain the preset calibration template image;

[0027] The first image information is obtained by performing connected component filtering on the preset calibration template image using morphological processing.

[0028] According to some embodiments of the first aspect of this application, the step of using morphological processing to perform connected component filtering on the preset calibration template image to obtain the first image information includes the following steps:

[0029] The calibration template image is binarized to obtain a binary image;

[0030] By performing hit-and-miss processing on the binary image, the edge regions of the binary image are removed;

[0031] The binary image with the edge regions removed is subjected to erosion followed by dilation to obtain the original connected components on the calibration template image.

[0032] Calculate the length and width of the original connected component;

[0033] The original connected components whose length and width meet the requirements are retained and used as the first reserved connected components.

[0034] According to some embodiments of the first aspect of this application, obtaining the transformation relationship between the pixel coordinate system and the world coordinate system based on the corner information includes the following steps:

[0035] Subpixel detection calculations are performed on the corner information;

[0036] Based on the results of the sub-pixel detection calculation, the pixel coordinates of the corner points are obtained;

[0037] Based on the mapping relationship between the pixel coordinates and the corresponding world coordinates, the transformation relationship between the pixel coordinate system and the world coordinate system is obtained.

[0038] A camera calibration system according to a second aspect of this application includes a camera, a memory, a processor, and a computer program stored in the memory and executable on the processor. The camera is connected to the memory, and the processor executes the computer program to implement the camera calibration method as described in the first aspect of the embodiment.

[0039] The camera calibration system according to the embodiments of this application has at least the following beneficial effects:

[0040] The camera calibration system includes a camera, a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the camera calibration method as described in the first aspect embodiment. This method can filter the connected components of a preset calibration template image, accurately reducing invalid connected components. As a result, when determining corner information, it can reduce the interference caused by invalid connected components, thereby reducing the complexity of corner recognition, reducing the computation time of the corner recognition algorithm, and improving the speed and accuracy of corner recognition. This simplifies the calibration work and optimizes the effect of camera calibration.

[0041] A computer-readable storage medium according to a third aspect of this application stores computer-executable instructions for causing a computer to perform a camera calibration method as described in the first aspect embodiment. The computer-executable instructions can filter connected components of a preset calibration template image, accurately reducing invalid connected components. This reduces interference from invalid connected components when determining corner information, thereby reducing the complexity of corner recognition, reducing the computation time of the corner recognition algorithm, and improving the speed and accuracy of corner recognition. It also simplifies the calibration process and optimizes the camera calibration effect.

[0042] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application. Attached Figure Description

[0043] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0044] Figure 1 A flowchart illustrating a camera calibration method provided in one embodiment of this application;

[0045] Figure 2 A flowchart for obtaining first image information provided in one embodiment of this application;

[0046] Figure 3 A flowchart illustrating the process of filtering connected components of a preset calibration template image using morphological processing, provided as an embodiment of this application;

[0047] Figure 4 A flowchart for filtering connected components based on the overlap ratio and overlapping area of ​​the minimum bounding rectangle and the positive bounding rectangle of each first reserved connected component in the first image information, provided as an embodiment of this application;

[0048] Figure 5 A flowchart for classifying and sorting second reserved connected components is provided as an embodiment of this application;

[0049] Figure 6 A flowchart for determining an orientation angle is provided as an embodiment of this application;

[0050] Figure 7 A flowchart illustrating the transformation relationship between pixel coordinate system and world coordinate system based on corner information, provided as an embodiment of this application;

[0051] Figure 8 A schematic diagram of a system architecture for performing a camera calibration method according to an embodiment of this application;

[0052] Figure 9 This is a schematic diagram of a preset calibration template provided in one embodiment of this application.

[0053] Figure label:

[0054] Camera calibration system 100; processor 110; memory 120. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0056] It should be noted that although a logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than that shown in the flowchart. The terminology in the specification, claims, and the foregoing drawings is used to distinguish similar objects and is not necessarily used to describe a specific order or sequence.

[0057] In the description of this application, the use of "first" and "second" is for the purpose of distinguishing technical features only, and should not be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated or the order of the technical features indicated.

[0058] In the description of this application, unless otherwise expressly defined, terms such as "setup," "installation," and "connection" should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in this application in conjunction with the specific content of the technical solution.

[0059] One of the fundamental tasks of computer vision is to calculate the geometric information of objects in three-dimensional space from image information acquired by a camera, and thereby reconstruct and identify the objects. The relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image is determined by the geometric model of the camera imaging; these geometric model parameters are the camera parameters. By photographing a flat plate with a fixed-spacing pattern array and performing calibration algorithms, the camera's geometric model can be obtained, leading to high-precision measurement and reconstruction results. Whether in image measurement or machine vision applications, camera parameter calibration is a crucial step; the accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the camera's output. Therefore, proper camera calibration is a prerequisite for successful subsequent work.

[0060] Traditional camera calibration methods require a calibration template of known size. Calibration is achieved by establishing a correspondence between the coordinates of corner points on the calibration template and their coordinates in the camera-captured image of the template. Existing camera calibration methods often involve first performing morphological processing on the acquired calibration template image, and then performing corner detection on the result to obtain the coordinates of the corner points in the calibration template image. However, the morphologically processed image still contains a lot of interference information, which affects the efficiency and accuracy of corner detection, thus impacting the effectiveness of camera calibration.

[0061] Based on this, this application provides a camera calibration method, a camera calibration system 100, and a computer-readable storage medium, which can reduce interference information, improve the speed and accuracy of corner point recognition, thereby simplifying the calibration work and optimizing the effect of camera calibration.

[0062] The embodiments of this application will be further described below with reference to the accompanying drawings.

[0063] like Figure 8 As shown, Figure 8 This is a schematic diagram of a system architecture for performing a camera calibration method according to an embodiment of this application. Figure 8 In the example, the camera calibration system 100 includes a memory 120 and a processor 110, which can be connected via a bus or other means. Figure 8Taking a bus connection as an example, memory 120, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory 120 may include high-speed random access memory 120, and may also include non-transitory memory 120, such as at least one disk storage device 120, flash memory device, or other non-transitory solid-state memory 120. In some embodiments, memory 120 may optionally include memory 120 remotely located relative to processor 110, and these remote memories 120 can be connected to the system architecture platform via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0064] It will be understood by those skilled in the art that Figure 8 The schematic diagram of the system architecture shown in the figure does not constitute a limitation on the embodiments of this application. It may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0065] Reference Figure 1 , Figure 1 This is a flowchart of a camera calibration method provided in one embodiment of the present application. The camera calibration method includes, but is not limited to, steps S100 to S500.

[0066] Step S100: Obtain first image information, which is used to characterize the image after coarse screening of connected components on the preset calibration template image.

[0067] It should be noted that a connected region refers to an image region composed of foreground pixels with the same pixel value and adjacent positions.

[0068] It should be noted that, in one embodiment, the preset calibration template is a checkerboard calibration template, as shown below. Figure 9 As shown, the checkerboard pattern includes alternating rectangular black areas and rectangular white areas. Coarse connectivity filtering is used to eliminate unclear or significantly different connected components from the checkerboard area in the calibration template image, thereby reducing interference from invalid connected components in corner detection. Step S100 enables coarse corner detection of the calibration template image.

[0069] Step S200: Based on the overlap ratio and overlapping area of ​​the minimum bounding rectangle and the positive bounding rectangle of each first reserved connected component in the first image information, the connected components are filtered to obtain several second reserved connected components.

[0070] It should be noted that the first reserved connected component is the connected component remaining after step S100, which excludes unclear connected components and those with significantly different areas from the checkerboard pattern in the calibration template image. A second connected component filtering is performed on the first reserved connected components based on the overlap ratio and overlapping area of ​​the minimum and positive bounding rectangles of each first reserved connected component. This further reduces invalid connected components that interfere with corner detection. The overlap ratio and overlapping area of ​​the minimum and positive bounding rectangles of the first reserved connected components are determined using geometric methods; the algorithm is relatively simple but highly accurate. Through these two different connected component filtering processes, the interference of invalid connected components on corner detection is significantly reduced, thereby shortening computation time, improving the accuracy and efficiency of corner detection, and optimizing the camera calibration effect.

[0071] Step S300: Classify and sort several second reserved connected components.

[0072] It should be noted that classifying the second reserved connected components is to group connected components with similar features into one category and sort the second reserved connected components of the same category. This can reduce the influence between different types of connected components and improve the accuracy of corner detection.

[0073] Step S400: Determine the corner information of several second reserved connected components based on the classification and sorting results.

[0074] It should be noted that, in one embodiment, the preset calibration template is a checkerboard calibration template, on which alternating rectangular black areas and rectangular white areas are provided. The corner points on the calibration template are the intersections of the rectangular black and rectangular white areas or the intersections of the two diagonals of the checkerboard. Those skilled in the art can selectively set the corner points according to actual needs.

[0075] Step S500: Based on the corner information, obtain the transformation relationship between the pixel coordinate system and the preset world coordinate system.

[0076] It should be noted that, in one embodiment, the corner information represents the coordinates of the corner points in the pixel coordinate system. The coordinate transformation relationship between the pixel coordinate system and the world coordinate system can be calculated based on the corner point coordinates in both systems. The world coordinate system is predefined and its coordinates can be calculated for each corner point. Specifically, a right-handed Cartesian coordinate system can be constructed as the world coordinate system, using the calibration template plane as the XY plane and the vertical direction of the calibration template as the Z-axis.

[0077] It should be noted that the preset calibration template image is obtained by capturing the preset calibration template with a camera, and the calibration template image includes several images of the calibration template at different positions and angles. Each image is considered a unit, and corner detection is performed on each unit; that is, steps S100 to S400 are performed on each image to filter out the second reserved connected components in each image, classify and sort the second reserved connected components in each image, and then determine the corner information in each image based on the classification and sorting results. In the subsequent step S500, the transformation relationship between the pixel coordinate system and the preset world coordinate system is obtained based on the corner information. The corner information used is the corner information detected from all images in the calibration template image.

[0078] Reference Figure 4 , Figure 4 This is a flowchart of a method for filtering connected components based on the overlap ratio and overlapping area of ​​the minimum bounding rectangle and the positive bounding rectangle of each first reserved connected component in the first image information, according to an embodiment of this application. In step S200, the connected components are filtered based on the overlap ratio and overlapping area of ​​the minimum bounding rectangle and the positive bounding rectangle of each first reserved connected component in the first image information to obtain a number of second reserved connected components, including but not limited to steps S210 to S240.

[0079] Step S210: Calculate the overlap ratio of the minimum bounding rectangle and the positive bounding rectangle of each first reserved connected component.

[0080] Step S220: Retain the first reserved connected components with an overlap ratio within a preset range and use them as the third reserved connected components respectively.

[0081] It should be noted that, in one embodiment, the preset range of the overlap ratio is 0.6-1. The preset calibration template is a checkerboard calibration template, where the rectangular black area and the rectangular white area each correspond to a checkerboard square. The larger the overlap ratio, the more likely the first reserved connected component is a connected component of the corresponding checkerboard square, thus allowing for more accurate exclusion of invalid connected components. It is understood that the preset range of the overlap ratio can be modified according to requirements.

[0082] Step S230: Calculate the overlapping area of ​​the minimum bounding rectangle and the positive bounding rectangle of each third reserved connected component.

[0083] Step S240: Retain the third reserved connected components with overlapping areas greater than or equal to the preset value and use them as the second reserved connected components respectively.

[0084] It should be noted that, in one embodiment, the preset calibration template is a chessboard calibration template, where both the rectangular black area and the rectangular white area correspond to a chessboard square. The preset value of the overlapping area is three-quarters of the area of ​​a single chessboard square. The larger the overlapping area, the more likely the first reserved connected component is a connected component of the corresponding chessboard square, thus further eliminating invalid connected components.

[0085] In one embodiment, after steps S210 to S240, a second reserved connected component is ultimately retained, where the overlap ratio of the circumscribed rectangle and the positive circumscribed rectangle is within a preset range, and the overlapping area is greater than or equal to a preset value. This allows for further connected component filtering of the calibration template image to perform precise corner detection on the calibration template. Subsequently, based on the results of the precise corner detection, classification, sorting, determination of corner coordinates, and calculation of the transformation relationship between the pixel coordinate system and the preset world coordinate system are performed. It is understood that the preset range of the overlap ratio and the preset value of the overlapping area can be adjusted according to actual needs, and are not specifically limited here.

[0086] Reference Figure 5 , Figure 5 The flowchart for classifying and sorting the second reserved connected components is provided in one embodiment of this application; the classification and sorting of the second reserved connected components in step S300 includes, but is not limited to, steps S310 to S330.

[0087] Step S310: Obtain the orientation angle of the calibration template image.

[0088] Step S320: Divide the second reserved connected component into a set of boundary connected components and a set of internal connected components.

[0089] It should be noted that the set of boundary connected components is a collection of boundary connected components; the set of internal connected components is a collection of internal connected components. In one embodiment, the third connected component is divided into a set of boundary connected components and a set of internal connected components based on whether each of its four vertices has a nearest neighbor vertex. An internal connected component is a third connected component whose four vertices have a nearest neighbor vertex; other third connected components are boundary connected components. A nearest neighbor vertex represents a vertex of another third connected component. The third connected component can also be divided based on the number of neighboring third connected components; a third connected component with four neighboring third connected components is an internal connected component; a third connected component with two neighboring third connected components is a boundary connected component.

[0090] Step S330: Sort the boundary connected domain set and the internal connected domain set according to the direction angle.

[0091] It should be noted that, in one embodiment, the direction of the orientation angle is used as the sorting direction, and the second reserved connected components are sorted according to the distance from the second reserved connected component to the starting point; here, the starting point is a feature point on a preset calibration template.

[0092] Reference Figure 6 , Figure 6 This is a flowchart illustrating the determination of an orientation angle according to an embodiment of this application. Step S310 involves obtaining the orientation angle of the calibration template image, including but not limited to steps S311 and S312.

[0093] Step S311: Calculate the orientation angle of the nearest connected region of the feature pattern on the calibration template image based on the first image information.

[0094] Step S312: Use the orientation angle of the nearest connected region of the feature pattern as the orientation angle.

[0095] It should be noted that, in one embodiment, the preset calibration template is a chessboard calibration template, wherein the combination of rectangular black areas and rectangular white areas to form a chessboard corresponds to one chessboard square on the chessboard. (Refer to...) Figure 9 As shown, black cross patterns are set as feature patterns on the white chessboard squares in the lower right and lower left corners of the chessboard. When calculating the direction angle of the nearest connected component of the feature pattern on the calibration template image, the area of ​​the black pattern on each second reserved connected component is first calculated. The pattern whose area matches the area of ​​the cross pattern is the feature pattern. The nearest connected component to the intersection point of the cross pattern is determined, resulting in two nearest connected components (i.e., the nearest connected component to the center point of the feature pattern). The average of the direction angles of the two nearest connected components is used as the direction angle for arrangement. Setting feature patterns facilitates the determination of the direction angle of the calibration template image. Furthermore, by setting two feature patterns, two nearest connected components can be obtained. Using the average of the direction angles of the two nearest connected components as the direction angle for arrangement can improve the accuracy of the direction angle calculation.

[0096] It should be noted that the feature pattern can also be other shapes, such as circular black blocks or square black blocks.

[0097] Reference Figure 2 , Figure 2 This is a flowchart illustrating the acquisition of first image information according to an embodiment of this application. The acquisition of first image information in step S100 includes, but is not limited to, steps S110 and S120.

[0098] Step S110: Obtain the preset calibration template image.

[0099] Step S120: Use morphological processing to filter the connected components of the preset calibration template image to obtain the first image information.

[0100] Reference Figure 2 , Figure 2 This is a flowchart illustrating the acquisition of first image information according to an embodiment of this application. Step S120 involves using morphological processing to filter connected components of a preset calibration template image to obtain the first image information, including but not limited to steps S121 to S125.

[0101] Step S121: Binarize the calibration template image to obtain a binary image;

[0102] Step S122: Remove edge regions of the binary image by performing hit / miss processing on the binary image;

[0103] Step S123: Perform erosion and dilation operations on the binary image with the edge regions removed to obtain the original connected components on the calibration template image;

[0104] Step S124: Calculate the length and width of the original connected component;

[0105] Step S125: Retain the original connected components whose length and width meet the requirements and use them as the first reserved connected components.

[0106] It should be noted that the original connected components can be extracted from the calibration template image through binarization, hit / miss processing, and erosion followed by dilation.

[0107] In one embodiment, the preset calibration template is a checkerboard calibration template, on which alternating rectangular black areas and rectangular white areas are arranged, and all checkerboard squares have the same length and width. The original connected components include connected components corresponding to the checkerboard squares and invalid connected components.

[0108] When filtering connected components in the calibration template image, original connected components with lengths and widths 0.7 to 1.3 times that of the checkerboard grid are retained as the first reserved connected components in the first image information. This is to exclude invalid connected components in the original connected components that differ significantly from the length and width features of the checkerboard grid. The closer the preset range of length and width is to the length and width features in the calibration template image, the more closely the retained first reserved connected components fit the rectangular black area and / or rectangular white area.

[0109] It is understandable that when filtering connected components in the calibration template image, features for comparison can be selected based on the preset calibration template, such as a combination of one or more of the length, width, and area features. Furthermore, preset values ​​for these features can be set according to the actual conditions of the preset calibration template. No specific limitations are made here regarding the specific features to be compared or their preset values.

[0110] Reference Figure 7 , Figure 7 This is a flowchart illustrating the transformation relationship between the pixel coordinate system and the world coordinate system based on corner information, according to one embodiment of this application. Step S500, which involves obtaining the transformation relationship between the pixel coordinate system and the world coordinate system based on corner information, includes, but is not limited to, steps S510 to S530.

[0111] Step S510: Perform sub-pixel detection calculation on the corner information.

[0112] It should be noted that corner information represents the pixel coordinates of the corner, which correspond to the corner's coordinates in the pixel coordinate system. Sub-pixel detection is employed, meaning that based on the identified corner's pixel coordinates, the pixel coordinates are iterated repeatedly to obtain the corner's pixel coordinate values ​​accurate to the sub-pixel level, so that the corner's pixel coordinates can more accurately correspond to the corner's coordinates in the pixel coordinate system.

[0113] Step S520: Obtain the pixel coordinates of the corner points based on the results of the sub-pixel detection operation.

[0114] It should be noted that, in one embodiment, the results of the subpixel detection operation are subjected to weight removal operation or least squares operation to obtain the pixel coordinates used in subsequent steps, so that the pixel coordinates of the corner point can more accurately correspond to the coordinates of the corner point in the pixel coordinate system.

[0115] Step S530: Based on the mapping relationship between pixel coordinates and the corresponding world coordinates, obtain the transformation relationship between the pixel coordinate system and the world coordinate system.

[0116] It should be noted that, in one embodiment, pixel coordinates refer to the coordinates of a corner point in the pixel coordinate system. The world coordinate system is fixed on the calibration template. Since the world coordinate system of the calibration template is predefined, the size of each grid on the calibration template is known, and the physical coordinates of each corner point in the world coordinate system can be calculated. Based on the pixel coordinates of each corner point and the physical coordinates of each corner point in the world coordinate system, the mapping relationship between the pixel coordinates and the corresponding world coordinates is calculated.

[0117] In one embodiment, this application also provides a camera calibration system 100, including a camera, a memory 120, a processor 110, and a computer program stored on the memory 120 and executable on the processor 110. The camera is connected to the memory 120, and the processor 110 executes the computer program to implement the camera calibration method as described in the first aspect embodiment. It can filter the connected components of a preset calibration template image, accurately reducing invalid connected components. This reduces interference from invalid connected components when determining corner information, thereby reducing the complexity of corner recognition, reducing the computation time of the corner recognition algorithm, and improving the speed and accuracy of corner recognition. It simplifies the calibration work and optimizes the effect of camera calibration.

[0118] Furthermore, one embodiment of this application provides a computer-readable storage medium storing computer-executable instructions for causing a computer to execute the camera calibration method as described in the first aspect embodiment. This method can filter connected components of a preset calibration template image, accurately reducing invalid connected components. This reduces interference from invalid connected components when determining corner information, thereby reducing the complexity of corner recognition, decreasing the computation time of the corner recognition algorithm, and improving the speed and accuracy of corner recognition. It simplifies the calibration process and optimizes the camera calibration effect.

[0119] For example, when executed by a processor 110 in the above-described camera calibration system 100 embodiment, the processor 110 can execute the camera calibration method described above, for example, the method described above. Figure 1 Method steps S100 to S500 Figure 2 Method steps S110 and S120, Figure 3 Method steps S121 to S125 in the above method Figure 4 Method steps S210 to S240, Figure 5 Method steps S310 to S330 in the text Figure 6 Method steps S311 to S312 in the text Figure 7 Method steps S510 to S530.

[0120] The camera calibration method of this application is described below with reference to an embodiment.

[0121] The preset calibration template is a checkerboard calibration template, which consists of alternating rectangular black areas and rectangular white areas, with each checkerboard square having equal length and width. Several images of the calibration template at different positions and angles are captured by a camera to form the calibration template image.

[0122] For one of the calibration template images, the following operations are performed: binarization, hit / miss processing, and erosion followed by dilation are applied to obtain the connected components in the calibration template image after edge removal, which are then used as the original connected components. The length and width features of the original connected components are compared with preset values, and those that meet the length and width requirements are retained as the first reserved connected components in the first image information, thus eliminating invalid connected components in the calibration template image whose length and width features differ significantly from the checkerboard pattern. The overlap ratio of the minimum bounding rectangle and the positive bounding rectangle of each first reserved connected component is calculated. For example, retain the first reserved connected components whose overlap ratio between the minimum bounding rectangle and the positive bounding rectangle is within 0.6-1, and use them as the third reserved connected components; calculate the overlapping area of ​​the minimum bounding rectangle and the positive bounding rectangle of each third reserved connected component; retain the third reserved connected components whose overlapping area is greater than or equal to three-quarters of the checkerboard area, and use them as the second reserved connected components; calculate the position of the cross pattern on the image; sort the second reserved connected components according to the direction angle of the nearest connected component of the cross pattern; use the intersection of the third connected components as the corner points of the calibration template. Use the above steps to detect each image and obtain the position information of the corner points in each image.

[0123] Then perform the following operations:

[0124] Sub-pixel detection is performed on the corner point location information; then, weight removal is performed on the sub-pixel detection results to obtain the optimal result; the pixel coordinates of the corner point in the image corresponding to the optimal result are retained as pixel coordinates, and the mapping relationship between the pixel coordinates and the corresponding world coordinates is calculated to obtain the transformation relationship between the pixel coordinate system and the world coordinate system; subsequently, the camera model parameters are calculated using the transformation relationship to complete the calibration. Calculating the camera model parameters based on the transformation relationship is existing technology and will not be further elaborated here. This embodiment calculates the overlap ratio and overlapping area of ​​each first reserved connected component through simple geometric operations and performs filtering, which can accurately reduce invalid connected components. Therefore, when determining corner point information, it can reduce the interference caused by invalid connected components, thereby reducing the complexity of corner point recognition, reducing the computation time of the corner point recognition algorithm, and improving the speed and accuracy of corner point recognition; it simplifies the calibration work and optimizes the camera calibration effect.

[0125] It will be understood by those skilled in the art that all or some of the steps and systems in the methods disclosed above can be implemented as software, firmware, hardware, and suitable combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, as is known to those skilled in the art, communication media typically contain computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0126] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A camera calibration method, characterized by, The method comprises the following steps: obtaining first image information, the first image information being used to represent an image after a coarse screening of connected domains of a preset calibration template image; performing connected domain screening according to an overlap ratio and an overlapping area of a minimum circumscribed rectangle and a circumscribed rectangle of each first reserved connected domain in the first image information, to obtain a plurality of second reserved connected domains; the overlap ratio of the minimum circumscribed rectangle of the second reserved connected domain and the corresponding circumscribed rectangle is within a preset range; the overlapping area between the minimum circumscribed rectangle of the second reserved connected domain and the corresponding circumscribed rectangle is greater than or equal to a preset value; classifying and sorting the plurality of second reserved connected domains; determining corner point information of the plurality of second reserved connected domains according to the classification and sorting results; the corner point information represents an intersection of two diagonal lines of a chessboard grid on a chessboard grid calibration template; obtaining a conversion relationship between a pixel coordinate system and a world coordinate system according to the corner point information; classifying and sorting the plurality of second reserved connected domains, comprising the following steps: obtaining a direction angle of the calibration template image; dividing the second reserved connected domains into a boundary connected domain set and an internal connected domain set; the boundary connected domain set is a set of boundary connected domains; the internal connected domain set is a set of internal connected domains; an internal connected domain represents a second connected domain whose four vertices all have a nearest-neighbor second connected domain; a boundary connected domain is a second connected domain other than the internal connected domains; sorting the boundary connected domain set and the internal connected domain set according to the direction angle, to classify connected domains with similar features into a category, and to sort second reserved connected domains in the same category.

2. The camera calibration method of claim 1, wherein, The connected domain screening according to the overlap ratio and the overlapping area of the minimum circumscribed rectangle and the circumscribed rectangle of each first reserved connected domain in the first image information, to obtain a plurality of second reserved connected domains, comprises the following steps: calculating the overlap ratio of the minimum circumscribed rectangle and the circumscribed rectangle of each first reserved connected domain; reserving the first reserved connected domains with the overlap ratio within a preset range as third reserved connected domains; calculating the overlapping area of the minimum circumscribed rectangle and the circumscribed rectangle of each third reserved connected domain; reserving the third reserved connected domains with the overlapping area greater than or equal to a preset value as the second reserved connected domains.

3. The camera calibration method of claim 2, wherein, The preset range is 0.6-1.

4. The camera calibration method of claim 1, wherein, The direction angle is determined by the following steps: calculating a direction angle of a nearest-neighbor connected domain of a feature pattern on the calibration template image according to the first image information; taking the direction angle of the nearest-neighbor connected domain as the direction angle of the calibration template image.

5. The camera calibration method of any one of claims 1 to 4, wherein, The first image information is obtained by the following steps: obtaining the calibration template image; performing connected domain screening on the calibration template image by using morphological processing, to obtain the first image information.

6. The camera calibration method of claim 5, wherein, The connected domain screening on the calibration template image by using morphological processing, to obtain the first image information, comprises the following steps: performing binaryzation processing on the calibration template image, to obtain a binary image; removing an edge region of the binary image by performing hit-miss processing on the binary image; The binary image with the edge region removed is subjected to a first-erosion-then-dilation operation to obtain a raw connected domain on the calibration template image; The length and width of the raw connected domain are calculated; The raw connected domain with the length and width meeting the requirements is reserved as the first reserved connected domain.

7. The camera calibration method of any one of claims 1 to 4, wherein, The conversion relationship between the pixel coordinate system and the world coordinate system according to the corner point information comprises the following steps: The corner point information is subjected to a sub-pixel detection operation; The pixel coordinates of the corner point are obtained according to the result of the sub-pixel detection operation; The conversion relationship between the pixel coordinate system and the world coordinate system is obtained according to the mapping relationship between the pixel coordinates and the corresponding world coordinates.

8. A camera calibration system characterized by, It comprises: A camera, a memory, a processor and a computer program stored on the memory and executable on the processor, the camera being connected to the memory, and the processor implementing the camera calibration method according to any one of claims 1 to 7 when executing the computer program.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer executable instructions for causing a computer to execute the camera calibration method according to any one of claims 1 to 7.

Images