A method for extracting feature regions of a calibration board, a calibration method, and a calibration board

Abstract

The present application provides a method for extracting feature regions of a calibration board, a calibration method, and a calibration board. The calibration board includes: five reference dots and a plurality of reference dots, and the dot regions and the background color are complementary colors; among them, among the five reference dots, the connection lines of four of the outer reference dots form a quadrilateral with a perspective relationship, serving as the reference for all reference dots, and the outer reference dots are arranged in the central region of the calibration board; the fifth reference point is located at the middle position between the two lower outer reference dots and is set as the origin of the calibration board coordinate system, used to record the pose information of the calibration board and the sorting information of the reference dots. The present application can avoid the complex operation of manually selecting four corner points of the image, improve the automation degree of calibration, has good robustness, can well adaptively and accurately extract the target region, and can improve the positioning accuracy of feature points by using the ellipse fitting algorithm based on the contour.

Description

Technical Field

[0001] The present application relates to the technical field of computer vision - camera calibration, and particularly to a method for extracting feature regions of a calibration board, a calibration method, and a calibration board. Background Art

[0002] Computer vision is a discipline on how to use cameras and computers to obtain the data and information of the objects to be photographed that we need, and camera calibration is to establish the mapping relationship between the pixel positions of the camera image and the corresponding objects in the world coordinate system. Therefore, camera calibration is the basis of computer vision, and the accuracy of the calibration result will directly affect the positioning accuracy of the entire vision system.

[0003] Currently, the mainstream camera calibration boards can be divided into checkerboard calibration boards and circular calibration boards according to the feature patterns thereon (as Figure 1 shown). Compared with the checkerboard calibration board, the dot calibration board has the following advantages:

[0004] ① It has good anti - noise and anti - blur performance.

[0005] ② When processing projector calibration, it can avoid the decrease in detection accuracy caused by the sharp change of black and white pixels at the corners of the checkerboard.

[0006] The calibration board types can also be divided into two categories according to whether they have a reference:

[0007] ① For example, a method for extracting feature points of a circular array calibration board (application number CN201710445416.8) in CN107274454B is a calibration board without a reference. When extracting features, it is necessary to manually select a reference, with low automation and inconvenient use. And due to the absence of a reference, there is no direction information, which is not suitable for calibrating multiple devices and the hand - eye calibration system (including mobile and fixed types) associated with actuators (robots).

[0008] ② For example, a camera calibration board and a method for collecting camera calibration data (application number CN201811147388.2) in CN109285194B is a calibration board with a reference, and it will not cause the solution failure of the mapping points due to partial occlusion of the calibration board or some areas exceeding the imaging range.

[0009] Currently, the mainstream feature point extraction algorithms for circular calibration boards are:

[0010] ① findCirclesGrid, HoughCircle, BlobDetector, SimpleBlobDetector, etc. in OpenCV.

[0011] ② Spot detection algorithms such as Difference of Gaussian, Laplacian of Gaussian, and Determinant of Hessian.

[0012] ③ Ellipse fitting algorithm based on contour trajectory. Summary of the Invention

[0013] Based on the above objectives, the present application proposes a calibration board, including:

[0014] Five reference dots and multiple reference dots, and the dot areas and the background color are opposite to each other; among them,

[0015] Among the five reference dots, the connection lines of four of the outer reference dots form a quadrilateral with perspective relationship, serving as the reference for all reference dots, and the outer reference dots are arranged in the central area of the calibration board; the fifth reference dot is located in the middle position between the two lower outer reference dots, and is set as the origin of the calibration board coordinate system, which is used to record the pose information of the calibration board and the sorting information of the reference dots.

[0016] Further, the reference dots on the calibration board are arranged in an array, with each row parallel, each column parallel, perpendicular to each other between rows and columns, and the reference dots are equally spaced from each other.

[0017] The present application also proposes a method for extracting the feature area of the calibration board using the above calibration board, including:

[0018] Place the calibration board within the field of view of the CCD camera, collect the calibration board images, and record multiple groups of images by changing the relative position and pose between the calibration board and the camera;

[0019] Perform binary processing and filtering operations on each group of images;

[0020] Use the Canny operator to extract the contours of each group of images, and select the inner contours according to the hierarchical relationship of the contours. Perform ellipse fitting based on the least squares method on the inner contours to obtain the ellipse contours;

[0021] Filter the ellipse contours;

[0022] According to the size of the filtered ellipse, calculate the average diameter of the reference dots and the reference dots, and calculate the adaptive proportional coefficient of morphology;

[0023] Perform morphological opening and closing operations on the circular features to obtain the region of interest, which is the preliminary exploration region of the calibration board;

[0024] Find the reference points in the preliminary exploration area, and calculate the centroid of the calibration board based on the 4 reference points on the periphery. According to the proportional relationship between the centroid of the ideal calibration board and the reference points and the four outermost corner points of the calibration board, scale up the reference points in the image by pixel distance, so as to calculate the accurate calibration board area.

[0025] For a mobile camera, that is, the camera is mounted on the actuator. When collecting images, the pose of the calibration board does not change. After each calibration board image is collected, control the pose change of the actuator, so as to realize the change of the relative pose between the camera and the calibration board.

[0026] For a fixed camera, that is, the position and pose of the camera are fixed. Each time when collecting images, change the position and pose of the calibration board.

[0027] Further, after obtaining the area where the calibration board is located, it further includes:

[0028] When the calibration board exceeds the field of view, perform boundary processing on the region of interest.

[0029] This application also provides a calibration method for the calibration board, including:

[0030] Use the above method to perform ellipse fitting based on contour features on the area where the calibration board is located, and obtain all the fitted ellipse contours on the calibration board area;

[0031] Calculate the ellipse position corresponding to the reference dot according to the dimension information, and judge the pose of the origin coordinate system of the calibration board and the positions of the four reference ellipses according to the cosine value of the vector angle;

[0032] According to the pose information of the calibration board coordinate system, sort the reference dots to obtain the image coordinates corresponding one by one to the world coordinates;

[0033] Complete camera calibration according to Zhang's calibration method.

[0034] Further, the method for sorting the reference dots includes:

[0035] Perform affine transformation on the reference dots, and then sort them in the way that the row and column coordinate values increase monotonically; or,

[0036] On the basis of the original image, according to the centroid sorting method of the vector angle.

[0037] Generally speaking, the advantages of this application and the experience brought to users are:

[0038] 1. Improve the automation degree of calibration

[0039] ① Compared with the existing calibration algorithms, through the adaptive area extraction algorithm of the calibration board, the complex operation of manually selecting the four corner points of the image is avoided.

[0040] ② By introducing a coordinate system, the directional problem of feature point sorting is solved, and the degree of automation is improved.

[0041] 2. Good robustness

[0042] ① In the case where the shooting background where the calibration board is located is cluttered and the color of the environmental background is close to or the same as the color of the calibration board background, the target area can still be accurately extracted well adaptively.

[0043] ② In the case where the calibration board has partial occlusion or partially exceeds the camera's field of view, the feature points of the target area can still be extracted.

[0044] ③ In the case where there is an angle between the plane where the calibration board is located and the image plane of the camera, the feature points of the target area can still be extracted.

[0045] 3. Ellipse fitting algorithm based on contour features

[0046] When there is an angle between the plane where the calibration board is located and the image plane of the camera, affected by the perspective transformation, the shape of the feature circular dots on the calibration board changes. Using the ellipse fitting algorithm based on the contour can improve the positioning accuracy of the feature points. Description of the drawings

[0047] In the drawings, unless otherwise specified, the same reference numerals throughout the several drawings denote the same or similar components or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed according to the present application and should not be regarded as limiting the scope of the present application.

[0048] Att Figure 1 Schematic diagrams of the checkerboard calibration board (left) and the circular calibration board (right) of the prior art.

[0049] Att Figure 2 Schematic diagram of a circular calibration board with a reference in the present application.

[0050] Att Figure 3 Schematic diagrams of the angle change between the calibration board and the camera plane and the extraction result schematic diagram.

[0051] Att Figure 4 Schematic diagram of the extraction result of the adaptive area of the calibration board exceeding the field of view.

[0052] Att Figure 5 Schematic diagram of the ellipse fitting image based on the graphic features (left: graphic contour, right: ellipse contour).

[0053] Att Figure 6 Schematic diagram of the center point sorting based on the vector angle.

[0054] Appendix Figure 7 It is a schematic diagram of the sorting result (left: before sorting; right: after sorting).

[0055] Appendix Figure 8 It is a flowchart for adaptively extracting the region of interest of the calibration board in this application.

[0056] Appendix Figure 9 It is a flowchart of a calibration method based on ellipse fitting in this application.

[0057] Appendix Figure 10 It shows a schematic diagram of the structure of an electronic device provided by an embodiment of this application;

[0058] Appendix Figure 11 It shows a schematic diagram of a storage medium provided by an embodiment of this application. Detailed implementation manners

[0059] The following further elaborates on this application in conjunction with the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the relevant invention and are not intended to limit the invention. Additionally, it should be noted that for ease of description, only parts related to the relevant invention are shown in the drawings.

[0060] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The following will elaborate on this application in detail with reference to the drawings and embodiments.

[0061] The objectives of this application include the following four points:

[0062] 1. Realize the functions of fully automated extraction of the target area of the calibration board and sorting of feature dots.

[0063] 2. Solve the problem that the traditional dot calibration algorithm fails to solve the mapping points due to occlusion in some areas or some areas exceeding the imaging range.

[0064] 3. Improve the robustness of feature point dot extraction:

[0065] When there is an angle between the plane where the calibration board is located and the image plane of the camera, the mainstream circular detection algorithm and blob detection algorithm based on OpenCV will fail to recognize the feature dots. Using the ellipse fitting algorithm based on the contour trajectory can well improve the robustness of feature point dot extraction.

[0066] 4. Improve the positioning accuracy of the reference dots

[0067] When there is an angle between the plane where the calibration board is located and the image plane of the camera, using the ellipse fitting algorithm based on the contour will improve the positioning accuracy of the feature dots.

[0068] The core of this application is to provide a circular calibration board with a reference, as well as a calibration board adaptive region extraction algorithm and an ellipse fitting algorithm based on the feature contours on the calibration board. The technical solutions in the embodiments of this application will be described below in conjunction with the accompanying drawings in the embodiments of this application (it should be noted that the protected content of this application is not limited to the following description scope):

[0069] Embodiment 1, a circular calibration board with a reference.

[0070] As Figure 2 shown is the illustration of the calibration board used in this application. The calibration board consists of the following aspects: it is composed of 5 reference dots and several reference dots (the number and arrangement shape of the reference points are not limited), and the dot area and the background color are opposite colors. Among the 5 reference dots, the connection lines of 4 of the outer reference dots can form a quadrilateral with a perspective relationship, which is the reference for all reference points and is arranged in the central area of the calibration board. And the fifth reference point is set as the origin of the calibration board coordinate system, which is used to record the pose information of the calibration board and the sorting information of the reference points. And the reference points on the calibration board are arranged in an array, with each row (column) parallel and perpendicular to each other between rows and columns, and the distance between reference points is equal.

[0071] Embodiment 2, extraction of the calibration board adaptive region.

[0072] Step1: Place the calibration board within the field of view of a CCD (Charge Coupled Device) camera, and collect calibration board images. By changing the relative position and pose of the calibration board and the camera, multiple groups of images are recorded. Due to different camera arrangement methods, it can be divided into mobile and fixed types:

[0073] For the mobile type, the camera is mounted on an actuator (robot arm). When collecting images, it is necessary to ensure that the pose of the calibration board does not change. After each calibration board image is collected, it is necessary to control the change of the pose of the actuator (robot arm) to achieve the change of the relative pose between the camera and the calibration board.

[0074] For the fixed type, the position and pose of the camera are fixed. Each time when collecting images, it is necessary to change the position and pose of the calibration board.

[0075] Step2: Use OTSU (Otsu method - maximum between-class variance method) to binarize the image. Assume that there is a threshold T in the image. By judging the size relationship between each pixel in the image and T, the pixels can be divided into two categories: background B and foreground F. When the optimal T threshold is selected, the difference between the background and the foreground is the largest. The implementation steps are as follows:

[0076] 1. Select the segmentation threshold T, and count the proportion p0 of the number of foreground pixel values less than T among all pixel points and the average gray value a0. Similarly, count p1 and a1 of the background pixels greater than T;

[0077] 2. Calculate the average gray value of all pixels;

[0078] A g = p0a0 + p1a1

[0079] 3. Calculate the between-class variance

[0080] g = p0(p0 - A g ) 2 + p1(a1 - A g ) 2

[0081] 4. Traverse all segmentation thresholds T and repeat steps 1 - 3 to obtain the threshold T with the maximum between-class variance g.

[0082] Step 3: Use an efficient median filtering function to eliminate salt-and-pepper noise in the image. Traditional filters need to perform pixel sorting according to the filter range at each pixel position, which is very time-consuming. When the window moves one column (row) along the row (column), the change in the window content is just that a new column (row) on the right replaces a column (row) on the left. For an m*n filter, (m*n - 2m) pixels do not change.

[0083] 1. Calculate the median position, where ceil is the ceiling function and t is the median position:

[0084]

[0085] 2. Sort the window content and establish a pixel histogram H, and count the number n of pixels with brightness less than the median (i.e., the pixel value corresponding to the t position) m .

[0086] 3. Move the filter, and for each pixel p with brightness p g that is removed from the left, perform a decrement operation on H pg which is the histogram of pixel p:

[0087] H pg = H pg - 1

[0088] n m = n m - 1

[0089] 4. For each pixel p with brightness p g that is moved in, perform an increment operation

[0090] H pg = Hpg +1

[0091] n m = n m +1

[0092] 5. According to the relationship between n m and t, discuss in different cases and perform boundary judgment, and finally complete the efficient median filtering operation.

[0093] Step 4: Use the Canny operator of OPENCV to extract the contours of the image, and select the inner contours according to the hierarchical relationship of the contours. On this basis, perform ellipse fitting based on the least squares method on the contours to obtain the elliptical contour:

[0094] 1. Given the ellipse equation

[0095] ax 2 + bxy + cy 2 + dx + ey = 1

[0096] 2. Let a = [a, b, c, d, e] T , x = [x 2 , xy, y 2 , x, y] T , then the equation can be expressed as

[0097] ax = 1

[0098] 3. The ellipse fitting optimization problem can be expressed as

[0099] min ||Sa|| 2

[0100] Subject to a T Ca = 1

[0101] where S is the sample set, a is the ellipse equation parameter, and the constant matrix C is

[0102]

[0103] 4. According to the Lagrange multiplier method, introduce the Lagrange factor λ to obtain the following equation:

[0104] 2S T Sa - 2λCa = 0

[0105] 5. Solve the eigenvalues and eigenvectors of S T Sa = λCa (λ i , u i ), and finally solve to obtain

[0106]

[0107] 6. Let ai = μ i u i , take λ i > 0 corresponding eigenvector u i can be used as the solution of the curve fitting equation.

[0108] Step5: Filter the circular features on the calibration board to obtain a relatively pure contour map (it should be noted that although circular features are used in this application, the protection scope should not be limited by the geometric shape of the graph); The filtering method in this step can use common filtering algorithms, such as filtering operations based on area, aspect ratio, and outliers, etc., so as to eliminate the interference of the elliptical contours in the background environment on the elliptical contours on the calibration board. According to the calibration board parameters, take effective filtering means to filter out the inappropriate elliptical trajectories in the fitted image.

[0109] Step6: According to the size of the filtered ellipse, calculate the average diameter of the reference circle and the reference circle, so as to calculate the adaptive proportionality coefficient of morphology; In this step, calculate the adaptive size of the calibration board according to the reference elliptical points in the calibration board.

[0110] Step7:

[0111] Use the elementSizeChangeCLOSE() and elementSizeChangeOPEN() functions in OPENCV to perform morphological opening and closing operations on the circular features to obtain the region of interest, which is the preliminary exploration region of the calibration board; Find the reference points in the preliminary exploration region, and calculate the center of gravity of the calibration board according to the 4 reference points on the periphery. According to the proportional relationship between the center of gravity of the ideal calibration board and the reference points and the four outermost corner points of the calibration board, scale the reference points in the image by pixel distance, so as to calculate the accurate calibration board region. In this step, according to the adaptively adjusted size information, perform morphological opening-closing or closing-opening operations on the filtered elliptical image to obtain the region where the calibration board is located in the image. By this step, the extraction work of the calibration board has been completed. Further, the following steps can also be included:

[0112] Step8: Perform boundary processing on the region of interest to handle the situation where the calibration board exceeds the field of view:

[0113] 1. Calculate the calibration center of gravity x c , y c , where x i , y i are the coordinates of the reference circle points P1, P2, P3, P4:

[0114]

[0115]

[0116] 2. Calculate the distances from the reference origin points P1, P2, P3, and P4 to the centroid

[0117]

[0118] 3. According to the layout of the reference points on the calibration board, appropriately magnify the reference points according to d in step 2 i to obtain new points as the contour reference of the calibration board.

[0119] Thus, the automatic extraction of the region of interest of the calibration board is completed, and the process is as Figure 8 shown.

[0120] The adaptive region extraction algorithm of the calibration board based on graphic features in this application has high robustness:

[0121] ① It can be applied to the situation where the shooting background is cluttered and the environmental background color is close to or the same as the calibration board background color, and still can extract the feature points of the target region.

[0122] ② In the case where there is an angle between the plane where the calibration board is located and the image plane of the camera, it can still extract the feature points of the target region (as Figure 3 shown).

[0123] ③ In the case where the calibration board has partial occlusion or partially exceeds the camera's field of view, it can still extract the feature points of the target region (as Figure 4 shown).

[0124] Example 3, the calibration process based on the ellipse fitting method of the graphic contour, as Figure 9 shown, includes the following steps:

[0125] Step1: Perform ellipse fitting based on contour features on the region where the calibration board is located to obtain all the fitted ellipse contours on the calibration board region (the implementation process refers to step 4 of Example 2); Attached Figure 5 is a schematic diagram of the ellipse fitting image based on graphic features (left: graphic contour, right: ellipse contour). The ellipse fitting algorithm based on contour features can, ① improve the positioning accuracy of feature points. ② In the case where there is an angle between the plane where the calibration board is located and the image plane of the camera, improve the robustness of the recognition of the reference circular points (as Figure 5 shown).

[0126] Step2: Calculate the ellipse positions corresponding to the reference origin points according to the dimension information, and judge the pose of the origin coordinate system of the calibration board and the positions of the four reference ellipses according to the cosine value of the vector angle;

[0127] 1. Cosine value of the vector angle

[0128]

[0129] In the formula, dot() is the dot product of vectors, and cross() is the cross product of vectors.

[0130] 2. Calculate the midpoint P from two vectors with θ = 0 or 180, which is the origin of the calibration board.

[0131] Step3: The reference point sorting method can be divided into two categories. One is to perform an affine transformation on the reference dot circles and then sort them in ascending order of the row (column) coordinate values. The other is to sort them according to the center of the vector angle on the basis of the original image. Finally, the image coordinates corresponding one by one to the world coordinates can be obtained;

[0132] Center sorting method (as shown in Figure 6 )

[0133] 1. Find the feature points on the boundaries AB and AC and sort them according to the length of the distance between two points. The points on AC are denoted as V i (i = 1, 2,.., n), and the points on AB are denoted as H i (i = 1, 2,.., n);

[0134] 2. Taking AB (AC) as the reference, with V i (H i ) as the starting point, select any point P in the center dot matrix except for the boundaries AB and AC i , and calculate the angle θ i P i (H i P i ) between the vector and AB (AC) vi (θ Hi );

[0135]

[0136]

[0137] 3. Select the group with the smallest value of θ vi (θ Hi ), and sort the reference dot circles in ascending order of P i V i (P i H i ). Attached Figure 7 is a schematic diagram of the sorting result (left: before sorting, right: after sorting).

[0138] Affine transformation method:

[0139] 1. Solve the affine transformation matrix according to the reference dot circle coordinate system and the calibration board workpiece coordinate system

[0140]

[0141] a i , where {i = 1, 2, 3, 4} are rotation parameters, and t x , t y is a translation parameter, x and y are the positions of the original image, and x' and y' are the positions after affine transformation.

[0142] 2. Perform an affine transformation on the reference origin point and arrange it in ascending order of x and y.

[0143] Step4: Complete camera calibration according to Zhang's calibration method.

[0144]

[0145] s c is the scale factor;

[0146] u and v are the horizontal and vertical coordinates of the pixel coordinate system respectively;

[0147] d x , d y is the physical size of the pixel on the x-axis and y-axis, with the unit of: mm / pixel;

[0148] f is the focal length of the camera;

[0149] r ij {i = 0, 1, 2; j = 0, 1, 2} is the rotation matrix;

[0150] T x , T y , T z is the translation vector;

[0151] x w , y w , z w are the point coordinates in the world coordinate system.

[0152] The embodiment of the present application also provides an electronic device corresponding to the calibration plate feature region extraction method and calibration method provided in the foregoing embodiment to execute the calibration plate feature region extraction method and calibration method. The embodiments of the present application are not limited.

[0153] Please refer to Figure 10 , which shows a schematic diagram of an electronic device provided in some embodiments of the present application. As Figure 10As shown, the electronic device 2 includes: a processor 200, a memory 201, a bus 202, and a communication interface 203. The processor 200, the communication interface 203, and the memory 201 are connected through the bus 202. A computer program that can run on the processor 200 is stored in the memory 201. When the processor 200 runs the computer program, it executes the calibration board feature area extraction method and the calibration method provided in any of the foregoing embodiments of the present application.

[0154] Among them, the memory 201 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. Through at least one communication interface 203 (which can be wired or wireless), a communication connection is established between this system network element and at least one other network element, and the Internet, wide area network, local area network, metropolitan area network, etc. can be used.

[0155] The bus 202 can be an ISA bus, a PCI bus, an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. Among them, the memory 201 is used to store a program. After receiving an execution instruction, the processor 200 executes the program. The calibration board feature area extraction method disclosed in any of the foregoing embodiments of the present application can be applied to or implemented by the processor 200.

[0156] The processor 200 may be an integrated circuit chip with the ability to process signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in the processor 200 or the instructions in the form of software. The above-mentioned processor 200 may be a general-purpose processor, including a central processing unit (CPU for short), a network processor (NP for short), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. It can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present application. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as being executed and completed by the hardware decoding processor, or executed and completed by a combination of the hardware and software modules in the decoding processor. The software module may be located in a mature storage medium in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. This storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and combines its hardware to complete the steps of the above method.

[0157] The electronic device provided by the embodiment of the present application and the method for extracting the characteristic area of the calibration board provided by the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by it.

[0158] The embodiment of the present application also provides a computer-readable storage medium corresponding to the method for extracting the characteristic area of the calibration board provided in the foregoing embodiment. Please refer to Figure 11 , which shows that the computer-readable storage medium is an optical disc 30, on which a computer program (i.e., a program product) is stored. When the computer program is run by the processor, it will execute the method for extracting the characteristic area of the calibration board and the calibration method provided in any of the foregoing embodiments.

[0159] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical and magnetic storage media, which will not be elaborated here one by one.

[0160] The computer-readable storage medium provided by the above embodiments of the present application and the calibration plate feature region extraction method and calibration method provided by the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run, or implemented by the application programs stored therein.

[0161] It should be noted that:

[0162] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings provided herein. The structure required to construct such systems will be apparent from the above description. In addition, the present application is not directed to any particular programming language. It should be understood that the content of the present application described herein can be implemented using various programming languages, and the description of a particular language above is for the purpose of disclosing the best mode of the present application.

[0163] In the specification provided herein, a large number of specific details are set forth. However, it can be understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0164] Similarly, it should be understood that, in order to streamline the present application and assist in understanding one or more of the various inventive aspects, in the above description of the exemplary embodiments of the present application, the various features of the present application are sometimes grouped together into a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting an intention that the claimed present application requires more features than are expressly recited in each claim. Rather, as reflected in the following claims, the inventive aspects lie in less than all the features of the single foregoing disclosed embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into the detailed description, with each claim standing on its own as a separate embodiment of the present application.

[0165] Those skilled in the art can understand that the modules in the devices in the embodiments can be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise explicitly stated, each feature disclosed in this specification (including the accompanying claims, abstract and drawings) can be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0166] In addition, those skilled in the art can understand that although some of the embodiments described herein include certain features included in other embodiments rather than other features, the combination of the features of different embodiments means that it is within the scope of this application and forms different embodiments. For example, in the following claims, any one of the claimed embodiments can be used in any combination.

[0167] Each component embodiment of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that in practice, a microprocessor or a digital signal processor (DSP) can be used to implement some or all of the functions of some or all of the components in the virtual machine creation system according to the embodiments of the present application. The present application can also be implemented as a device or system program (for example, a computer program and a computer program product) for executing part or all of the methods described herein. Such a program for implementing the present application can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0168] It should be noted that the above embodiments are illustrative of the present application rather than restrictive of the present application, and those skilled in the art can design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word "comprising" does not exclude the presence of elements or steps not listed in the claim. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application can be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In the unit claims listing several systems, several of these systems may be embodied by the same item of hardware. The use of the words first, second, and third, etc. does not denote any order. These words may be interpreted as names.

[0169] As described above, the foregoing is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present application can easily conceive of various changes or substitutions, and these should all be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A method for extracting the feature region of a calibration board using a calibration board, the calibration board comprising: Five reference dots and multiple reference dots, and the dot area and the background color are complementary colors; among them, among the five reference dots, the connection lines of four of the outer reference dots form a quadrilateral with a perspective relationship, serving as the reference for all reference dots, and the outer reference dots are arranged in the central area of the calibration board; the fifth reference dot is located in the middle position between the two outer reference dots on the lower side, and is set as the origin of the calibration board coordinate system, used to record the pose information of the calibration board and the sorting information of the reference dots. It is characterized in that it includes: Place the calibration board within the field of view of the CCD camera, collect the calibration board image, and record multiple groups of images by changing the relative position and pose of the calibration board and the camera. Perform binary processing and filtering operations on each group of images. Use the Canny operator to extract the contours of each group of images, and select the inner contours according to the hierarchical relationship of the contours. Perform ellipse fitting based on the least squares method on the inner contours to obtain the ellipse contours. Filter the ellipse contours. According to the filtered ellipse size, statistically calculate the average diameter of the reference dots and the reference dots, and calculate the adaptive proportionality coefficient of morphology. Perform morphological opening and closing operations on the circular features to obtain the region of interest, which is the preliminary exploration region of the calibration board. Search for the reference point in the preliminary exploration region, and calculate the centroid of the calibration board according to the 4 outer reference dots; according to the proportional relationship between the centroid of the ideal calibration board and the reference dots and the four outermost corner points of the calibration board, scale up the reference dots in the image by pixel distance, so as to calculate the accurate calibration board region.

2. The method according to claim 1, characterized in that The reference dots on the calibration board are arranged in an array, each row is parallel, each column is parallel, the rows and columns are perpendicular to each other, and the reference dots are spaced at equal distances.

3. The method according to claim 2, characterized in that The camera is mounted on the actuator. When collecting images, the pose of the calibration board does not change. After each calibration board image is collected, control the pose change of the actuator to realize the change of the relative pose between the camera and the calibration board.

4. The method according to claim 2, characterized in that The position and pose of the camera are fixed, and the position and pose of the calibration board are changed each time when collecting images.

5. The method according to claim 2, wherein After obtaining the area where the calibration board is located, it further includes: When the calibration board exceeds the field of view, perform boundary processing on the region of interest.

6. A calibration method for a calibration plate, characterized in that, It includes: Use the method of any one of claims 1-5 to perform ellipse fitting based on contour features on the area where the calibration board is located to obtain all the fitted ellipse contours on the calibration board area; Calculate the ellipse position corresponding to the reference dot according to the size information, and judge the pose of the origin coordinate system of the calibration board and the positions of the four reference ellipses according to the cosine value of the vector angle; Sort the reference dots according to the pose information of the calibration board coordinate system to obtain the image coordinates corresponding one by one to the world coordinates; Complete camera calibration according to the Zhang's calibration method.

7. The method according to claim 6, characterized in that The reference dot sorting method includes: Perform an affine transformation on the reference dot, and then sort it in the way that the row and column coordinate values increase monotonically; or, Based on the original image, according to the method of sorting the center of the vector angle.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor runs the computer program to implement the method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, The program is executed by the processor to implement the method according to any one of claims 1-7.

Images