Depth camera calibration method, system, device and medium

Abstract

The present invention provides a depth camera calibration method, system, device and medium, including: obtaining a checkerboard image, binarizing the checkerboard image, generating a first checkerboard binarized image; extracting multiple first contours from the first checkerboard binarized image, and determining the central checkerboard area of ​​the checkerboard image according to the first contours; binarizing the target checkerboard image to generate a second checkerboard binarized image, and extracting multiple second contours from the second checkerboard binarized image; screening out grid contours from the multiple second contours, and then determining the center point of each grid contour, and determining the corner points of each grid area according to the center points of four adjacent grid contours; according to the corner points of each grid area, each grid area is numbered with the central checkerboard area as the center, and the internal and external parameters of the depth camera are determined according to the corner points and the numbers. The present invention can automatically calculate the internal and external parameters of the camera and realize automatic calibration of the depth camera.

Description

Background Art

[0002] In the image measurement process and machine vision applications, in order to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, a geometric model of camera imaging must be established. These geometric model parameters are the camera parameters. Under most conditions, these parameters must be obtained through experiments and calculations. The process of solving the parameters is called camera calibration.

[0003] When calibrating a camera, a calibration plate is usually used. By shooting a flat plate with a fixed-pitch pattern array and calculating it with a calibration algorithm, the geometric model of the camera can be obtained, thereby obtaining high-precision measurement and reconstruction results.

[0004] Commonly used calibration plates include checkerboard calibration plates, circular mark point calibration plates, QR code calibration plates, and coded mark point calibration plates. When using a checkerboard calibration plate for camera calibration, the corner points of the captured checkerboard image need to be manually marked in the prior art, or when the number of grids in the checkerboard image is small and only occupies part of the camera's field of view, the corner points can be detected by image recognition to identify the corner points of the checkerboard image. However, when the checkerboard image fills the entire camera's field of view, errors often occur when identifying the corner points, so that each grid area in the checkerboard image cannot be labeled. Summary of the invention

[0005] In view of the defects in the prior art, an object of the present invention is to provide a depth camera calibration method, system, device and medium.

[0006] The depth camera calibration method provided by the present invention comprises the following steps:

[0007] Step S1: obtaining a checkerboard image, and performing binarization processing on the checkerboard image to generate a first checkerboard binary image;

[0008] Step S2: extracting a plurality of first contours from the first checkerboard binary image, and determining a central checkerboard area of ​​the checkerboard image according to the first contours, wherein the central checkerboard area has a preset marker;

[0009] Step S3: after inverting the marker area, generating a target checkerboard image, binarizing the target checkerboard image to generate a second checkerboard binary image, and extracting a plurality of second contours from the second checkerboard binary image;

[0010] Step S4: selecting a grid contour from the plurality of second contours, and then determining the center point of each grid contour, and determining the corner point of each grid area according to the center points of four adjacent grid contours;

[0011] Step S5: labeling each of the grid areas according to the corner points of each grid area with the central chessboard area as the center, and determining the internal and external parameters of the depth camera according to the corner points and the labels.

[0012] Preferably, step S2 comprises the following steps:

[0013] Step S201: obtaining the grayscale values ​​of the center point of each of the first contours and four points above, below, left and right with a Manhattan distance L from the center point, wherein the center checkerboard is black and the marker is preset to be white;

[0014] Step S202: determining whether the ratio of the grayscale values ​​of the four upper, lower, left, and right points to the grayscale value of the center point is greater than a preset ratio threshold, and if it is greater than a preset distance threshold, determining that the contour is a central chessboard area;

[0015] Step S203: Repeat steps S201 to S202 until the central chessboard area is determined.

[0016] Preferably, step S3 comprises the following steps:

[0017] Step S301: binarizing the target checkerboard image by a preset first gray value threshold to generate a second checkerboard binary image A, wherein the outline of the white checkerboard area in the second checkerboard binary image A is isolated by the outline of the black checkerboard area, and the black checkerboard areas are connected to form an integral area;

[0018] Step S302: binarizing the checkerboard image by a preset second gray value threshold to generate a second checkerboard binary image B, wherein the outline of the black checkerboard area in the second checkerboard binary image B is isolated by the outline of the white checkerboard area, and the white checkerboard area is connected to form an integral area;

[0019] Step S303: extracting the second contour of each white checkerboard area in the first checkerboard binary image A, and extracting the second contour of each black checkerboard area in the second checkerboard binary image B.

[0020] Preferably, step S4 comprises the following steps:

[0021] Step S401: Calculate the area A of each second contour;

[0022] Step S402: Area A is in the interval [0.2A median , A median ] The second contour is extracted, A median is the median of all second contour areas;

[0023] Step S403: Screening the extracted second contour to determine the grid contour according to a preset area screening formula, wherein the area screening formula is:

[0024] A median -2σ<A<A median +2σ,σ x <0.02×L 3 , σ y <0.02×L 3 ;

[0025] Among them, σ x is the variance of the second profile in the X direction, σ y is the variance of the second profile in the Y direction, σ is the variance of the areas of all second profiles,

[0026] Preferably, the method further comprises the following steps:

[0027] Step S6: Determine the pixel distance of the checkerboard image and the spatial distance of the corresponding checkerboard calibration plate according to the corner points and the labels, and determine the reference distance of the checkerboard calibration plate from the depth camera according to the spatial distance of the checkerboard and the internal and external parameters of the depth camera.

[0028] Preferably, step S4 comprises the following steps:

[0029] Step S401: selecting a grid contour from a plurality of the second contours, and determining a center point of each grid contour according to the grid contours;

[0030] Step S402: determining the coarse corner point of each grid area according to the centroid of the center points of four adjacent grid contours;

[0031] Step S403: extracting sub-pixels of each coarse corner point, and determining the corner points of each grid area according to the sub-pixels of the coarse corner point.

[0032] Preferably, step S5 comprises the following steps:

[0033] Step S501: searching for white checkerboard areas that are adjacent to the central checkerboard area with the central checkerboard area as the central area, and marking the adjacent white checkerboard areas, wherein the number of adjacent white checkerboard areas is less than four, and the distance between the center of the adjacent white checkerboard area and the center of the central checkerboard area is less than L, Among them, area is the area of ​​the central area;

[0034] Step S502: using the white checkerboard area of ​​each of the neighboring cells to be found as the central area to find the black checkerboard area of ​​the neighboring cells and labeling the black checkerboard area of ​​the neighboring cells, and then using the black checkerboard area of ​​each of the neighboring cells to be found as the central area to find the white checkerboard area of ​​the neighboring cells and labeling the white checkerboard area of ​​the neighboring cells;

[0035] Step S503: Repeat step S502 until no white checkerboard area or black checkerboard area that can be grown is found.

[0036] The depth camera calibration system provided by the present invention includes the following modules:

[0037] A binarization processing module is used to obtain a checkerboard image, perform binarization processing on the checkerboard image, and generate a first checkerboard binary image;

[0038] a center determination module, configured to extract a plurality of first contours from the first checkerboard binary image, and determine a central checkerboard area of ​​the checkerboard image according to the first contours, wherein the central checkerboard area has a preset marker;

[0039] Contour extraction module: after inverting the marker area, generating a target chessboard image, binarizing the target chessboard image to generate a second chessboard binary image, and extracting a plurality of second contours from the second chessboard binary image;

[0040] A corner point extraction module, used to screen out grid contours from the plurality of second contours, and then determine the center point of each grid contour, and determine the corner point of each grid area according to the center points of four adjacent grid contours;

[0041] The parameter calculation module is used to label each of the grid areas according to the corner points of each grid area with the central chessboard area as the center, and determine the internal and external parameters of the depth camera according to the corner points and the labels.

[0042] The depth camera calibration device provided by the present invention includes:

[0043] processor;

[0044] a memory storing executable instructions of the processor;

[0045] The processor is configured to perform the steps of the depth camera calibration method by executing the executable instructions.

[0046] The computer-readable storage medium provided according to the present invention is used to store a program, and when the program is executed, the steps of any one of the depth camera calibration methods are implemented.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] In the present invention, the camera to be calibrated can be carried on an automatically controlled rail, and after automatically photographing a chessboard image, the corner points of each grid area can be automatically extracted from the photographed chessboard image, and each grid area can be labeled, so that the internal and external parameters of the camera can be obtained, thereby realizing automatic calibration of the depth camera;

[0049] The present invention can also obtain a reference distance between the checkerboard calibration plate and the depth camera according to the internal and external parameters. The reference distance can be used as the real distance between the checkerboard calibration plate and the depth camera, and can be used as a reference distance for calibration of the depth camera, thus solving the problem that the real distance cannot be obtained in general depth camera evaluation.

[0050] In the present invention, multiple contours are extracted from a checkerboard binary image, and grid contours are screened out from the multiple contours, and then the center point of each grid contour is determined, so that the coarse corner point of each grid area can be determined, and then the target corner point is determined, thereby realizing automatic extraction of the corner points of the checkerboard without knowing the number of checkerboards in the checkerboard. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following will briefly introduce the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings in the following descriptions are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without creative work. By reading the detailed description of the non-limiting embodiments with reference to the following drawings, other features, purposes and advantages of the present invention will become more obvious:

[0052] Figure 1 is a flowchart of the steps of the depth camera calibration method in an embodiment of the present invention;

[0053] Figure 2 is a flowchart of the steps of a depth camera calibration method in a modified example of the present invention;

[0054] Figure 3 A flowchart of the steps of determining a central chessboard area in an embodiment of the present invention;

[0055] Figure 4 A flowchart of the steps of extracting a plurality of second contours in an embodiment of the present invention;

[0056] Figure 5 A flowchart of the steps of selecting a grid outline from a plurality of the second outlines in an embodiment of the present invention;

[0057] Figure 6A flowchart of the steps of determining the corner points of each grid area in an embodiment of the present invention;

[0058] Figure 7 is a flowchart of the steps of labeling a plurality of grid areas in an embodiment of the present invention;

[0059] Figure 8 Schematic diagram of a module of a depth camera calibration system according to an embodiment of the present invention;

[0060] Fig. 9 is a schematic diagram of the structure of a depth camera calibration device in an embodiment of the present invention; and

[0061] Fig.10 Schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. DETAILED DESCRIPTION

[0062] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several variations and improvements may be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0063] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein, for example, can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0064] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0065] The depth camera calibration method provided by the present invention is intended to solve the problems existing in the prior art.

[0066] The following specific embodiments are used to describe in detail the technical solutions of the present invention and how the technical solutions of the present application solve the above-mentioned technical problems. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0067] Figure 1 is a flowchart of the steps of the depth camera calibration method in an embodiment of the present invention. Figure 1 As shown, the depth camera calibration method provided by the present invention comprises the following steps:

[0068] Step S1: obtaining a checkerboard image, and performing binarization processing on the checkerboard image to generate a first checkerboard binary image;

[0069] Step S2: extracting a plurality of first contours from the first checkerboard binary image, and determining a central checkerboard area of ​​the checkerboard image according to the first contours, wherein the central checkerboard area has a preset marker;

[0070] Step S3: after inverting the marker area, generating a target checkerboard image, binarizing the target checkerboard image to generate a second checkerboard binary image, and extracting a plurality of second contours from the second checkerboard binary image;

[0071] Step S4: selecting a grid contour from the plurality of second contours, and then determining the center point of each grid contour, and determining the corner point of each grid area according to the center points of four adjacent grid contours;

[0072] Step S5: labeling each of the grid areas according to the corner points of each grid area with the central chessboard area as the center, and determining the internal and external parameters of the depth camera according to the corner points and the labels.

[0073] In the embodiment of the present invention, the camera to be calibrated can be mounted on an automatically controlled guide rail, and after automatically capturing a checkerboard image, the corner points of each grid area can be automatically extracted from the captured checkerboard image, and each grid area can be numbered, so that the internal and external parameters of the camera can be obtained, thereby realizing automatic calibration of the depth camera.

[0074] In the embodiment of the present invention, the coarse corner point is determined according to the center of gravity of an area formed by the center points of four adjacent grid contours.

[0075] Figure 2 is a flowchart of the steps of the depth camera calibration method in a modified example of the present invention, such as Figure 2 As shown, the depth camera calibration method provided by the present invention also includes the following steps:

[0076] Step S6: Determine the pixel distance of the checkerboard image and the spatial distance of the corresponding checkerboard calibration plate according to the corner points and the labels, and determine the reference distance of the checkerboard calibration plate from the depth camera according to the spatial distance of the checkerboard and the internal and external parameters of the depth camera.

[0077] The present invention can also obtain a reference distance between the checkerboard calibration plate and the depth camera according to the internal and external parameters. The reference distance can be used as the actual distance between the checkerboard calibration plate and the depth camera, and can be used as a reference distance for calibration of the depth camera, thus solving the problem that the actual distance cannot be obtained in general depth camera evaluation.

[0078] Figure 3 FIG. 1 is a flow chart of the steps of determining the central checkerboard area of ​​the checkerboard binary image in an embodiment of the present invention. Figure 3 As shown, step S2 includes the following steps:

[0079] Step S201: obtaining the grayscale values ​​of the center point of each of the first contours and four points above, below, left and right with a Manhattan distance L from the center point, wherein the center checkerboard is black and the marker is preset to be white;

[0080] Step S202: determining whether the ratio of the grayscale values ​​of the four upper, lower, left, and right points to the grayscale value of the center point is greater than a preset ratio threshold, and if it is greater than a preset distance threshold, determining that the contour is a central chessboard area;

[0081] Step S203: Repeat steps S201 to S202 until the central chessboard area is determined.

[0082] In the embodiment of the present invention, the ratio threshold is 2. The marker is preset to white and pasted on the black checkerboard. The area of ​​the marker is smaller than the black checkerboard and is contained by the black checkerboard. In order to ensure that the grayscale values ​​of the four points above, below, left and right of the center point are also relatively large, to prevent the center point from being a salt noise point. To ensure that the grayscale value I of the center point is greater than 100, and the sum of the intensities of the four surrounding points is greater than 2I, when all the center points that meet this condition are obtained, a centroid is finally obtained, which is the center point position.

[0083] Figure 4 FIG. 1 is a flowchart of steps for extracting a plurality of second contours in an embodiment of the present invention. Figure 4 As shown, step S3 includes the following steps:

[0084] Step S301: binarizing the target checkerboard image by a preset first gray value threshold to generate a second checkerboard binary image A, wherein the outline of the white checkerboard area in the second checkerboard binary image A is isolated by the outline of the black checkerboard area, and the black checkerboard areas are connected to form an integral area;

[0085] Step S302: binarizing the checkerboard image by a preset second gray value threshold to generate a second checkerboard binary image B, wherein the outline of the black checkerboard area in the second checkerboard binary image B is isolated by the outline of the white checkerboard area, and the white checkerboard area is connected to form an integral area;

[0086] Step S303: extracting the second contour of each white checkerboard area in the first checkerboard binary image A, and extracting the second contour of each black checkerboard area in the second checkerboard binary image B.

[0087] In the embodiment of the present invention, the first gray value threshold may be set to 200, and the second gray value threshold may be set to 50.

[0088] Figure 5 FIG. 1 is a flow chart of the steps of selecting a grid contour from a plurality of second contours in an embodiment of the present invention. Figure 5 As shown, step S4 includes the following steps:

[0089] Step S401: Calculate the area A of each second contour;

[0090] Step S402: Area A is in the interval [0.2A median , A median ] to extract the contour within A median is the median of all second contour areas;

[0091] Step S403: Screening the extracted second contour to determine the grid contour according to a preset area screening formula, wherein the area screening formula is:

[0092] A median -2σ<A<A median +2σ,σ x <0.02×L 3 , σ y <0.02×L 3 ;

[0093] Among them, σ x is the variance of the second profile in the X direction, σ y is the variance of the second profile in the Y direction, σ is the variance of the areas of all second profiles,

[0094] In the embodiment of the present invention, formula A median -2σ<A<A median +2σ is used to screen the area of ​​the second contour, the formula σ x <0.02×L 3 , σ y <0.02×L 3 Used to determine whether the second contour is a rectangle.

[0095] Figure 6 FIG. 1 is a flowchart of steps for determining the corner points of each grid area in an embodiment of the present invention. Figure 6 As shown, step S4 includes the following steps:

[0096] Step S401: selecting a grid contour from a plurality of the second contours, and determining a center point of each grid contour according to the grid contours;

[0097] Step S402: determining the coarse corner point of each grid area according to the centroid of the center points of four adjacent grid contours;

[0098] Step S403: extracting sub-pixels of each coarse corner point, and determining the corner points of each grid area according to the sub-pixels of the coarse corner point.

[0099] In an embodiment of the present invention, the marker area may be inverted to black or painted black to generate a target checkerboard image, and then the target checkerboard image is binarized to generate a first checkerboard binary image and a second checkerboard binary image.

[0100] Figure 7 FIG. 1 is a flowchart of steps for labeling a plurality of grid regions in an embodiment of the present invention, such as Figure 7 As shown, step S5 includes the following steps:

[0101] Step S501: searching for white checkerboard areas that are adjacent to the central checkerboard area with the central checkerboard area as the central area, and marking the adjacent white checkerboard areas, wherein the number of adjacent white checkerboard areas is less than four, and the distance between the center of the adjacent white checkerboard area and the center of the central checkerboard area is less than L, Among them, area is the area of ​​the central area;

[0102] Step S502: using the white checkerboard area of ​​each of the neighboring cells to be found as the central area to find the black checkerboard area of ​​the neighboring cells and labeling the black checkerboard area of ​​the neighboring cells, and then using the black checkerboard area of ​​each of the neighboring cells to be found as the central area to find the white checkerboard area of ​​the neighboring cells and labeling the white checkerboard area of ​​the neighboring cells;

[0103] Step S503: Repeat step S502 until no white checkerboard area or black checkerboard area that can be grown is found.

[0104] In the embodiment of the present invention, the size of the checkerboard is 31×31, and the number of adjacent black checkerboard areas and adjacent white checkerboard areas is less than or equal to four. When labeling, for a checkerboard area with coordinates (m, n), key=31*m+n, where key is the label of the checkerboard area.

[0105] Figure 8 FIG. 1 is a schematic diagram of a module of a depth camera calibration system according to an embodiment of the present invention. Figure 8 As shown, the depth camera calibration system provided by the present invention includes the following modules:

[0106] A binarization processing module is used to obtain a checkerboard image, perform binarization processing on the checkerboard image, and generate a first checkerboard binary image;

[0107] a center determination module, configured to extract a plurality of first contours from the first checkerboard binary image, and determine a central checkerboard area of ​​the checkerboard image according to the first contours, wherein the central checkerboard area has a preset marker;

[0108] Contour extraction module: after inverting the marker area, generating a target chessboard image, binarizing the target chessboard image to generate a second chessboard binary image, and extracting a plurality of second contours from the second chessboard binary image;

[0109] A corner point extraction module, used to screen out grid contours from the plurality of second contours, and then determine the center point of each grid contour, and determine the corner point of each grid area according to the center points of four adjacent grid contours;

[0110] The parameter calculation module is used to label each of the grid areas according to the corner points of each grid area with the central chessboard area as the center, and determine the internal and external parameters of the depth camera according to the corner points and the labels.

[0111] An embodiment of the present invention further provides a depth camera calibration device, comprising a processor and a memory storing executable instructions of the processor, wherein the processor is configured to execute the steps of the depth camera calibration method by executing the executable instructions.

[0112] As described above, in this embodiment, the present invention extracts multiple contours from the checkerboard binary image, selects the grid contour from the multiple contours, and then determines the center point of each grid contour, so as to determine the coarse corner point of each grid area, and then determine the target corner point, thereby realizing automatic extraction of the corner points of the checkerboard without knowing the number of checkerboards in the checkerboard.

[0113] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0114] Fig. 9 Schematic diagram of the structure of the depth camera calibration device of the present invention. Fig. 9 The electronic device 600 according to this embodiment of the present invention is described. Fig. 9 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0115] like Fig. 9 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0116] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps of various exemplary embodiments of the present invention described in the depth camera calibration method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0117] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0118] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0119] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0120] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), one or more devices that enable a user to interact with the electronic device 600, and / or any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 through the bus 630. It should be understood that although Fig. 9 Not shown, other hardware and / or software modules may be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0121] In an embodiment of the present invention, a computer-readable storage medium is also provided for storing a program, and the steps of the depth camera calibration method are implemented when the program is executed. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the above depth camera calibration method section of this specification.

[0122] As shown above, when the program of the computer-readable storage medium of this embodiment is executed, the present invention extracts multiple contours from the checkerboard binary image, selects the grid contours from the multiple contours, and then determines the center point of each grid contour, so as to determine the coarse corner points of each grid area, and then determine the target corner points, thereby realizing automatic extraction of the corner points of the checkerboard without knowing the number of checkerboards in the checkerboard.

[0123] Fig.10 Schematic diagram of the structure of the computer-readable storage medium of the present invention. Fig.10As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0124] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0125] Computer readable storage media may include data signals propagated in baseband or as part of a carrier wave, wherein readable program codes are carried. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or device. The program codes contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination thereof.

[0126] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0127] In the embodiment of the present invention, a camera to be calibrated can be mounted on an automatically controlled guide rail, and after automatically capturing a checkerboard image, the corner points of each grid area can be automatically extracted from the captured checkerboard image, and each of the grid areas can be labeled, so that the internal and external parameters of the camera can be obtained, thereby realizing automatic calibration of the depth camera; in the embodiment of the present invention, a reference distance of a checkerboard calibration plate from a depth camera can be obtained based on the internal and external parameters, and the reference distance can be used as the real distance of the checkerboard calibration plate from the depth camera, and can be used as a reference distance for calibration of the depth camera, thereby solving the problem that the real distance cannot be obtained in general depth camera evaluation; in the embodiment of the present invention, a plurality of contours are extracted from a checkerboard binary image, and a grid contour is selected from the plurality of contours, and then the center point of each grid contour is determined, so that the coarse corner points of each grid area can be determined, and then the target corner points are determined, thereby realizing automatic extraction of the corner points of the checkerboard, and there is no need to know the number of checkerboards in the checkerboard.

[0128] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same and similar parts between the embodiments can be referred to each other. The above description of the disclosed embodiments enables professionals and technicians in this field to implement or use the present invention. Various modifications to these embodiments will be obvious to professionals and technicians in this field, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to these embodiments shown in this article, but will comply with the widest range consistent with the principles and novel features disclosed herein.

[0129] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art may make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A depth camera calibration method, characterized in that: The steps include: Step S1: obtaining a checkerboard image, and performing binarization processing on the checkerboard image to generate a first checkerboard binary image; Step S2: extracting a plurality of first contours from the first checkerboard binary image, and determining a central checkerboard area of ​​the checkerboard image according to the first contours, wherein the central checkerboard area has a preset marker; Step S3: after inverting the marker area, generating a target checkerboard image, binarizing the target checkerboard image to generate a second checkerboard binary image, and extracting a plurality of second contours from the second checkerboard binary image; Step S4: selecting a grid contour from the plurality of second contours, and then determining the center point of each grid contour, and determining the corner point of each grid area according to the center points of four adjacent grid contours; Step S5: labeling each of the grid areas according to the corner points of each grid area with the central chessboard area as the center, and determining the internal and external parameters of the depth camera according to the corner points and the labels; Wherein, the step S3 comprises the following steps: Step S301: binarizing the target checkerboard image by a preset first gray value threshold to generate a second checkerboard binary image A, wherein the outline of the white checkerboard area in the second checkerboard binary image A is isolated by the outline of the black checkerboard area, and the black checkerboard areas are connected to form an integral area; Step S302: binarizing the checkerboard image by a preset second gray value threshold to generate a second checkerboard binary image B, wherein the outline of the black checkerboard area in the second checkerboard binary image B is isolated by the outline of the white checkerboard area, and the white checkerboard area is connected to form an integral area; Step S303: extracting the second contour of each white checkerboard area in the first checkerboard binary image A, and extracting the second contour of each black checkerboard area in the second checkerboard binary image B.

2. The depth camera calibration method according to claim 1, characterized in that: The step S2 comprises the following steps: Step S201: obtaining the grayscale values ​​of the center point of each of the first contours and four points above, below, left and right with a Manhattan distance L from the center point, wherein the center checkerboard is black and the marker is preset to be white; Step S202: determining whether the ratio of the grayscale values ​​of the four upper, lower, left, and right points to the grayscale value of the center point is greater than a preset ratio threshold, and if it is greater than a preset distance threshold, determining that the contour is a central chessboard area; Step S203: Repeat steps S201 to S202 until the central chessboard area is determined.

3. The depth camera calibration method according to claim 1, characterized in that: The step S4 comprises the following steps: Step S401: Calculate the area A of each second contour; Step S402: Area A is in the interval [0.2A median , A median ] The second contour is extracted, A median is the median of all second contour areas; Step S403: Screening the extracted second contour to determine the grid contour according to a preset area screening formula, wherein the area screening formula is: A median -2s <A<A median +2σ,σ x <0.02×L 3 ,s y <0.02×L 3 ; Among them, σ x is the variance of the second profile in the X direction, σ y is the variance of the second profile in the Y direction, σ is the variance of the areas of all second profiles, 4. The depth camera calibration method according to claim 1, characterized in that: The following steps are also included: Step S6: Determine the pixel distance of the checkerboard image and the spatial distance of the corresponding checkerboard calibration plate according to the corner points and the labels, and determine the reference distance of the checkerboard calibration plate from the depth camera according to the spatial distance of the checkerboard and the internal and external parameters of the depth camera.

5. The depth camera calibration method according to claim 3, characterized in that: The step S4 comprises the following steps: Step S401: selecting a grid contour from a plurality of the second contours, and determining a center point of each grid contour according to the grid contours; Step S402: determining the coarse corner point of each grid area according to the centroid of the center points of four adjacent grid contours; Step S403: extracting sub-pixels of each coarse corner point, and determining the corner points of each grid area according to the sub-pixels of the coarse corner point.

6. The depth camera calibration method according to claim 5, characterized in that: The step S5 comprises the following steps: Step S501: searching for white checkerboard areas that are adjacent to the central checkerboard area with the central checkerboard area as the central area, and marking the adjacent white checkerboard areas, wherein the number of adjacent white checkerboard areas is less than four, and the distance between the center of the adjacent white checkerboard area and the center of the central checkerboard area is less than L, Among them, area is the area of ​​the central area; Step S502: using the white checkerboard area of ​​each of the neighboring cells to be found as the central area to find the black checkerboard area of ​​the neighboring cells and labeling the black checkerboard area of ​​the neighboring cells, and then using the black checkerboard area of ​​each of the neighboring cells to be found as the central area to find the white checkerboard area of ​​the neighboring cells and labeling the white checkerboard area of ​​the neighboring cells; Step S503: Repeat step S502 until no white checkerboard area or black checkerboard area that can be grown is found.

7. A depth camera calibration system, characterized in that: Includes the following modules: A binarization processing module is used to obtain a checkerboard image, perform binarization processing on the checkerboard image, and generate a first checkerboard binary image; a center determination module, configured to extract a plurality of first contours from the first checkerboard binary image, and determine a central checkerboard area of ​​the checkerboard image according to the first contours, wherein the central checkerboard area has a preset marker; Contour extraction module: after inverting the marker area, generating a target chessboard image, binarizing the target chessboard image to generate a second chessboard binary image, and extracting a plurality of second contours from the second chessboard binary image; A corner point extraction module, used to screen out grid contours from the plurality of second contours, and then determine the center point of each grid contour, and determine the corner point of each grid area according to the center points of four adjacent grid contours; A parameter calculation module, used to label each of the grid areas according to the corner points of each grid area, with the central chessboard area as the center, and determine the internal and external parameters of the depth camera according to the corner points and the labels; The contour extraction module comprises the following steps during processing: Step S301: binarizing the target checkerboard image by a preset first gray value threshold to generate a second checkerboard binary image A, wherein the outline of the white checkerboard area in the second checkerboard binary image A is isolated by the outline of the black checkerboard area, and the black checkerboard areas are connected to form an integral area; Step S302: binarizing the checkerboard image by a preset second gray value threshold to generate a second checkerboard binary image B, wherein the outline of the black checkerboard area in the second checkerboard binary image B is isolated by the outline of the white checkerboard area, and the white checkerboard area is connected to form an integral area; Step S303: extracting the second contour of each white checkerboard area in the first checkerboard binary image A, and extracting the second contour of each black checkerboard area in the second checkerboard binary image B.

8. A depth camera calibration device, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to perform the steps of the depth camera calibration method according to any one of claims 1 to 6 by executing the executable instructions.

9. A computer-readable storage medium for storing a program, characterized in that: When the program is executed, the steps of the depth camera calibration method described in any one of claims 1 to 6 are implemented.

Images