Camera calibration system

Abstract

The application provides a camera calibration system, comprising: an image acquisition module, which acquires a target calibration board image relative to a camera orthographic projection, and cuts out a plurality of first ROI regions in the target calibration board image, each first ROI region comprising a calibration icon; a sub-pixel edge determination module, which is used for extracting a gradient map and an edge pixel point of the calibration icon, finding a gradient maximum value point on the edge pixel point according to the gradient map, and determining a sub-pixel edge point of the calibration icon according to the gradient maximum value point; a pixel coordinate generation module, which is used for fitting and determining a center point pixel of the calibration icon according to the sub-pixel edge point of the calibration icon, and further generating a center point pixel coordinate relative to a camera inverse projection; and a parameter calculation module, which is used for acquiring a label and a center point pixel coordinate of each calibration icon, and determining target internal and external parameters of the camera according to the label and the center point pixel coordinate of the calibration icon. In the application, the precision of camera calibration is improved, and the camera is convenient to apply to a picking robot.

Description

Technical Field

[0001] This invention relates to cameras, and more specifically, to a camera calibration system. Background Technology

[0002] In image measurement and machine vision applications, to determine the relationship between the three-dimensional geometric position of a point on the surface of a spatial object and its corresponding point in the image, a geometric model of camera imaging must be established. These geometric model parameters are the camera parameters. Under most conditions, these camera parameters must be obtained through experimentation and calculation; this process of solving for the parameters is called camera calibration. Whether in image measurement or machine vision applications, camera parameter calibration is a crucial step, as the accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the camera.

[0003] When calibrating a camera, a calibration plate is typically used. By photographing a plate with a fixed-spaced pattern array and performing calculations using a calibration algorithm, the camera's geometric model can be derived, resulting in high-precision measurement and reconstruction results.

[0004] Commonly used calibration boards include checkerboard calibration boards, circular marker calibration boards, QR code calibration boards, and coded marker calibration boards. When using a circular marker calibration board for camera calibration, the circular markers often form elliptical images due to factors such as shooting angle and image distortion. If the circular markers are still treated as circles for center extraction, errors in center position extraction often occur, affecting calibration accuracy. Summary of the Invention

[0005] In view of the shortcomings of the prior art, the purpose of this invention is to provide a camera calibration system.

[0006] The camera calibration system provided by the present invention includes the following modules:

[0007] The image acquisition module is used to acquire a target calibration board image projected relative to the camera, and to extract multiple first ROI regions from the target calibration board image, each first ROI region including a calibration icon;

[0008] The subpixel edge determination module is used to extract the gradient map and edge pixels of the calibration icon, find the maximum gradient point on the edge pixels according to the gradient map, and determine the subpixel edge point of the calibration icon according to the maximum gradient point.

[0009] A pixel coordinate generation module is used to determine the center pixel of the calibration icon based on the fitting of the sub-pixel edge points of the calibration icon, and then generate the center pixel coordinates by back projection relative to the camera.

[0010] The parameter calculation module is used to obtain the label of each calibration icon and the pixel coordinates of the center point, and to determine the target intrinsic and extrinsic parameters of the camera based on the label of the calibration icon and the pixel coordinates of the center point.

[0011] Preferably, the image acquisition module includes the following units:

[0012] An image acquisition unit is used to acquire multiple pre-acquired calibration board images, which are acquired in the camera coordinate system.

[0013] The camera parameter unit is used to acquire the preliminary intrinsic and extrinsic parameters generated by the initial calibration of the camera.

[0014] The first image projection unit is used to project the calibration board image onto the calibration board coordinate system according to the preliminary internal and external parameters to generate a target calibration board image, and to make the target calibration board image in a frontal view state in the calibration board coordinate system.

[0015] Preferably, the subpixel edge determination module includes the following units:

[0016] An edge pixel extraction module is used to extract the gradient map and edge pixels of the calibration icon;

[0017] The gradient value point extraction module is used to find the maximum gradient point on the edge pixel point according to the gradient map, and select a second ROI region with the maximum gradient point as the center.

[0018] A subpixel determination module is used to take the centroid of the second ROI region as the subpixel edge point.

[0019] Preferably, the pixel coordinate generation module includes the following units:

[0020] The preliminary coordinate generation module is used to determine the center point pixel of the calibration icon based on the sub-pixel edge points of the calibration icon and generate the preliminary coordinates of the center point pixel;

[0021] The camera parameter unit is used to acquire the preliminary intrinsic and extrinsic parameters generated by the initial calibration of the camera.

[0022] The second image projection unit is used to project the preliminary coordinates of the center point pixel onto the camera coordinate system based on the preliminary intrinsic and extrinsic parameters to generate the center point pixel coordinates.

[0023] Preferably, it also includes an iterative execution module, which controls the image acquisition module, subpixel edge determination module, pixel coordinate generation module and parameter calculation module to iteratively calculate the target intrinsic and extrinsic parameters multiple times until the number of executions reaches a preset execution number threshold, or until the error of the target intrinsic and extrinsic parameters of the camera converges to the minimum error value.

[0024] Preferably, the labeling of the calibration icon is generated by a labeling generation module, which includes the following units:

[0025] The grayscale value calculation unit is used to generate multiple third ROI regions with each of the center point pixels as the center, and to calculate the average pixel grayscale value of each of the third ROI regions.

[0026] The icon differentiation unit is used to sort the average pixel gray values ​​to generate a gray value sequence, and to determine the first calibration icon region and the second calibration icon region in the calibration board image based on the gray value sequence.

[0027] The first calibration icon determination unit is used to calculate the center point pixel distance between each pair of multiple first calibration icon regions, and to number the first calibration icon regions according to the multiple center point pixel distances;

[0028] The corner icon determination unit is used to determine the second calibration icon area located at the four corners of the calibration board based on the distance between the center point pixel of the second calibration icon area and the line connecting the two first calibration icon areas;

[0029] The labeling unit is used to label all calibration icon areas according to the second calibration icon areas located at the four corners of the calibration board.

[0030] Preferably, the diameter of the third ROI region is determined based on the average distance between the centers of all adjacent calibrated icon regions.

[0031] Preferably, the minimum error value is defined as the error of the target intrinsic and extrinsic parameters calculated in the previous iteration being less than a preset difference threshold.

[0032] Preferably, the first calibration icon area is a large circle area, the second calibration icon area is a small circle area, and the first few calibration icon areas are the pixel positions of the large circle area.

[0033] Preferably, when the first calibration icon determination module numbers the first calibration icon area, it includes:

[0034] Calculate the distance between each pair of circle centers in the large circular region to determine multiple distance values, sort the multiple distance values, and find the coordinates of the two pairs of circle centers with the smallest and largest distance values ​​respectively;

[0035] The pair of center points with the smallest distance value are denoted as the second and third large circles, and the pair of center points with the largest distance value are denoted as the fourth and fifth large circles. The remaining large circle area is the first large circle.

[0036] Take the midpoints of the fourth and fifth large circles and form a straight line with the first large circle. Calculate the distances from the centers of the second and third large circles to the first straight line, and denote the circle with the shorter distance as the second large circle and the circle with the longer distance as the third large circle.

[0037] Take the first and third large circles to form a second straight line. Calculate the distances from the centers of the fourth and fifth large circles to the second straight line. The circle with the shorter distance is designated as the fourth large circle, and the circle with the longer distance is designated as the fifth large circle.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] In this invention, the calibration board image is transformed into a target calibration board image projected orthogonally relative to the camera. Then, the sub-pixel edge points and center point pixels are determined based on the gradient map of the target calibration board image. The center point pixel coordinates are then generated by back-projection, thereby generating and calculating the target intrinsic and extrinsic parameters of the camera. This improves the accuracy of camera calibration and facilitates the application of the camera in picking robots. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0041] Figure 1 This is a schematic diagram of the camera calibration system in an embodiment of the present invention;

[0042] Figure 2 This is a schematic diagram of the image acquisition module in an embodiment of the present invention;

[0043] Figure 3 This is a schematic diagram of the subpixel edge determination module in an embodiment of the present invention;

[0044] Figure 4 This is a schematic diagram of the pixel coordinate generation module in an embodiment of the present invention;

[0045] Figure 5 This is a schematic diagram of the label generation module in an embodiment of the present invention;

[0046] Figure 6 (a) is a schematic diagram of the first ROI region in an embodiment of the present invention;

[0047] Figure 6 (b) is a schematic diagram of the gradient of the calibration icon in an embodiment of the present invention;

[0048] Figure 6 (c) is a schematic diagram of the pixel edge points of the calibrated icon in an embodiment of the present invention;

[0049] Figure 7 (a) is a schematic diagram of the target calibration plate image after orthographic projection in an embodiment of the present invention;

[0050] Figure 7 (b) is a schematic diagram of the calibration plate image before orthographic projection in an embodiment of the present invention;

[0051] Figure 8 This is a schematic diagram of a calibration plate image with labels in an embodiment of the present invention;

[0052] Figure 9 This is a schematic diagram of the structure of the item picking robot that applies the camera calibration method in an embodiment of the present invention;

[0053] Figure 10 This is a schematic diagram of the camera calibration device in an embodiment of the present invention;

[0054] Figure 11 This is a schematic diagram of the camera calibration device in an embodiment of the present invention; and

[0055] Figure 12 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Detailed Implementation

[0056] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0057] The terms “first,” “second,” “third,” “fourth,” etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that embodiments of the invention described herein can be implemented, for example, in orders other than those illustrated or described herein. Furthermore, the terms “comprising” and “having,” and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0058] The technical solution of the present invention will be described in detail below with reference to specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0059] The technical solutions of the present invention and how they solve the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will now be described with reference to the accompanying drawings.

[0060] Figure 1 This is a schematic diagram of the camera calibration system in an embodiment of the present invention, such as... Figure 1 As shown, the camera calibration system provided by the present invention includes:

[0061] The image acquisition module is used to acquire a target calibration board image projected relative to the camera, and to extract multiple first ROI regions from the target calibration board image. Each first ROI region includes a calibration icon, such as... Figure 6 As shown in (a), the calibration icon is an elliptical area.

[0062] Figure 2 This is a schematic diagram of the image acquisition module in an embodiment of the present invention, as shown below. Figure 2 As shown, the image acquisition module includes the following units:

[0063] The image acquisition unit is used to acquire multiple pre-acquired calibration board images, which are acquired in the camera coordinate system, such as... Figure 7 As shown in (a);

[0064] The camera parameter unit is used to acquire the preliminary intrinsic and extrinsic parameters generated by the initial calibration of the camera.

[0065] The first image projection unit is used to project the calibration board image onto the calibration board coordinate system according to the preliminary intrinsic and extrinsic parameters to generate a target calibration board image, such that the target calibration board image is in a frontal view state in the calibration board coordinate system. Figure 7 As shown in (b).

[0066] In this embodiment of the invention, the camera coordinate system is based on the lens optical center as the origin O. C The row and column directions parallel to the pixel are X and X, respectively. C axis and Y C The Z-axis direction is determined according to the right-hand coordinate system rule. C A three-dimensional rectangular coordinate system is established along the axial direction. The calibration plate coordinate system has its origin O at the upper left corner of the calibration plate. U The row and column directions of the calibration plate are X and X respectively. U axis and Y U Axial direction.

[0067] The calibration plate image is subjected to perspective transformation through a homography matrix to obtain a distortion-free target calibration plate image. The homography matrix is ​​generated at least by the preliminary intrinsic and extrinsic parameters.

[0068] The subpixel edge determination module is used to extract the gradient map and edge pixels of the calibration icon, find the maximum gradient point on the edge pixels according to the gradient map, and determine the subpixel edge point of the calibration icon according to the maximum gradient point.

[0069] Figure 3 This is a schematic diagram of the sub-pixel edge determination module in an embodiment of the present invention, as shown below. Figure 3 As shown, the sub-pixel edge determination module includes the following units:

[0070] An edge pixel extraction module is used to extract the gradient map and edge pixels of the calibration icon;

[0071] The gradient value point extraction module is used to find the maximum gradient point on the edge pixel point according to the gradient map, and select a second ROI region with the maximum gradient point as the center.

[0072] A subpixel determination module is used to take the centroid of the second ROI region as the subpixel edge point.

[0073] In this embodiment of the invention, the Sobel operator is used to generate the gradient map of the edge of the calibration icon, such as... Figure 6 As shown in (b), the Canny operator is then used to generate integer pixel edge pixels for the calibration icon, as follows. Figure 6 As shown in (c), the edge pixels are integer pixels.

[0074] The second ROI region is a 5×5 or 7×7 pixel range centered on the point where the gradient maximum value is located.

[0075] A pixel coordinate generation module is used to determine the center pixel of the calibration icon based on the fitting of the sub-pixel edge points of the calibration icon, and then generate the center pixel coordinates by back projection relative to the camera.

[0076] Figure 4 This is a schematic diagram of the pixel coordinate generation module in an embodiment of the present invention, as shown below. Figure 4 As shown, the pixel coordinate generation module includes the following units:

[0077] The preliminary coordinate generation module is used to determine the center point pixel of the calibration icon based on the sub-pixel edge points of the calibration icon and generate the preliminary coordinates of the center point pixel;

[0078] The camera parameter unit is used to acquire the preliminary intrinsic and extrinsic parameters generated by the initial calibration of the camera.

[0079] The second image projection unit is used to project the preliminary coordinates of the center point pixel onto the camera coordinate system based on the preliminary intrinsic and extrinsic parameters to generate the center point pixel coordinates.

[0080] In this embodiment of the invention, at least one matrix generated by the initial intrinsic and extrinsic parameters can reverse the initial coordinates of the center point pixel to generate the center point pixel coordinates, that is, convert them into the center point pixel coordinates in the distorted calibration plate image.

[0081] The parameter calculation module is used to obtain the label of each calibration icon and the pixel coordinates of the center point, and to determine the target intrinsic and extrinsic parameters of the camera based on the label of the calibration icon and the pixel coordinates of the center point.

[0082] In this embodiment of the invention, the camera calibration system provided by the present invention further includes an iterative execution module, which controls the image acquisition module, the sub-pixel edge determination module, the pixel coordinate generation module and the parameter calculation module to iteratively calculate the intrinsic and extrinsic parameters of the target multiple times until the number of executions reaches a preset execution number threshold, or until the error of the intrinsic and extrinsic parameters of the camera converges to the minimum error value.

[0083] The execution count threshold can be set to 4 to 10 times.

[0084] The minimum error is defined as the error of the target intrinsic and extrinsic parameters calculated in the previous iteration being less than the difference between the error of the target intrinsic and extrinsic parameters calculated in the current iteration and the error of the target intrinsic and extrinsic parameters calculated in the previous iteration.

[0085] In this embodiment of the invention, the difference threshold can be set to 0.001.

[0086] Figure 5 This is a schematic diagram of the module that generates the label number of the calibration icon in an embodiment of the present invention, as shown below. Figure 5 As shown, the labels of the calibration icons are generated by a label generation module, which includes the following units:

[0087] The grayscale value calculation unit is used to generate multiple third ROI regions with each of the center point pixels as the center, and to calculate the average pixel grayscale value of each of the third ROI regions.

[0088] The icon differentiation unit is used to sort the average pixel gray values ​​to generate a gray value sequence, and to determine the first calibration icon region and the second calibration icon region in the calibration board image based on the gray value sequence.

[0089] like Figure 8 As shown, the first calibration icon area is a large circle, and the second calibration icon area is a small circle. The first five calibration icon areas are the pixel positions of the five large circles.

[0090] The first calibration icon determination unit is used to calculate the center point pixel distance between each pair of multiple first calibration icon regions, and to number the first calibration icon regions according to the multiple center point pixel distances;

[0091] More specifically, first, calculate the distance between the centers of each pair of large circular regions, resulting in a total of 10 distance values. Sort these 10 distance values ​​and identify the two pairs of center coordinates with the smallest and largest distances. The pair with the smallest distance is the bottom two large circles, denoted as large circle 2 and large circle 3. The pair with the largest distance is the two large circles in the middle row, denoted as large circle 4 and large circle 5. The remaining large circular region is large circle 1.

[0092] Then, it's necessary to further distinguish between large circles 2 and 3, and large circles 4 and 5. For large circles 2 and 3, take the midpoints of large circles 4 and 5 and form a first straight line with large circle 1. Calculate the distances from the centers of large circles 2 and 3 to this first straight line; the shorter distance is designated as large circle 2, and the longer distance as large circle 3. For large circles 4 and 5, take large circles 1 and 3 to form a second straight line. Calculate the distances from the centers of large circles 4 and 5 to this second straight line; the shorter distance is designated as large circle 4, and the longer distance as large circle 5. The completed large circle regions are then numbered as follows: Figure 8 As shown.

[0093] The corner icon determination unit is used to determine the second calibration icon area located at the four corners of the calibration board based on the distance between the center point pixel of the second calibration icon area and the line connecting the two first calibration icon areas;

[0094] Specifically, connect large circle 1 and large circle 5 to form a third straight line, calculate the distance from the center coordinates of all circles in the figure to the third straight line, and the one with the longest distance is the center coordinate of the lower right circle in the figure, denoted as small circle A; similarly, connect large circle 1 and large circle 4, large circle 2 and large circle 4, and large circle 2 and large circle 5 with a straight line to determine the center pixel coordinates of small circle B in the lower left corner, small circle C in the upper left corner, and small circle D in the upper right corner.

[0095] The labeling unit is used to label all calibration icon areas according to the second calibration icon areas located at the four corners of the calibration board.

[0096] Specifically, connect small circle A and small circle D to form a straight line AD. Calculate the distance between the center coordinates of all calibration icon areas and the straight line AD. Assuming the calibration board's circular array is 7 rows and 9 columns, take the 7 center coordinates with the smallest distance as the rightmost column of circles. Then calculate the distance to the center point of small circle A from the 7 center coordinates. Sort the 7 circles from farthest to closest to small circle A to form small circle 9, small circle 18, small circle 27, small circle 36, small circle 45, small circle 54, and small circle 63.

[0097] Connect small circles A and B to form a straight line AB. Then, traverse small circles 9, 18, 27, 36, 45, 54, and 63, and draw a straight line parallel to AB through each circle in turn. Recalculate the distance from the center of all circles to the straight line AD and sort them to obtain the sequence number of the 9 circles in each row.

[0098] In this embodiment of the invention, the diameter of the third ROI region is determined based on the average distance between the centers of all adjacent calibrated icon regions.

[0099] Figure 9 This is a schematic diagram of the structure of an item picking robot that applies a camera calibration method in an embodiment of the present invention, as shown below. Figure 9 As shown, the camera calibration device provided by the present invention further includes:

[0100] Unit 1 and Unit 2 are used for storing and / or transporting materials;

[0101] A depth camera 300, whose visual scanning area at least covers the first unit for storing or transporting the material, is used to visually scan the material, acquire a depth image of the material, and generate the pose information and storage location of the material based on the depth image.

[0102] The robot unit 100 is communicatively connected to the depth camera 300, and is used to receive the pose information and storage position, determine the placement state of the target object based on the pose and storage position, and pick up the target object based on the placement state.

[0103] In one embodiment of the present invention, the first unit may be configured as a storage unit 200;

[0104] The storage unit 200 is used to store materials, which are the target objects, such as metal products, boxes, or any other items.

[0105] The robot unit 100 is communicatively connected to the depth camera 300. It is used to receive the pose information and storage position, determine the placement state of the target object based on the pose and storage position, and pick up the target object according to the placement state and transfer it to the second unit.

[0106] The second unit can be configured to transport or store the picked materials, such as a support rack to facilitate the neat arrangement of items.

[0107] The second unit may also include a transport unit, enabling the robot unit 100 to move the target object on the support frame to the transport unit.

[0108] The depth camera 300 is mounted on the camera bracket 500 and is not shown due to obstruction by the crossbeam of the camera bracket 500.

[0109] The robot unit 100 includes a processor configured to execute the steps of the camera calibration method by executing executable instructions. This involves transforming the calibration board image into a target calibration board image projected orthographically relative to the depth camera 300, determining sub-pixel edge points and center pixel points based on a gradient map in the target calibration board image, generating center pixel coordinates through back projection, and then calculating the target intrinsic and extrinsic parameters of the camera. This improves the accuracy of camera calibration and facilitates the application of the camera in picking robots.

[0110] Figure 10 This is a flowchart illustrating the steps of the camera calibration method in an embodiment of the present invention, as follows: Figure 10 As shown, the camera calibration method provided by the present invention includes the following steps:

[0111] Step S1: Obtain the target calibration board image projected onto the calibration board coordinate system, and extract multiple first ROI regions from the target calibration board image, each first ROI region including a calibration icon;

[0112] Step S2: Extract the gradient map and edge pixels of the calibration icon, find the maximum gradient point on the edge pixels according to the gradient map, and determine the sub-pixel edge point of the calibration icon according to the maximum gradient point;

[0113] Step S3: Determine the preliminary coordinates of the center pixel of the calibration icon based on the fitting of the sub-pixel edge points of the calibration icon, and then project the preliminary coordinates of the center pixel onto the camera coordinate system to generate the center pixel coordinates;

[0114] Step S4: Obtain the label of each calibration icon and the center point pixel coordinates, and determine the target intrinsic and extrinsic parameters of the camera based on the label of the calibration icon and the center point pixel coordinates.

[0115] This invention also provides a camera calibration device, including a processor and a memory. The memory stores executable instructions for the processor. The processor is configured to perform camera calibration method steps by executing the executable instructions.

[0116] As described above, in this embodiment, by transforming the calibration board image into a target calibration board image projected orthogonally relative to the camera, and then determining the sub-pixel edge points and center point pixels based on the gradient map in the target calibration board image, and then back-projecting to generate the coordinates of the center point pixel, the target intrinsic and extrinsic parameters of the camera are calculated, thereby improving the accuracy of camera calibration and facilitating the application of the camera in picking robots.

[0117] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "platform."

[0118] Figure 11 This is a schematic diagram of the camera calibration device in an embodiment of the present invention. Refer to the following... Figure 11 To describe an electronic device 600 according to this embodiment of the present invention. Figure 11 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0119] like Figure 11 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0120] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the camera calibration method section of this specification, based on various exemplary embodiments of the present invention. For example, the processing unit 610 can perform actions such as... Figure 1 The steps are shown in the figure.

[0121] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0122] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0123] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0124] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, camera, depth camera, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although... Figure 11 As not shown in the diagram, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0125] This invention also provides a computer-readable storage medium for storing a program that, when executed, implements the steps of a camera calibration method. In some possible implementations, various aspects of the invention can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the camera calibration method section of this specification according to various exemplary embodiments of the invention.

[0126] As shown above, when the program of the computer-readable storage medium of this embodiment is executed, it transforms the calibration board image into a target calibration board image projected orthogonally relative to the camera, then determines the sub-pixel edge points and center point pixels in the target calibration board image based on the gradient map, and then generates the coordinates of the center point pixel through back projection, thereby generating and calculating the target intrinsic and extrinsic parameters of the camera, which improves the accuracy of camera calibration and facilitates the application of the camera in picking robots.

[0127] Figure 12 This is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. (Reference) Figure 12 As shown, a program product 800 for implementing the above-described method according to an embodiment of the present invention is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0128] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0129] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0130] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0131] In this embodiment of the invention, the calibration board image is transformed into a target calibration board image projected orthogonally relative to the camera. Then, the sub-pixel edge points and center point pixels are determined based on the gradient map in the target calibration board image. The center point pixel coordinates are then generated by back projection, thereby generating and calculating the target intrinsic and extrinsic parameters of the camera. This improves the accuracy of camera calibration and facilitates the application of the camera in picking robots.

[0132] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0133] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A camera calibration system, characterized in that, Includes the following modules: The image acquisition module is used to acquire a target calibration board image projected relative to the camera, and to extract multiple first ROI regions from the target calibration board image, each first ROI region including a calibration icon; The subpixel edge determination module is used to extract the gradient map and edge pixels of the calibration icon, find the maximum gradient point on the edge pixels according to the gradient map, and determine the subpixel edge point of the calibration icon according to the maximum gradient point. A pixel coordinate generation module is used to determine the center pixel of the calibration icon based on the fitting of the sub-pixel edge points of the calibration icon, and then generate the center pixel coordinates by back projection relative to the camera. The parameter calculation module is used to obtain the label of each calibration icon and the pixel coordinates of the center point, and to determine the target intrinsic and extrinsic parameters of the camera based on the label of the calibration icon and the pixel coordinates of the center point.

2. The camera calibration system according to claim 1, characterized in that, The image acquisition module includes the following units: an image acquisition unit, used to acquire multiple pre-acquired calibration board images, the calibration board images being acquired in the camera coordinate system; and a camera parameter unit, used to acquire preliminary intrinsic and extrinsic parameters generated by preliminary calibration of the camera. The first image projection unit is used to project the calibration board image onto the calibration board coordinate system according to the preliminary internal and external parameters to generate a target calibration board image, and to make the target calibration board image in a frontal view state in the calibration board coordinate system.

3. The camera calibration system according to claim 1, characterized in that, The subpixel edge determination module includes the following units: an edge pixel extraction module, used to extract the gradient map and edge pixel points of the calibration icon; a gradient value point extraction module, used to find the maximum gradient point on the edge pixel points according to the gradient map, and select a second ROI region centered on the maximum gradient point; and a subpixel determination module, used to take the centroid of the second ROI region as the subpixel edge point.

4. The camera calibration system according to claim 1, characterized in that, The pixel coordinate generation module includes the following units: a preliminary coordinate generation module, used to determine the center point pixel of the calibration icon based on the sub-pixel edge points of the calibration icon and generate the preliminary coordinates of the center point pixel; and a camera parameter unit, used to obtain the preliminary intrinsic and extrinsic parameters generated by the preliminary calibration of the camera. The second image projection unit is used to project the preliminary coordinates of the center point pixel onto the camera coordinate system based on the preliminary intrinsic and extrinsic parameters to generate the center point pixel coordinates.

5. The camera calibration system according to claim 1, characterized in that, It also includes an iterative execution module, which controls the image acquisition module, subpixel edge determination module, pixel coordinate generation module and parameter calculation module to iteratively calculate the intrinsic and extrinsic parameters of the target multiple times until the number of executions reaches a preset execution number threshold, or until the error of the camera's intrinsic and extrinsic parameters converges to the minimum error value.

6. The camera calibration system according to claim 1, characterized in that, The labeling of the calibration icon is generated by a labeling generation module, which includes the following units: a grayscale value calculation unit, used to generate multiple third ROI regions with each center pixel as the center, and calculate the average pixel grayscale value of each third ROI region; and an icon differentiation unit, used to sort the average pixel grayscale values ​​to generate a grayscale value sequence, and determine the first calibration icon region and the second calibration icon region in the calibration board image based on the grayscale value sequence. The first calibration icon determination unit is used to calculate the center point pixel distance between each pair of multiple first calibration icon regions and number the first calibration icon regions according to the multiple center point pixel distances; the corner icon determination unit is used to determine the second calibration icon regions located at the four corners of the calibration board according to the distance between the center point pixel of the second calibration icon region and the line connecting the two first calibration icon regions; the labeling unit is used to label all calibration icon regions according to the second calibration icon regions located at the four corners of the calibration board.

7. The camera calibration system according to claim 6, characterized in that, The diameter of the third ROI region is determined based on the average distance between the centers of all adjacent calibrated icon regions.

8. The camera calibration system according to claim 5, characterized in that, The minimum error is defined as the error of the target intrinsic and extrinsic parameters calculated in the previous iteration being less than the difference between the error of the target intrinsic and extrinsic parameters calculated in the current iteration and the error of the target intrinsic and extrinsic parameters calculated in the previous iteration.

9. The camera calibration system according to claim 6, characterized in that, The first calibration icon area is a large circle area, the second calibration icon area is a small circle area, and the top-ranked calibration icon areas are the pixel positions of the large circle area.

10. The camera calibration system according to claim 9, characterized in that, When the first calibration icon determination module numbers the first calibration icon area, it includes: calculating the distance between the centers of each pair of large circles in the large circle area to determine multiple distance values; sorting the multiple distance values; finding the coordinates of the two pairs of center points with the smallest and largest distance values ​​respectively; the pair of center points with the smallest distance value is recorded as large circle number 2 and large circle number 3, and the pair of center points with the largest distance value is recorded as large circle number 4 and large circle number 5, and the remaining large circle area is large circle number 1; taking the midpoint of large circle number 4 and large circle number 5 and large circle number 1 to form a first straight line, calculating the distance from the center of large circle number 2 and large circle number 3 to the first straight line respectively, and recording the shorter distance as large circle number 2 and the longer distance as large circle number 3; taking large circle number 1 and large circle number 3 to form a second straight line, calculating the distance from the center of large circle number 4 and large circle number 5 to the second straight line respectively, and recording the shorter distance as large circle number 4 and the longer distance as large circle number 5.

Images