Calibration method, electronic device, computer-readable storage medium, and calibration system

Abstract

The present application discloses a calibration method, an electronic device, a computer-readable storage medium and a calibration system. The calibration method is applied to a calibration system, and the calibration system includes a camera, a calibration device and a calibration card. The method includes: in response to a camera being installed on the calibration device, the calibration device is moved to an initial position, and the calibration card is photographed at the initial position by the camera to obtain an initial image; based on a camera coordinate system corresponding to the camera and the calibration card in the initial image, an initial shooting posture corresponding to the calibration device is determined; based on the initial shooting posture and at least one preset target point, a set of calibration postures is obtained; wherein the target point is a boundary point of an initial calibration image photographed by the camera under the initial shooting posture; based on the calibration posture, the camera is calibrated. The above scheme can improve the reliability of the camera calibration result.

Description

Technical Field

[0001] The present application relates to the field of camera calibration, and in particular to a calibration method, electronic equipment, computer-readable storage medium and calibration system. Background Art

[0002] Camera calibration is an important process in computer vision, which is used to determine the intrinsic and extrinsic parameters of the camera, thereby associating two-dimensional image points with three-dimensional world points. Existing camera calibration schemes usually set a series of fixed shooting postures. When the camera is installed on the calibration equipment, it will take pictures of the target according to these fixed shooting postures and then calibrate. The reliability of the calibration results is low. Summary of the invention

[0003] The main technical problem solved by the present application is to provide a calibration method, an electronic device, a computer-readable storage medium and a calibration system, which can improve the reliability of camera calibration results.

[0004] To solve the above technical problems, the first aspect of the present application provides a calibration method, which is applied to a calibration system, wherein the calibration system includes a camera, a calibration device and a calibration card, and the method includes: in response to the camera being installed on the calibration device, moving the calibration device to an initial position, using the camera to shoot the calibration card at the initial position to obtain an initial image; based on a camera coordinate system corresponding to the camera and the calibration card in the initial image, determining an initial shooting posture corresponding to the calibration device; based on the initial shooting posture and at least one preset target point, obtaining a set of calibration postures; wherein the target point is a boundary point of an initial calibration image shot by the camera under the initial shooting posture; and based on the calibration posture, calibrating the camera.

[0005] To solve the above technical problems, the second aspect of the present application provides an electronic device, comprising a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the calibration method described in the first aspect.

[0006] In order to solve the above technical problem, the third aspect of the present application provides a computer-readable storage medium, which stores program instructions that can be executed by a processor, and the program instructions are used to implement the calibration method described in the first aspect above.

[0007] To solve the above technical problems, the fourth aspect of the present application provides a calibration system, including a camera, a calibration device and a calibration card, and the electronic device described in the above second aspect, and the electronic device is coupled to the camera and the calibration device.

[0008] In the above scheme, when the camera is installed on the calibration device, the calibration device is moved to the initial position, and the camera is used to capture an image of the calibration card at the initial position to obtain an initial image, and based on the camera coordinate system corresponding to the camera and the calibration card in the initial image, the shooting posture of the calibration device is adjusted to determine the initial shooting posture corresponding to the calibration device, and based on the initial shooting posture corresponding to the calibration device and at least one preset target point, a set of calibration postures that can be used for camera calibration is obtained, and the camera is calibrated based on the obtained calibration postures, and by adjusting the initial shooting posture of the calibration device and obtaining the corresponding calibration posture based on the adjusted initial shooting posture, the center fluctuation range of any calibration posture when the camera and the calibration device are calibrated together is compressed, thereby improving the reliability of the camera calibration result. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0010] Figure 1 It is a flow chart of an implementation method of the calibration method of the present application;

[0011] Figure 2 It is a flow chart of another implementation method of the calibration method of the present application;

[0012] Figure 3 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0013] Figure 4 It is a schematic diagram of the structure of an embodiment of a computer-readable storage medium of the present application;

[0014] Figure 5 It is a structural schematic diagram of an implementation method of a calibration system of the present application. DETAILED DESCRIPTION

[0015] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and different implementation methods can be adaptively combined. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0016] The terms "system" and "network" are often used interchangeably in this article. The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship. In addition, "many" in this article means two or more than two.

[0017] See also Figure 1 , Figure 1 The figure is a flow chart of an implementation method of the present application. The method is applied to a calibration system, which includes a camera, a calibration device and a calibration chart, wherein the calibration method includes:

[0018] S101: In response to a camera being installed on a calibration device, the calibration device is moved to an initial position, and a calibration chart is photographed at the initial position using the camera to obtain an initial image.

[0019] Specifically, when the camera is installed on the calibration device, the calibration device is moved to an initial position, and the camera is used to capture an image of the calibration chart at the initial position to obtain an initial image.

[0020] Optionally, when the calibration device moves to the initial position, the calibration chart is approximately located at the center of the camera.

[0021] In one application, the calibration device is installed on a mechanical device, and the mechanical device is controlled by a preset program to move the calibration device to an initial position, thereby ensuring that the calibration device can be moved to the same initial position each time with high accuracy, and a camera is used to capture an image of the calibration card at the initial position to obtain an initial image.

[0022] In another application, an obvious mark, such as a reflective mark, a color mark, etc., is set at the initial position in advance. When the calibration device is moved to the initial position, the sensor detects the mark and triggers the camera to capture the calibration card, thereby obtaining the initial image.

[0023] S102: Determine an initial shooting posture corresponding to the calibration device based on a camera coordinate system corresponding to the camera and a calibration chart in the initial image.

[0024] Specifically, based on a camera coordinate system corresponding to the camera and a calibration chart in the initial image, the shooting posture of the calibration device is adjusted to determine an initial shooting posture corresponding to the calibration device.

[0025] In one application, key points of a calibration chart in an initial image are identified, and based on a camera coordinate system corresponding to the camera and the identified key points, a shooting posture of a calibration device is adjusted to determine an initial shooting posture corresponding to the calibration device.

[0026] In another application, image features of the calibration chart in the initial image are obtained, and based on the camera coordinate system and image features corresponding to the camera, the shooting posture of the calibration device is adjusted to determine an initial shooting posture corresponding to the calibration device.

[0027] S103: Obtain a set of calibration postures based on the initial shooting posture and at least one preset target point; wherein the target point is a boundary point of an initial calibration image shot by the camera under the initial shooting posture.

[0028] Specifically, based on an initial shooting posture corresponding to the calibration device and at least one preset target point, a set of calibration postures that can be used for camera calibration is obtained.

[0029] In one application, the coordinates of the edge corner points of the calibration chart in the initial calibration image are obtained, and the coordinates of the preset target points are obtained, and the calibration posture is obtained according to the coordinates of the edge corner points, the coordinates of the target points and the initial shooting posture.

[0030] In another application method, the coordinates of multiple edge corner points of the calibration card in the initial calibration image are obtained, and the coordinates of multiple preset target points are obtained, the distances between the multiple edge corner points and the target points are calculated, and the edge corner points that meet the distance threshold are screened out, and the calibration posture is obtained based on the screened edge corner points and the corresponding target points, as well as the initial shooting posture.

[0031] S104: Calibrate the camera based on the calibration posture.

[0032] Specifically, the camera is calibrated based on the obtained calibration posture.

[0033] In one application, multiple images corresponding to multiple different calibration postures are obtained, and the camera is calibrated according to multiple corner points of the calibration chart in the multiple images.

[0034] In another application mode, a calibration image corresponding to a single calibration posture is obtained, and the camera is calibrated according to the center coordinates of each feature point of the calibration chart in the calibration image.

[0035] In the above scheme, when the camera is installed on the calibration device, the calibration device is moved to the initial position, and the camera is used to capture an image of the calibration card at the initial position to obtain an initial image, and based on the camera coordinate system corresponding to the camera and the calibration card in the initial image, the shooting posture of the calibration device is adjusted to determine the initial shooting posture corresponding to the calibration device, and based on the initial shooting posture corresponding to the calibration device and at least one preset target point, a set of calibration postures that can be used for camera calibration is obtained, and the camera is calibrated based on the obtained calibration postures, and by adjusting the initial shooting posture of the calibration device and obtaining the corresponding calibration posture based on the adjusted initial shooting posture, the center fluctuation range of any calibration posture when the camera and the calibration device are calibrated together is compressed, thereby improving the reliability of the camera calibration result.

[0036] See also Figure 2 , Figure 2 1 is a flow chart of another embodiment of the calibration method of the present application. The calibration method is applied to a calibration system, which includes a camera, a calibration device and a calibration chart, wherein the calibration method includes:

[0037] S201: In response to a camera being installed on a calibration device, the calibration device is moved to an initial position, and a calibration chart is photographed at the initial position using the camera to obtain an initial image.

[0038] Specifically, when the camera is installed on the calibration device, the calibration device is moved to an initial position, and the camera is used to capture an image of the calibration chart at the initial position to obtain an initial image.

[0039] S202: Acquire image features in the initial image, and obtain first center coordinates and a roll angle of the initial image based on the image features.

[0040] Specifically, the calibration card in the initial image contains multiple image features. The first image feature in the image features is obtained, and based on the first image feature, the first center coordinate is obtained. The second image feature in the image features is obtained, and based on the second image feature, the roll angle is obtained. By extracting the image features for calculation, a more accurate initial value can be provided for the subsequent calibration process.

[0041] It should be noted that after the calibration chart is projected onto the camera screen, it must have the necessary clarity so that the corresponding image features can be recognized.

[0042] S203: Determine an initial shooting posture based on the first center coordinates of the initial image, the second center coordinates of the camera coordinate system corresponding to the camera, and the rolling angle.

[0043] Specifically, based on the first center coordinate of the initial image, the second center coordinate of the camera coordinate system corresponding to the camera, and the roll angle, the shooting posture corresponding to the calibration device is adjusted to determine the initial shooting posture.

[0044] In one implementation scenario, step S203 specifically includes: obtaining an offset based on the first center coordinate of the initial image and the second center coordinate of the camera coordinate system corresponding to the camera; adjusting the shooting posture of the calibration device based on the offset and the roll angle, and determining the adjusted shooting posture as the initial shooting posture.

[0045] Specifically, based on the first center coordinate of the initial image and the second center coordinate of the camera coordinate system corresponding to the camera, the offset between the two center coordinates is calculated, and the shooting posture of the calibration device is adjusted according to the offset and the roll angle, and the adjusted shooting posture is determined as the initial shooting posture of the calibration device.

[0046] Optionally, after the initial shooting posture of the calibration device is determined, the camera can be used to take another image in the initial shooting posture, and whether the first center coordinate and the second center coordinate coincide with each other can be determined based on the image. If they coincide, the shooting posture is determined to be the initial shooting posture; if they do not coincide, the shooting posture is readjusted until they coincide, thereby ensuring the accuracy of the subsequent calibration posture obtained with the initial shooting posture.

[0047] In a specific implementation scenario, the camera includes a chip, and the resolution of the chip is X*Y. Then the horizontal coordinate X of the second center coordinate of the camera coordinate system corresponding to the camera is 0 =X / 2, ordinate Y 0 =Y / 2, the first center coordinate of the initial image is (X', Y'), and the offset calculated based on the first center coordinate and the second center coordinate is (δ X , δ Y ), where δ X =X / 2-X',δ Y =Y / 2-Y', the roll angle is θ, according to the offset (δ X , δ Y ) and the rolling angle θ to adjust the shooting posture of the calibration device, and determine the adjusted shooting posture as the initial shooting posture of the calibration device.

[0048] It can be understood that, in general, the calibration device is a six-axis robot, and the control parameters of the motion system corresponding to the shooting posture of the six-axis robot are (x n ,y n , z n , P n , Y n , R n), where the control parameters corresponding to the initial shooting posture of the calibration device are (0,0,0,0,0,0).

[0049] S204: Obtain a set of calibration postures based on the initial shooting posture and at least one preset target point.

[0050] Specifically, based on an initial shooting posture corresponding to the calibration device and at least one preset target point, a set of calibration postures that can be used for camera calibration is obtained.

[0051] In one implementation scenario, step S204 specifically includes: obtaining the coordinates of the edge corner points of the calibration card in the initial calibration image that meet the preset distance conditions with the target point; obtaining the deflection angle information based on the coordinates of the edge corner points and the coordinates of the target point; and obtaining a set of calibration postures based on the initial shooting posture and multiple deflection angle information.

[0052] Specifically, the coordinates of the edge corner points of the calibration chart and the target point in the initial calibration image that meet the preset distance conditions are obtained, wherein the preset target point is the boundary point of the initial calibration image taken by the camera under the initial shooting posture, and the deflection angle information is calculated based on the coordinates of the edge corner points and the coordinates of the target point. Based on the initial shooting posture of the calibration device and multiple calculated deflection angle information, a set of calibration postures is obtained. By approaching the target point with the edge corner points, the coverage rate of the feature points obtained by subsequent identification can be further improved, thereby achieving a more reliable and accurate camera calibration result.

[0053] In a specific implementation scenario, it is assumed that the coordinates of the edge corner points are (a n ,h n ), the corresponding pre-set target points close to the edge corner point are (L, U), (P, Q), etc. The coordinates of the center point are (X 0 , Y 0 ), the horizontal field of view of the camera HFOV=α, the number of pixels in the horizontal direction is X. Generally speaking, in the optical system, the center of the camera is the lens. The farther away from the central axis, the larger the corresponding field of view angle. However, due to the existence of camera distortion, the degree of distance and the angle of the field of view cannot always achieve a linear relationship. In order to simplify the calculation, the camera can still be approximately considered as an ideal camera model, that is, the two are considered to be in a linear relationship. Then each pixel can be considered to correspond linearly to a certain HFOV, and the lens is a rotating body, which conforms to the same characteristics in all directions. The angle corresponding to 1 pixel can be calculated using the following formula:

[0054] (1)

[0055] Among them, k is a coefficient determined based on the characteristics of the lens, generally between 0 and 1, α is the horizontal field of view, and X is the number of pixels in the horizontal direction.

[0056] Furthermore, the edge corner points need to satisfy the following distance conditions: |a n -X 0 |≤|LX 0 |,|h n -Y 0 |≤|LY 0 |, that is, the edge corner point must be between the target point and the center point to avoid the calibration chart moving out of the camera field of view, and the distance between the edge corner point and the target point does not exceed z pixels, where z is the preset distance threshold and its value is not restricted and depends on the actual situation.

[0057] Furthermore, after obtaining the edge corner points that meet the above conditions, the deflection angle information can be calculated by the following formula transformed from formula (1):

[0058] (2)

[0059] (3)

[0060] Therefore, as long as the pre-set target points are sufficient and reasonable, multiple deflection angle information Y can be obtained. n and P n , and based on multiple deflection angle information Y n and P n , and the initial shooting posture (0,0,0,0,0,0), we get a set of calibration postures (0,0,0,P n , Y n , 0).

[0061] S205: Using a camera to photograph the calibration card in a calibration posture to obtain a plurality of reference calibration images.

[0062] Specifically, when the calibration device moves to a corresponding calibration posture, the calibration chart is photographed by a camera to obtain a plurality of reference calibration images.

[0063] Optionally, the calibration chart may be photographed multiple times by a camera at each calibration posture to obtain a clearer reference calibration image. By selecting a reference calibration image with higher definition, the accuracy of subsequent camera calibration results can be further improved.

[0064] S206: Calibrate the camera based on at least a plurality of reference calibration images.

[0065] Specifically, the camera is calibrated at least based on multiple reference calibration images taken by the camera. More image information can be obtained through reference calibration images corresponding to multiple different calibration postures, which helps to better identify and correct the distortion caused by lens characteristics, thereby improving the accuracy of camera calibration. Moreover, a more accurate three-dimensional space model can be constructed through reference calibration images at multiple angles.

[0066] In one implementation, after step S102, the method further includes: acquiring an initial calibration image in an initial shooting posture, and step S206 specifically includes: calibrating the camera based on the initial calibration image and a plurality of reference calibration images.

[0067] Specifically, the camera is calibrated based on an initial calibration image under the initial shooting posture corresponding to the calibration device and a set of multiple reference calibration images under the calibration posture. By adding the initial calibration image, the intrinsic and extrinsic parameters of the camera can be determined more accurately, thereby further improving the accuracy of camera calibration.

[0068] In one implementation scenario, the step of calibrating the camera based on the initial calibration image and multiple reference calibration images specifically includes: extracting feature points of the calibration chart in the initial calibration image and the multiple reference calibration images; and calibrating the camera based on the multiple feature points.

[0069] Specifically, the corner points of the calibration chart in the initial calibration image and multiple reference calibration images are extracted as feature points, and the camera is calibrated based on multiple feature points. The more feature points there are, the higher the coverage rate is, and the more accurate the camera calibration result is.

[0070] See also Figure 3 , Figure 3: is a structural diagram of an embodiment of an electronic device of the present application, the electronic device 30 includes a memory 300 and a processor 302 coupled to each other, wherein the memory 300 stores program data (not shown), and the processor 302 calls the program data to implement the method in any of the above embodiments. For the description of the relevant content, please refer to the detailed description of the above method embodiment, which will not be repeated here. Specifically, the electronic device 30 includes but is not limited to: a desktop computer, a laptop computer, a tablet computer, a server, etc., which are not limited here. In addition, the processor 302 can also be called a CPU (Center Processing Unit). The processor 302 may be an integrated circuit chip with signal processing capabilities. The processor 302 can also be a general-purpose processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field-programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. In addition, the processor 302 can be implemented by an integrated circuit chip.

[0071] See also Figure 4 , Figure 4 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. The computer-readable storage medium 40 stores program data 400. When the program data 400 is executed by a processor, the method in any of the above embodiments is implemented. For descriptions of related contents, please refer to the detailed description of the above method embodiments, which will not be repeated here.

[0072] See also Figure 5 , Figure 5 It is a structural diagram of an implementation method of a calibration system of the present application. The calibration system 50 includes a camera 500, a calibration device 501 and a calibration chart 502, as well as the electronic device 30 described above, and the electronic device 30 is coupled to the camera 500 and the calibration device 501.

[0073] It should be noted that the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present implementation scheme.

[0074] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0075] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor to execute all or part of the steps of each implementation method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc., and other media that can store program codes.

[0076] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A calibration method, characterized in that: Applied to a calibration system, the calibration system includes a camera, a calibration device and a calibration chart, and the method includes: In response to the camera being installed on the calibration device, the calibration device is moved to an initial position, and the calibration chart is photographed at the initial position by the camera to obtain an initial image; Based on the camera coordinate system corresponding to the camera and the calibration chart in the initial image, determining the initial shooting posture corresponding to the calibration device; wherein, based on the camera coordinate system corresponding to the camera and the calibration chart in the initial image, determining the initial shooting posture corresponding to the calibration device comprises: acquiring image features in the initial image, and obtaining the first center coordinates and the roll angle of the initial image based on the image features; determining the initial shooting posture based on the first center coordinate of the initial image, the second center coordinates of the camera coordinate system corresponding to the camera, and the roll angle; Based on the initial shooting posture and at least one preset target point, a set of calibration postures is obtained; wherein the target point is a boundary point of an initial calibration image shot by the camera under the initial shooting posture; wherein, based on the initial shooting posture and at least one preset target point, a set of calibration postures is obtained, including: obtaining the coordinates of the edge corner points of the calibration chart in the initial calibration image that meet a preset distance condition with the target point; based on the coordinates of the edge corner points and the coordinates of the target point, obtaining deflection angle information; based on the initial shooting posture and a plurality of the deflection angle information, a set of calibration postures is obtained; Based on the calibration posture, the camera is calibrated.

2. The calibration method according to claim 1, characterized in that: The determining the initial shooting posture based on the first center coordinate of the initial image, the second center coordinate of the camera coordinate system corresponding to the camera, and the rolling angle includes: Obtaining an offset based on a first center coordinate of the initial image and a second center coordinate of a camera coordinate system corresponding to the camera; Based on the offset and the rolling angle, the shooting posture of the calibration device is adjusted, and the adjusted shooting posture is determined as the initial shooting posture.

3. The calibration method according to claim 1, characterized in that: The step of calibrating the camera based on the calibration posture includes: Using the camera to photograph the calibration chart in the calibration posture to obtain a plurality of reference calibration images; The camera is calibrated based on at least a plurality of the reference calibration images.

4. The calibration method according to claim 3, characterized in that: After determining the initial shooting posture corresponding to the calibration device based on the camera coordinate system corresponding to the camera and the calibration chart in the initial image, the method further includes: Acquire the initial calibration image under the initial shooting posture; The step of calibrating the camera based at least on the plurality of reference calibration images comprises: The camera is calibrated based on the initial calibration image and the plurality of reference calibration images.

5. The calibration method according to claim 4, characterized in that: The calibrating the camera based on the initial calibration image and the plurality of reference calibration images includes: Extracting feature points of the calibration chart in the initial calibration image and the plurality of reference calibration images; The camera is calibrated based on the multiple feature points.

6. An electronic device, characterized in that: It comprises a memory and a processor coupled to each other, wherein the memory stores program instructions, and the processor is used to execute the program instructions to implement the calibration method according to any one of claims 1 to 5.

7. A computer-readable storage medium, characterized in that: Program instructions that can be executed by a processor are stored, and the program instructions are used to implement the calibration method described in any one of claims 1-5.

8. A calibration system, characterized in that: The invention comprises a camera, a calibration device and a calibration chart, and an electronic device as claimed in claim 6, wherein the electronic device is coupled to the camera and the calibration device.

Images