A camera calibration method, device and movable carrier

Abstract

The present invention provides a method, vehicle and device for camera calibration, the method comprising: obtaining first coordinate information of a calibration pattern preset on a movable vehicle in a world coordinate system; obtaining second coordinate information of the calibration pattern on the movable vehicle in an image captured by a camera of the movable vehicle; and obtaining calibration parameters of the camera based on the first coordinate information and the second coordinate information. The method obtains the world coordinates and pixel coordinates of the calibration pattern of the movable vehicle in real time, thereby calculating the calibration parameters of the camera, and realizing automatic calibration of the camera. The entire calibration process does not require manual participation, has high calibration accuracy, improves calibration efficiency, and saves labor costs.

Description

Technical Field

[0001] The present invention relates to the technical field of camera calibration, and in particular to a camera calibration method, device and movable carrier. Background Art

[0002] With the increase in the number of cars, more and more parking garages are built underground in order to make full use of the space. In order to monitor the specific conditions of the garage as much as possible, a large number of surveillance cameras need to be installed in different locations of the garage. At present, some projects need to use the camera installed in the garage to locate the vehicle, and locate the actual coordinates of the vehicle through the image captured by the camera. Therefore, the camera must know its precise installation coordinates and angles to convert the vehicle position on the image into coordinates in the real world. Since there will inevitably be deviations when installing the camera, it is necessary to measure the installation position and angle of each camera, that is, to calibrate the camera.

[0003] At present, the calibration of surveillance cameras in garages is mostly done manually. During calibration, a calibration pattern needs to be laid out for each camera. The schematic diagram of laying out the calibration pattern can be referred to Figure 1 As shown, the position of the calibration pattern is measured, which is time-consuming and laborious, and it is not possible to use one camera to take many calibration photos. It can be seen that the existing calibration method has the problems of complex operation process, low calibration accuracy, and inaccurate judgment of calibration results by manual participation. Summary of the invention

[0004] In view of the above problems, the present invention is proposed to provide a camera calibration method, device and movable carrier that overcome the above problems or at least partially solve the above problems.

[0005] In a first aspect, an embodiment of the present invention provides a method for camera calibration, including: obtaining first coordinate information of a preset calibration pattern on a movable carrier in a world coordinate system; obtaining second coordinate information of the calibration pattern on the movable carrier in an image taken by a camera of the movable carrier; and obtaining calibration parameters of the camera based on the first coordinate information and the second coordinate information.

[0006] In one embodiment, the step of obtaining first coordinate information of a preset calibration pattern on the movable carrier in a world coordinate system includes:

[0007] The coordinate value of the locator on the movable carrier in the world coordinate system is obtained; and the first coordinate information of the calibration pattern is determined according to the relative position relationship between the preset locator on the movable carrier and the calibration pattern.

[0008] In one embodiment, the movable carrier is further provided with a processor and a display screen connected to the processor; the processor is used to collect the coordinate value of the preset locator in the world coordinate system in real time and encode it, and display the encoded pattern obtained by encoding through the display screen;

[0009] The step of obtaining the coordinate value of the preset locator on the movable vehicle in the world coordinate includes:

[0010] By identifying the coding pattern on the vehicle in the image taken by the camera, the coordinate value of the preset locator corresponding to the coding pattern in the world coordinate is obtained.

[0011] In one embodiment, the movable carrier is further provided with a wireless communication module, and the wireless communication module is used to collect the coordinate values ​​of the preset locator in the world coordinate system in real time and send the coordinate values ​​by wireless communication;

[0012] The step of obtaining the coordinate value of the preset locator on the movable vehicle in the world coordinate includes:

[0013] Acquire the coordinate value of the preset locator in the world coordinate system sent by the wireless communication module.

[0014] In one embodiment, the acquiring of the second coordinate information of the calibration pattern on the movable carrier in the image captured by the camera of the movable carrier includes:

[0015] Respectively obtain images of the movable carrier at different positions taken by a camera, and respectively obtain pixel coordinate values ​​of at least one feature point on the calibration pattern at different positions through the images at different positions;

[0016] The obtaining the calibration parameters of the camera based on the first coordinate information and the second coordinate information includes:

[0017] Determining the world coordinate value of the at least one feature point on the calibration pattern according to the first coordinate information of the calibration pattern at the different positions on the movable carrier;

[0018] According to the first coordinate information and pixel coordinate value of at least one feature point on the calibration pattern at different positions, a multi-point perspective imaging PNP algorithm is used to construct multiple equations and jointly solve the installation parameters of the camera, where the installation parameters are the calibration parameters.

[0019] In one embodiment, the method further comprises: acquiring a plurality of images taken by a camera of the movable vehicle at each of the positions;

[0020] The pixel coordinate value of at least one feature point on the calibration pattern at each position is obtained by:

[0021] Based on multiple images taken at the same position, the pixel coordinate value of at least one feature point of the calibration pattern on the movable carrier in each image is calculated; based on the pixel coordinate value of at least one feature point of the calibration pattern on the movable carrier in each image, the pixel coordinate value of at least one feature point on the calibration pattern at each position in the image is determined.

[0022] In one embodiment, after obtaining the calibration parameters of the camera based on the first coordinate information and the second coordinate information, the method further includes:

[0023] Acquire an initial image, where the initial image is an image taken by a calibrated camera;

[0024] Acquire the detection image captured by the camera in the daily calibration operation after the calibration is completed;

[0025] According to the pixel information of the same marker in the initial image and the detection image, the change value of the calibration parameter of the camera when the relative calibration is completed is determined by the stream light method; and the calibration parameter of the camera when the calibration is completed is adjusted according to the change value.

[0026] In a second aspect, an embodiment of the present invention provides a movable carrier, comprising: a carrier body, a processor, a display screen and a locator; the processor, the display screen and the locator are arranged on the carrier body; the display screen and the locator are respectively connected to the processor;

[0027] The locator is used to locate its own coordinate value in the world coordinate system in real time and transmit the coordinate value to the processor;

[0028] The processor is used to encode the coordinate value, generate a corresponding encoding pattern and display it on the display screen;

[0029] At least one calibration pattern is also provided on the carrier body, and the calibration pattern is drawn on the carrier body or displayed on the carrier body through the display screen.

[0030] In one embodiment, the locator is used to obtain the coordinate value of its own location in real time by any one or more of the following methods:

[0031] UWB-based positioning, laser positioning or total station positioning.

[0032] In a third aspect, an embodiment of the present invention provides a camera calibration device, comprising: a first acquisition module, configured to acquire first coordinate information of a preset calibration pattern on a movable vehicle in a world coordinate system;

[0033] A second acquisition module, used to acquire second coordinate information of a calibration pattern on the movable vehicle in an image captured by a camera of the movable vehicle;

[0034] A calibration module is used to obtain calibration parameters of the camera based on the first coordinate information and the second coordinate information.

[0035] In a fourth aspect, an embodiment of the present invention provides a server, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned camera calibration method when executing the program.

[0036] In a fifth aspect, an embodiment of the present invention provides a camera, characterized in that it includes: a camera body, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned camera calibration method when executing the program.

[0037] In a sixth aspect, an embodiment of the present invention provides a camera calibration system, comprising a movable carrier, a camera and the above-mentioned server;

[0038] The camera is used to photograph the calibration pattern on the movable carrier and transmit the photographed image to the server.

[0039] In a seventh aspect, an embodiment of the present invention provides a camera calibration system, including: a mobile vehicle and a camera as described above;

[0040] The camera is used to photograph the calibration pattern on the movable carrier and perform camera calibration operations.

[0041] The beneficial effects of the above technical solution provided by the embodiment of the present invention include at least:

[0042] The camera calibration method provided by the embodiment of the present invention utilizes a movable carrier, sets a calibration pattern on the movable carrier, and infers the calibration parameters of the camera by obtaining the first coordinate information (i.e., world coordinate value) of the calibration pattern on the movable carrier in the world coordinates, and by obtaining the second coordinate information (i.e., pixel coordinate value) of the calibration pattern in the image captured by the movable carrier. The above process can be automatically completed by the camera or the algorithm server connected to the camera, and the entire calibration process can be performed without human intervention, and accurate calibration of the camera can be achieved, thereby improving the efficiency of camera calibration and saving labor costs. The problem of complex manual calibration operation process, low calibration accuracy, and inaccurate judgment of calibration results by manual participation in the prior art is avoided.

[0043] The movable carrier provided by an embodiment of the present invention includes a carrier body, a processor, a display screen and a locator; when the movable carrier moves to different positions, the coordinate values ​​of different positions of the calibration pattern on the movable carrier can be timely transmitted to the processor through the positioning of the locator, so as to display the real-time position coding pattern of the calibration pattern on the display screen, and by identifying the coding pattern, the coordinates of the world coordinate system of the locator can be obtained, and then the coordinates of at least one feature point on the calibration pattern in the world coordinate system can be determined, which provides a simple information acquisition means for determining the coordinate value of at least one feature point on the calibration pattern in the world coordinate system.

[0044] The carrier body of the movable carrier can be moved, which is convenient for collecting multiple images of the calibration pattern at the same position, and collecting images of the calibration pattern at multiple different positions. Multiple images of the calibration pattern at the same position can be used to avoid the influence of the external environment and internal parameter errors existing when the camera takes pictures on the calculation of the pixel coordinate value;

[0045] By combining the world coordinate value of at least one feature point in the calibration pattern at different positions, the world coordinate value and pixel coordinate value of the feature point in the image at different positions are calculated using a preset algorithm to determine the installation parameters of the camera, which can also reduce the influence of measurement errors and camera internal parameter errors.

[0046] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings.

[0047] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0049] Figure 1 A schematic diagram of laying a calibration pattern in the prior art;

[0050] Figure 2 One of the network architecture diagrams applicable to the camera calibration method in the embodiment of the present invention;

[0051] Figure 3 The second network architecture diagram applicable to the camera calibration method in the embodiment of the present invention;

[0052] Figure 4 A flowchart of a camera calibration method according to an embodiment of the present invention;

[0053] Figure 5 Schematic diagram of the structure of a movable carrier in an embodiment of the present invention (front view);

[0054] Figure 6 Schematic diagram of the structure of a movable carrier in an embodiment of the present invention (top view);

[0055] Figure 7 Schematic diagram of the structure of a camera calibration device in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] The exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present invention and to enable the scope of the present invention to be fully communicated to those skilled in the art.

[0057] It should be understood that the terms described in the present invention are only for describing a particular embodiment and are not intended to limit the present invention. In addition, for the numerical range in the present invention, it should be understood that each intermediate value between the upper and lower limits of the scope is also specifically invented. Each smaller range between the intermediate value in any stated value or stated range and any other stated value or intermediate value in the described range is also included in the present invention. The upper and lower limits of these smaller ranges can be independently included or excluded in the scope.

[0058] Unless otherwise indicated, all technical and scientific terms used herein have the same meanings as commonly understood by those skilled in the art to which the invention belongs. Although the present invention describes only preferred methods and materials, any methods and materials similar or equivalent to those described herein may also be used in the implementation or testing of the present invention. All documents mentioned in this specification are incorporated by reference to invent and describe methods and / or materials related to the documents. In the event of a conflict with any incorporated document, the content of this specification shall prevail.

[0059] In the description of the present invention, it should be noted that the terms "include", "including", "have", "contain" and the like are open terms, meaning including but not limited to.

[0060] The inventor of the present invention found in his work that in the process of camera calibration, it is necessary to manually lay out a calibration pattern for each camera and measure the position of the calibration pattern, which is time-consuming and laborious. In order to solve this problem, the inventor of the present invention has developed a movable carrier for camera calibration and a camera calibration method, which can achieve accurate calibration of camera installation parameters by obtaining the world coordinate value and pixel coordinate value of the calibration pattern on the movable carrier. The entire calibration process can be implemented by the camera or the algorithm server in the background connected to the camera, which does not require manual participation, saves labor costs, and has high calibration efficiency.

[0061] Before describing the camera calibration method provided by the embodiment of the present invention, the network architecture involved in the camera calibration method provided by the embodiment of the present invention is described:

[0062] Reference Figure 2 As shown, in some embodiments, the camera to be calibrated only has the function of taking images, and transmits the taken image data. Specifically, for example, the taken image data can be transmitted to an algorithm server; the algorithm server can communicate with the camera to be calibrated in a wireless or wired manner, and the camera can transmit the image data taken containing the movable vehicle to the algorithm server; the algorithm server executes the operation of calibrating the camera.

[0063] Or refer to Figure 3 As shown, in some embodiments, the camera to be calibrated has a conventional shooting function and a certain computing capability. At this time, the camera can shoot the movable vehicle and perform subsequent processing on the image data containing the movable vehicle to complete the calibration operation of the camera.

[0064] For the camera calibration method provided by the embodiment of the present invention, the execution subject may be a camera, or may be an algorithm server. The camera calibration method provided by the embodiment of the present invention is described in detail below.

[0065] Reference Figure 4 As shown, the camera calibration method provided by the embodiment of the present invention includes the following steps:

[0066] S41, obtaining first coordinate information of a preset calibration pattern on the movable carrier in a world coordinate system.

[0067] See also Figure 5 As shown, the movable carrier provided by the embodiment of the present invention includes a carrier body, a processor (not shown in the figure), a display screen and a locator; the processor, the display screen and the locator are arranged on the carrier body; the display screen and the locator are respectively connected to the processor; the locator is used to locate its own coordinate value in the world coordinate system in real time, and pass the coordinate value to the processor; the processor is used to encode the coordinate value, generate a corresponding coding pattern and display it on the display screen; at least one calibration pattern is also arranged on the carrier body, and the calibration pattern can be drawn on the carrier body, or displayed on the carrier body through the display screen.

[0068] For example, a locator can determine the coordinates of its own location in the following ways:

[0069] Ultra Wide Band (UWB) base station positioning, laser positioning or total station positioning.

[0070] The coding pattern is displayed on the display screen, and the camera or the algorithm server connected to the camera can identify the coordinate value of the locator itself recorded in the coding pattern through the captured picture.

[0071] The coding pattern can adopt any common coding method in the prior art, such as a QR code or other coding methods. For example, the size of the QR code is 25cm×25cm. Correspondingly, the size of the QR code will affect the size of the display screen. On the basis of ensuring the recognition of the QR code, the area of ​​the car occupied by the display screen is minimized as much as possible.

[0072] Reference Figure 6 As shown, from a top view, the calibration pattern can be set at the four corners and / or four sides of the movable carrier display screen, and the display screen capable of displaying a QR code can be set in the center of the movable carrier. Of course, the embodiment of the present invention is not limited to the above arrangement, and the display screen can be set at other suitable positions on the movable carrier for easy reading, and the calibration pattern can be distributed on one side or both sides of the display screen, or other arrangements can be adopted. The locator can be set on one side of the movable carrier.

[0073] The calibration pattern can be a common calibration pattern of black and white squares. One or more feature points are formed at the intersection of the black and white squares. The world coordinates and pixel coordinates of these feature points can be used to infer the installation parameters of the camera according to a preset algorithm. When multiple calibration patterns are set on the movable vehicle, the multiple calibration patterns can be the same or different, for example Figure 6 For example, the sizes of the calibration patterns set at the four corners and four sides of the display screen are different.

[0074] Preferably, the carrier body of the movable carrier can be various types of movable vehicles, such as a flatbed cart. In specific implementation, the flatbed cart can be moved manually, or the server can send control instructions to control the flatbed cart to move along a set route; in this way, the calibration pattern on the movable carrier can be controlled to move quickly from one position to another, and the operation is convenient and quick.

[0075] In a possible implementation, the calibration pattern may also be displayed on the movable carrier via the display screen.

[0076] Generally speaking, the flatbed cart and the calibration pattern set on the flatbed cart are realized by rigid objects, or the calibration pattern is also displayed on the display screen. Since the relative position relationship between the preset locator and the different feature points in the calibration pattern is fixed, you only need to know the world coordinates of the preset locator to accurately calculate the world coordinates and pixel coordinates of the feature points on the calibration pattern on the movable carrier.

[0077] In order to ensure the accurate positioning of the locator, the positioning method can be through ultra-wide band (UWB) positioning, laser positioning or using a total station positioning.

[0078] For example, the locator is used to obtain the coordinate value of its own location in real time by interacting with the UWB base station, which can ensure that the positioning accuracy reaches the centimeter level for indoor positioning, thereby improving the positioning accuracy.

[0079] On the movable carrier, multiple calibration patterns can be set, and these calibration patterns can be completely the same, or the calibration patterns can be the same pattern, or the multiple calibration patterns are not completely the same. By using the coordinate values ​​of different feature points on different calibration patterns (whether world coordinate values ​​or pixel coordinate values), the installation parameters of the camera are calibrated, which can avoid the influence of factors such as motion blur or focus blur on the results of the captured image when the camera is taking pictures, and improve the calibration accuracy.

[0080] In one embodiment, the coordinate value of the preset locator on the movable carrier itself in the world coordinate system is obtained, and the first coordinate information of the calibration pattern is determined according to the relative position relationship between the preset locator on the movable carrier and the calibration pattern.

[0081] Furthermore, the coordinate value of the preset locator on the movable vehicle in the world coordinate can be obtained in the following two ways:

[0082] Method 1: By identifying the coding pattern on the movable carrier in the image captured by the camera, the coordinate value of the preset locator corresponding to the coding pattern in the world coordinate is obtained.

[0083] As mentioned above, a processor and a display screen connected to the processor can also be set on the movable carrier; the processor is used to collect and encode the coordinate values ​​of the preset locator in the world coordinate system in real time, and display the encoded pattern through the display screen.

[0084] As the movable vehicle moves, the positioning results of the locator will also be updated in real time. Correspondingly, the display screen can also directly display the updated positioning results, that is, the world coordinates of the positioning can be observed more intuitively. The camera or algorithm server can obtain the real-time world coordinate value of the locator based on the recognition of the coded pattern in the acquired image, and can use the real-time world coordinate value of the locator to perform calculations to obtain the world coordinate value of the feature point on the calibration pattern in the world coordinate system.

[0085] Method 2: obtaining the coordinate value of the preset locator in the world coordinate system sent by the wireless communication module set on the movable carrier.

[0086] In this way, the movable carrier may not be provided with a display screen, but a wireless communication module may be provided. Through the wireless communication module, the coordinate value of the preset locator in the world coordinate system can be obtained in real time, and the coordinate value can be sent through wireless communication, for example, through wireless local area network (wifi), Bluetooth, infrared, etc. The wireless communication module adopts a communication module of the corresponding standard.

[0087] S42, obtaining second coordinate information of a calibration pattern on the movable carrier in an image captured by a camera of the movable carrier.

[0088] S43. Obtain calibration parameters of the camera based on the first coordinate information and the second coordinate information.

[0089] In one embodiment, through the movement of the movable vehicle, the camera can take pictures of the movable vehicle at different positions.

[0090] The images of the movable vehicle at different positions taken by the camera are respectively obtained, and the pixel coordinate values ​​of at least one feature point on the calibration pattern at different positions are respectively obtained through the images at different positions;

[0091] Correspondingly, in the above step S43, the calibration parameters of the camera can be calculated by a preset algorithm based on the first coordinate information and the second coordinate information.

[0092] After the calibration parameters are obtained, they can be stored for subsequent use in positioning and other scenarios.

[0093] Specifically, according to the first coordinate information of the calibration pattern at different positions on the movable carrier, the world coordinate value of at least one feature point on the calibration pattern is determined;

[0094] According to the first coordinate information and pixel coordinate value of at least one feature point on the calibration pattern at different positions, a multi-point perspective imaging (Perspective-n-Point, PnP) algorithm is used to construct multiple equations and jointly solve the installation parameters of the camera. The above installation parameters are calibration parameters.

[0095] The above installation parameters include but are not limited to: three spatial coordinate values ​​(x, y and z axes) and three Euler angles (ie, the angles between the optical center ray of the camera and the x, y, z axes).

[0096] The reason why the first coordinate information (world coordinate value) and pixel coordinate value of the feature points at different positions are needed is to use the PNP algorithm (similar to the least squares fitting method). At each position, the first coordinate information and pixel coordinate value of each feature point have a corresponding functional relationship with the installation parameters of the camera. This is to reduce the influence of measurement errors and camera intrinsic parameter errors. The PnP algorithm is used to calculate the camera extrinsic parameters (calculate the camera's position and posture (rotation matrix R and translation matrix t)) of photos at different positions, which can reduce the influence of measurement errors and camera intrinsic parameter errors, making the solved camera installation parameters more accurate.

[0097] In the PNP algorithm, the coordinates of the object in the world coordinate system and the pixel coordinates of the object in the camera's image plane are known to solve the camera's pose. The PNP algorithm can use multiple points, and the results obtained are more reliable.

[0098] The embodiment of the present invention may also use the minimum multiplication method to solve the camera's position and posture.

[0099] In order to further reduce the error of the pixel coordinate value (ie, the world coordinate value) of each feature point at each of the above-mentioned positions, multiple pictures can be taken for the movable vehicle at the same position at each of the above-mentioned multiple positions.

[0100] Specifically, the camera or algorithm server further includes executing the following steps: obtaining a plurality of images captured by the camera of the movable vehicle at each position;

[0101] The pixel coordinate value of at least one feature point on the calibration pattern at each position is obtained by:

[0102] Based on multiple images taken at the same position, the pixel coordinate value of at least one feature point of the calibration pattern on the movable carrier in each image is calculated; based on the pixel coordinate value of at least one feature point of the calibration pattern on the movable carrier in each image, the pixel coordinate value of at least one feature point on the calibration pattern at each position in the image is determined.

[0103] The pixel coordinate values ​​of the same feature point at the same position in different images are theoretically consistent, but are affected by various factors, such as the shooting environment, the camera's internal parameter error, measurement error, etc. The pixel coordinate values ​​of the same feature point in different images may actually be inconsistent. In order to obtain accurate pixel coordinate values, the images can be superimposed and calculated, and the pixel coordinate values ​​of each feature point at the position can be determined by taking the average value, etc., to reduce the impact of errors so that the final external parameters of the camera are more accurate.

[0104] The camera calibration method provided in the embodiment of the present invention obtains the world coordinates and pixel coordinates of the calibration pattern of the movable carrier in real time, thereby calculating the calibration parameters of the camera, thereby realizing automatic calibration of the camera. The entire calibration process does not require human intervention, has high calibration accuracy, improves calibration efficiency, and saves labor costs.

[0105] In the above process, after the camera calibration process is completed, if the camera is displaced due to the passage of time, such as being slightly loose, the embodiment of the present invention can also perform daily adjustments to the calibration parameters.

[0106] Specifically, for example, the following operations can be performed:

[0107] Obtain an initial image, which is an image taken by a calibrated camera. That is, the image taken by the calibrated camera is used as the initial image;

[0108] Obtain the detection images captured by the camera during the daily calibration operation after the calibration is completed;

[0109] According to the pixel information of the same marker in the initial image and the detection image, the change value of the calibration parameter of the camera when the relative calibration is completed is determined by the optical flow method; and the calibration parameter of the camera when the calibration is completed is adjusted according to the change value.

[0110] The above-mentioned daily calibration process is to adjust the calibration parameters of the calibrated camera, which can reduce the workload of recalibrating the camera.

[0111] The optical flow method is used to calculate the offset of feature lines or points on the same marker (marker line, fixed object) in the camera image in the initial image and the detection image taken by the camera during daily calibration operations. The offset of the camera angle can be inferred from the distance of this offset.

[0112] It should be noted that the markers used in daily calibration may be fixed markers in the garage, for example, the markers may be speed reduction lines or sidewalk lines in the garage, or they may be signs on the walls of the garage, etc. The offset of the camera angle may be inferred through calculation of the optical flow method based on the offset angle of the speed reduction lines, sidewalks or signboards.

[0113] Based on the same inventive concept, an embodiment of the present invention further provides a camera calibration device, a server, a camera and a camera calibration system. Since the principles of the problems solved by the camera calibration device, the server, the camera and the camera calibration system are similar to those of the aforementioned camera calibration method, the implementation of the device, the server, the camera and the camera calibration system can refer to the implementation of the aforementioned method, and the repeated parts will not be repeated.

[0114] The embodiment of the present invention also provides a camera calibration device, referring to Figure 7 As shown, including:

[0115] A first acquisition module 71 is used to acquire first coordinate information of a preset calibration pattern on the movable carrier in a world coordinate system;

[0116] A second acquisition module 72 is used to acquire second coordinate information of a calibration pattern on the movable vehicle in an image of the movable vehicle captured by a camera;

[0117] The calibration module 73 is used to obtain calibration parameters of the camera based on the first coordinate information and the second coordinate information.

[0118] In some optional embodiments, the first acquisition module 71 is further used to obtain the coordinate value of the locator itself on the movable carrier in the world coordinate system; and determine the first coordinate information of the calibration pattern based on the relative position relationship between the preset locator on the movable carrier and the calibration pattern.

[0119] In some optional embodiments, the movable carrier is also provided with a processor and a display screen connected to the processor; the processor is used to collect and encode the coordinate value of the preset locator in the world coordinate system in real time, and display the encoded coding pattern through the display screen; accordingly, the above-mentioned first acquisition module 71 is further used to identify the coding pattern on the carrier in the image taken by the camera, and obtain the coordinate value of the preset locator in the world coordinate system corresponding to the coding pattern.

[0120] In some optional embodiments, a wireless communication module is also provided on the movable carrier, and the wireless communication module is used to collect the coordinate value of the preset locator in the world coordinate system in real time, and send the coordinate value via wireless communication; the first acquisition module 71 is further used to obtain the coordinate value of the preset locator in the world coordinate system sent by the wireless communication module.

[0121] In some optional embodiments, the second acquisition module 72 is used to respectively acquire images of the movable vehicle at multiple different positions taken by the camera, and respectively acquire pixel coordinate values ​​of at least one feature point on the calibration pattern at different positions through the images at different positions;

[0122] Correspondingly, the above-mentioned calibration module 73 is used to determine the world coordinate value of at least one feature point on the calibration pattern according to the first coordinate information of the calibration pattern at the different positions on the movable carrier; according to the first coordinate information and pixel coordinate value of at least one feature point on the calibration pattern at the different positions, using the multi-point perspective imaging PNP algorithm, construct multiple equations and jointly solve the installation parameters of the camera, and the installation parameters are the calibration parameters.

[0123] In some optional embodiments, the second acquisition module is further used to acquire multiple images of the movable vehicle taken by a camera at each of the positions; calculate the pixel coordinate value of at least one feature point of the calibration pattern on the movable vehicle in each image based on the multiple images taken at the same position; determine the pixel coordinate value of at least one feature point on the calibration pattern at each position in the image based on the pixel coordinate value of at least one feature point of the calibration pattern on the movable vehicle in each image.

[0124] In some optional embodiments, the above-mentioned device also includes: the calibration module 73 is also used to use the image taken by the camera after the calibration is completed as the initial image; obtain the detection image taken by the camera in the daily calibration operation after the calibration is completed; determine the change value of the calibration parameters of the camera relative to the completion of the calibration through the optical flow method based on the pixel information of the same marker in the initial image and the detection image; and adjust the calibration parameters of the camera when the calibration is completed according to the change value.

[0125] Based on the inventive concept of the present invention, an embodiment of the present invention further provides a server, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above camera calibration method is implemented when the processor executes the program.

[0126] Based on the inventive concept of the present invention, an embodiment of the present invention also provides a camera, including: a camera body, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the above camera calibration method is implemented when the processor executes the program.

[0127] Based on the inventive concept of the present invention, an embodiment of the present invention further provides a vehicle, which includes the camera as described in the above embodiment.

[0128] Based on the inventive concept of the present invention, an embodiment of the present invention further provides a camera calibration system, comprising: a movable carrier, a camera and the above server;

[0129] The camera is used to capture the calibration pattern on the movable carrier and transmit the captured image to the server.

[0130] Based on the inventive concept of the present invention, an embodiment of the present invention further provides a camera calibration system, including: a movable carrier and a camera as above; the camera is used to photograph a calibration pattern on the movable carrier and perform camera calibration operations.

[0131] It should be understood that the specific order or hierarchy of steps in the inventive process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process can be rearranged without departing from the scope of the present invention. The attached method claims present the elements of the various steps in an exemplary order and are not intended to be limited to the specific order or hierarchy described.

[0132] In the above detailed description, various features are grouped together in a single embodiment to simplify the invention. This inventive approach should not be interpreted as reflecting an intention that the embodiments of the claimed subject matter require more features than are clearly stated in each claim. On the contrary, as reflected in the appended claims, the invention is in a state of having less than all the features of a single embodiment invented. Therefore, the appended claims are hereby expressly incorporated into the detailed description, with each claim standing alone as a separate preferred embodiment of the invention.

[0133] The above description includes examples of one or more embodiments. Of course, it is impossible to describe all possible combinations of components or methods for the purpose of describing the above embodiments, but it should be recognized by those skilled in the art that the various embodiments can be further combined and arranged. Therefore, the embodiments described herein are intended to cover all such changes, modifications and variations that fall within the scope of protection of the appended claims. In addition, with respect to the term "comprising" used in the specification or claims, the word is covered in a manner similar to the term "including", just as "including," explained as a transitional word in the claims. In addition, any term "or" used in the specification of the claims is intended to represent "non-exclusive or". The terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance.

Claims

1. A camera calibration method, characterized in that: include: Obtaining first coordinate information of a preset calibration pattern on the movable carrier in a world coordinate system; Acquire second coordinate information of a calibration pattern on the movable carrier in an image captured by a camera of the movable carrier; Obtaining calibration parameters of the camera based on the first coordinate information and the second coordinate information; The obtaining of first coordinate information of a preset calibration pattern on the movable carrier in a world coordinate system includes: Get the coordinate value of the locator itself on the movable vehicle in the world coordinate system; Determining first coordinate information of the calibration pattern according to a relative position relationship between the preset locator on the movable carrier and the calibration pattern; The acquiring of the second coordinate information of the calibration pattern on the movable carrier in the image captured by the camera of the movable carrier includes: The images of the movable carrier at different positions taken by the camera are respectively obtained, and the pixel coordinate values ​​of at least one feature point on the calibration pattern at different positions are respectively obtained through the images at different positions. The pixel coordinate values ​​of at least one feature point on the calibration pattern at different positions are the second coordinate information.

2. The method according to claim 1, characterized in that The movable carrier is also provided with a processor and a display screen connected to the processor; the processor is used to collect the coordinate value of the preset locator in the world coordinate system in real time and encode it, and display the encoded pattern obtained by encoding through the display screen; The step of obtaining the coordinate value of the preset locator on the movable vehicle in the world coordinate includes: By identifying the coding pattern on the vehicle in the image taken by the camera, the coordinate value of the preset locator corresponding to the coding pattern in the world coordinate is obtained.

3. The method according to claim 1, characterized in that The movable carrier is also provided with a wireless communication module, which is used to collect the coordinate values ​​of the preset locator in the world coordinate system in real time and send the coordinate values ​​by wireless communication; The step of obtaining the coordinate value of the preset locator on the movable vehicle in the world coordinate includes: Acquire the coordinate value of the preset locator in the world coordinate system sent by the wireless communication module.

4. The method according to any one of claims 1 to 3, characterized in that: The obtaining the calibration parameters of the camera based on the first coordinate information and the second coordinate information includes: Determining the world coordinate value of the at least one feature point on the calibration pattern according to the first coordinate information of the calibration pattern at the different positions on the movable carrier; According to the first coordinate information and the second coordinate information of the at least one feature point on the calibration pattern at the different positions, a multi-point perspective imaging PNP algorithm is used to construct multiple equations and jointly solve the installation parameters of the camera, and the installation parameters are the calibration parameters.

5. The method according to claim 4, characterized in that Also includes: Acquire a plurality of images taken by a camera of the movable vehicle at each of the positions; The pixel coordinate value of at least one feature point on the calibration pattern at each position is obtained by: Calculating the pixel coordinate value of at least one feature point of the calibration pattern on the movable carrier in each image according to a plurality of images taken at the same position; According to the pixel coordinate value of at least one feature point of the calibration pattern on the movable carrier in each image, the pixel coordinate value of at least one feature point on the calibration pattern at each position in the image is determined.

6. The method according to any one of claims 1 to 5, characterized in that: After obtaining the calibration parameters of the camera based on the first coordinate information and the second coordinate information, the method further includes: Acquire an initial image, where the initial image is an image taken by a calibrated camera; Acquire the detection image captured by the camera in the daily calibration operation after the calibration is completed; According to the pixel information of the same marker in the initial image and the detection image, the change value of the calibration parameter of the camera when the relative calibration is completed is determined by the optical flow method; and the calibration parameter of the camera when the calibration is completed is adjusted according to the change value.

7. A camera calibration device, characterized in that: include: A first acquisition module, used to acquire first coordinate information of a preset calibration pattern on the movable carrier in a world coordinate system; The obtaining of the first coordinate information of the preset calibration pattern on the movable carrier in the world coordinate system includes: obtaining the coordinate value of the locator itself on the movable carrier in the world coordinate system; determining the first coordinate information of the calibration pattern according to the relative position relationship between the preset locator on the movable carrier and the calibration pattern; A second acquisition module is used to acquire second coordinate information of the calibration pattern on the movable vehicle in the image taken by the camera of the movable vehicle; the acquiring the second coordinate information of the calibration pattern on the movable vehicle in the image taken by the camera of the movable vehicle comprises: respectively acquiring images of the movable vehicle at different positions taken by the camera, and respectively acquiring pixel coordinate values ​​of at least one feature point on the calibration pattern at different positions through the images at different positions, so as to obtain the second coordinate information of the calibration pattern; A calibration module is used to obtain calibration parameters of the camera based on the first coordinate information and the second coordinate information.

8. A server, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the camera calibration method according to any one of claims 1 to 6 is implemented.

9. A camera, characterized in that: include: A camera body, a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the camera calibration method as described in any one of claims 1 to 6 when executing the program.

10. A vehicle, characterized in that: Comprising the camera as claimed in claim 9.

11. A camera calibration system, characterized in that: include: A movable carrier, a camera and a server as claimed in claim 8; The camera is used to photograph the calibration pattern on the movable carrier and transmit the photographed image to the server.

12. A camera calibration system, characterized in that: include: A mobile carrier and a camera as claimed in claim 9; The camera is used to photograph the calibration pattern on the movable carrier and perform camera calibration operations.

Images