A camera calibration method and system in a free vision system

Abstract

This invention proposes a camera calibration method and system for a free vision system, comprising: obtaining the center point position of the camera and determining the camera coordinate system with the center point as the origin; obtaining the XY axis origin of the calibration board in the camera's image; and aligning the camera's coordinate system with the XY axis of the calibration board based on the X-axis and Y-axis distances of the camera's center point position from the origin of the calibration board, thereby calibrating the camera. Camera calibration is completed simply by aligning the coordinate system of the calibration board with the coordinate system of the camera, making the operation convenient.

Description

Technical Field

[0001] This invention belongs to the field of camera calibration, and particularly relates to a camera calibration method and system in a free vision system. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] With the development and rise of 5G technology and short videos, many new ways to stream e-commerce and sports events have emerged, with free-viewpoint being one of the most representative. Free-viewpoint allows users to watch the streamer, products, and the stadium from different angles, increasing user engagement and the fun of live video streaming.

[0004] The free-view system consists of dozens of cameras and a corresponding computing server. The center of the lens of each of the dozens of cameras needs to be aligned with the center of the arc, and the pose of the cameras needs to be calibrated at the same time.

[0005] Traditional camera positioning relies on visual estimation to determine the camera's horizontal and vertical alignment. More professional methods involve using expensive levels and laser equipment for calibration. The center position and XY axes are determined by attaching a laser to the camera's center and manually leveling the camera. This method is not only time-consuming but also prone to errors. Furthermore, the use of laser equipment is limited by the shape of the camera. Summary of the Invention

[0006] To overcome the shortcomings of the prior art, the present invention provides a camera calibration method and system in a free vision system. The camera calibration can be completed simply by aligning the coordinate system of the calibration plate with the coordinate system of the camera, which is easy to operate.

[0007] To achieve the above objectives, one or more embodiments of the present invention provide the following technical solution: a camera calibration method in a free vision system, comprising:

[0008] Obtain the center point position of the camera and determine the camera coordinate system with the center point as the origin;

[0009] Obtain the XY axis origin of the calibration board in the camera's image;

[0010] By aligning the camera's center point with the origin of the calibration board's XY axes, the camera's coordinate system is made to coincide with the calibration board's XY axes, thus achieving camera calibration.

[0011] Secondly, embodiments of the present invention provide a camera calibration system in a free vision system, comprising: a camera, a calibration board, and a calibration device, wherein the calibration board has coordinate axes;

[0012] The calibration device is used to calibrate the camera by aligning the camera's coordinate system with the XY axis of the calibration plate based on the distance relationship between the center point of the camera and the origin of the XY axis of the calibration plate.

[0013] Thirdly, embodiments of the present invention provide a computer device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the camera calibration method in the above-described free vision system are performed.

[0014] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the camera calibration method in a free vision system as described above.

[0015] The above one or more technical solutions have the following beneficial effects:

[0016] In this invention, camera calibration can be completed simply by aligning the coordinate system of the calibration plate with the coordinate system of the camera, making the operation convenient.

[0017] In this invention, the requirements for the camera are low, while the laser calibration method has higher requirements for the camera hardware. If relying solely on an additional laser device, the laser device itself is quite expensive. If the camera itself is required to have its own laser device, it would be a customized product, and the price would be far higher than the camera device used in this invention.

[0018] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0019] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0020] Figure 1 This is a schematic diagram of the calibration device in Embodiment 2 of the present invention;

[0021] Figure 2 This is a schematic diagram of the calibration process of the calibration device in Embodiment 2 of the present invention;

[0022] Figure 3This is a schematic diagram of the calibration process in Embodiment 2 of the present invention. Detailed Implementation

[0023] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular implementations only and is not intended to limit the exemplary implementations of the present invention.

[0025] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0026] Example 1

[0027] This embodiment discloses a camera calibration method in a free vision system, including:

[0028] Obtain the center point position of the image captured by the camera, and determine the origin of the camera image coordinate system with the center point of the image captured by the camera as the origin, while setting the XY axis along the origin;

[0029] Obtain the XY axis origin of the calibration board in the image captured by the camera;

[0030] Based on the distance between the center point of the captured image and the X-axis and Y-axis distance between the center point and the origin of the calibration board in the captured image, the camera's coordinate system is manually aligned to coincide with the XY axis of the calibration board. That is, by operating the camera to change its position, the software interface is observed to confirm whether the center point of the captured image coincides with the origin of the calibration board in the captured image, thus achieving camera calibration.

[0031] In this embodiment, a calibration plate with coordinate axes drawn on it is used, wherein the distance between the calibration plate and the camera is in the range of 2-20m, the camera is mounted on a tripod, and calibration software is used.

[0032] In this embodiment, the location of the center point of the captured image and the coordinate system of the camera, which is determined by the center point of the captured image as the origin, are specifically as follows:

[0033] The calculation of the center point of the image captured by the camera and the drawing of the coordinate axes are mainly achieved by finding the resolution center point of the image captured by the camera and drawing two perpendicular line segments parallel to the width and height of the captured image at the center point.

[0034] The following formula is used to calculate the center point of the image captured by the camera:

[0035] W_center = W_real / 2

[0036] H_center = H_real / 2

[0037] In the above formula, W_center and H_center are the width and height pixel values ​​of the center point of the camera's captured image, respectively. W_real and H_real are the width and height values ​​of the camera's captured image, respectively. The width and height values ​​of the captured image are typically 1920*1080, set manually. The software implements the above calculation formula in code. By inputting the preset width and height values ​​of the captured image, the pixel value of the center point of the camera's captured image can be calculated. Using the pixel value of the center point of the camera's captured image as the origin, horizontal and vertical lines are added to the captured image to achieve the XY axis, and the image is then input to the GPU graphics card for rendering on the monitor.

[0038] In this embodiment, the coordinate lines of the camera image and the coordinate lines on the cardboard in the captured image are aligned and calibrated. As long as the two coordinate systems coincide, the accurate free-view pose can be calibrated.

[0039] Example 2

[0040] This embodiment discloses a camera calibration system in a free vision system, including: a camera, a calibration board and a calibration device, wherein the calibration board has coordinate axes;

[0041] The calibration device is used to calibrate the camera by aligning the camera's coordinate system with the XY axis of the calibration plate based on the distance relationship between the center point of the camera and the origin of the XY axis of the calibration plate.

[0042] like Figures 1-3 As shown, in this embodiment, calibration software is used for the calibration device, including the initialization and configuration of the camera SDK, the acquisition and decoding of H264 / H265 video streams, the video stream buffer algorithm (to ensure that the video stream is not interrupted and is real-time), the calculation of the center point position of the camera captured image, the drawing of the camera coordinate axis in the video, the real-time display of the coordinate axis and the captured video image.

[0043] The initial configuration of the camera SDK mainly involves encoder selection, bitrate setting, and resolution setting. This embodiment uses a 1080P 8Mbps H.265 encoding as the final configuration. This is because most mainstream cameras support 1080P resolution, and the H.265 encoder offers over 40% higher compression efficiency than the H.264 encoder. For 1080P resolution, an 8Mbps bitrate maximizes color fidelity.

[0044] H.265 video stream acquisition and decoding includes registering the video stream callback interface (encapsulated in a C++ class), initializing and configuring the HEVC hardware decoder, and asynchronous decoding using threads. HEVC hardware decoding fully utilizes the NVIDIA graphics card's hardware decoder to achieve high-speed, real-time video decoding; decoding one frame takes approximately 10ms. Utilizing threads involves storing the obtained H.265 bitstream in a buffer within the registered interface and then using a separate thread to call and decode the HEVC decoder, achieving asynchronous operation, saving time, and enhancing real-time performance.

[0045] The video buffer algorithm refers to storing the decoded video into a queue buffer. The algorithm calculates and maintains the size of the queue, because a queue that is too large will cause video stuttering and latency, while a queue that is too small will cause video frame loss. The following formula is used:

[0046] Q = B_i / B_s + C

[0047] The above formula is for the dynamic queue size, where Q is the actual queue size, B_i is the HEVC decoder's decoding frame rate, B_s is the playback frame rate, and C is a constant to ensure the queue is not empty. By dynamically maintaining the size of the video buffer, real-time, stutter-free video streaming can be provided.

[0048] Example 3

[0049] The purpose of this embodiment is to provide a computing device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the above-described method.

[0050] Example 4

[0051] The purpose of this embodiment is to provide a computer-readable storage medium.

[0052] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the steps of the above method.

[0053] The steps and methods involved in the apparatuses of Embodiments 2, 3, and 4 above correspond to those in Embodiment 1. For specific implementation details, please refer to the relevant description section of Embodiment 1. The term "computer-readable storage medium" should be understood as a single medium or multiple media including one or more instruction sets; it should also be understood as including any medium capable of storing, encoding, or carrying an instruction set for execution by a processor and enabling the processor to perform any of the methods in this invention.

[0054] Those skilled in the art will understand that the modules or steps of the present invention described above can be implemented using general-purpose computer devices. Optionally, they can be implemented using computer-executable program code, thereby allowing them to be stored in a storage device for execution by a computer device, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. The present invention is not limited to any particular combination of hardware and software.

[0055] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A camera calibration method in a free vision system, characterized in that, include: Obtain the center point position of the camera and determine the camera coordinate system with the center point as the origin; Specifically, the camera coordinate system is determined by the center point of the image captured by the camera and by drawing two perpendicular line segments parallel to the width and height of the captured image at the center point. Obtain the XY axis origin of the calibration board in the camera's image; By aligning the camera's center point with the origin of the calibration board's XY axes, the camera's coordinate system is made to coincide with the calibration board's XY axes, thus achieving camera calibration. By manipulating the camera to make the center point of the captured image coincide with the origin of the calibration board in the captured image, the camera coordinate system coincides with the XY axis origin of the calibration board, thus achieving camera calibration.

2. The camera calibration method in a free vision system as described in claim 1, characterized in that, The distance between the camera and the calibration plate is 2-20m.

3. A camera calibration system in a free vision system, employing the calibration method as described in any one of claims 1-2, characterized in that, include: A camera, a calibration plate, and a calibration device, wherein the calibration plate has coordinate axes; The calibration device is used to calibrate the camera by aligning the camera's coordinate system with the XY axis of the calibration plate based on the distance relationship between the center point of the camera and the origin of the XY axis of the calibration plate.

4. The camera calibration system in a free vision system as described in claim 3, characterized in that, The calibration device uses calibration software, which calculates the origin of the camera coordinate system based on the center point of the resolution of the image captured by the camera.

5. The camera calibration system in a free vision system as described in claim 3, characterized in that, By manipulating the camera to make the center point of the captured image coincide with the origin of the calibration board in the captured image, the camera coordinate system coincides with the XY axis origin of the calibration board, thus achieving camera calibration.

6. The camera calibration system in a free vision system as described in claim 3, characterized in that, The distance between the camera and the calibration plate is 2-20m.

7. A computer device, characterized in that, include: The system includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, they perform the steps of a camera calibration method in a free vision system as described in any one of claims 1 to 2.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of a camera calibration method in a free vision system as described in any one of claims 1 to 2.

Images