A method and system for improving the tracking accuracy of a camera

Abstract

This application relates to the field of video processing, and particularly to a method and system for improving the tracking accuracy of a camera. The method includes: obtaining the motion data of the camera, where the motion data includes a first translation vector and a first Euler angle; based on the first translation vector, the first Euler angle, and a conversion formula, obtaining the first motion pose of the camera based on the world coordinate system, where the conversion formula is a formula for converting the first Euler angle into a first rotation matrix; obtaining the video stream collected by the camera, and obtaining the coordinate matrix of each corner point of the Aruco code in the video stream; based on the coordinate matrix and the camera internal parameter matrix, obtaining the second motion pose of the camera; and performing weighted fusion on the first motion pose and the second motion pose to obtain the target motion pose of the camera. This application can improve the tracking accuracy of the camera, ensure the accuracy and continuity of the tracking data, and at the same time reduce the input cost.

Description

Technical Field

[0001] This application relates to the field of video processing, and particularly to a method and system for improving the tracking accuracy of a camera. Background Art

[0002] In the shooting of virtual studio programs, the shooting method of a moving camera is often used. The camera moves on a slide rail or a jib to shoot a foreground (such as a host) against a specific background color (usually blue or green). The foreground is keyed out from the background color by a keyer, and the foreground is superimposed with a virtual background (picture, video or 3D scene) generated by a computer. When the camera moves, in order to keep the virtual background in sync with the foreground captured in the moving picture of the camera, it is necessary to track the motion pose (rotation matrix and translation vector) of the camera in real time.

[0003] The traditional method is to install sensors on the slide rail or the jib to measure the position of the camera, and install gear sensors at the position of the moving part on the slide rail or the jib and the rotating shaft connected to the camera, so as to calculate the world coordinates and shooting angle of the camera, so that the virtual background moves synchronously. However, due to the limitations of the accuracy of the mechanical and sensors, it is impossible to solve the tracking data errors caused by problems such as jitter caused by inertia and non-rigid structures, resulting in inaccurate tracking data.

[0004] Currently, in order to improve the problem of inaccurate tracking data, some manufacturers install laser receivers on the camera and install multiple laser transmitters in the studio to obtain multiple laser angle data within a certain period of time, and thus obtain a relatively accurate camera tracking result. However, since the laser is easily blocked by the camera and its connection structure, it is difficult to ensure the continuity of the tracking data. At the same time, it is necessary to install laser receivers and transmitters, thus increasing the investment cost. Summary of the Invention

[0005] In order to solve or partially solve the problems existing in the related art, a method and system for improving the tracking accuracy of a camera disclosed in this application can improve the tracking accuracy of the camera, ensure the accuracy and continuity of the tracking data, and at the same time reduce the investment cost.

[0006] First aspect, the present application provides a method for improving the tracking accuracy of a camera, adopting the following technical solutions: A method for improving the tracking accuracy of a camera, comprising: obtaining motion data of the camera, wherein the motion data includes a first translation vector and a first Euler angle; based on the first translation vector, the first Euler angle and a conversion formula, obtaining a first motion pose of the camera based on the world coordinate system, wherein the conversion formula is a formula for converting the first Euler angle into a first rotation matrix; obtaining a video stream collected by the camera, and obtaining a coordinate matrix of each corner point of the Aruco code in the video stream; based on the coordinate matrix and the camera internal parameter matrix, obtaining a second motion pose of the camera; and performing weighted fusion on the first motion pose and the second motion pose to obtain a target motion pose of the camera.

[0007] By adopting the above technical solutions, by obtaining the first translation vector and the first Euler angle and using the formula for converting the first Euler angle into the first rotation matrix, a first motion pose of the camera based on the world coordinate system can be obtained. By obtaining the coordinate matrix of each corner point of the Aruco code and the camera internal parameter matrix from the video stream, a second motion pose of the camera can be obtained, and then by performing weighted fusion on the first motion pose and the second motion pose, a target motion pose can be obtained. The overall technical solution can improve the tracking accuracy of the camera, ensure the accuracy and continuity of the tracking data, and at the same time reduce the input cost.

[0008] Optionally, the obtaining the initial video stream collected by the camera and identifying the coordinate matrix of each corner point of the Aruco code includes: obtaining the video stream collected by the camera; performing distortion correction on the video stream based on a preset distortion correction matrix to obtain a target video stream; obtaining the coordinate matrix of each corner point of the Aruco code in the target video stream, wherein the coordinate matrix includes a first coordinate matrix and a second coordinate matrix; the first coordinate matrix is the coordinate matrix of each corner point of the Aruco code on the plane of the target video stream, and the second coordinate matrix is the coordinate matrix of each corner point of the Aruco code in the world coordinate system.

[0009] By adopting the above technical solutions, by performing distortion correction on the collected video stream, the accuracy of the coordinate matrix of each corner point of the Aruco code obtained subsequently can be ensured, thereby ensuring the accuracy of the target motion pose and improving the tracking accuracy of the camera; by obtaining the first coordinate matrix of each corner point of the Aruco code in the target video stream on the plane of the target video stream and the second coordinate matrix in the world coordinate system, it is convenient for calculating the second motion pose.

[0010] Optionally, before obtaining the motion data of the camera, it further includes: determining the origin of the world coordinate system; wherein, the first translation vector is the amount of position change relative to the origin, and the first Euler angle is the angle of rotation about the x-axis, y-axis, and z-axis relative to the world coordinate system.

[0011] By adopting the above technical solution, by determining the origin of the world coordinate system, it is convenient to determine the first translation vector and the first Euler angle to calculate the first motion posture of the camera, and it is also convenient to determine the coordinate matrix of each corner point of the Aruco code to calculate the second motion posture of the camera.

[0012] Optionally, the calculation formula for the first motion posture is: M’ = [f(R’)|T’]; where M’ is the first motion posture, R’ is the first Euler angle, f is the formula for converting the Euler angle into a rotation matrix; T’ is the first translation vector.

[0013] By adopting the above technical solution, based on M’ = [f(R’)|T’], the first motion posture can be directly obtained from the acquired first Euler angle and the first translation vector, thereby reflecting the simplicity of obtaining the first motion posture data.

[0014] Optionally, the second motion posture is calculated by: M image = M camera · [R A ’|T A · M world ; where [R A ’|T A is the second motion posture, R A ’ is the second rotation matrix, T A is the second translation vector, M image is the first coordinate matrix, M camera is the camera internal parameter matrix, and M world is the second coordinate matrix.

[0015] By adopting the above technical solution, by substituting the first coordinate matrix, the camera internal parameter matrix, and the second coordinate matrix into M image = M camera · [R A ’|T A · M world can directly obtain the second motion posture, thereby reflecting the simplicity of obtaining the second motion posture data.

[0016] Optionally, M A = [f(R A )|T A ; R A’ = f(R A );

[0017] θ x = atan2(r 32 , r 33 ); θ z = atan2(r 21 , r 11 ); R A = (θ x , θ y , θ z ); where, [R A ’|T A and M A are both the second motion postures; R A is the second Euler angle, θ x is the pitch angle of the camera, θ y is the yaw angle of the camera, θ z is the roll angle of the camera.

[0018] By adopting the above technical solution, through the conversion relationship between the formulas, the second translation vector T A and the second Euler angle R A can be calculated, so as to calculate the second motion posture, so as to facilitate the weighted fusion of the first motion posture and the second motion posture to obtain the target motion posture of the camera.

[0019] Optionally, the formula for the weighted fusion is: M = [f(α·R A + (1 - α)·R’)|β·T A + (1 - β)·T’]; where, M is the target motion posture, α is the first preset weight value, and β is the second preset weight value.

[0020] By adopting the above technical solution, through the formula for weighted fusion, the accurate target motion posture of the camera can be obtained based on the known second Euler angle R A , the first Euler angle R’, the formula f for converting the Euler angle to the rotation matrix, the first translation vector T’, the second translation vector T A , the first preset weight value α and the second preset weight value β.

[0021] Second aspect, the present application provides a system for improving the tracking accuracy of a camera, adopting the following technical solutions: A system for improving the tracking accuracy of a camera includes: a data acquisition module for acquiring the motion data of the camera, where the motion data includes a first translation vector and a first Euler angle; a first motion attitude acquisition module for obtaining the first motion attitude of the camera based on the world coordinate system based on the first translation vector, the first Euler angle, and a conversion formula, where the conversion formula is used to convert the first Euler angle into a first rotation matrix; a coordinate matrix acquisition module for acquiring the video stream collected by the camera and identifying the coordinate matrix of each corner point of the Aruco code in the video stream; a second motion attitude acquisition module for obtaining the second motion attitude of the camera based on the coordinate matrix and the camera internal parameter matrix; a target motion attitude acquisition module for performing weighted fusion on the first motion attitude and the second motion attitude to obtain the target motion attitude of the camera.

[0022] By adopting the above technical solutions, the first translation vector and the first Euler angle can be obtained through the data acquisition module, and then through the first motion attitude acquisition module, using the formula for converting the first Euler angle into the first rotation matrix, the first motion attitude of the camera based on the world coordinate system can be obtained. The coordinate matrix acquisition module obtains the coordinate matrix of each corner point of the Aruco code based on the video stream, and then through the second motion attitude acquisition module, based on the coordinate matrix and the camera internal parameter matrix, the second motion attitude of the camera can be obtained. Then, through the target motion attitude acquisition module, the first motion attitude and the second motion attitude can be weighted and fused to obtain the target motion attitude. Its overall technical solution can improve the tracking accuracy of the camera, ensure the accuracy and continuity of the tracking data, and at the same time reduce the input cost.

[0023] Optionally, the coordinate matrix acquisition module includes: a video stream acquisition unit for acquiring the video stream collected by the camera; a distortion correction unit for performing distortion correction on the video stream based on a preset distortion correction matrix to obtain a target video stream; a matrix acquisition unit for acquiring the coordinate matrix of each corner point of the Aruco code in the target video stream, where the coordinate matrix includes a first coordinate matrix and a second coordinate matrix; the first coordinate matrix is the coordinate matrix of each corner point of the Aruco code on the plane of the target video stream, and the second coordinate matrix is the coordinate matrix of each corner point of the Aruco code in the world coordinate system.

[0024] By adopting the above technical solution, the video stream acquisition unit can acquire the video stream collected by the camera, and the distortion correction unit can perform distortion correction on the video stream based on a preset distortion correction matrix to obtain a target video stream, so as to ensure the accuracy of the target motion posture and improve the accuracy of camera tracking; the matrix acquisition unit can acquire the first coordinate matrix of each corner point of the Aruco code in the target video stream on the plane of the target video stream and the second coordinate matrix in the world coordinate system, so as to facilitate the calculation of the second motion posture.

[0025] Optionally, a system for improving the accuracy of camera tracking further includes an origin determination module for determining the origin of the world coordinate system; wherein, the first translation vector is the amount of position change relative to the origin, and the first Euler angle is the angle of rotation around the x-axis, y-axis, and z-axis relative to the world coordinate system.

[0026] By adopting the above technical solution, the origin determination module can determine the origin of the world coordinate system, so as to facilitate the determination of the first translation vector and the first Euler angle to calculate the first motion posture of the camera, and facilitate the determination of the coordinate matrix of each corner point of the Aruco code to calculate the second motion posture of the camera.

[0027] In summary, the present application includes at least one of the following beneficial technical effects:

[0028] 1. By obtaining the first translation vector and the first Euler angle and using the formula for converting the first Euler angle into the first rotation matrix, the first motion posture of the camera based on the world coordinate system can be obtained. By obtaining the coordinate matrix of each corner point of the Aruco code and the camera internal parameter matrix from the video stream, the second motion posture of the camera can be obtained. Then, by weighted fusion of the first motion posture and the second motion posture, the target motion posture can be obtained. The overall technical solution can improve the accuracy of camera tracking, ensure the accuracy and continuity of tracking data, and reduce the input cost at the same time.

[0029] 2. By performing distortion correction on the collected video stream, the accuracy of the coordinate matrix of each corner point of the subsequent obtained Aruco code can be ensured, thereby ensuring the accuracy of the target motion posture and improving the accuracy of camera tracking; by obtaining the first coordinate matrix of each corner point of the Aruco code in the target video stream on the plane of the target video stream and the second coordinate matrix in the world coordinate system, the calculation of the second motion posture can be facilitated.

[0030] 3. By determining the origin of the world coordinate system, it is convenient to determine the first translation vector and the first Euler angle to calculate the first motion posture of the camera, and it is also convenient to determine the coordinate matrix of each corner point of the Aruco code to calculate the second motion posture of the camera.

[0031] 4. By using the obtained first Euler angles and the first translation vector, the first motion posture can be directly obtained based on M’ = [f(R’)|T’], thus reflecting the simplicity of obtaining the first motion posture data. Description of the Drawings

[0032] Figure 1 FIG. is a schematic diagram of a scene in a simulated virtual studio for applying a method for improving the tracking accuracy of a camera disclosed in an embodiment of the present application;

[0033] Figure 2 FIG. is a flowchart of a method for improving the tracking accuracy of a camera disclosed in an embodiment of the present application;

[0034] Figure 3 FIG. is another flowchart of a method for improving the tracking accuracy of a camera disclosed in an embodiment of the present application;

[0035] Figure 4 FIG. is a schematic diagram of modules of a system for improving the tracking accuracy of a camera disclosed in another embodiment of the present application;

[0036] Figure 5 FIG. is a schematic diagram of the structure of an electronic device shown in another embodiment of the present application.

[0037] Description of the Reference Numerals:

[0038] 10. Data acquisition module; 20. First motion posture acquisition module; 30. Coordinate matrix acquisition module; 40. Second motion posture acquisition module; 50. Motion posture acquisition module. Detailed Embodiments

[0039] The embodiments of the present application will be described in more detail below with reference to the drawings. Although the embodiments of the present application are shown in the drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0040] The terms used in the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. The singular forms of "a", "the" and "said" used in the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0041] It should be understood that although the terms "first", "second", etc. may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of this application, the meaning of "a plurality of" is two or more, unless otherwise specifically defined.

[0042] In the related art, in order to improve the problem of inaccurate tracking data, some manufacturers install laser receivers on cameras and install multiple laser transmitters in the studio to obtain multiple laser angle data within a certain period of time, and thus obtain a relatively accurate camera tracking result. However, since the laser is easily blocked by the camera and its connection structure, it is difficult to ensure the continuity of the tracking data. At the same time, it is necessary to install laser receivers and transmitters, which increases the input cost.

[0043] Therefore, in order to solve the above technical problems, this application discloses a method and system for improving the tracking accuracy of a camera, which can save the user's label production time and at the same time reduce the error rate of label production, so as to facilitate the user's use and improve the user experience.

[0044] The following will Figures 1-5 describe in detail the technical solutions of the embodiments of this application.

[0045] See Figure 1 , which is a scene in a simulated virtual studio. Aruco codes for visual recognition and positioning are set on the background of the studio. In order not to affect the later matte extraction and the recognition of the Aruco codes, the background and the Aruco codes can be set to different shades of green. Among them, in order to better recognize the Aruco codes, the outer edge of the Aruco code can be set to dark green (R:0 G:150 / 255 B:0), and the inside can be set to light green (R:0 G:250 / 255 B:0). The camera can move on the slide rail to collect the foreground (person or object) in the background. It should be noted here that in this embodiment, the settings of the colors and depths of the background and the Aruco codes are not limited.

[0046] See Figure 2 , which is a schematic flowchart of a method for improving the tracking accuracy of a camera in an embodiment of this application, and specifically includes the following steps:

[0047] S10. Obtain the motion data of the camera;

[0048] Among them, the motion data includes the first translation vector and the first Euler angles. The origin position relative to the first translation vector can be Figure 1 the position where the camera shown in Figure 1 is located at the leftmost side of the slide rail. The first Euler angles are the angles by which the camera rotates around the x-axis, y-axis, and z-axis of the world coordinate system established at the leftmost side of the slide rail. Of course, here, the origin position and the establishment position of the world coordinate system are not limited and can be set according to the actual situation.

[0049] S20. Based on the first translation vector, the first Euler angles, and the conversion formula, obtain the first motion pose of the camera based on the world coordinate system;

[0050] Among them, the conversion formula can be understood as a formula for converting the first Euler angles into the first rotation matrix. For example, let the Euler angles be (roll, pitch, yaw), and the rotation matrix be R, R = R x (rolI)·R y (pitch)·R z (yaw), where R x 、R y 、R z are the rotation matrices around the x, y, and z axes respectively, and the specific forms can be as follows:

[0051] R x (roll) = [[1, 0, 0], [0, cos(roll), -sin(roll)], [0, sin(roll), cos(roll)]];

[0052] R y (pitch) = [[cos(pitch), 0, sin(pitch)], [0, 1, 0], [-sin(pitch), 0, cos(pitch)]];

[0053] R z (yaw) = [[cos(yaw), -sin(yaw), 0], [sin(yaw), cos(yaw), 0], [0, 0, 1]];

[0054] Among them, the angle units in the sin and cos functions are radians.

[0055] The acquisition of the first motion pose can be obtained by directly measuring the first translation vector and the first Euler angles and using the conversion formula.

[0056] S30. Obtain the video stream collected by the camera and obtain the coordinate matrix of each corner point of the Aruco code in the video stream;

[0057] Among them, the coordinate matrix of each corner point of the Aruco code can be measured according to its position in the collected video stream and the established world coordinate system (for example, the origin is located at Figure 1 the leftmost position of the slide rail). Among them, the coordinate matrix obtained according to the position in the collected video stream can be recognized by using the Aruco library in OpenCv. By obtaining the positions of each corner point of the Aruco code, the foreground can be located and recognized. This positioning and recognition method can be recognized by using the solvePNP method of opencv to accurately track the foreground, so as to facilitate subsequent synchronization of the virtual background with the foreground captured during the movement of the camera.

[0058] S40. Obtain the second motion pose of the camera based on the coordinate matrix and the camera internal parameter matrix;

[0059] Among them, the camera internal parameter matrix is obtained by pre-calibrating the camera. In this embodiment, it is a known variable. The internal parameter matrix of the camera is related technology and will not be elaborated here.

[0060] S50. Perform weighted fusion on the first motion pose and the second motion pose to obtain the target motion pose of the camera.

[0061] Among them, weighted fusion can be understood as fusing and calculating the first motion pose and the second motion pose through a weighted fusion algorithm to obtain the target motion pose of the camera.

[0062] In another embodiment, referring to Figure 3 , step S30 includes:

[0063] S31. Obtain the video stream collected by the camera;

[0064] Among them, the video stream can be understood as the foreground video captured by the camera in a stationary and / or moving state.

[0065] S32. Perform distortion correction on the video stream based on a preset distortion correction matrix to obtain a target video stream;

[0066] For example, V’ = V·M distortion , V’ is the corrected video stream, that is, the target video stream; V is the video stream captured by the camera; M distortion is the preset distortion correction matrix of the camera. It should be noted here that the preset distortion correction matrix can be obtained by calibrating the camera using the Zhang's calibration method and will not be elaborated here.

[0067] S33. Obtain the coordinate matrix of each corner point of the Aruco code in the target video stream;

[0068] Among them, the coordinate matrix includes a first coordinate matrix and a second coordinate matrix. The first coordinate matrix is the coordinate matrix of each corner point of the Aruco code on the plane of the target video stream; the second coordinate matrix is the coordinate matrix of each corner point of the Aruco code in the world coordinate system, so as to accurately identify and locate the foreground.

[0069] In another embodiment, before step S10, it further includes:

[0070] S01. Determine the origin of the world coordinate system;

[0071] Among them, the first translation vector is the amount of position change relative to the origin, and the first Euler angle is the angle of rotation around the x-axis, y-axis, and z-axis relative to the world coordinate system. By first confirming the origin of the world coordinate system, it is convenient to determine the first translation vector and the first Euler angle to calculate the first motion posture of the camera, and it is also convenient to determine the coordinate matrix of each corner point of the Aruco code to calculate the second motion posture of the camera.

[0072] In another embodiment, the calculation formula for the first motion posture is: M’ = [f(R’)|T’];

[0073] Among them, M’ is the first motion posture, R’ is the first Euler angle, f is the formula for converting the Euler angle into a rotation matrix, which can refer to the description of the conversion formula in step S20 above; T’ is the first translation vector.

[0074] In another embodiment, the second motion posture is calculated by: M image = M camera ·[R A ’|T A ·M world ;

[0075] Among them, [R A ’|T A is the second motion posture, R A ’ is the second rotation matrix, T A is the second translation vector, M image is the first coordinate matrix, M camera is the camera internal parameter matrix, M world is the second coordinate matrix, so as to realize obtaining the second motion posture [R image ’|T camera through the known M world , M A and M A .

[0076] In another embodiment, M A = [f(R A )|T A; R A ’ = f(R A );

[0077] θ x = atan2(r 32 , r 33 ); θ z = atan2(r 21 , r 11 ); R A = (θ x , θ y , θ z );

[0078] Wherein, [R A ’|T A and MA both represent the second motion posture; R A is the second Euler angle, θ x is the pitch angle of the camera, θ y is the yaw angle of the camera, θ z is the roll angle of the camera, to calculate R A ’ through [R A ’|T A and MA, that is, to obtain the values of r 11 , r 21 , r 31 , r 32、 r 33 , and then substitute them into the formula of θ x , θ y , θ z to obtain θ x , θ y , θ z , and further obtain the second Euler angle R A and the second translation vector T A .

[0079] In another embodiment, the formula for weighted fusion can be: M = [f(α·R A +(1 - α)·R’)|β·T A +(1 - β)·T’]; wherein, M is the target motion posture, α is the first preset weight value, β is the second preset weight value. Combining the first Euler angle R’, the first translation vector T’ obtained from the above measurement and the second Euler angle R A and the second translation vector T A calculated above, the target motion posture of the camera can be calculated. Regarding the first preset weight value α and the second preset weight value β as known variables, it can be understood that the operator pre-sets the values according to the actual situation, and is not limited in this embodiment.

[0080] Please refer to Figure 4 In another embodiment of the present application, a system for improving the tracking accuracy of a camera is disclosed, including: a data acquisition module 10, a first motion attitude acquisition module 20, a coordinate matrix acquisition module 30, a second motion attitude acquisition module 40, and a motion attitude acquisition module 50.

[0081] Among them, the data acquisition module 10 is used to acquire the motion data of the camera, and the motion data includes a first translation vector and a first Euler angle; the first motion attitude acquisition module 20 is used to obtain the first motion attitude of the camera based on the world coordinate system based on the first translation vector, the first Euler angle, and a conversion formula, where the conversion formula is a formula for converting the first Euler angle into a first rotation matrix; the coordinate matrix acquisition module 30 is used to acquire the video stream collected by the camera and identify the coordinate matrix of each corner point of the Aruco code in the video stream; the second motion attitude acquisition module 40 is used to obtain the second motion attitude of the camera based on the coordinate matrix and the camera internal parameter matrix; the target motion attitude acquisition module 50 performs weighted fusion on the first motion attitude and the second motion attitude to obtain the target motion attitude of the camera.

[0082] In another embodiment, the coordinate matrix acquisition module 30 includes: a video stream acquisition unit 31 for acquiring the video stream collected by the camera; a distortion correction unit 32 for performing distortion correction on the video stream based on a preset distortion correction matrix to obtain a target video stream; a matrix acquisition unit 33 for acquiring the coordinate matrix of each corner point of the Aruco code in the target video stream, where the coordinate matrix includes a first coordinate matrix of each corner point of the Aruco code on the plane of the target video stream and a second coordinate matrix in the world coordinate system.

[0083] In another embodiment, a system for improving the tracking accuracy of a camera further includes: an origin determination module 60 for determining the origin of the world coordinate system; among them, the first translation vector is the amount of position change relative to the origin, and the first Euler angle is the angle of rotation around the x-axis, y-axis, and z-axis relative to the world coordinate system.

[0084] It should be noted that the system for improving the tracking accuracy of a camera disclosed in this embodiment implements a method for improving the tracking accuracy of a camera as described in the above embodiment, so it will not be described in detail here. Optionally, each module in this embodiment and the above other operations or functions are respectively for implementing the method in the foregoing embodiment.

[0085] Refer to Figure 5 In another embodiment of the present application, an electronic device is shown. An electronic device includes a memory 210 and a processor 220.

[0086] The processor 220 may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0087] The general-purpose processor may be a microprocessor or the processor may also be any conventional processor. The memory 210 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices.

[0088] Among them, the ROM may store static data or instructions required by the processor 220 or other modules of the computer. The permanent storage device may be a readable and writable storage device. The permanent storage device may be a non-volatile storage device that does not lose the stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device uses a mass storage device (such as a magnetic or optical disk, flash memory) as the permanent storage device.

[0089] In some other embodiments, the permanent storage device may be a removable storage device (such as a floppy disk, optical drive). The system memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory. The system memory may store some or all of the instructions and data required by the processor during operation.

[0090] In addition, the memory 210 may include any combination of computer-readable storage media, including various types of semiconductor storage chips (such as DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and magnetic disks and / or optical disks may also be used.

[0091] In some embodiments, the memory 210 may include a removable storage device that is readable and / or writable, such as a compact disc (CD), read-only digital versatile disc (such as DVD-ROM, dual-layer DVD-ROM), read-only Blu-ray disc, super density disc, flash memory card (such as SD card, minSD card, Micro-SD card, etc.), magnetic floppy disk, etc. The computer-readable storage medium does not include carrier waves and instantaneous electronic signals transmitted wirelessly or by wire. Executable code is stored on the memory 210, and when the executable code is processed by the processor 220, it can cause the processor 220 to execute some or all of the methods described above.

[0092] In addition, the method according to the present application can also be implemented as a computer program or a computer program product, which includes computer program code instructions for performing some or all of the steps in the above method of the present application.

[0093] Alternatively, the present application can also be implemented as a computer-readable storage medium (or a non-transitory machine-readable storage medium or a machine-readable storage medium), on which executable code (or a computer program or computer instruction code) is stored. When the executable code (or the computer program or computer instruction code) is executed by a processor of an electronic device (or a server, etc.), the processor is caused to execute some or all of the steps of the above method according to the present application.

[0094] The various embodiments of the present application have been described above. The above description is exemplary and not exhaustive, and is also not limited to the disclosed embodiments. Many modifications and variations are obvious to those of ordinary skill in the art in the technical field without departing from the scope and spirit of the described embodiments. The selection of the terms used herein is intended to best explain the principles of the embodiments, the practical application, or the improvement of the technology in the market, or to enable other ordinary skill in the art in the technical field to understand the disclosed embodiments.

Claims

1. A method for improving the tracking accuracy of a camera, characterized in that, Including: Obtain the motion data of the camera, where the motion data includes a first translation vector and a first Euler angle; Based on the first translation vector, the first Euler angle, and a conversion formula, obtain the first motion pose of the camera based on the world coordinate system, where the conversion formula is a formula for converting the first Euler angle into a first rotation matrix; Obtain the video stream collected by the camera, and obtain the coordinate matrix of each corner point of the Aruco code in the video stream; Based on the coordinate matrix and the camera internal parameter matrix, obtain the second motion pose of the camera; Perform weighted fusion on the first motion pose and the second motion pose to obtain the target motion pose of the camera; Wherein, the obtaining the coordinate matrix of each corner point of the Aruco code in the initially collected video stream of the camera includes: Obtain the video stream collected by the camera; Based on a preset distortion correction matrix, perform distortion correction on the video stream to obtain a target video stream; Obtain the coordinate matrix of each corner point of the Aruco code in the target video stream, where the coordinate matrix includes a first coordinate matrix and a second coordinate matrix; the first coordinate matrix is the coordinate matrix of each corner point of the Aruco code on the plane of the target video stream, and the second coordinate matrix is the coordinate matrix of each corner point of the Aruco code in the world coordinate system; The calculation formula of the first motion pose is: M’ = [f(R’)|T’]; Wherein, M’ is the first motion pose, R’ is the first Euler angle, f is the formula for converting the Euler angle into a rotation matrix; T’ is the first translation vector; The second motion posture is calculated by: M image =M camera ·[R A ’|T A ·M world for calculation; Among them, [R A ’|T A is the second motion posture, R A ’ is the second rotation matrix, T A is the second translation vector, M image is the first coordinate matrix, M camera is the camera internal parameter matrix, M world is the second coordinate matrix.

2. The method for improving the camera tracking accuracy according to claim 1, wherein Before obtaining the motion data of the camera, it further includes: Determine the origin of the world coordinate system; Wherein, the first translation vector is the position change amount relative to the origin, and the first Euler angle is the angle of rotation around the x-axis, y-axis, and z-axis relative to the world coordinate system.

3. The method for improving the camera tracking accuracy according to claim 1, wherein ； ； ； ； ； ； ； Among them, and are both the second motion postures; is the second Euler angle, is the pitch angle of the camera, is the yaw angle of the camera, is the roll angle of the camera.

4. The method for improving the tracking accuracy of a camera according to claim 3, wherein The formula for the weighted fusion is: M = [f(α • R A + (1 - α) • R’) | β • T A + (1 - β) • T’]; Wherein, M is the target motion pose, α is the first preset weight value, and β is the second preset weight value.

5. A system for improving the tracking accuracy of a camera, characterized in that, Applied to the method for improving the tracking accuracy of a camera according to any one of claims 1 to 4 above, it includes: A data acquisition module for acquiring the motion data of the camera, where the motion data includes a first translation vector and a first Euler angle; A first motion pose acquisition module for obtaining the first motion pose of the camera based on the world coordinate system based on the first translation vector, the first Euler angle, and a conversion formula, where the conversion formula is a formula for converting the first Euler angle into a first rotation matrix; A coordinate matrix acquisition module for acquiring the video stream collected by the camera and identifying the coordinate matrix of each corner point of the Aruco code in the video stream; A second motion pose acquisition module for obtaining the second motion pose of the camera based on the coordinate matrix and the camera internal parameter matrix; A target motion pose acquisition module for performing weighted fusion on the first motion pose and the second motion pose to obtain the target motion pose of the camera.

6. The system for improving the tracking accuracy of a camera according to claim 5, wherein, The coordinate matrix acquisition module includes: A video stream acquisition unit for acquiring the video stream collected by the camera; A distortion correction unit for performing distortion correction on the video stream based on a preset distortion correction matrix to obtain a target video stream; A matrix acquisition unit for acquiring a coordinate matrix of each corner point of the Aruco code in the target video stream, where the coordinate matrix includes a first coordinate matrix and a second coordinate matrix; the first coordinate matrix is the coordinate matrix of each corner point of the Aruco code on the plane of the target video stream, and the second coordinate matrix is the coordinate matrix of each corner point of the Aruco code in the world coordinate system.

7. The system for improving the tracking accuracy of a camera according to claim 5, wherein It further includes: An origin determination module for determining the origin of the world coordinate system; Wherein, the first translation vector is the amount of position change relative to the origin, and the first Euler angle is the angle of rotation around the x-axis, y-axis, and z-axis relative to the world coordinate system.

Images