An XR virtual studio camera tracking and positioning method and system

Abstract

The present invention discloses an XR virtual studio camera tracking and positioning method and system. The method includes: capturing real-time position and attitude information of a physical object to be photographed through feature matching of a virtual object, and determining the shooting angle and shooting position coordinates of each camera according to the real-time position and attitude information; detecting the acceleration and angular velocity of each camera, and determining the inertial parameters of the camera according to the acceleration and angular velocity; determining the optimal moving shooting attitude and the optimal moving placement position of each camera according to the shooting angle, shooting position coordinates and inertial parameters of each camera; and adjusting the shooting trajectory and shooting parameters of each camera according to the optimal moving shooting attitude and the optimal moving placement position. By combining computer vision and IMU, precise tracking and positioning of the camera can be achieved, which can maximize the reduction of the attitude position error of the camera and ensure the shooting effect and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of device tracking and positioning, and particularly to an XR virtual studio camera tracking and positioning method and system. Background Art

[0002] Currently, with the rapid development of the digital entertainment industry, XR (Extended Reality) technology, as a new digital media technology, has been widely applied in fields such as movies, television, and advertising. Among them, XR virtual studio technology has brought revolutionary changes to content production with its unique advantages, such as real-time interaction, precise tracking, and virtual-real fusion. However, in an XR virtual studio, how to achieve precise tracking and positioning of cameras has become one of the key factors affecting the shooting effect.

[0003] Traditional camera tracking and positioning methods mainly rely on mechanical devices and optical sensors, but these methods have many limitations, such as low accuracy, susceptibility to interference, and complex operation. Therefore, developing an efficient, accurate, and easy-to-use XR virtual studio camera tracking and positioning device is of great significance for improving the shooting effect of XR virtual studios. Summary of the Invention

[0004] In view of the problems shown above, the present invention provides an XR virtual studio camera tracking and positioning method and system to solve the problems in the background art that there are many limitations in camera tracking and positioning relying on mechanical devices and optical sensors, such as low accuracy, susceptibility to interference, and complex operation.

[0005] An XR virtual studio camera tracking and positioning method includes the following steps:

[0006] Capture the real-time position and attitude information of the shooting entity object through feature matching of virtual objects, and determine the shooting angle and shooting position coordinates of each camera according to the real-time position and attitude information;

[0007] Detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity;

[0008] Determine the optimal moving shooting attitude and the optimal moving placement position of each camera according to the shooting angle and shooting position coordinates of each camera and the inertial parameters;

[0009] Adjust the shooting trajectory and shooting parameters of each camera according to the optimal moving shooting attitude and the optimal moving placement position.

[0010] Preferably, the step of capturing the real-time position and attitude information of the shooting entity object through feature matching of virtual objects and determining the shooting angle and shooting position coordinates of each camera according to the real-time position and attitude information includes:

[0011] Detect the feature points of the virtual object and label them, and determine the distribution position of the virtual object in the virtual scene rendering through feature point matching;

[0012] Capture the real-time position and attitude information of the shooting entity object according to the interaction parameters of the virtual object and the shooting entity object and the distribution position of the virtual object in the virtual scene rendering;

[0013] Determine the shooting requirements, determine multiple key shooting perspective ranges based on the shooting requirements, and obtain the visual effects at each shooting angle of each coordinate point within each key shooting perspective range;

[0014] Based on the computer vision system, select the position coordinates and shooting angles with the best visual effects within each key shooting perspective range as the shooting angles and shooting position coordinates for placing the camera within each key shooting perspective range.

[0015] Preferably, detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity, including:

[0016] Detect the acceleration of each camera through the inertial detection unit;

[0017] Obtain the internal sensor data of each camera, and integrate the internal sensor data to obtain the angular velocity of each camera;

[0018] Determine the rotation matrix and translation matrix of each camera according to the acceleration and angular velocity of each camera;

[0019] Determine the inertial parameters of each camera according to the rotation matrix and translation matrix of each camera.

[0020] Preferably, determine the best moving shooting posture and the best moving placement position of each camera according to the shooting angle and shooting position coordinates of each camera and the inertial parameters, including:

[0021] Determine the optimal placement direction of each camera according to the shooting angle and inertial parameters of each camera, and determine the horizontal and vertical coordinate values of the best placement position of the camera according to the optimal placement direction and the shooting position;

[0022] Determine the best placement height of each camera according to the horizontal and vertical coordinate values of the best placement position of each camera, and determine the optimal moving path of each camera according to the best placement height and the shooting area range of each camera;

[0023] Determine the best moving shooting posture of each camera according to the optimal moving path, the best placement height of each camera, and the optimal placement direction;

[0024] Determine the optimal moving and placement positions of each camera according to the path points of the optimal moving path.

[0025] Preferably, adjusting the shooting trajectories and shooting parameters of each camera according to the optimal moving shooting postures and optimal moving placement positions includes:

[0026] Determine the moving path points of each camera according to the optimal moving shooting postures and optimal moving placement positions;

[0027] Determine the visual occlusion parameters at the moving path points of each camera, and determine the conflict factors between the visual occlusion parameters and the shooting trajectories of each camera;

[0028] Determine the correction parameters according to the conflict factors, and adjust the shooting trajectories of each camera based on the correction parameters;

[0029] Determine the distance change parameters between each camera and the shooting entity object according to the optimal moving shooting postures and optimal moving placement positions;

[0030] Adjust the shooting parameters of each camera according to the distance change parameters, and the shooting parameters include: exposure time, focus distance, and zoom range.

[0031] An XR virtual studio camera tracking and positioning system, the system includes:

[0032] The first determination module is used to capture the real-time position and pose information of the shooting entity object through feature matching of the virtual object, and determine the shooting angle and shooting position coordinates of each camera according to the real-time position and pose information;

[0033] The second determination module is used to detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity;

[0034] The third determination module is used to determine the optimal moving shooting posture and optimal moving placement position of the camera according to the shooting angle and shooting position coordinates of each camera and the inertial parameters;

[0035] The adjustment module is used to adjust the shooting trajectories and shooting parameters of each camera according to the optimal moving shooting postures and optimal moving placement positions.

[0036] Preferably, the first determination module includes:

[0037] The first determination sub-module is used to detect the feature points of the virtual object and label them, and determine the distribution position of the virtual object in the virtual scene rendering through feature point matching;

[0038] A capture sub-module, configured to capture the real-time position and attitude information of the shooting entity object according to the interaction parameters between the virtual object and the shooting entity object and the distribution position of the virtual object in the virtual scene rendering;

[0039] A first acquisition sub-module, configured to determine the shooting requirements, determine multiple key shooting perspective ranges based on the shooting requirements, and acquire the visual effects at each shooting angle of each coordinate point within each key shooting perspective range;

[0040] A selection sub-module, configured to select, based on the computer vision system, the position coordinates and shooting angle with the best visual effect within each key shooting perspective range as the shooting angle and shooting position coordinates for installing the camera within each key shooting perspective range.

[0041] Preferably, the second determination module includes:

[0042] A second determination sub-module, configured to detect the acceleration of each camera through an inertial detection unit;

[0043] A second acquisition sub-module, configured to acquire the internal sensor data of each camera, and integrate the internal sensor data to obtain the angular velocity of each camera;

[0044] A third determination sub-module, configured to determine the rotation matrix and translation matrix of each camera according to the acceleration and angular velocity of each camera;

[0045] A fourth determination sub-module, configured to determine the inertial parameters of each camera according to the rotation matrix and translation matrix of each camera.

[0046] Preferably, the third determination module includes:

[0047] A fifth determination sub-module, configured to determine the optimal placement direction of each camera according to the shooting angle and inertial parameters of each camera, and determine the horizontal coordinate value and vertical coordinate value of the best placement position of the camera according to the optimal placement direction and the shooting position;

[0048] A sixth determination sub-module, configured to determine the best placement height of each camera according to the horizontal coordinate value and vertical coordinate value of the best placement position of each camera, and determine the optimal movement path of each camera according to the best placement height and the shooting area range of each camera;

[0049] A seventh determination sub-module, configured to determine the best moving shooting attitude of each camera according to the optimal movement path, the best placement height of each camera, and the optimal placement direction;

[0050] An eighth determination sub-module, configured to determine the best moving placement position of each camera according to the path points of the optimal movement path.

[0051] Preferably, the adjustment module includes:

[0052] A ninth determination sub-module, configured to determine the movement path points of each camera according to the optimal moving shooting posture and the optimal moving placement position;

[0053] A tenth determination sub-module, configured to determine the visual occlusion parameters at the movement path points of each camera, and determine the conflict factors between the visual occlusion parameters and the shooting trajectories of each camera;

[0054] An adjustment sub-module, configured to determine correction parameters according to the conflict factors, and adjust the shooting trajectories of each camera based on the correction parameters;

[0055] An eleventh determination sub-module, configured to determine the distance change parameters between each camera and the shooting entity object according to the optimal moving shooting posture and the optimal moving placement position;

[0056] An adjustment sub-module, configured to adjust the shooting parameters of each camera according to the distance change parameters, where the shooting parameters include: exposure time, focusing distance, and zoom range.

[0057] Other features and advantages of the present invention will be described in the subsequent description, and, in part, will be obvious from the description, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained by the structures specifically pointed out in the written description and the drawings.

[0058] Next, through the drawings and embodiments, the technical solutions of the present invention will be further described in detail. Description of the Drawings

[0059] The drawings are used to provide a further understanding of the present invention, and constitute a part of the description. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention.

[0060] Figure 1 It is a working flowchart of an XR virtual studio camera tracking and positioning method provided by the present invention;

[0061] Figure 2 It is another working flowchart of an XR virtual studio camera tracking and positioning method provided by the present invention;

[0062] Figure 3 It is a structural schematic diagram of an XR virtual studio camera tracking and positioning system provided by the present invention;

[0063] Figure 4 It is a structural schematic diagram of the first determination module in an XR virtual studio camera tracking and positioning system provided by the present invention. Detailed Embodiments

[0064] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0065] Currently, with the rapid development of the digital entertainment industry, XR (Extended Reality) technology, as an emerging digital media technology, is widely used in fields such as movies, television, and advertising. Among them, XR virtual studio technology, with its unique advantages such as real-time interaction, precise tracking, and virtual-real fusion, has brought revolutionary changes to content production. However, in an XR virtual studio, how to achieve precise tracking and positioning of cameras has become one of the key factors affecting the shooting effect.

[0066] Traditional camera tracking and positioning methods mainly rely on mechanical devices and optical sensors, but these methods have many limitations, such as low accuracy, susceptibility to interference, and complex operation. Therefore, developing an efficient, accurate, and easy-to-use XR virtual studio camera tracking and positioning device is of great significance for improving the shooting effect of XR virtual studios. To solve the above problems, this embodiment discloses an XR virtual studio camera tracking and positioning method.

[0067] An XR virtual studio camera tracking and positioning method, as Figure 1 shown, includes the following steps:

[0068] Step S101: Capture the real-time position and attitude information of the shooting entity object through feature matching of virtual objects, and determine the shooting angle and shooting position coordinates of each camera according to the real-time position and attitude information;

[0069] Step S102: Detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity;

[0070] Step S103: Determine the optimal moving shooting attitude and the optimal moving placement position of each camera according to the shooting angle and shooting position coordinates of each camera and the inertial parameters;

[0071] Step S104: Adjust the shooting trajectory and shooting parameters of each camera according to the optimal moving shooting attitude and the optimal moving placement position.

[0072] In this embodiment, the shooting angle and shooting position coordinates of each camera are determined based on a computer vision system;

[0073] In this embodiment, the acceleration and angular velocity of each camera are detected by an IMU (Inertial Measurement Unit).

[0074] The working principle of the above technical solution is as follows: The real-time position and pose information of the captured physical object are captured through feature matching of virtual objects, and the shooting angle and shooting position coordinates of each camera are determined according to the real-time position and pose information; the acceleration and angular velocity of each camera are detected, and the inertial parameters of the camera are determined according to the acceleration and angular velocity; according to the shooting angle and shooting position coordinates of each camera and the inertial parameters, the optimal moving shooting pose and the optimal moving placement position of the camera are determined; according to the optimal moving shooting pose and the optimal moving placement position, the shooting trajectory and shooting parameters of each camera are adjusted.

[0075] The beneficial effects of the above technical solution are as follows: By combining computer vision and IMU, the precise tracking and positioning of the camera can be realized, which can maximize the reduction of the pose position error of the camera, ensure the shooting effect and accuracy, and improve the stability and reliability of the system. The automatic control of the movement trajectory and shooting parameters of the camera is realized, without manual intervention, which reduces the usage difficulty and solves the problems mentioned in the prior art that there are many limitations in the tracking and positioning of the camera relying on mechanical devices and optical sensors, such as low accuracy, susceptibility to interference, and complex operation.

[0076] In this embodiment, before capturing the real-time position and pose information of the captured physical object through feature matching of virtual objects and determining the shooting angle and shooting position coordinates of each camera according to the real-time position and pose information, it further includes:

[0077] Determine the motion matrix of the captured physical object according to the real-time position and pose information of the captured physical object;

[0078] Determine the motion amplitude and motion complexity of the captured physical object according to the motion matrix, and obtain the parameter requirements for the shooting video frame rate based on the motion amplitude and motion complexity;

[0079] Determine the shooting metric distance range of the camera according to the frame rate requirement parameter and the motion compensation parameter of the camera, and collect the standard motion videos of the camera for the physical shooting object within each shooting metric distance difference based on the shooting metric distance range;

[0080] Obtain the ghost frame images from the standard motion videos and extract the common image features, and construct the view difference sparse feature matrix of the motion videos according to the common image features;

[0081] Determine the motion offset vector of the camera within each shooting exclusive distance difference according to the view difference sparse feature matrix;

[0082] The best shooting metric distance between the camera and the shooting entity object is filtered based on the motion offset vector;

[0083] Based on the best shooting metric distance, a reasonable shooting position range of the camera is determined. It is detected whether the current shooting position of each camera is within the reasonable shooting position range. If so, a reminder that the current shooting position is qualified is sent. If not, a reminder that the current shooting position is unqualified is sent and the position difference between the current shooting position and the nearest reasonable shooting position within the reasonable shooting position range is calculated;

[0084] The position of the target camera with an unqualified current shooting position is adjusted according to the position difference.

[0085] The beneficial effects of the above technical solution are as follows: By determining the shooting metric distance range of the camera, the image frame quality and motion compensation quality of the video shot by the camera can be guaranteed. Further, by determining the best shooting metric distance, not only the high quality of the shot video can be guaranteed, but also the current shooting position of each camera can be intelligently adjusted to ensure the shooting effect, improving the practicability, stability and reliability.

[0086] In one embodiment, as Figure 2 shown, capturing the real-time position and pose information of the shooting entity object through feature matching of the virtual object, and determining the shooting angle and shooting position coordinates of each camera according to the real-time position and pose information, includes:

[0087] Step S201: Detect the feature points of the virtual object and label them, and determine the distribution position of the virtual object in the virtual scene rendering through feature point matching;

[0088] Step S202: Capture the real-time position and pose information of the shooting entity object according to the interaction parameters between the virtual object and the shooting entity object and the distribution position of the virtual object in the virtual scene rendering;

[0089] Step S203: Determine the shooting requirements, determine multiple key shooting perspective ranges based on the shooting requirements, and obtain the visual effects at each shooting angle of each coordinate point within each key shooting perspective range;

[0090] Step S204: Based on the computer vision system, select the position coordinates and shooting angle with the best visual effect within each key shooting perspective range as the shooting angle and shooting position coordinates for installing the camera within each key shooting perspective range.

[0091] The beneficial effects of the above technical solution are as follows: By matching the feature points of the virtual object, the distribution position of the virtual object in the virtual scene rendering is determined. Combining the interaction parameters of the virtual object and the photographed entity object, the real-time position and attitude information of the photographed entity object are captured. At the same time, according to the shooting requirements, the visual effects under the shooting angles of each camera are determined, so as to arrange the shooting angles and shooting position coordinates of the cameras, which can ensure the accurate positioning and shooting of the cameras for the photographed entity object, improve the stability, and at the same time, the angles and position coordinates with the best shooting vision can be selected, further ensuring the shooting effect.

[0092] In one embodiment, detecting the acceleration and angular velocity of each camera, and determining the inertial parameters of the camera according to the acceleration and angular velocity, includes:

[0093] Detecting the acceleration of each camera through an inertial detection unit;

[0094] Obtaining the internal sensor data of each camera, and integrating the internal sensor data to obtain the angular velocity of each camera;

[0095] Determining the rotation matrix and translation matrix of the camera according to the acceleration and angular velocity of each camera;

[0096] Determining the inertial parameters of each camera according to the rotation matrix and translation matrix of each camera.

[0097] The beneficial effects of the above technical solution are as follows: According to the acceleration and angular velocity of each camera, the rotation matrix and translation matrix of the camera are determined, so as to determine the inertial parameters of each camera, improving the accuracy and precision of controlling the motion attitude of the camera, and being able to more accurately track the motion trajectory of the camera.

[0098] In one embodiment, determining the optimal moving shooting attitude and the optimal moving placement position of each camera according to the shooting angle, shooting position coordinates and inertial parameters of each camera, includes:

[0099] Determining the optimal placement direction of each camera according to the shooting angle and inertial parameters of each camera, and determining the horizontal coordinate value and vertical coordinate value of the optimal placement position of the camera according to the optimal placement direction and the shooting position;

[0100] Determining the optimal placement height of each camera according to the horizontal coordinate value and vertical coordinate value of the optimal placement position of each camera, and determining the optimal moving path of each camera according to the optimal placement height and the shooting area range of the camera;

[0101] Determining the optimal moving shooting attitude of each camera according to the optimal moving path, the optimal placement height and the optimal placement direction of each camera;

[0102] Determine the optimal moving and placement positions of each camera according to the path points of the optimal moving path.

[0103] The beneficial effects of the above technical solution are as follows: Determine the optimal placement direction and the optimal placement height of each camera according to the shooting angle and inertial parameters of each camera, determine the optimal moving path of each camera, and thus determine the optimal moving shooting posture of each camera, which can ensure that each camera can capture videos in an optimized manner, thereby improving the accuracy and real-time performance of the content captured by the camera.

[0104] In one embodiment, adjusting the shooting trajectories and shooting parameters of each camera according to the optimal moving shooting posture and the optimal moving placement position includes:

[0105] Determine the moving path points of each camera according to the optimal moving shooting posture and the optimal moving placement position;

[0106] Determine the visual occlusion parameters at the moving path points of each camera, and determine the conflict factors between the visual occlusion parameters and the shooting trajectories of each camera;

[0107] Determine the correction parameters according to the conflict factors, and adjust the shooting trajectories of each camera based on the correction parameters;

[0108] Determine the distance change parameters between each camera and the shooting entity object according to the optimal moving shooting posture and the optimal moving placement position;

[0109] Adjust the shooting parameters of each camera according to the distance change parameters, and the shooting parameters include: exposure time, focusing distance, and zoom range.

[0110] The beneficial effects of the above technical solution are as follows: Determine the visual occlusion parameters at the moving path points of each camera according to the optimal moving shooting posture and the optimal moving placement position, thereby determining the conflict factors of the shooting trajectories, and adjusting the shooting trajectories of each camera, which ensures that the camera can timely adjust its position and orientation according to the surrounding conflict factors, avoiding blind spots and missed shots. At the same time, adjust the shooting parameters of each camera according to the distance change parameters, and the camera can automatically adjust its parameters according to the distance change, avoiding problems such as blurring or defocusing caused by the distance change.

[0111] In one embodiment, this embodiment also discloses an XR virtual studio camera tracking and positioning system, as Figure 3 shown, and this system includes:

[0112] The first determination module 301 is configured to capture the real-time position and attitude information of the shooting entity object through the feature matching of the virtual object, and determine the shooting angle and shooting position coordinates of each camera according to the real-time position and attitude information;

[0113] The second determination module 302 is configured to detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity;

[0114] The third determination module 303 is configured to determine the optimal moving shooting attitude and the optimal moving placement position of each camera according to the shooting angle and shooting position coordinates of each camera and the inertial parameters;

[0115] The adjustment module 304 is configured to adjust the shooting trajectory and shooting parameters of each camera according to the optimal moving shooting attitude and the optimal moving placement position.

[0116] The working principle and beneficial effects of the above technical solution have been described in the method embodiment, and will not be elaborated here.

[0117] In one embodiment, as Figure 4 shown, the first determination module 301 includes:

[0118] The first determination sub-module 3011 is configured to detect the feature points of the virtual object and label them, and determine the distribution position of the virtual object in the virtual scene rendering through feature point matching;

[0119] The capture sub-module 3012 is configured to capture the real-time position and attitude information of the shooting entity object according to the interaction parameters of the virtual object and the shooting entity object and the distribution position of the virtual object in the virtual scene rendering;

[0120] The first acquisition sub-module 3013 is configured to determine the shooting requirements, determine multiple key shooting perspective ranges based on the shooting requirements, and acquire the visual effects at each shooting angle of each coordinate point within each key shooting perspective range;

[0121] The selection sub-module 3014 is configured to select, based on the computer vision system, the position coordinates and shooting angle with the best visual effect within each key shooting perspective range as the shooting angle and shooting position coordinates for placing the camera within each key shooting perspective range.

[0122] In one embodiment, the second determination module includes:

[0123] The second determination sub-module is configured to detect the acceleration of each camera through the inertial detection unit;

[0124] The second acquisition sub-module is configured to acquire the internal sensor data of each camera, and integrate the internal sensor data to acquire the angular velocity of each camera;

[0125] A third determination sub-module, configured to determine a rotation matrix and a translation matrix of each camera according to the acceleration and angular velocity of each camera;

[0126] A fourth determination sub-module, configured to determine the inertial parameters of each camera according to the rotation matrix and translation matrix of each camera.

[0127] In one embodiment, the third determination module includes:

[0128] A fifth determination sub-module, configured to determine the optimal placement direction of each camera according to the shooting angle and inertial parameters of each camera, and determine the horizontal coordinate value and vertical coordinate value of the optimal placement position of the camera according to the optimal placement direction and shooting position;

[0129] A sixth determination sub-module, configured to determine the optimal placement height of each camera according to the horizontal coordinate value and vertical coordinate value of the optimal placement position of each camera, and determine the optimal movement path of each camera according to the optimal placement height and the shooting area range of each camera;

[0130] A seventh determination sub-module, configured to determine the optimal movement shooting posture of each camera according to the optimal movement path, the optimal placement height of each camera, and the optimal placement direction;

[0131] An eighth determination sub-module, configured to determine the optimal movement placement position of each camera according to the path points of the optimal movement path.

[0132] In one embodiment, the adjustment module includes:

[0133] A ninth determination sub-module, configured to determine the movement path points of each camera according to the optimal movement shooting posture and the optimal movement placement position;

[0134] A tenth determination sub-module, configured to determine the visual occlusion parameter at the movement path points of each camera, and determine the conflict factor between the visual occlusion parameter and the shooting trajectory of each camera;

[0135] An adjustment sub-module, configured to determine a correction parameter according to the conflict factor, and adjust the shooting trajectory of each camera based on the correction parameter;

[0136] An eleventh determination sub-module, configured to determine the distance change parameter between each camera and the shooting entity object according to the optimal movement shooting posture and the optimal movement placement position;

[0137] An adjustment sub-module, configured to adjust the shooting parameters of each camera according to the distance change parameter, where the shooting parameters include: exposure time, focus distance, and zoom range.

[0138] Those skilled in the art should understand that the first and second in the present invention only refer to different application stages.

[0139] After considering the specification and practicing the disclosure herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. This application is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and examples are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0140] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for tracking and positioning a camera in an XR virtual studio, characterized in that: The following steps are involved: The real-time position and posture information of the physical object is captured by matching the features of the virtual object, and the shooting angle and shooting position coordinates of each camera are determined according to the real-time position and posture information; Detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity; Determine the best mobile shooting posture and best mobile placement position of each camera according to the shooting angle, shooting position coordinates and inertial parameters of the camera; Adjust the shooting trajectory and shooting parameters of each camera according to the best mobile shooting posture and the best mobile placement position; Determining the best mobile shooting posture and the best mobile placement position of each camera according to the shooting angle, shooting position coordinates and inertial parameters of the camera includes: Determine the optimal placement direction of each camera according to the shooting angle and inertia parameters of the camera, and determine the horizontal coordinate value and vertical coordinate value of the optimal placement position of the camera according to the optimal placement direction and shooting position; Determine the optimal placement height of each camera according to the horizontal coordinate value and the vertical coordinate value of the optimal placement position of the camera, and determine the optimal moving path of each camera according to the optimal placement height and the shooting area of ​​the camera; Determine the best mobile shooting posture of the camera according to the optimal moving path, the optimal placement height and the optimal placement direction of each camera; The best mobile placement position of each camera is determined according to the path points of the optimal mobile path.

2. The XR virtual studio camera tracking and positioning method according to claim 1, characterized in that: The real-time position and posture information of the physical object is captured by matching the features of the virtual object, and the shooting angle and shooting position coordinates of each camera are determined according to the real-time position and posture information, including: Detect and annotate the feature points of virtual objects, and determine the distribution position of virtual objects in virtual scene rendering through feature point matching; Capturing the real-time position and posture information of the photographed physical object according to the interaction parameters between the virtual object and the photographed physical object and the distribution position of the virtual object in the virtual scene rendering; Determine shooting requirements, determine multiple key shooting angle ranges based on the shooting requirements, and obtain the visual effect of each coordinate point in each key shooting angle range under each shooting angle; Based on the computer vision system, the position coordinates and shooting angle of the best visual effect within each key shooting angle range are selected as the shooting angle and shooting position coordinates of the camera placed within the key shooting angle range.

3. The XR virtual studio camera tracking and positioning method according to claim 1, characterized in that: The detecting the acceleration and angular velocity of each camera and determining the inertial parameters of the camera according to the acceleration and angular velocity includes: Detect the acceleration of each camera through an inertial detection unit; Obtain the internal sensor data of each camera, and integrate the internal sensor data to obtain the angular velocity of each camera; Determine the rotation matrix and translation matrix of each camera according to the acceleration and angular velocity of the camera; Determine the inertial parameters of each camera based on its rotation matrix and translation matrix.

4. The XR virtual studio camera tracking and positioning method according to claim 1, characterized in that: The step of adjusting the shooting trajectory and shooting parameters of each camera according to the optimal mobile shooting posture and the optimal mobile placement position includes: Determine the moving path point of each camera according to the best moving shooting posture and the best moving placement position; Determine the visual occlusion parameters at the moving path points of each camera, and determine the conflict factors between the visual occlusion parameters and the shooting trajectory of each camera; Determine a correction parameter according to the conflict factor, and adjust the shooting trajectory of each camera based on the correction parameter; Determine the distance variation parameter between each camera and the photographed entity object according to the optimal mobile shooting posture and the optimal mobile placement position; The shooting parameters of each camera are adjusted according to the distance variation parameters, and the shooting parameters include: exposure time, focus distance and zoom range.

5. An XR virtual studio camera tracking and positioning system, characterized in that: The system includes: A first determination module is used to capture the real-time position and posture information of the photographed physical object through feature matching of the virtual object, and determine the shooting angle and shooting position coordinates of each camera according to the real-time position and posture information; The second determination module is used to detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity; The third determination module is used to determine the best mobile shooting posture and the best mobile placement position of each camera according to the shooting angle and shooting position coordinates and inertial parameters of the camera; An adjustment module, used for adjusting the shooting trajectory and shooting parameters of each camera according to the best mobile shooting posture and the best mobile placement position; The third determining module comprises: The fifth determination submodule is used to determine the optimal placement direction of each camera according to the shooting angle and inertia parameters of the camera, and determine the horizontal coordinate value and the vertical coordinate value of the optimal placement position of the camera according to the optimal placement direction and the shooting position; The sixth determination submodule is used to determine the optimal placement height of each camera according to the horizontal coordinate value and the vertical coordinate value of the optimal placement position of each camera, and determine the optimal moving path of each camera according to the optimal placement height and the shooting area range of the camera; A seventh determination submodule is used to determine the optimal mobile shooting posture of each camera according to the optimal moving path and the optimal placement height and optimal placement direction of each camera; The eighth determination submodule is used to determine the optimal mobile placement position of each camera according to the path points of the optimal moving path.

6. The XR virtual studio camera tracking and positioning system according to claim 5, characterized in that: The first determining module includes: The first determination submodule is used to detect and mark the feature points of the virtual object, and determine the distribution position of the virtual object in the virtual scene rendering by matching the feature points; A capture submodule, used for capturing the real-time position and posture information of the photographed physical object according to the interaction parameters between the virtual object and the photographed physical object and the distribution position of the virtual object in the virtual scene rendering; The first acquisition submodule is used to determine shooting requirements, determine multiple key shooting angle ranges based on the shooting requirements, and obtain the visual effect of each coordinate point in each key shooting angle range under each shooting angle; The selection submodule is used to select the position coordinates and shooting angle of the best visual effect in each key shooting angle range based on the computer vision system as the shooting angle and shooting position coordinates of the camera placed in the key shooting angle range.

7. The XR virtual studio camera tracking and positioning system according to claim 5, characterized in that: The second determining module includes: A second determination submodule, configured to detect the acceleration of each camera through an inertial detection unit; The second acquisition submodule is used to acquire the internal sensor data of each camera, and integrate the internal sensor data to acquire the angular velocity of each camera; A third determination submodule is used to determine the rotation matrix and translation matrix of each camera according to the acceleration and angular velocity of the camera; The fourth determination submodule is used to determine the inertial parameters of each camera according to the rotation matrix and the translation matrix of each camera.

8. The XR virtual studio camera tracking and positioning system according to claim 5, characterized in that: The adjustment module comprises: A ninth determination submodule, used to determine the moving path point of each camera according to the best moving shooting posture and the best moving placement position; A tenth determination submodule is used to determine the visual occlusion parameter on the moving path point of each camera, and determine the conflict factor between the visual occlusion parameter and the shooting trajectory of each camera; An adjustment submodule, used for determining a correction parameter according to the conflict factor, and adjusting the shooting trajectory of each camera based on the correction parameter; An eleventh determination submodule is used to determine a distance variation parameter between each camera and the photographed entity object according to the optimal mobile shooting posture and the optimal mobile placement position; The adjustment submodule is used to adjust the shooting parameters of each camera according to the distance change parameters, and the shooting parameters include: exposure time, focus distance and zoom range.

Images