XR virtual shed camera tracking and positioning method and system

Abstract

The invention discloses an XR virtual shed camera tracking and positioning method and system, and the method comprises the steps: capturing the real-time position and posture information of a shooting entity object through the feature matching of a virtual object, and determining the shooting angle and the shooting position coordinate of each camera according to the real-time position and posture information; detecting the acceleration and the angular velocity of each camera, and determining the inertial parameters of the camera according to the acceleration and the angular velocity; according to the shooting angle, the shooting position coordinate and the inertial parameter of each camera, determining the optimal moving shooting posture and the optimal moving placing position of the camera; and adjusting the shooting track and shooting parameters of each camera according to the optimal movement shooting posture and the optimal movement placing position. The mode of combining computer vision and IMU is adopted, accurate tracking and positioning of the camera can be achieved, the posture position error of the camera can be reduced to the maximum degree, and the shooting effect and precision are guaranteed.

Description

Technical Field

[0001] The present invention relates to the field of equipment tracking and positioning technology, and in particular to an XR virtual studio camera tracking and positioning method and system. Background Art

[0002] At present, with the rapid development of the digital entertainment industry, XR (extended reality) technology, as an emerging digital media technology, has been widely used in the fields of film, television, advertising, etc. Among them, XR virtual studio technology has brought revolutionary changes to content production with its unique advantages, such as real-time interaction, precise tracking, and virtual-real fusion. However, in the XR virtual studio, how to achieve accurate tracking and positioning of the camera has become one of the key factors affecting the shooting effect.

[0003] Traditional camera tracking and positioning methods mainly rely on mechanical devices and optical sensors, but these methods have many limitations, such as low accuracy, susceptibility to interference, and complex operation. Therefore, developing an efficient, accurate, and easy-to-use XR virtual studio camera tracking and positioning device is of great significance to improving the shooting effect of XR virtual studios. Summary of the invention

[0004] In response to the above-mentioned problems, the present invention provides an XR virtual studio camera tracking and positioning method and system to solve the many limitations of camera tracking and positioning relying on mechanical devices and optical sensors mentioned in the background technology, such as low accuracy, susceptibility to interference, and complex operation.

[0005] A method for tracking and positioning an XR virtual studio camera includes the following steps: The real-time position and posture information of the physical object is captured by matching the features of the virtual object, and the shooting angle and shooting position coordinates of each camera are determined according to the real-time position and posture information; Detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity; Determine the best mobile shooting posture and best mobile placement position of each camera according to the shooting angle, shooting position coordinates and inertial parameters of the camera; Adjust the shooting trajectory and shooting parameters of each camera according to the best mobile shooting posture and the best mobile placement position.

[0006] Preferably, the real-time position and posture information of the physical object is captured by matching the features of the virtual object, and the shooting angle and shooting position coordinates of each camera are determined according to the real-time position and posture information, including: Detect and annotate the feature points of virtual objects, and determine the distribution position of virtual objects in virtual scene rendering through feature point matching; Capturing the real-time position and posture information of the photographed physical object according to the interaction parameters between the virtual object and the photographed physical object and the distribution position of the virtual object in the virtual scene rendering; Determine shooting requirements, determine multiple key shooting angle ranges based on the shooting requirements, and obtain the visual effects of each coordinate point within each key shooting angle range under each shooting angle; Based on the computer vision system, the position coordinates and shooting angle of the best visual effect within each key shooting angle of view are selected as the shooting angle and shooting position coordinates of the camera placed within the key shooting angle of view.

[0007] Preferably, the detecting the acceleration and angular velocity of each camera and determining the inertial parameters of the camera according to the acceleration and angular velocity comprises: Detect the acceleration of each camera through an inertial detection unit; Obtain the internal sensor data of each camera, and integrate the internal sensor data to obtain the angular velocity of each camera; Determine the rotation matrix and translation matrix of each camera according to the acceleration and angular velocity of the camera; Determine the inertial parameters of each camera based on its rotation matrix and translation matrix.

[0008] Preferably, determining the best mobile shooting posture and best mobile placement position of each camera according to the shooting angle, shooting position coordinates and inertial parameters of the camera includes: Determine the optimal placement direction of each camera according to the shooting angle and inertia parameters of the camera, and determine the horizontal coordinate value and vertical coordinate value of the optimal placement position of the camera according to the optimal placement direction and shooting position; Determine the optimal placement height of each camera according to the horizontal coordinate value and the vertical coordinate value of the optimal placement position of the camera, and determine the optimal moving path of each camera according to the optimal placement height and the shooting area of ​​the camera; Determine the best mobile shooting posture of the camera according to the optimal moving path, the optimal placement height and the optimal placement direction of each camera; The best mobile placement position of each camera is determined according to the path points of the optimal mobile path.

[0009] Preferably, adjusting the shooting trajectory and shooting parameters of each camera according to the optimal mobile shooting posture and the optimal mobile placement position includes: Determine the moving path point of each camera according to the best moving shooting posture and the best moving placement position; Determine the visual occlusion parameters at the moving path points of each camera, and determine the conflict factors between the visual occlusion parameters and the shooting trajectory of each camera; Determine a correction parameter according to the conflict factor, and adjust the shooting trajectory of each camera based on the correction parameter; Determine the distance variation parameter between each camera and the photographed entity object according to the optimal mobile shooting posture and the optimal mobile placement position; The shooting parameters of each camera are adjusted according to the distance variation parameters, and the shooting parameters include: exposure time, focus distance and zoom range.

[0010] An XR virtual studio camera tracking and positioning system, the system comprising: A first determination module is used to capture the real-time position and posture information of the photographed physical object through feature matching of the virtual object, and determine the shooting angle and shooting position coordinates of each camera according to the real-time position and posture information; The second determination module is used to detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity; The third determination module is used to determine the best mobile shooting posture and the best mobile placement position of each camera according to the shooting angle and shooting position coordinates and inertial parameters of the camera; The adjustment module is used for adjusting the shooting trajectory and shooting parameters of each camera according to the best mobile shooting posture and the best mobile placement position.

[0011] Preferably, the first determining module includes: The first determination submodule is used to detect and mark the feature points of the virtual object, and determine the distribution position of the virtual object in the virtual scene rendering by matching the feature points; A capture submodule, used for capturing the real-time position and posture information of the photographed physical object according to the interaction parameters between the virtual object and the photographed physical object and the distribution position of the virtual object in the virtual scene rendering; The first acquisition submodule is used to determine shooting requirements, determine multiple key shooting angle ranges based on the shooting requirements, and obtain the visual effect of each coordinate point within each key shooting angle range under each shooting angle; The selection submodule is used to select the position coordinates and shooting angles with the best visual effect within each key shooting angle of view based on the computer vision system as the shooting angle and shooting position coordinates of the camera placed within the key shooting angle of view.

[0012] Preferably, the second determining module includes: A second determination submodule, configured to detect the acceleration of each camera through an inertial detection unit; The second acquisition submodule is used to acquire the internal sensor data of each camera, and integrate the internal sensor data to acquire the angular velocity of each camera; A third determination submodule is used to determine the rotation matrix and translation matrix of each camera according to the acceleration and angular velocity of the camera; The fourth determination submodule is used to determine the inertial parameters of each camera according to the rotation matrix and the translation matrix of each camera.

[0013] Preferably, the third determination module includes: The fifth determination submodule is used to determine the optimal placement direction of each camera according to the shooting angle and inertia parameters of the camera, and determine the horizontal coordinate value and the vertical coordinate value of the optimal placement position of the camera according to the optimal placement direction and the shooting position; The sixth determination submodule is used to determine the optimal placement height of each camera according to the horizontal coordinate value and the vertical coordinate value of the optimal placement position of each camera, and determine the optimal moving path of each camera according to the optimal placement height and the shooting area range of the camera; A seventh determination submodule is used to determine the optimal mobile shooting posture of each camera according to the optimal moving path and the optimal placement height and optimal placement direction of each camera; The eighth determination submodule is used to determine the optimal mobile placement position of each camera according to the path points of the optimal moving path.

[0014] Preferably, the adjustment module includes: A ninth determination submodule, used to determine the moving path point of each camera according to the best moving shooting posture and the best moving placement position; A tenth determination submodule, used to determine the visual occlusion parameter on the moving path point of each camera, and determine the conflict factor between the visual occlusion parameter and the shooting trajectory of each camera; An adjustment submodule, used for determining a correction parameter according to the conflict factor, and adjusting the shooting trajectory of each camera based on the correction parameter; An eleventh determination submodule is used to determine a distance variation parameter between each camera and the photographed entity object according to the optimal mobile shooting posture and the optimal mobile placement position; The adjustment submodule is used to adjust the shooting parameters of each camera according to the distance change parameters, and the shooting parameters include: exposure time, focus distance and zoom range.

[0015] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0016] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0018] Figure 1 A workflow diagram of an XR virtual studio camera tracking and positioning method provided by the present invention; Figure 2 Another workflow diagram of the XR virtual studio camera tracking and positioning method provided by the present invention; Figure 3 A schematic diagram of the structure of an XR virtual studio camera tracking and positioning system provided by the present invention; Figure 4 This is a schematic diagram of the structure of a first determination module in an XR virtual studio camera tracking and positioning system provided by the present invention. DETAILED DESCRIPTION

[0019] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0020] At present, with the rapid development of the digital entertainment industry, XR (extended reality) technology, as an emerging digital media technology, has been widely used in the fields of film, television, advertising, etc. Among them, XR virtual studio technology has brought revolutionary changes to content production with its unique advantages, such as real-time interaction, precise tracking, and virtual-real fusion. However, in the XR virtual studio, how to achieve accurate tracking and positioning of the camera has become one of the key factors affecting the shooting effect.

[0021] Traditional camera tracking and positioning methods mainly rely on mechanical devices and optical sensors, but these methods have many limitations, such as low accuracy, susceptibility to interference, and complex operation. Therefore, developing an efficient, accurate, and easy-to-use XR virtual studio camera tracking and positioning device is of great significance for improving the shooting effect of the XR virtual studio. In order to solve the above problems, this embodiment discloses an XR virtual studio camera tracking and positioning method.

[0022] A method for tracking and positioning a camera in an XR virtual studio, such as Figure 1As shown, the following steps are included: Step S101, capturing the real-time position and posture information of the physical object through feature matching of the virtual object, and determining the shooting angle and shooting position coordinates of each camera according to the real-time position and posture information; Step S102, detecting the acceleration and angular velocity of each camera, and determining the inertial parameters of the camera according to the acceleration and angular velocity; Step S103, determining the best mobile shooting posture and best mobile placement position of each camera according to the shooting angle, shooting position coordinates and inertia parameters of the camera; Step S104: adjusting the shooting trajectory and shooting parameters of each camera according to the optimal mobile shooting posture and the optimal mobile placement position.

[0023] In this embodiment, the shooting angle and shooting position coordinates of each camera are determined based on a computer vision system; In this embodiment, the acceleration and angular velocity of each camera are detected by an IMU (Inertial Measurement Unit).

[0024] The working principle of the above technical solution is: capture the real-time position and posture information of the physical object through feature matching of the virtual object, and determine the shooting angle and shooting position coordinates of each camera according to the real-time position and posture information; detect the acceleration and angular velocity of each camera, and determine the inertia parameters of the camera according to the acceleration and angular velocity; determine the best mobile shooting posture and the best mobile placement position of the camera according to the shooting angle and shooting position coordinates and inertia parameters of each camera; adjust the shooting trajectory and shooting parameters of each camera according to the best mobile shooting posture and the best mobile placement position.

[0025] The beneficial effects of the above technical solution are: by combining computer vision and IMU, accurate tracking and positioning of the camera can be achieved, which can minimize the camera's posture position error, ensure shooting effect and accuracy, and improve the stability and reliability of the system. Automatic control of the camera's motion trajectory and shooting parameters is achieved without manual intervention, which reduces the difficulty of use and solves the problems mentioned in the prior art that the camera tracking and positioning that rely on mechanical devices and optical sensors have many limitations, such as low accuracy, susceptibility to interference, and complex operation.

[0026] In this embodiment, before capturing the real-time position and posture information of the physical object through feature matching of the virtual object and determining the shooting angle and shooting position coordinates of each camera according to the real-time position and posture information, the method further includes: Determine the motion matrix of the photographed entity object according to the real-time position and posture information of the photographed entity object; Determine the motion amplitude and motion complexity of the photographed entity object according to the motion matrix, and obtain and determine the required parameters for the shooting video frame distance based on the motion amplitude and motion complexity; Determine the shooting measurement distance range of the camera according to the frame distance requirement parameter and the motion compensation parameter of the camera, and collect the standard motion video of the camera for the physical shooting object within each shooting measurement distance difference based on the shooting measurement distance range; Obtain ghost frame images according to standard motion video and extract common features of the images, and construct a view difference sparse feature matrix of the motion video according to the common features of the images; Determine the motion offset vector of the camera within each shooting exclusive distance difference according to the view difference sparse feature matrix; Filter out the best shooting measurement distance between the camera and the shooting entity object based on the motion offset vector; Determine the reasonable shooting position interval of the camera according to the optimal shooting measurement distance, detect whether the current shooting position of each camera is within the reasonable shooting position interval, if so, issue a qualified reminder of the current shooting position, if not, issue an unqualified reminder of the current shooting position and calculate the position difference between the current shooting position and the nearest reasonable shooting position within the reasonable shooting position interval; The position of the target camera whose current shooting position is unqualified is adjusted according to the position difference.

[0027] The beneficial effects of the above technical solution are as follows: by determining the shooting measurement distance range of the camera, the image frame quality and motion compensation quality of the camera's captured video can be guaranteed. Furthermore, by determining the optimal shooting measurement distance, the high quality of the captured video can be guaranteed while the current shooting position of each camera can be intelligently adjusted to ensure the shooting effect, thereby improving practicality, stability and reliability.

[0028] In one embodiment, Figure 2 As shown, the real-time position and posture information of the physical object is captured by matching the features of the virtual object, and the shooting angle and shooting position coordinates of each camera are determined according to the real-time position and posture information, including: Step S201, detecting and marking feature points of a virtual object, and determining the distribution position of the virtual object in a virtual scene rendering by matching the feature points; Step S202: capturing the real-time position and posture information of the photographed physical object according to the interaction parameters between the virtual object and the photographed physical object and the distribution position of the virtual object in the virtual scene rendering; Step S203: determining shooting requirements, determining multiple key shooting angle ranges based on the shooting requirements, and obtaining visual effects at each shooting angle of each coordinate point within each key shooting angle range; Step S204: Based on the computer vision system, the position coordinates and shooting angle of the best visual effect within each key shooting angle range are selected as the shooting angle and shooting position coordinates of the camera placed within the key shooting angle range.

[0029] The beneficial effects of the above technical solution are: determining the distribution position of the virtual object in the virtual scene rendering by matching the feature points of the virtual object, capturing the real-time position and posture information of the photographed physical object in combination with the interaction parameters of the virtual object and the photographed physical object, and at the same time, determining the visual effect of each camera shooting angle according to the shooting requirements, thereby placing the camera's shooting angle and shooting position coordinates, which can ensure the camera's accurate positioning and shooting of the photographed physical object, improve stability, and at the same time can also select the best shooting visual angle and position coordinates, further ensuring the shooting effect.

[0030] In one embodiment, detecting the acceleration and angular velocity of each camera and determining the inertial parameters of the camera according to the acceleration and angular velocity includes: Detect the acceleration of each camera through an inertial detection unit; Obtain the internal sensor data of each camera, and integrate the internal sensor data to obtain the angular velocity of each camera; Determine the rotation matrix and translation matrix of each camera according to the acceleration and angular velocity of the camera; Determine the inertial parameters of each camera based on its rotation matrix and translation matrix.

[0031] The beneficial effects of the above technical solution are: the rotation matrix and translation matrix of each camera are determined according to the acceleration and angular velocity of each camera, thereby determining the inertia parameters of each camera, improving the accuracy and precision of controlling the camera's motion posture, and being able to track the camera's motion trajectory more accurately.

[0032] In one embodiment, determining the best mobile shooting posture and best mobile placement position of each camera according to the shooting angle, shooting position coordinates and inertial parameters of the camera includes: Determine the optimal placement direction of each camera according to the shooting angle and inertia parameters of the camera, and determine the horizontal coordinate value and vertical coordinate value of the optimal placement position of the camera according to the optimal placement direction and shooting position; Determine the optimal placement height of each camera according to the horizontal coordinate value and the vertical coordinate value of the optimal placement position of the camera, and determine the optimal moving path of each camera according to the optimal placement height and the shooting area of ​​the camera; Determine the best mobile shooting posture of the camera according to the optimal moving path, the optimal placement height and the optimal placement direction of each camera; The best mobile placement position of each camera is determined according to the path points of the optimal mobile path.

[0033] The beneficial effects of the above technical solution are: determining the optimal placement direction and optimal placement height of each camera according to the shooting angle and inertia parameters of the camera, determining the optimal movement path of each camera, and thus determining the optimal mobile shooting posture of the camera, which can ensure that each camera can capture video in the most optimized way, thereby improving the accuracy and real-time performance of the camera's shooting content.

[0034] In one embodiment, adjusting the shooting trajectory and shooting parameters of each camera according to the optimal mobile shooting posture and the optimal mobile placement position includes: Determine the moving path point of each camera according to the best moving shooting posture and the best moving placement position; Determine the visual occlusion parameters at the moving path points of each camera, and determine the conflict factors between the visual occlusion parameters and the shooting trajectory of each camera; Determine a correction parameter according to the conflict factor, and adjust the shooting trajectory of each camera based on the correction parameter; Determine the distance variation parameter between each camera and the photographed entity object according to the optimal mobile shooting posture and the optimal mobile placement position; The shooting parameters of each camera are adjusted according to the distance variation parameters, and the shooting parameters include: exposure time, focus distance and zoom range.

[0035] The beneficial effects of the above technical solution are: determining the visual occlusion parameters on the moving path points of each camera according to the optimal mobile shooting posture and the optimal mobile placement position, thereby determining the conflict factor of the shooting trajectory, and adjusting the shooting trajectory of each camera, ensuring that the camera can adjust its position and orientation in time according to the surrounding conflict factors to avoid blind spots and missed shots. At the same time, the shooting parameters of each camera are adjusted according to the distance change parameters. The camera can automatically adjust its parameters according to the distance change to avoid blur or out of focus problems caused by distance changes.

[0036] In one embodiment, this embodiment also discloses an XR virtual studio camera tracking and positioning system, such as Figure 3 As shown, the system includes: The first determination module 301 is used to capture the real-time position and posture information of the photographed physical object through feature matching of the virtual object, and determine the shooting angle and shooting position coordinates of each camera according to the real-time position and posture information; The second determination module 302 is used to detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity; The third determination module 303 is used to determine the best mobile shooting posture and the best mobile placement position of each camera according to the shooting angle and shooting position coordinates and inertia parameters of the camera; The adjustment module 304 is used to adjust the shooting trajectory and shooting parameters of each camera according to the optimal mobile shooting posture and the optimal mobile placement position.

[0037] The working principle and beneficial effects of the above technical solution have been explained in the method embodiment and will not be repeated here.

[0038] In one embodiment, Figure 4 As shown, the first determining module 301 includes: The first determination submodule 3011 is used to detect and mark feature points of virtual objects, and determine the distribution position of virtual objects in virtual scene rendering by matching feature points; A capture submodule 3012 is used to capture the real-time position and posture information of the photographed physical object according to the interaction parameters between the virtual object and the photographed physical object and the distribution position of the virtual object in the virtual scene rendering; The first acquisition submodule 3013 is used to determine shooting requirements, determine multiple key shooting angle ranges based on the shooting requirements, and obtain the visual effect of each coordinate point in each key shooting angle range under each shooting angle; The selection submodule 3014 is used to select the position coordinates and shooting angles with the best visual effect within each key shooting angle range based on the computer vision system as the shooting angle and shooting position coordinates of the camera placed within the key shooting angle range.

[0039] In one embodiment, the second determining module includes: A second determination submodule, configured to detect the acceleration of each camera through an inertial detection unit; The second acquisition submodule is used to acquire the internal sensor data of each camera, and integrate the internal sensor data to acquire the angular velocity of each camera; A third determination submodule is used to determine the rotation matrix and translation matrix of each camera according to the acceleration and angular velocity of the camera; The fourth determination submodule is used to determine the inertial parameters of each camera according to the rotation matrix and the translation matrix of each camera.

[0040] In one embodiment, the third determining module includes: The fifth determination submodule is used to determine the optimal placement direction of each camera according to the shooting angle and inertia parameters of the camera, and determine the horizontal coordinate value and the vertical coordinate value of the optimal placement position of the camera according to the optimal placement direction and the shooting position; The sixth determination submodule is used to determine the optimal placement height of each camera according to the horizontal coordinate value and the vertical coordinate value of the optimal placement position of each camera, and determine the optimal moving path of each camera according to the optimal placement height and the shooting area range of the camera; A seventh determination submodule is used to determine the optimal mobile shooting posture of each camera according to the optimal moving path and the optimal placement height and optimal placement direction of each camera; The eighth determination submodule is used to determine the optimal mobile placement position of each camera according to the path points of the optimal moving path.

[0041] In one embodiment, the adjustment module includes: A ninth determination submodule, used to determine the moving path point of each camera according to the best moving shooting posture and the best moving placement position; A tenth determination submodule, used to determine the visual occlusion parameter on the moving path point of each camera, and determine the conflict factor between the visual occlusion parameter and the shooting trajectory of each camera; An adjustment submodule, used for determining a correction parameter according to the conflict factor, and adjusting the shooting trajectory of each camera based on the correction parameter; An eleventh determination submodule is used to determine a distance variation parameter between each camera and the photographed entity object according to the optimal mobile shooting posture and the optimal mobile placement position; The adjustment submodule is used to adjust the shooting parameters of each camera according to the distance change parameters, and the shooting parameters include: exposure time, focus distance and zoom range.

[0042] Those skilled in the art should understand that the first and second in the present invention merely refer to different application stages.

[0043] Other embodiments of the present disclosure will be readily apparent to those skilled in the art after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0044] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for tracking and positioning a camera in an XR virtual studio, characterized in that: The following steps are involved: The real-time position and posture information of the physical object is captured by matching the features of the virtual object, and the shooting angle and shooting position coordinates of each camera are determined according to the real-time position and posture information; Detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity; Determine the best mobile shooting posture and best mobile placement position of each camera according to the shooting angle, shooting position coordinates and inertial parameters of the camera; Adjust the shooting trajectory and shooting parameters of each camera according to the best mobile shooting posture and the best mobile placement position.

2. The XR virtual studio camera tracking and positioning method according to claim 1, characterized in that: The real-time position and posture information of the physical object is captured by matching the features of the virtual object, and the shooting angle and shooting position coordinates of each camera are determined according to the real-time position and posture information, including: Detect and annotate the feature points of virtual objects, and determine the distribution position of virtual objects in virtual scene rendering through feature point matching; Capturing the real-time position and posture information of the photographed physical object according to the interaction parameters between the virtual object and the photographed physical object and the distribution position of the virtual object in the virtual scene rendering; Determine shooting requirements, determine multiple key shooting angle ranges based on the shooting requirements, and obtain the visual effects of each coordinate point within each key shooting angle range under each shooting angle; Based on the computer vision system, the position coordinates and shooting angle of the best visual effect within each key shooting angle of view are selected as the shooting angle and shooting position coordinates of the camera placed within the key shooting angle of view.

3. The XR virtual studio camera tracking and positioning method according to claim 1, characterized in that: The detecting the acceleration and angular velocity of each camera and determining the inertial parameters of the camera according to the acceleration and angular velocity includes: Detect the acceleration of each camera through an inertial detection unit; Obtain the internal sensor data of each camera, and integrate the internal sensor data to obtain the angular velocity of each camera; Determine the rotation matrix and translation matrix of each camera according to the acceleration and angular velocity of the camera; Determine the inertial parameters of each camera based on its rotation matrix and translation matrix.

4. The XR virtual studio camera tracking and positioning method according to claim 1, characterized in that: Determining the best mobile shooting posture and the best mobile placement position of each camera according to the shooting angle, shooting position coordinates and inertial parameters of the camera includes: Determine the optimal placement direction of each camera according to the shooting angle and inertia parameters of the camera, and determine the horizontal coordinate value and vertical coordinate value of the optimal placement position of the camera according to the optimal placement direction and shooting position; Determine the optimal placement height of each camera according to the horizontal coordinate value and the vertical coordinate value of the optimal placement position of the camera, and determine the optimal moving path of each camera according to the optimal placement height and the shooting area of ​​the camera; Determine the best mobile shooting posture of the camera according to the optimal moving path, the optimal placement height and the optimal placement direction of each camera; The best mobile placement position of each camera is determined according to the path points of the optimal mobile path.

5. The XR virtual studio camera tracking and positioning method according to claim 1, characterized in that: The step of adjusting the shooting trajectory and shooting parameters of each camera according to the optimal mobile shooting posture and the optimal mobile placement position includes: Determine the moving path point of each camera according to the best moving shooting posture and the best moving placement position; Determine the visual occlusion parameters at the moving path points of each camera, and determine the conflict factors between the visual occlusion parameters and the shooting trajectory of each camera; Determine a correction parameter according to the conflict factor, and adjust the shooting trajectory of each camera based on the correction parameter; Determine the distance variation parameter between each camera and the photographed entity object according to the optimal mobile shooting posture and the optimal mobile placement position; The shooting parameters of each camera are adjusted according to the distance variation parameters, and the shooting parameters include: exposure time, focus distance and zoom range.

6. An XR virtual studio camera tracking and positioning system, characterized in that: The system includes: A first determination module is used to capture the real-time position and posture information of the photographed physical object through feature matching of the virtual object, and determine the shooting angle and shooting position coordinates of each camera according to the real-time position and posture information; The second determination module is used to detect the acceleration and angular velocity of each camera, and determine the inertial parameters of the camera according to the acceleration and angular velocity; The third determination module is used to determine the best mobile shooting posture and the best mobile placement position of each camera according to the shooting angle and shooting position coordinates and inertial parameters of the camera; The adjustment module is used for adjusting the shooting trajectory and shooting parameters of each camera according to the best mobile shooting posture and the best mobile placement position.

7. The XR virtual studio camera tracking and positioning system according to claim 6, characterized in that: The first determining module includes: The first determination submodule is used to detect and mark the feature points of the virtual object, and determine the distribution position of the virtual object in the virtual scene rendering by matching the feature points; A capture submodule, used for capturing the real-time position and posture information of the photographed physical object according to the interaction parameters between the virtual object and the photographed physical object and the distribution position of the virtual object in the virtual scene rendering; The first acquisition submodule is used to determine shooting requirements, determine multiple key shooting angle ranges based on the shooting requirements, and obtain the visual effect of each coordinate point within each key shooting angle range under each shooting angle; The selection submodule is used to select the position coordinates and shooting angles with the best visual effect within each key shooting angle of view based on the computer vision system as the shooting angle and shooting position coordinates of the camera placed within the key shooting angle of view.

8. The XR virtual studio camera tracking and positioning system according to claim 6, characterized in that: The second determining module includes: A second determination submodule, configured to detect the acceleration of each camera through an inertial detection unit; The second acquisition submodule is used to acquire the internal sensor data of each camera, and integrate the internal sensor data to acquire the angular velocity of each camera; A third determination submodule is used to determine the rotation matrix and translation matrix of each camera according to the acceleration and angular velocity of the camera; The fourth determination submodule is used to determine the inertial parameters of each camera according to the rotation matrix and the translation matrix of each camera.

9. The XR virtual studio camera tracking and positioning system according to claim 6, characterized in that: The third determining module comprises: The fifth determination submodule is used to determine the optimal placement direction of each camera according to the shooting angle and inertia parameters of the camera, and determine the horizontal coordinate value and the vertical coordinate value of the optimal placement position of the camera according to the optimal placement direction and the shooting position; The sixth determination submodule is used to determine the optimal placement height of each camera according to the horizontal coordinate value and the vertical coordinate value of the optimal placement position of each camera, and determine the optimal moving path of each camera according to the optimal placement height and the shooting area range of the camera; A seventh determination submodule is used to determine the optimal mobile shooting posture of each camera according to the optimal moving path and the optimal placement height and optimal placement direction of each camera; The eighth determination submodule is used to determine the optimal mobile placement position of each camera according to the path points of the optimal moving path.

10. The XR virtual studio camera tracking and positioning system according to claim 6, characterized in that: The adjustment module comprises: A ninth determination submodule, used to determine the moving path point of each camera according to the best moving shooting posture and the best moving placement position; A tenth determination submodule is used to determine the visual occlusion parameter on the moving path point of each camera, and determine the conflict factor between the visual occlusion parameter and the shooting trajectory of each camera; An adjustment submodule, used for determining a correction parameter according to the conflict factor, and adjusting the shooting trajectory of each camera based on the correction parameter; An eleventh determination submodule is used to determine a distance variation parameter between each camera and the photographed entity object according to the optimal mobile shooting posture and the optimal mobile placement position; The adjustment submodule is used to adjust the shooting parameters of each camera according to the distance change parameters, and the shooting parameters include: exposure time, focus distance and zoom range.

Images