A real-time compensation method for camera pose errors in 3D rendering

Abstract

The present invention relates to the field of computer graphics technology, and specifically to a real-time compensation method for camera pose errors in a three-dimensional rendering process, comprising the following steps: S1, acquiring and updating the projection parameters of a main camera and a user interface UI camera in real time; S2, obtaining a scale factor based on the projection parameters of the user interface UI camera, and dynamically correcting the user input coordinates; when used, the present invention, in contrast to the prior art that only solves the coordinate correction under single-camera scaling, covers the input coordinate asynchronous problem caused by the independent projection parameters of multiple cameras. In the application environment of a digital twin platform, the operator needs to observe and monitor the status of the device model through the main camera. At the same time, he needs to interact with the control panel rendered by the UI camera in real time. When the UI camera adjusts the scaling ratio, the present method can ensure that the button click accuracy on the control panel is not affected, thereby ensuring the accuracy and smoothness of the operation.

Description

Technical Field

[0001] The present invention relates to the technical field of computer graphics, and in particular to a real-time compensation method for camera posture errors in a three-dimensional rendering process. Background Art

[0002] In the multi-camera collaborative rendering architecture of complex graphics systems (such as digital twins and game engines), the main camera is mainly responsible for dynamically rendering three-dimensional scenes to ensure the realism and smoothness of the scenes, while the independent UI camera is specifically responsible for processing the rendering output of interface elements to ensure the clarity and responsiveness of the user interface.

[0003] In existing technologies, multi-camera collaborative rendering methods face significant technical bottlenecks and challenges in practical applications:

[0004] 1. Existing technologies have obstacles in synchronizing input coordinates with multi-camera viewports. When users perform interactive operations, traditional coordinate conversion methods are often designed based on the reference frame of a single camera and cannot effectively respond to the independent projection parameters of the UI camera. This can lead to a series of problems such as click position offset and interaction failure, seriously affecting the user experience.

[0005] 2. The existing technology lacks an effective cross-camera coordinate mapping mechanism and a dynamic coordinate association rule between the main camera and the UI camera. When the main camera's perspective changes, UI elements cannot maintain the correct spatial reference position, resulting in misalignment or display anomalies.

[0006] 3. Existing technologies lack dynamic UI compensation capabilities. When faced with the combined effects of UI offsets required for screen adaptation (such as floating window position adjustment) and camera zoom parameters, existing technologies struggle to automatically integrate and correct coordinate systems, failing to ensure accurate display of UI elements at different viewing angles and zoom levels.

[0007] The above technical pain points not only limit the performance and stability of the multi-camera collaborative rendering architecture, but also severely restrict the user experience in complex interactive scenarios.

[0008] In summary, developing a real-time compensation method for camera pose errors in the three-dimensional rendering process is still a key issue that needs to be solved urgently in the field of computer graphics technology. Summary of the Invention

[0009] The purpose of the present invention is to solve the problem in the prior art that not only limits the performance and stability of the multi-camera collaborative rendering architecture, but also seriously restricts the user experience in complex interactive scenarios.

[0010] To achieve the above object, the present invention provides the following technical solutions:

[0011] The present invention provides a real-time compensation method for camera pose error in a three-dimensional rendering process, comprising the following steps:

[0012] S1. Obtain and update the projection parameters of the main camera and the user interface UI camera in real time;

[0013] S2. Obtain a scale factor based on the projection parameters of the user interface UI camera, and dynamically correct the user input coordinates;

[0014] S3. Obtaining a UI offset according to the projection parameters;

[0015] S4. Generate viewport coordinate data according to the screen coordinates input by the user, the UI offset, and the scale factor;

[0016] S5. Map the corrected viewport coordinate data to the coordinate system of the main camera.

[0017] Furthermore, in step S1, the method for obtaining and updating the projection parameters of the main camera and the user interface UI camera in real time is:

[0018] The real-time monitoring user interface UI camera adopts the viewport height parameter of orthogonal projection, and the viewport height parameter is set to , continuously monitored as a function of time, the orthogonal projection matrix expression:

[0019] ,

[0020] Where, Indicates at time The current orthographic projection size parameters of the user interface UI camera, Indicates at time The orthographic projection matrix, The width parameter represents the viewport size, Indicates the camera's far clipping plane distance, Indicates the near clipping plane distance of the camera, express The scaling factor in the axis direction maps the viewport width to Within the interval, express The scaling factor in the axis direction maps the viewport height to Within the interval, express The scaling factor in the axis direction will reduce the depth range Map to , express Axis translation offset item, used to move the midpoint to the origin so that the depth value is linearly mapped to , This indicates the requirement to maintain the x, y, and z axes unchanged in the matrix. By continuously monitoring changes in the viewport height parameter, the world coordinate system transformation matrix of the main camera is obtained. The world coordinate system transformation matrix records the specific position, rotation angle, and scaling of the main camera in three-dimensional space, and serves as the basic data for scene rendering and coordinate transformation.

[0021] Furthermore, in step S2, the scale factor is obtained according to the projection parameters of the user interface UI camera, and the method for dynamically correcting the user input coordinates is as follows:

[0022] According to the ratio between the current orthogonal projection size parameter of the projection parameter user interface UI camera and the initially set orthogonal projection parameter, a scaling factor for adjusting the view display range is calculated and named as the scale factor. The expression is:

[0023] ,

[0024] Where, Indicates at time The calculated scaling factor, Indicates at time The current orthographic projection size parameters of the user interface UI camera, Indicates the initial settings of the orthographic projection parameters.

[0025] Furthermore, in step S2, the scale factor is obtained according to the projection parameters of the user interface UI camera, and the method for dynamically correcting the user input coordinates is as follows:

[0026] Based on the dynamic adaptation mechanism of scale factor and mathematical model, the interactive coordinates input by the user are intelligently and dynamically adjusted. Through real-time transformation and compensation of the coordinate system, the relative position and display ratio of the target UI elements have a corresponding relationship under various view size changes, introducing a disturbance term. Model the UI scaling error and build a dynamic adaptation model with disturbance compensation terms. The expression is:

[0027] ,

[0028] Where, Indicates that after dynamic adaptation adjustment at time The target coordinates, is with The transformation matrix of the factor correlation at time Function to perform the input coordinates based on Logical transformation, represents the original interaction coordinates entered by the user, represents the homogeneous terms of homogeneous coordinates, represents the disturbance term, represents the disturbance term Obey the mean of 0, in the real number field One of the normal distributions on is the symbol for the normal distribution.

[0029] Furthermore, in step S3, the method for obtaining the UI offset according to the projection parameters is:

[0030] Get the complete geometric information of the axis-aligned bounding box of the target UI element in the canvas space from the projection parameters, and then extract the coordinates of the lower left corner of the axis-aligned bounding box of the target UI element and the coordinates of the upper right vertex ,expression:

[0031] ,

[0032] Where, Represents the coordinates of all vertices that make up the geometry of the UI element. yes In its expanded form, yes In its expanded form, The coordinate value is composed of the minimum component value of the three axes x, y, and z in the canvas coordinate system of the UI element, representing the lower left spatial position of the UI element in the canvas space; The coordinate value is composed of the maximum component value of the UI element in the three axes, which defines the upper right boundary position of the UI element in the canvas space. The coordinates of the two diagonal vertices are mathematically defined to fully describe the minimum enclosing rectangular area of ​​the UI element in the canvas space in the form of floating-point numbers. It not only contains the absolute position information of the UI element, but also implicitly encodes the actual size data of the UI element in the canvas. In the subsequent coordinate conversion and spatial calculation process, this pair of boundary coordinate data will serve as the core basic parameter, providing a reliable geometric basis for the key processing links of the UI element's world space position calculation, collision detection, and layout optimization.

[0033] Furthermore, in step S3, the method for obtaining the UI offset according to the projection parameters is:

[0034] Based on the optimized memory access mechanism and cache strategy, all basic geometric data are accessed and updated to adapt to various layout changes in dynamic UI scenes. In the process of calculating the spatial size of UI elements, an algorithm based on the three-dimensional Euclidean distance formula is used to measure the actual physical width of UI elements in the world coordinate system. At the same time, the calculation process of the spatial size of the entire UI element adopts high-precision floating-point operations and introduces an anti-jitter filtering algorithm. First, the coordinate system is converted and processed. and The world coordinate values ​​of the two vertices are then input into the optimized Euclidean distance calculation module to calculate the component differences of the two vertices on each coordinate axis. , , , then square each component difference, sum it up and take the square root, and finally output the straight-line distance value. This calculation result is cached as the UI offset, expression:

[0035] ,

[0036] Where, express offset, Indicates the straight-line distance related to the spatial size of the UI element calculated based on the three-dimensional Euclidean distance. Is the symbol for the Euclidean norm, indicating that the two vertices are in the world coordinate system The difference of the components on the coordinate axis, Indicates that the two vertices are in the world coordinate system The difference of the components on the coordinate axis, Indicates that the two vertices in the world coordinate system are The component difference on the coordinate axis, the UI offset not only includes the absolute width information of the UI element, but is also normalized to adapt to display requirements under different resolutions.

[0037] Furthermore, in step S4, the method for generating viewport coordinate data according to the screen coordinates input by the user, the UI offset and the scale factor is:

[0038] The screen coordinates input by the user are converted and processed, and the two-dimensional screen coordinates generated by the user through mouse clicks, touch operations and keyboard input are mathematically transformed and normalized and then mapped to the viewport coordinate system defined by the user interface UI camera. The conversion process integrates multiple calculation steps of screen resolution adaptation, device pixel ratio compensation and view projection inverse transformation to obtain coordinate correspondences including but not limited to different display devices and screen sizes.

[0039] Furthermore, in step S4, the method for generating viewport coordinate data according to the screen coordinates input by the user, the UI offset and the scale factor is:

[0040] The x component of the screen coordinates input by the user is added to the UI offset. The coordinate value generated by the addition operation reflects the relative position of the UI element with respect to the screen. After the addition operation is completed, it is multiplied by the scale factor to further adjust the coordinate value to adapt to the UI layout at different zoom levels. Finally, a corrected viewport coordinate is generated. The expression is:

[0041] ,

[0042] Where, Represents the corrected viewport coordinates, yes After expansion, it represents the viewport coordinates Quantity and The form of the portion, Refers to the corrected viewport coordinates component, is the scaling factor, Represents the screen coordinates of the user input Quantity, express offset, Is the corrected viewport coordinates Quantity, The screen coordinates entered by the user Quantity.

[0043] Furthermore, in step S5, the method of mapping the corrected viewport coordinate data to the coordinate system of the main camera is:

[0044] According to the conversion of the viewport coordinate data into world coordinates, the mathematical principle of matrix transformation is adopted to realize spatial mapping through the world coordinate system conversion matrix of the main camera. Specifically, the spatial mapping process combines the corrected viewport coordinate data and the depth value z into homogeneous coordinates, and then multiplies the homogeneous coordinates with the world coordinate system conversion matrix of the main camera through matrix multiplication to obtain a composite matrix, which is expressed as follows:

[0045] ,

[0046] Where, is the four-dimensional homogeneous world coordinate, with the superscript Indicates that it is in homogeneous coordinate form, is the transformation matrix of the main camera, are homogeneous viewport coordinates, is the projection matrix of the main camera, is the view matrix of the main camera, Indicates that the two-dimensional coordinates on the viewport plane correspond to the horizontal and vertical positions of the screen or viewport space. It is a depth-related coordinate used to represent the information of the point in the depth direction of the viewport.

[0047] Furthermore, in step S5, the method of mapping the corrected viewport coordinate data to the coordinate system of the main camera is:

[0048] The composite matrix contains information about the camera's spatial position, rotation state, and projection parameters. Finally, perspective division is performed to convert the homogeneous coordinates into world coordinates. The expression is:

[0049] ,

[0050] Where, The actual position of an object in three-dimensional space is converted from four-dimensional homogeneous world coordinates through perspective division. represents the three-dimensional real space, 、 、 They are four-dimensional homogeneous world coordinates The x, y, and z components of Represents four-dimensional homogeneous world coordinates The homogeneous components of the image are obtained, and a reverse verification mechanism is introduced to reproject the obtained world coordinates back to the screen space, perform a closed-loop verification with the original input coordinates, and control the conversion error to less than 0.5 pixels.

[0051] Beneficial effects:

[0052] Compared with the known public technology, the technical solution provided by the present invention has the following beneficial effects:

[0053] When in use, compared with the existing technology that only solves the coordinate correction under single camera scaling, this method covers the input coordinate asynchronous problem caused by independent projection parameters of multiple cameras. In the application environment of the digital twin platform, the operator needs to observe and monitor the status of the device model through the main camera. At the same time, it is also necessary to interact with the control panel rendered by the UI camera in real time. When the UI camera adjusts the zoom ratio, this method can ensure that the button click accuracy on the control panel is not affected, thereby ensuring the accuracy and smoothness of the operation.

[0054] When in use, the present invention has the function of dynamically correcting the input screen coordinates according to the changes in the camera viewing angle, which facilitates the automatic adjustment and optimization of coordinate data under different viewing angles and zoom states, so that the input screen coordinates are highly consistent with the actual displayed content, thereby improving the user interaction experience and operating efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 This is a flowchart of a real-time compensation method for camera pose error in a three-dimensional rendering process according to the present invention.

[0056] Figure 2 This is a schematic diagram of an input coordinate correction method based on a real-time compensation method for camera pose errors in a three-dimensional rendering process according to the present invention. DETAILED DESCRIPTION

[0057] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0058] It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having," as well as any variations thereof, are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus comprising a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to these processes, methods, products, or apparatuses.

[0059] The present invention is described in further detail below with reference to the accompanying drawings:

[0060] like Figure 1-Figure 2 As shown, the present invention provides a real-time compensation method for camera pose error in a three-dimensional rendering process, comprising the following steps:

[0061] S1. Obtain and update the projection parameters of the main camera and the user interface UI camera in real time;

[0062] Furthermore, in step S1, the method for obtaining and updating the projection parameters of the main camera and the user interface UI camera in real time is:

[0063] The real-time monitoring user interface UI camera adopts the viewport height parameter of orthogonal projection, and the viewport height parameter is set to , continuously monitored as a function of time, the orthogonal projection matrix expression:

[0064] ,

[0065] Where, Indicates at time The current orthographic projection size parameters of the user interface UI camera, Indicates at time The orthographic projection matrix, The width parameter represents the viewport size, Indicates the camera's far clipping plane distance, Indicates the near clipping plane distance of the camera, express The scaling factor in the axis direction maps the viewport width to Within the interval, express The scaling factor in the axis direction maps the viewport height to Within the interval, express The scaling factor in the axis direction will reduce the depth range Map to , express Axis translation offset item, used to move the midpoint to the origin so that the depth value is linearly mapped to , This indicates the requirement to maintain the x, y, and z axes unchanged in the matrix. By continuously monitoring changes in the viewport height parameter, the world coordinate system transformation matrix of the main camera is obtained. The world coordinate system transformation matrix records the specific position, rotation angle, and scaling of the main camera in three-dimensional space, and serves as the basic data for scene rendering and coordinate transformation.

[0066] In this embodiment, this method obtains and updates the projection parameters of the main camera and the user interface UI camera in real time as the basis for subsequent coordinate calculation and image rendering. The user interface UI camera adopts an orthogonal projection method. By synchronously updating the parameters of the main camera and the UI camera, the coordination between the three-dimensional view and the two-dimensional interface is enhanced, which facilitates the realization of multi-view linkage. The projection parameters are updated in real time to ensure rendering consistency and geometric accuracy under different resolutions and devices, which facilitates the improvement of rendering accuracy.

[0067] S2. Obtain a scale factor based on the projection parameters of the user interface UI camera, and dynamically correct the user input coordinates;

[0068] Furthermore, in step S2, the scale factor is obtained according to the projection parameters of the user interface UI camera, and the method for dynamically correcting the user input coordinates is as follows:

[0069] According to the ratio between the current orthogonal projection size parameter of the projection parameter user interface UI camera and the initially set orthogonal projection parameter, a scaling factor for adjusting the view display range is calculated and named as the scale factor. The expression is:

[0070] ,

[0071] Where, Indicates at time The calculated scaling factor, Indicates at time The current orthographic projection size parameters of the user interface UI camera, Indicates the initial settings of the orthographic projection parameters.

[0072] Furthermore, in step S2, the scale factor is obtained according to the projection parameters of the user interface UI camera, and the method for dynamically correcting the user input coordinates is as follows:

[0073] Based on the dynamic adaptation mechanism of scale factor and mathematical model, the interactive coordinates input by the user are intelligently and dynamically adjusted. Through real-time transformation and compensation of the coordinate system, the relative position and display ratio of the target UI elements have a corresponding relationship under various view size changes, introducing a disturbance term. Model the UI scaling error and build a dynamic adaptation model with disturbance compensation terms. The expression is:

[0074] ,

[0075] Where, Indicates that after dynamic adaptation adjustment at time The target coordinates, is with The transformation matrix of the factor correlation at time Function to perform the input coordinates based on Logical transformation, represents the original interaction coordinates entered by the user, represents the homogeneous terms of homogeneous coordinates, represents the disturbance term, represents the disturbance term Obey the mean of 0, in the real number field One of the normal distributions on is the symbol for the normal distribution.

[0076] In this embodiment, this method calculates the scale factor of the current view based on the projection parameters of the user interface UI camera and dynamically corrects the interaction coordinates input by the user accordingly. This process is the core step of view scaling adaptation, ensuring that user interaction behaviors can still be accurately mapped to the original design position after the interface is scaled. By dynamically correcting the input coordinates, the user's click accuracy and interaction accuracy in different scaling states are improved, achieving high-precision interaction calibration. Through disturbance item modeling and compensation mechanism, the boundary error and UI dislocation caused by scaling are effectively reduced.

[0077] S3. Obtaining a UI offset according to the projection parameters;

[0078] Furthermore, in step S3, the method for obtaining the UI offset according to the projection parameters is:

[0079] Get the complete geometric information of the axis-aligned bounding box of the target UI element in the canvas space from the projection parameters, and then extract the coordinates of the lower left corner of the axis-aligned bounding box of the target UI element and the coordinates of the upper right vertex ,expression:

[0080] ,

[0081] Where, Represents the coordinates of all vertices that make up the geometry of the UI element. yes In its expanded form, yes In its expanded form, The coordinate value is composed of the minimum component value of the three axes x, y, and z in the canvas coordinate system of the UI element, representing the lower left spatial position of the UI element in the canvas space; The coordinate value is composed of the maximum component value of the UI element in the three axes, which defines the upper right boundary position of the UI element in the canvas space. The coordinates of the two diagonal vertices are mathematically defined to fully describe the minimum enclosing rectangular area of ​​the UI element in the canvas space in the form of floating-point numbers. It not only contains the absolute position information of the UI element, but also implicitly encodes the actual size data of the UI element in the canvas. In the subsequent coordinate conversion and spatial calculation process, this pair of boundary coordinate data will serve as the core basic parameter, providing a reliable geometric basis for the key processing links of the UI element's world space position calculation, collision detection, and layout optimization.

[0082] Furthermore, in step S3, the method for obtaining the UI offset according to the projection parameters is:

[0083] Based on the optimized memory access mechanism and cache strategy, all basic geometric data are accessed and updated to adapt to various layout changes in dynamic UI scenes. In the process of calculating the spatial size of UI elements, an algorithm based on the three-dimensional Euclidean distance formula is used to measure the actual physical width of UI elements in the world coordinate system. At the same time, the calculation process of the spatial size of the entire UI element adopts high-precision floating-point operations and introduces an anti-jitter filtering algorithm. First, the coordinate system is converted and processed. and The world coordinate values ​​of the two vertices are then input into the optimized Euclidean distance calculation module to calculate the component differences of the two vertices on each coordinate axis. , , , then square each component difference, sum it up and take the square root, and finally output the straight-line distance value. This calculation result is cached as the UI offset, expression:

[0084] ,

[0085] Where, express offset, Indicates the straight-line distance related to the spatial size of the UI element calculated based on the three-dimensional Euclidean distance. is the symbol for the Euclidean norm, Indicates that the two vertices are in the world coordinate system The difference of the components on the coordinate axis, Indicates that the two vertices are in the world coordinate system The difference of the components on the coordinate axis, Indicates that the two vertices in the world coordinate system are The component difference on the coordinate axis, the UI offset not only includes the absolute width information of the UI element, but is also normalized to adapt to display requirements under different resolutions.

[0086] In this embodiment, this method automatically extracts the geometric bounding information of the target UI element within its canvas space based on its projection parameters, and uses this information to calculate the UI offset used for subsequent coordinate mapping. This offset not only reflects the relative position of the UI element within the canvas, but also implicitly includes its actual spatial dimensions. Through mathematical calculations of the bounding box vertices, the system understands the actual spatial position and dimensions of the UI element, facilitating spatial perception and precise positioning. By automatically normalizing the coordinates and offsets, the system improves performance consistency across different terminal devices and facilitates cross-platform adaptability.

[0087] S4. Generate viewport coordinate data according to the screen coordinates input by the user, the UI offset, and the scale factor;

[0088] Furthermore, in step S4, the method for generating viewport coordinate data according to the screen coordinates input by the user, the UI offset and the scale factor is:

[0089] The screen coordinates input by the user are converted and processed, and the two-dimensional screen coordinates generated by the user through mouse clicks, touch operations and keyboard input are mathematically transformed and normalized and then mapped to the viewport coordinate system defined by the user interface UI camera. The conversion process integrates multiple calculation steps of screen resolution adaptation, device pixel ratio compensation and view projection inverse transformation to obtain coordinate correspondences including but not limited to different display devices and screen sizes.

[0090] Furthermore, in step S4, the method for generating viewport coordinate data according to the screen coordinates input by the user, the UI offset and the scale factor is:

[0091] The x component of the screen coordinates input by the user is added to the UI offset. The coordinate value generated by the addition operation reflects the relative position of the UI element with respect to the screen. After the addition operation is completed, it is multiplied by the scale factor to further adjust the coordinate value to adapt to the UI layout at different zoom levels. Finally, a corrected viewport coordinate is generated. The expression is:

[0092] ,

[0093] Where, Represents the corrected viewport coordinates, yes After expansion, it represents the viewport coordinates Quantity and The form of the portion, Refers to the corrected viewport coordinates component, is the scaling factor, Represents the screen coordinates of the user input Quantity, express offset, Is the corrected viewport coordinates Quantity, The screen coordinates entered by the user Quantity.

[0094] In this embodiment, this method completes the coordinate transformation processing based on the two-dimensional screen coordinates, UI offset and scale factor input by the user, and finally generates the viewport coordinate data under the current user interface UI camera definition. It is a key bridge from user physical operations (such as mouse clicks, touch, keyboard input) to interface response, ensuring that different input sources and terminal devices correctly present interaction results in a unified coordinate system, supporting unified mapping of input coordinates of different terminals such as mobile phones, tablets, and PCs, improving system compatibility, facilitating precise cross-platform interaction, integrating resolution and pixel density compensation mechanisms, ensuring consistent performance on high-DPI or low-resolution devices, introducing a scaling factor to support UI multi-level zoom display scenarios, and effectively improving operation accuracy and interface response consistency.

[0095] S5. Mapping the corrected viewport coordinate data to the coordinate system of the main camera;

[0096] Furthermore, in step S5, the method of mapping the corrected viewport coordinate data to the coordinate system of the main camera is:

[0097] According to the conversion of the viewport coordinate data into world coordinates, the mathematical principle of matrix transformation is adopted to realize spatial mapping through the world coordinate system conversion matrix of the main camera. Specifically, the spatial mapping process combines the corrected viewport coordinate data and the depth value z into homogeneous coordinates, and then multiplies the homogeneous coordinates with the world coordinate system conversion matrix of the main camera through matrix multiplication to obtain a composite matrix, which is expressed as follows:

[0098] ,

[0099] Where, is the four-dimensional homogeneous world coordinate, with the superscript Indicates that it is in homogeneous coordinate form, is the transformation matrix of the main camera, are homogeneous viewport coordinates, is the projection matrix of the main camera, is the view matrix of the main camera, Indicates that the two-dimensional coordinates on the viewport plane correspond to the horizontal and vertical positions of the screen or viewport space. It is a depth-related coordinate used to represent the information of the point in the depth direction of the viewport.

[0100] Furthermore, in step S5, the method of mapping the corrected viewport coordinate data to the coordinate system of the main camera is:

[0101] The composite matrix contains information about the camera's spatial position, rotation state, and projection parameters. Finally, perspective division is performed to convert the homogeneous coordinates into world coordinates. The expression is:

[0102] ,

[0103] Where, The actual position of an object in three-dimensional space is converted from four-dimensional homogeneous world coordinates through perspective division. represents the three-dimensional real space, 、 、 They are four-dimensional homogeneous world coordinates The x, y, and z components of Represents four-dimensional homogeneous world coordinates The homogeneous components of the image are obtained, and a reverse verification mechanism is introduced to reproject the obtained world coordinates back to the screen space, perform a closed-loop verification with the original input coordinates, and control the conversion error to less than 0.5 pixels.

[0104] In this embodiment, this method maps the corrected viewport coordinate data to the world coordinate system of the primary camera, employing a strict homogeneous coordinate space transformation and matrix multiplication mechanism to improve the accuracy of the mapping between screen interaction coordinates and 3D world coordinates. It also introduces UI offsets and scale factors to achieve cross-device, multi-resolution coordinate standardization, facilitating adaptation to multi-resolution and scaling environments. This allows for accurate click responses and object selection even in dynamic UI layouts or complex viewing angles, thereby enhancing the user interaction experience.

[0105] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A real-time compensation method for camera pose error in a three-dimensional rendering process, characterized in that: The following steps are involved: S1. Obtain and update the projection parameters of the main camera and the user interface UI camera in real time; S2. Obtain a scale factor based on the projection parameters of the user interface UI camera, and dynamically correct the user input coordinates; S3. Obtaining a UI offset according to the projection parameters; S4. Generate viewport coordinate data according to the screen coordinates input by the user, the UI offset, and the scale factor; S5. Mapping the corrected viewport coordinate data to the coordinate system of the main camera; In step S2, the scale factor is obtained according to the projection parameters of the user interface UI camera, and the method for dynamically correcting the user input coordinates is as follows: Based on the dynamic adaptation mechanism of scale factor and mathematical model, the interactive coordinates input by the user are intelligently and dynamically adjusted. Through real-time transformation and compensation of the coordinate system, the relative position and display ratio of the target UI elements have a corresponding relationship under various view size changes, introducing a disturbance term. Model the UI scaling error and build a dynamic adaptation model with disturbance compensation terms. The expression is: ， Where, Indicates that after dynamic adaptation adjustment at time The target coordinates, is with The transformation matrix of the factor correlation at time Function to perform the input coordinates based on Logical transformation, represents the original interaction coordinates entered by the user, represents the homogeneous terms of homogeneous coordinates, represents the disturbance term, represents the disturbance term Obey the mean of 0, in the real number field One of the normal distributions on is the symbol of normal distribution; In step S3, the method for obtaining the UI offset according to the projection parameters is: Get the complete geometric information of the axis-aligned bounding box of the target UI element in the canvas space from the projection parameters, and then extract the coordinates of the lower left corner of the axis-aligned bounding box of the target UI element and the coordinates of the upper right vertex ,expression: ， Where, Represents the coordinates of all vertices that make up the geometry of the UI element. yes In its expanded form, yes In its expanded form, The coordinate value is composed of the minimum component value of the three axes x, y, and z in the canvas coordinate system of the UI element, representing the lower left spatial position of the UI element in the canvas space; The coordinate value is composed of the maximum component value of the UI element in the three axes, which defines the upper right boundary position of the UI element in the canvas space. The coordinates of the two diagonal vertices are mathematically defined to fully describe the minimum enclosing rectangular area of ​​the UI element in the canvas space in the form of floating-point numbers. It not only contains the absolute position information of the UI element, but also implicitly encodes the actual size data of the UI element in the canvas. In the subsequent coordinate conversion and spatial calculation process, this pair of boundary coordinate data will serve as the core basic parameter, providing a reliable geometric basis for the key processing links of the UI element's world space position calculation, collision detection, and layout optimization.

2. The method for real-time compensation of camera pose error in a three-dimensional rendering process according to claim 1, characterized in that: In step S1, the method for obtaining and updating the projection parameters of the main camera and the user interface UI camera in real time is: The real-time monitoring user interface UI camera adopts the viewport height parameter of orthogonal projection, and the viewport height parameter is set to , continuously monitored as a function of time, the orthogonal projection matrix expression: ， Where, Indicates at time The current orthographic projection size parameters of the user interface UI camera, Indicates at time The orthographic projection matrix, The width parameter represents the viewport size, Indicates the camera's far clipping plane distance, Indicates the near clipping plane distance of the camera, express The scaling factor in the axis direction maps the viewport width to Within the interval, express The scaling factor in the axis direction maps the viewport height to Within the interval, express The scaling factor in the axis direction will reduce the depth range Map to , express Axis translation offset item, used to move the midpoint to the origin so that the depth value is linearly mapped to , Indicates that the matrix is ​​maintained 、 、 The axially invariant construction requirement is to obtain the world coordinate system transformation matrix of the main camera by continuously monitoring the changes in the viewport height parameters. The world coordinate system transformation matrix records the specific position, rotation angle and scaling ratio of the main camera in three-dimensional space, which serves as the basic data in the scene rendering and coordinate transformation process.

3. The method for real-time compensation of camera pose error in a three-dimensional rendering process according to claim 2, characterized in that: In step S2, the scale factor is obtained according to the projection parameters of the user interface UI camera, and the method for dynamically correcting the user input coordinates is as follows: According to the ratio between the current orthogonal projection size parameter of the projection parameter user interface UI camera and the initially set orthogonal projection parameter, a scaling factor for adjusting the view display range is calculated and named as the scale factor. The expression is: ， Where, Indicates at time The calculated scaling factor, Indicates at time The current orthographic projection size parameters of the user interface UI camera, Indicates the initial settings of the orthographic projection parameters.

4. The method for real-time compensation of camera pose error in a three-dimensional rendering process according to claim 3, characterized in that: In step S3, the method for obtaining the UI offset according to the projection parameters is: Based on the optimized memory access mechanism and cache strategy, all basic geometric data are accessed and updated to adapt to various layout changes in dynamic UI scenes. In the process of calculating the spatial size of UI elements, an algorithm based on the three-dimensional Euclidean distance formula is used to measure the actual physical width of UI elements in the world coordinate system. At the same time, the calculation process of the spatial size of the entire UI element adopts high-precision floating-point operations and introduces an anti-jitter filtering algorithm. First, the coordinate system is converted and processed. and The world coordinate values ​​of the two vertices are then input into the optimized Euclidean distance calculation module to calculate the component differences of the two vertices on each coordinate axis. , , , then square each component difference, sum it up and take the square root, and finally output the straight-line distance value. This calculation result is cached as the UI offset, expression: ， Where, express offset, Indicates the straight-line distance related to the spatial size of the UI element calculated based on the three-dimensional Euclidean distance. is the symbol for the Euclidean norm, Indicates that the two vertices are in the world coordinate system The difference of the components on the coordinate axis, Indicates that the two vertices are in the world coordinate system The difference of the components on the coordinate axis, Indicates that the two vertices in the world coordinate system are The component difference on the coordinate axis, the UI offset not only includes the absolute width information of the UI element, but is also normalized to adapt to display requirements under different resolutions.

5. The method for real-time compensation of camera pose error in a three-dimensional rendering process according to claim 4, characterized in that: In step S4, the method for generating viewport coordinate data according to the screen coordinates input by the user, the UI offset and the scale factor is as follows: The screen coordinates input by the user are converted and processed, and the two-dimensional screen coordinates generated by the user through mouse clicks, touch operations and keyboard input are mathematically transformed and normalized and then mapped to the viewport coordinate system defined by the user interface UI camera. The conversion process integrates multiple calculation steps of screen resolution adaptation, device pixel ratio compensation and view projection inverse transformation to obtain coordinate correspondences including but not limited to different display devices and screen sizes.

6. The method for real-time compensation of camera pose error in a three-dimensional rendering process according to claim 5, characterized in that: In step S4, the method for generating viewport coordinate data according to the screen coordinates input by the user, the UI offset and the scale factor is as follows: The x component of the screen coordinates input by the user is added to the UI offset. The coordinate value generated by the addition operation reflects the relative position of the UI element with respect to the screen. After the addition operation is completed, it is multiplied by the scale factor to further adjust the coordinate value to adapt to the UI layout at different zoom levels. Finally, a corrected viewport coordinate is generated. The expression is: , Where, Represents the corrected viewport coordinates, yes After expansion, it represents the viewport coordinates Quantity and The form of the portion, Refers to the corrected viewport coordinates component, is the scaling factor, Represents the screen coordinates of the user input Quantity, express offset, Is the corrected viewport coordinates Quantity, The screen coordinates entered by the user Quantity.

7. The method for real-time compensation of camera pose error in a three-dimensional rendering process according to claim 6, characterized in that: In step S5, the method of mapping the corrected viewport coordinate data to the coordinate system of the main camera is as follows: According to the conversion of the viewport coordinate data into world coordinates, the mathematical principle of matrix transformation is adopted to realize spatial mapping through the world coordinate system conversion matrix of the main camera. Specifically, the spatial mapping process combines the corrected viewport coordinate data and the depth value z into homogeneous coordinates, and then multiplies the homogeneous coordinates with the world coordinate system conversion matrix of the main camera through matrix multiplication to obtain a composite matrix, which is expressed as follows: ， Where, is the four-dimensional homogeneous world coordinate, with the superscript Indicates that it is in homogeneous coordinate form, is the transformation matrix of the main camera, are homogeneous viewport coordinates, is the projection matrix of the main camera, is the view matrix of the main camera, Indicates that the two-dimensional coordinates on the viewport plane correspond to the horizontal and vertical positions of the screen or viewport space. It is a depth-related coordinate used to represent the information of the point in the depth direction of the viewport.

8. The method for real-time compensation of camera pose error in a three-dimensional rendering process according to claim 7, characterized in that: In step S5, the method of mapping the corrected viewport coordinate data to the coordinate system of the main camera is as follows: The composite matrix contains information about the camera's spatial position, rotation state, and projection parameters. Finally, perspective division is performed to convert the homogeneous coordinates into world coordinates. The expression is: ， Where, The actual position of an object in three-dimensional space is converted from four-dimensional homogeneous world coordinates through perspective division. represents the three-dimensional real space, They are four-dimensional homogeneous world coordinates of 、 、 Quantity, Represents four-dimensional homogeneous world coordinates The homogeneous components of the image are obtained, and a reverse verification mechanism is introduced to reproject the obtained world coordinates back to the screen space, perform a closed-loop verification with the original input coordinates, and control the conversion error to less than 0.5 pixels.

Images