Bare eye 3D immersive interaction method and system applied in a mega sphere building

Abstract

The application discloses a naked-eye 3D immersive interaction method and system applied to a giant ball building, and relates to the technical field of naked-eye 3D interaction; the naked-eye 3D immersive interaction method and system are characterized in that the geometric parameters of the giant ball building and the display screen array distribution parameters are collected, a regular icosahedron is used to construct a spherical surface partition and a viewing area mapping, multi-source collaborative positioning is realized in combination with an ultra-wideband and an inertial measurement unit, and errors are compensated, a parallax adaptation image is generated based on the user position, the line of sight and the virtual object depth, motion prediction and pre-rendering are used to optimize the interaction response, the positioning accuracy and the delay evaluation are synchronously completed, the problems of inaccurate naked-eye 3D interaction positioning, poor parallax adaptation and high response delay in the giant ball building are effectively solved, and the positioning accuracy, the parallax adaptation accuracy, the interaction response speed and the user immersive experience are significantly improved.

Description

Technical Field

[0001] This invention relates to the field of naked-eye 3D interaction technology, specifically to a naked-eye 3D immersive interaction method and system applied to giant spherical buildings. Background Technology

[0002] With the rapid development of cultural tourism technology and immersive display fields, giant spherical buildings, with their grand spatial scale and fully enveloping visual experience, have become an important application scenario for naked-eye 3D interactive technology. These buildings, using the inner surface of a sphere as the core display carrier, need to achieve deep immersive interaction between users and virtual scenes, and meet the demand for accurate response in situations where multiple people are watching simultaneously and moving freely. Their technological application value is increasing daily. However, within the vast space of giant spherical buildings, the realization of naked-eye 3D immersive interaction faces many technical bottlenecks:

[0003] First, establishing spatial geometric benchmarks is difficult. The curvature of the inner surface of the giant sphere is uniform and covers a wide range. Traditional measurement techniques are unable to accurately capture the core geometric parameters of the sphere. At the same time, the distributed naked-eye 3D display array has problems such as spatial dispersion and strong parameter heterogeneity. It lacks a unified and accurate parameter calibration system, resulting in a lack of reliable spatial benchmark support for subsequent interaction and rendering processes.

[0004] Secondly, the adaptation between display partitions and viewing areas is insufficient. The omnidirectional display requirements of spherical space require that partitioning not only conform to spatial geometric laws, but also dynamically match the user's real-time viewing range. Existing partitioning methods are mostly based on planar logical extensions, which easily lead to waste of display resources or distortion of the visual effect in some areas.

[0005] Third, the accuracy of user positioning and tracking is limited. In large spaces, electromagnetic wave propagation is easily affected by factors such as multipath reflection and curved surface obstruction, making it difficult for a single positioning technology to achieve real-time and accurate capture of user location.

[0006] Fourth, the dynamic adaptation capability of parallax rendering is lacking. As the user moves within the spherical space, the viewing angle and distance will continuously change. Traditional naked-eye 3D rendering technology cannot dynamically adjust the parallax parameters according to the user's real-time status and the depth information of the virtual scene, often resulting in visual problems such as ghosting, insufficient stereoscopic effect, and screen tearing.

[0007] Fifth, it is difficult to guarantee the consistency of interactive response latency and user experience. The amount of data transmission and rendering computation in the giant spherical space is huge. In addition, the user's movement state changes dynamically in real time. Existing technology is unable to balance rendering accuracy and response speed, resulting in high interactive latency.

[0008] To address the aforementioned shortcomings, a technical solution is provided. Summary of the Invention

[0009] The purpose of this invention is to address the problems of inaccurate positioning, poor parallax adaptation, and high response latency in naked-eye 3D interaction within giant spherical buildings, and to propose a naked-eye 3D immersive interaction method and system applicable to giant spherical buildings.

[0010] The objective of this invention can be achieved through the following technical solutions:

[0011] Naked-eye 3D immersive interaction methods applied within giant spherical buildings include:

[0012] S1. Spatial geometric parameter acquisition: The spherical geometric parameters of the giant spherical building and the distribution parameters of the naked-eye 3D display array are obtained through the spatial measurement unit;

[0013] S2. Spherical Partition Mapping Construction: Based on the geometric parameters of the sphere, the display partition is divided using the icosahedral spherical subdivision method, and a visibility mapping matrix between the partition and the user's viewing area is constructed.

[0014] S3, Multi-source cooperative positioning and tracking: Through a multi-type sensor array deployed in a spherical space, the user's three-dimensional position coordinates and head posture information are acquired in real time, and spherical geometric error compensation is performed;

[0015] S4. Parallax Adaptive Rendering: Based on the user's position, viewing direction, and virtual object depth, calculate the baseline and additional parallax angles and convert them into pixel offsets to generate naked-eye 3D images.

[0016] S5. Predictive Interaction Response and Evaluation: Pre-rendering is triggered by motion trend prediction, and a rapid response is achieved by combining gesture interaction recognition. At the same time, the evaluation area is divided to complete the evaluation of positioning accuracy and interaction latency.

[0017] As a further improvement of the present invention, the specific operation steps of S1 are as follows:

[0018] A global measurement control network is set up in a spherical space. The static laser tracker deployed at the center of the sphere is used as the origin of the spatial reference system. The measurement base station array composed of total stations is calibrated by the multi-station intersection measurement method to establish a global coordinate system.

[0019] Three-dimensional laser scanning is performed on the inner surface of the sphere to obtain point cloud data. The three-dimensional coordinates of the sphere center and the actual effective radius are obtained based on the least squares method. The validity of the parameters is verified by the root mean square error.

[0020] For each physical display unit, parametric measurements are performed, including spatial position measurement, physical size measurement, pixel parameter reading, and raster parameter calibration;

[0021] All display unit parameters are fused and coordinates are unified to form a display array distribution database.

[0022] As a further improvement of the present invention, the specific operation steps of S2 are as follows:

[0023] Based on the three-dimensional coordinates of the center of the giant spherical building and its actual effective radius, a reference sphere is generated in a spherical coordinate system with the center of the sphere as the origin.

[0024] Using the spherical subdivision method of a regular icosahedron, starting from a regular icosahedron inscribed in a sphere, each equilateral triangle face is recursively subdivided to obtain the vertex set, the triangle face set, the total number of vertices and the total number of partitions.

[0025] The three-dimensional coordinates of the center point of each physical display unit and its corresponding normal vector are projected onto the reference sphere. Based on the weight of the physical display unit and each spherical triangular facet, the triangular facet with the largest weight is assigned as the logical partition, and the physical units of the same logical partition are merged into a virtual display partition.

[0026] Extract the center coordinates, normal vector, area, and pixel resources of each display partition, and discretize the effective viewing area of ​​the user into a dense sampling point network; construct a visibility mapping matrix based on the visibility coefficients of each partition and sampling point.

[0027] Based on the average visibility of the partitions calculated using the visibility mapping matrix, and combined with the partition pixel resources, the partition rendering priority coefficient is obtained, and rendering resources are allocated accordingly.

[0028] As a further improvement of the present invention, the specific operation steps of S3 are as follows:

[0029] Based on the spherical geometric parameters of the giant spherical building, combined with the vertex distribution pattern of the truncated icosahedron, a spatial positioning reference network is constructed.

[0030] The user-worn interactive terminal has built-in ultra-wideband tags, inertial measurement units, and visual markers. The ultra-wideband tags send ranging signals at fixed intervals, and the distance values ​​from the ultra-wideband tags to each base station are obtained through two-way time-of-flight ranging, and are corrected based on the constructed spherical distance correction network.

[0031] Based on the corrected ranging values, the user's initial position coordinates are obtained through iterative optimization using the least squares method; the user's fused position coordinates are obtained by fusing the triaxial acceleration and angular velocity data obtained by the inertial measurement unit and calculating the relative motion position coordinates through the flight path.

[0032] Construct a spherical spatial electromagnetic wave propagation delay distribution map to obtain the additional time delay of the reflected signal, and perform direct signal separation based on the time difference between the inherent propagation delay of the direct signal and the additional time delay of the reflected signal;

[0033] A spatial position error distribution field is established by using multiple calibration beacons to perform online compensation for the fused user position; based on the user's line of sight, a quaternion representation of the head pose is constructed and converted into Euler angles; the compensated user position coordinates and head pose information are output.

[0034] As a further improvement of the present invention, the specific operation steps of S4 are as follows:

[0035] Based on the compensated user position coordinates and head posture information obtained from multi-source collaborative localization and tracking, combined with the user's interpupillary distance and head yaw angle, the spatial positions of the user's left and right eyes in the global coordinate system are obtained.

[0036] For each visible display partition within the user's field of vision, the binocular reference parallax is obtained by combining the distance from the user's head to the center point of the corresponding display partition with the interpupillary distance;

[0037] Based on the depth information of the virtual object, the additional disparity angle is obtained; the reference disparity and the additional disparity are superimposed and converted into the corresponding pixel offset.

[0038] Images with horizontal parallax are generated for each visible display partition, and the images are interleaved based on the grating array coding rules to obtain the encoded naked-eye 3D image.

[0039] As a further improvement of the present invention, the specific operation steps of S5 are as follows:

[0040] A multi-level caching mechanism is established to store user position coordinates and head pose information in a circular cache, extract motion velocity and acceleration vectors, and use second-order kinematic equations to predict user position in advance.

[0041] Deploy a distributed depth camera array to capture gestures, extract the 3D coordinates of hand skeleton points to construct gesture feature vectors, and identify the type of interaction intent;

[0042] Based on the predicted location, the visible display partition is extracted and parallax adaptive pre-rendering is performed. If the prediction confidence exceeds the threshold, it is stored in the cache.

[0043] The spherical interaction space is divided into evaluation areas. The real coordinates of the test points are calibrated, and the user positioning error and interaction delay are collected. The average error and delay of each area are statistically analyzed. If all are less than the preset threshold, the evaluation is considered satisfactory. For areas that do not meet the standard, the weight of the ultra-wideband base station and the error compensation parameters are adjusted and the evaluation is re-evaluated. An interaction performance report including positioning accuracy, delay and stability indicators is generated.

[0044] A second aspect of the present invention provides a naked-eye 3D immersive interactive system for use in giant spherical buildings, comprising:

[0045] Spatial geometric parameter acquisition module: Deploys measurement control network and base station array, scans the fitted geometric parameters of the inner surface of the sphere and verifies their effectiveness, measures the spatial position, size, pixel and raster parameters of each display unit; outputs a database of sphere geometry and display array distribution;

[0046] Spherical Partition Mapping Construction Module: Generates a reference sphere based on sphere parameters, divides it into logical partitions for display units, and constructs a visibility mapping matrix and rendering priority;

[0047] Multi-source cooperative positioning and tracking module: Deploys ultra-wideband base stations, integrates terminal ranging data and inertial measurement data, obtains user position coordinates through path correction, signal separation and error compensation, extracts head posture Euler angles and outputs them;

[0048] The parallax adaptive rendering module obtains the binocular coordinates based on the user's position and pose, calculates the baseline and additional parallax and converts them into pixel offsets, generates a parallax image and outputs a naked-eye 3D image based on raster coding rules;

[0049] Predictive Interaction Response and Evaluation Module: Caches user motion data and predicts location, captures gestures to recognize interaction intent, pre-renders and caches images; divides the evaluation area, collects positioning errors and interaction delays, performs statistical evaluation, and outputs a performance report.

[0050] Compared with the prior art, the beneficial effects of the present invention are:

[0051] This invention collects precise parameters of the geometry of a giant spherical building and the display array, constructs a mapping between spherical partitions and viewing areas using icosahedral subdivision, integrates ultra-wideband and inertial measurement units to achieve multi-source collaborative positioning and compensate for errors, dynamically adapts parallax rendering based on user position, line of sight, and virtual object depth, optimizes interactive response through motion prediction pre-rendering and gesture recognition, and simultaneously completes full-space interactive performance evaluation. This significantly improves positioning accuracy, parallax adaptation precision, and interactive response speed, enhances display resource utilization efficiency and consistency of the full-space interactive experience, and greatly enhances the user's naked-eye 3D immersive interactive experience. Attached Figure Description

[0052] Figure 1 This is a flowchart of the method of the present invention. Detailed Implementation

[0053] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0054] Example:

[0055] like Figure 1 As shown, the naked-eye 3D immersive interaction method applied to giant spherical buildings includes spatial geometric parameter acquisition, spherical partition mapping construction, multi-source collaborative localization and tracking, parallax adaptive rendering, and predictive interactive response and evaluation.

[0056] S1. Spatial Geometric Parameter Acquisition: The spherical geometric parameters of the giant spherical building and the distribution parameters of the naked-eye 3D display array are acquired through spatial measurement units; the specific implementation process is as follows:

[0057] Geometric parameter measurement of a sphere: A global measurement control network is set up in the spherical space, and a static laser tracker is deployed at the center of the sphere as the origin of the spatial reference system; multiple high-precision total stations are uniformly set up along the equatorial plane of the sphere to form a measurement base station array; the spatial coordinates of the measurement base station array are mutually calibrated by the multi-station intersection measurement method to establish a global coordinate system;

[0058] Based on the deployed measurement base station array, a three-dimensional laser scan is performed on the inner surface of the giant spherical building to obtain dense point cloud data; the three-dimensional coordinates of the center of the giant spherical building and the actual effective radius of the sphere are obtained by least squares fitting.

[0059] Through formula The root mean square error is calculated, where, Represents the three-dimensional coordinates of the sphere's center. This represents the actual effective radius of the sphere. Indicates the first Coordinates of each scan point Indicates the total number of scan points;

[0060] When the root mean square error is less than the preset difference, the geometric parameters of the sphere are deemed valid.

[0061] Display unit parameter acquisition: For each physical display unit constituting the display array, the following parametric measurements are performed:

[0062] Spatial position measurement: The three-dimensional coordinates of multiple non-coplanar feature points on the outer frame of the display unit are measured using a laser tracker. The equation of the sphere on which the display unit is located is obtained through plane fitting, and the plane normal vector is obtained. Combined with the measured coordinates of the center point of the display unit, the spatial position of the current single physical display unit is obtained.

[0063] Physical dimension measurement: Using a high-resolution industrial camera with a calibration board, the visible width and visible height of the display unit are measured;

[0064] Pixel parameter reading: The horizontal and vertical pixel resolutions are obtained directly from the display unit driver board through the display control interface;

[0065] Grating parameter calibration: By displaying a specific test pattern and using a camera to capture and analyze moiré fringes from multiple predetermined viewpoints, the pitch and tilt angle of the naked-eye 3D grating are obtained;

[0066] Parameter fusion and unification: All measured display unit position parameters are unified from the local coordinate system at the time of measurement to the global coordinate system through coordinate transformation;

[0067] Based on the center point of the display unit, the projection point is obtained by projecting the center point of the display unit onto the fitted sphere. The spherical angular coordinates corresponding to the center of the display unit are then calculated using the following formula. :

[0068] , ;

[0069] in, This indicates the three-dimensional coordinates of the center point of the display unit;

[0070] Based on the physical dimensions of the display unit and the actual effective radius of the sphere, using the formula... The approximate solid angle of the sphere covered by the display unit is obtained by estimation. ,in, These represent the visible width and visible height of the display unit, respectively.

[0071] All data that has been integrated and unified will be stored in a structured format to form a display array distribution database, which records the core information of each display unit, including the unit's unique identifier ID, the center point coordinates and normal vector in the global coordinate system, physical size, pixel resolution, raster parameters, and the corresponding approximate solid angle of the covered sphere.

[0072] Finally, the complete database of sphere geometry parameters and display array distribution is output.

[0073] S2. Spherical Partition Mapping Construction: Based on the geometric parameters of the sphere, the inner surface of the sphere is divided into multiple display partitions, and a mapping relationship between each display partition and the user's viewing area is established; the specific implementation process is as follows:

[0074] Based on the three-dimensional coordinates of the sphere's center and the sphere's actual effective radius, a reference sphere is generated in the spherical coordinate system; the spherical coordinate system refers to a three-dimensional spatial reference system with the three-dimensional coordinates of the giant spherical building's center as its origin;

[0075] Using the spherical subdivision method of a regular icosahedron, starting from a regular icosahedron inscribed in a sphere, each equilateral triangular face of the regular icosahedron is recursively subdivided to obtain a vertex set and a triangular facet set; where each triangular facet is represented by the index numbers of the three vertices in the vertex set;

[0076] Through formula The total number of vertices is calculated.

[0077] Through formula The total number of partitions is calculated.

[0078] in, This represents the number of times each equilateral triangular face of a regular icosahedron is recursively subdivided.

[0079] Based on the display array distribution database, the three-dimensional coordinates of the center point of each physical display unit and its corresponding normal vector are projected onto a reference sphere, and then processed using the formula... The weights assigned to the center point of the physical display unit and each spherical triangular facet are calculated. ,in, Indicates the first Three-dimensional coordinates of the center point of each physical display unit Indicates the first The three-dimensional coordinates of the center point of the physical display unit are vertically projected onto the first physical display unit. The three-dimensional coordinates of the projection points obtained on the plane containing the reference spherical triangular facets. This represents the alignment smoothing coefficient, a preset positive constant. Represents an exponential function. They represent the first The physical display unit and the first The unit normal vector of the plane of a reference spherical triangular facet;

[0080] Each physical display unit is assigned a reference spherical triangular facet with the highest weight as its logical partition; if multiple physical display units belong to the same logical partition, they are merged into a single virtual display partition.

[0081] For each display partition generated, key parameters are extracted, including partition center coordinates, partition normal vector, partition area, and partition pixel resources.

[0082] Partition pixel resources refer to the total horizontal and vertical pixels of all physical display units belonging to the current display partition, which is used to obtain the total pixel resolution of the partition.

[0083] Define the user's effective viewing area as a sphere with its center as the origin and a radius of 3D coordinates. The spherical space; discretize the spherical space into a dense network of sampling points;

[0084] For each display partition and viewing sampling point, the formula is used. The visibility coefficient is calculated. , Represents the Heaviside step function. Indicates the first The partition normal vector of each display partition. Indicates the first The three-dimensional spatial coordinates of each viewing sampling point in the global coordinate system. Indicates the first The three-dimensional spatial coordinates of the center point of each display partition in the global coordinate system. This indicates the angle between the line of sight and the normal, i.e. point to The line-of-sight vector and the partition normal vector The angle between them Indicates the viewing angle attenuation coefficient;

[0085] Based on the visibility coefficients of display partitions and viewing sampling points, a visibility mapping matrix between partitions and viewing points is constructed; the matrix elements are the visibility coefficients. , indicating the first The display partition for the first Visible intensity at each viewing sampling point;

[0086] Based on the visibility mapping matrix, the average visibility of each display partition across all viewing points is calculated using the formula. The partition rendering priority coefficient is calculated. ,in, This indicates the total number of viewing sampling points. They represent the first The total horizontal and vertical pixel count of each display partition Indicates the first The spherical space area of ​​each display partition; Represents a logarithmic function;

[0087] Display partitions with higher rendering priority coefficients will be allocated more real-time rendering computing resources. Meanwhile, display partitions that are not visible from any valid viewing point will be marked as inactive partitions and will not participate in real-time rendering.

[0088] Generate and output a spherical partition mapping database, including the following structured data:

[0089] Partition geometry list: Each displayed partition includes its identifier, center coordinates, normal vector, area, and the index of its associated vertex;

[0090] Partition display resource list: a list of physical display unit IDs associated with each partition, and the total pixel resolution;

[0091] Visibility mapping matrix: the visibility coefficients of compressed storage partitions and viewing points;

[0092] Partition Priority List: A list of partition IDs sorted by rendering priority and their corresponding rendering priority coefficients.

[0093] S3. Multi-source cooperative positioning and tracking: By deploying a multi-type sensor array in a spherical space, the system acquires the user's three-dimensional position coordinates and head posture information in real time, and performs spherical geometric error compensation; the specific implementation process is as follows:

[0094] On the inner surface of the spherical space, based on the geometric parameters of the sphere and the distribution pattern of the vertices of the truncated icosahedron, several ultra-wideband positioning base stations are deployed to form a spatial positioning reference network.

[0095] The user-worn interactive terminal has built-in ultra-wideband tags, inertial measurement units, and visual markers. The ultra-wideband tags send ranging signals to each ultra-wideband positioning base station at fixed intervals, and the distance values ​​from the ultra-wideband tags to each ultra-wideband positioning base station are obtained through two-way time-of-flight ranging.

[0096] To address the multipath effect and surface blocking effect of electromagnetic wave propagation in a spherical space, a spherical distance correction network is established to correct the ranging value using the spherical path.

[0097] Based on the corrected ultra-wideband ranging values, using the formula The positional residuals are calculated, where, This represents the total number of ultra-wideband positioning base stations. This indicates the coordinates of the user's location to be calculated. Indicates the first The location coordinates of an ultra-wideband positioning base station. Indicates the first Corrected ultra-wideband ranging values ​​of an ultra-wideband positioning base station. Indicates the first Weighting coefficients of each ultra-wideband positioning base station. Denotes the Euclidean norm;

[0098] The user's initial position coordinates are obtained by iterative optimization using the least squares method to minimize the position residual.

[0099] The inertial measurement unit acquires triaxial acceleration and triaxial angular velocity data, and the user's relative motion position coordinates are calculated from the flight path.

[0100] Based on the user's initial position and relative motion position, using the formula The user's location coordinates after fusion are calculated. ,in, Indicates the user's initial location coordinates. Represents the user's relative position coordinates. The fusion weighting coefficient represents the user's relative motion position coordinates, adjusted based on the real-time accuracy ratio of ultra-wideband positioning and inertial navigation;

[0101] An electromagnetic wave propagation delay distribution map is constructed in a spherical space. Based on the reflection characteristics of the spherical surface, the inner surface of the sphere is divided into a hot reflection zone and a cold reflection zone. This is achieved using the formula... The additional time delay of the reflected signal is calculated. ;in, This represents the straight-line distance from the user's location coordinates to the center of the sphere after merging. This indicates the speed at which electromagnetic waves propagate in the air;

[0102] Based on the inherent propagation delay of the direct signal and the additional delay of the reflected signal, the time difference between the direct signal and the reflected signal is obtained. ;

[0103] When the time difference between the direct signal and the reflected signal is less than a preset threshold, the signal superposition decomposition technique is used to separate the direct signal component from the received mixed signal.

[0104] Through formula The amplitude of the direct signal is calculated, where, This represents the total amplitude of the received signal, that is, the amplitude of the mixed signal after the direct signal and the reflected signal are superimposed. This represents the electromagnetic wave attenuation coefficient of the interior space of a giant spherical building.

[0105] Calibration beacons are set at different locations in a spherical space. The calibration beacons periodically emit positioning signals at known positions. By comparing the positioning results of the calibration beacons with the actual positions, the positioning error value at each calibration beacon is obtained.

[0106] Based on the positioning error values ​​at all calibration beacons, a spatial position error distribution field is constructed; this is then expressed using the formula... Define any point within the interior space of a giant spherical building The estimated positioning error vector; where, Indicates the total number of calibration beacons. Indicates the first Error data of each calibration beacon to the point The weighting coefficient of the error value at that point Indicates the first The positioning error vector obtained from the actual measurement at each calibration beacon;

[0107] The fused user location is substituted into the spatial location error distribution field for online compensation to obtain the compensated user location coordinates.

[0108] The user's gaze direction is extracted based on gyroscope data from the inertial measurement unit; a quaternion representation of the head posture is constructed and converted into Euler angles as the gaze direction parameter; finally, the compensated user position coordinates and head posture information are output.

[0109] S4, Parallax Adaptive Rendering:

[0110] Based on the compensated user position coordinates and gaze direction, the positions of the left and right eyes are obtained using the following formula:

[0111] Left eye position: ;

[0112] Right eye position: ;

[0113] in, This represents the three-dimensional coordinates of the user's head center point in the global coordinate system. Indicates the user's interpupillary distance. Indicates the user's head yaw angle;

[0114] For each visible display partition within the user's field of view, using the formula The reference disparity angles of the two eyes are calculated. ;

[0115] Based on the depth information of virtual objects in naked-eye 3D scenes, using the formula The additional disparity angle is calculated, where, This represents the straight-line distance from the user's head position to the center point of the current display partition. This indicates the distance from an object in the virtual scene to the user's head;

[0116] The reference disparity angle and the additional disparity angle are summed and converted into pixel offsets;

[0117] Based on the transformed pixel offset, a pair of images with horizontal parallax are generated for each visible display partition, including a left-eye image and a right-eye image; and the images are interleaved based on the encoding rules of the raster array to obtain the encoded naked-eye 3D image.

[0118] S5. Predictive Interaction Response and Evaluation:

[0119] A multi-level caching mechanism for user motion states is established, storing the user's position coordinates and head pose information within the most recent time window in a circular buffer; based on the historical position sequence within the time window, the user's velocity and acceleration vectors are extracted, and the instantaneous velocity vector is obtained through the forward difference method. and instantaneous acceleration vector ;

[0120] A motion trend prediction mechanism is constructed, using second-order kinematic equations to predict user positions in advance; through formulas... The user's predicted location is obtained through calculation, where, Indicates the timeframe for advance prediction;

[0121] To capture user gestures, a distributed depth camera array is deployed; the 3D coordinates of the skeletal points of the hand are extracted using computer vision algorithms to construct a gesture feature vector.

[0122] The interaction intent type is identified based on the changing trend of the gesture feature vector. The interaction intent types include pointing, grasping, swiping, and zooming. When a pointing gesture is identified, the position of the interaction target is determined based on the intersection of the finger pointing direction and the spherical display surface. A ray is constructed from the starting point of the hand and solved simultaneously with the spherical equation. The intersection parameter is obtained by using the quadratic formula. The intersection point corresponding to the smallest positive value of the intersection parameter is selected as the position of the interaction target.

[0123] Based on the user's predicted location and the location of the interaction target, a predictive rendering task is triggered:

[0124] Based on the visibility mapping matrix, the set of visible display partitions corresponding to the predicted position is extracted; S4 parallax adaptive rendering is performed on the set of visible display partitions to generate a naked-eye 3D image suitable for the predicted position;

[0125] Meanwhile, after obtaining the user's predicted position from the second-order kinematic equation, the prediction confidence of this motion prediction is obtained by combining the fitting characteristics of historical motion data within the effective time window with the stability of the user's motion state through multi-index weighted calculation.

[0126] If the prediction confidence is greater than the preset confidence threshold, the pre-rendered image frame is stored in the high-speed frame buffer. When the user actually moves to the predicted position or triggers an interaction, the pre-rendered result is directly read from the high-speed frame buffer for display.

[0127] Interaction accuracy assessment:

[0128] The spherical interactive space, with the three-dimensional coordinates of the sphere's center as the origin and the radius as the actual effective radius of the sphere, is divided into several evaluation regions.

[0129] Within each evaluation area, several test points are selected, and the three-dimensional coordinates of all test points are pre-calibrated using a laser tracker to obtain the true position coordinates of the test points, which serve as the benchmark for evaluation.

[0130] During the evaluation phase, the user stands still at each test point for at least 3 seconds, and the user's position coordinates and positioning error value after compensation at the current test point are recorded; all test points are traversed to complete the collection of positioning accuracy data;

[0131] At each test point, a standardized sequence of interactive actions is executed. The physical start time of each interactive action is recorded by a camera device. At the same time, the response time from recognition to completion of the interactive intent is recorded. The interaction delay is obtained based on the difference between the start time and the response time.

[0132] The positioning accuracy data and interaction delay of all test points in each evaluation area are statistically analyzed to obtain the mean position error and mean interaction delay of each area;

[0133] For any evaluation area, if both the average location error and the average interaction delay are less than the corresponding preset thresholds, then the current evaluation area is deemed to have met the evaluation criteria.

[0134] If there are any substandard evaluation areas, locate the evaluation area with the largest average location error, adjust the weight parameters and error compensation parameters of the ultra-wideband positioning base stations in the evaluation area, and re-execute the evaluation until all evaluation areas meet the standards; after the evaluation is completed, generate an interactive performance report, record the positioning accuracy, interactive latency and stability indicators of each evaluation area, and store them in the interactive database and upload them to the display terminal simultaneously.

[0135] A naked-eye 3D immersive interactive system applied within a giant spherical building includes:

[0136] Spatial geometric parameter acquisition module: A global measurement and control network is set up. With the static laser tracker at the center of the sphere as the origin, a global coordinate system is established through multi-station intersection measurement. After three-dimensional laser scanning of the inner surface of the sphere, the coordinates of the center of the sphere and the effective radius are fitted using the least squares method. The validity of the parameters is verified by the root mean square error. For each physical display unit, the spatial position and physical size are measured, the pixel resolution is read, the grating parameters are calibrated, all parameters are unified to the global coordinate system, and the sphere geometric parameters and the display array distribution database are output.

[0137] Spherical Partition Mapping Construction Module: Generates a reference sphere based on sphere parameters, recursively subdivides triangular faces using the icosahedral spherical subdivision method, assigns logical partitions to physical display units and merges units in the same area into virtual partitions; extracts key partition parameters, discretizes the user viewing area and calculates the visibility coefficient, constructs a visibility mapping matrix between partitions and viewing points, calculates rendering priority by combining average visibility and pixel resources, and outputs a spherical partition mapping relationship database;

[0138] Multi-source cooperative positioning and tracking module: Ultra-wideband positioning base stations are deployed according to the vertices of a truncated icosahedron. The user terminal obtains the distance value through bidirectional time-of-flight ranging, and the multipath and occlusion errors are corrected by a spherical distance correction network. The module integrates the preliminary position of the ultra-wideband positioning with the trajectory calculation results of the inertial measurement unit to construct an electromagnetic wave propagation delay distribution map to separate the direct signal. The module establishes a spatial position error distribution field by calibrating beacons to compensate for the user's position error. The module extracts gyroscope data and converts it into Euler angles of head attitude to output accurate position coordinates and attitude information.

[0139] The parallax adaptive rendering module calculates the baseline parallax angle for the visible display partition based on the spatial position of both eyes, and obtains the additional parallax angle by combining the depth information of the virtual object. After superposition, it is converted into a pixel offset. It generates left and right eye parallax images for each partition, interweaves the images according to the raster array encoding rules, and outputs naked-eye 3D images.

[0140] Predictive Interaction Response and Evaluation Module: Stores recent user motion data in a circular buffer, extracts velocity and acceleration vectors, and predicts user position using second-order kinematic equations; deploys a depth camera array to capture gestures and recognize interaction intentions, pre-renders and caches images based on the predicted position; divides a spherical interaction space into evaluation areas, calibrates the real coordinates of test points, collects positioning errors and interaction delays, calculates the mean of the area and judges the interaction performance; for areas that do not meet the standards, adjusts parameters and re-evaluates, generating an interaction performance report.

[0141] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A naked-eye 3D immersive interactive method applied within a giant spherical building, characterized in that, include: S1. Spatial geometric parameter acquisition: The spherical geometric parameters of the giant spherical building and the distribution parameters of the naked-eye 3D display array are obtained through the spatial measurement unit; S2. Spherical Partition Mapping Construction: Based on the geometric parameters of the sphere, the display partition is divided using the icosahedral spherical subdivision method, and a visibility mapping matrix between the partition and the user's viewing area is constructed. The specific operation steps of S2 are as follows: Based on the three-dimensional coordinates of the center of the giant spherical building and its actual effective radius, a reference sphere is generated in a spherical coordinate system with the center of the sphere as the origin. Using the spherical subdivision method of a regular icosahedron, starting from a regular icosahedron inscribed in a sphere, each equilateral triangle face is recursively subdivided to obtain the vertex set, the triangle face set, the total number of vertices and the total number of partitions. The three-dimensional coordinates of the center point of each physical display unit and its corresponding normal vector are projected onto the reference sphere. Based on the weight of the physical display unit and each spherical triangular facet, the triangular facet with the largest weight is assigned as the logical partition, and the physical units of the same logical partition are merged into a virtual display partition. Extract the center coordinates, normal vector, area, and pixel resources of each display partition, and discretize the effective viewing area of ​​the user into a dense sampling point network; construct a visibility mapping matrix based on the visibility coefficients of each partition and sampling point. Based on the average visibility of the partitions calculated using the visibility mapping matrix, and combined with the partition pixel resources, the partition rendering priority coefficient is obtained, and rendering resources are allocated accordingly. S3, Multi-source cooperative positioning and tracking: Through a multi-type sensor array deployed in a spherical space, the user's three-dimensional position coordinates and head posture information are acquired in real time, and spherical geometric error compensation is performed; S4. Parallax Adaptive Rendering: Based on the user's position, viewing direction, and virtual object depth, calculate the baseline and additional parallax angles and convert them into pixel offsets to generate naked-eye 3D images. S5. Predictive Interaction Response and Evaluation: Pre-rendering is triggered by motion trend prediction, and a rapid response is achieved by combining gesture interaction recognition. At the same time, the evaluation area is divided to complete the evaluation of positioning accuracy and interaction latency.

2. The naked-eye 3D immersive interactive method applied to a giant spherical building according to claim 1, characterized in that, The specific operation steps of S1 are as follows: A global measurement control network is set up in a spherical space. The static laser tracker deployed at the center of the sphere is used as the origin of the spatial reference system. The measurement base station array composed of total stations is calibrated by the multi-station intersection measurement method to establish a global coordinate system. Three-dimensional laser scanning is performed on the inner surface of the sphere to obtain point cloud data. The three-dimensional coordinates of the sphere center and the actual effective radius are obtained based on the least squares method. The validity of the parameters is verified by the root mean square error. For each physical display unit, parametric measurements are performed, including spatial position measurement, physical size measurement, pixel parameter reading, and raster parameter calibration; All display unit parameters are fused and coordinates are unified to form a display array distribution database.

3. The naked-eye 3D immersive interactive method applied to a giant spherical building according to claim 1, characterized in that, The specific operation steps of S3 are as follows: Based on the spherical geometric parameters of the giant spherical building, combined with the vertex distribution pattern of the truncated icosahedron, a spatial positioning reference network is constructed. The user-worn interactive terminal has built-in ultra-wideband tags, inertial measurement units, and visual markers. The ultra-wideband tags send ranging signals at fixed intervals, and the distance values ​​from the ultra-wideband tags to each base station are obtained through two-way time-of-flight ranging, and are corrected based on the constructed spherical distance correction network. Based on the corrected ranging values, the user's initial position coordinates are obtained through iterative optimization using the least squares method. The user's position coordinates are obtained by fusing the triaxial acceleration and angular velocity data acquired by the inertial measurement unit and calculating the relative motion position coordinates through the flight path. Construct a spherical spatial electromagnetic wave propagation delay distribution map to obtain the additional time delay of the reflected signal, and perform direct signal separation based on the time difference between the inherent propagation delay of the direct signal and the additional time delay of the reflected signal; A spatial position error distribution field is established by using multiple calibration beacons to perform online compensation for the fused user position; based on the user's line of sight, a quaternion representation of the head pose is constructed and converted into Euler angles; the compensated user position coordinates and head pose information are output.

4. The naked-eye 3D immersive interactive method applied to a giant spherical building according to claim 1, characterized in that, The specific operation steps of S4 are as follows: Based on the compensated user position coordinates and head posture information obtained from multi-source collaborative localization and tracking, combined with the user's interpupillary distance and head yaw angle, the spatial positions of the user's left and right eyes in the global coordinate system are obtained. For each visible display partition within the user's field of vision, the binocular reference parallax is obtained by combining the distance from the user's head to the center point of the corresponding display partition with the interpupillary distance; Based on the depth information of the virtual object, the additional disparity angle is obtained; the reference disparity and the additional disparity are superimposed and converted into the corresponding pixel offset. Images with horizontal parallax are generated for each visible display partition, and the images are interleaved based on the grating array coding rules to obtain the encoded naked-eye 3D image.

5. The naked-eye 3D immersive interactive method applied to a giant spherical building according to claim 1, characterized in that, The specific operation steps of S5 are as follows: A multi-level caching mechanism is established to store user position coordinates and head pose information in a circular cache, extract motion velocity and acceleration vectors, and use second-order kinematic equations to predict user position in advance. Deploy a distributed depth camera array to capture gestures, extract the 3D coordinates of hand skeleton points to construct gesture feature vectors, and identify the type of interaction intent; Based on the predicted location, the visible display partition is extracted and parallax adaptive pre-rendering is performed. If the prediction confidence exceeds the threshold, it is stored in the cache. The spherical interaction space is divided into evaluation areas. The real coordinates of the test points are calibrated, and the user positioning error and interaction delay are collected. The average error and delay of each area are statistically analyzed. If all are less than the preset threshold, the evaluation is considered satisfactory. For areas that do not meet the standard, the weight of the ultra-wideband base station and the error compensation parameters are adjusted and the evaluation is re-evaluated. An interaction performance report including positioning accuracy, delay and stability indicators is generated.

6. A system for applying the naked-eye 3D immersive interactive method described in any one of claims 1-5 within a giant spherical building, comprising: Spatial geometric parameter acquisition module: Deploys measurement control network and base station array, scans the fitted geometric parameters of the inner surface of the sphere and verifies their effectiveness, measures the spatial position, size, pixel and raster parameters of each display unit; outputs a database of sphere geometry and display array distribution; Spherical Partition Mapping Construction Module: Generates a reference sphere based on sphere parameters, divides it into logical partitions for display units, and constructs a visibility mapping matrix and rendering priority; Multi-source cooperative positioning and tracking module: Deploys ultra-wideband base stations, integrates terminal ranging data and inertial measurement data, obtains user position coordinates through path correction, signal separation and error compensation, extracts head posture Euler angles and outputs them; The parallax adaptive rendering module obtains the binocular coordinates based on the user's position and pose, calculates the baseline and additional parallax and converts them into pixel offsets, generates a parallax image and outputs a naked-eye 3D image based on raster coding rules; Predictive interaction response and evaluation module: caches user motion data and predicts location, captures gestures to recognize interaction intent, pre-renders images and caches them; Divide the evaluation area, collect positioning errors and interaction delays, perform statistical evaluations, and output a performance report.

Images