Virtual-real interaction method and system for naked-eye 3D large screen

Abstract

The invention discloses a virtual-real interaction method and system for a naked-eye 3D large screen, and relates to the technical field of large screen interaction. According to the method, a crosstalk distance value, a crosstalk discrete value and a crosstalk deviation value three-dimensional evaluation system are introduced, a crosstalk general merit value is calculated after three-dimensional indexes are subjected to normalization processing, and a screening inclination angle with a relatively high crosstalk general merit value is selected as an optimal lenticular grating inclination angle of a corresponding grid unit; the crosstalk rate level, the fluctuation stability and the high-interference occurrence frequency are considered, and the problem that in the prior art, parameters are screened only based on the average crosstalk rate single index, and the stability and the reliability of the crosstalk rate are not considered is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of large - screen interaction, and particularly relates to a method and system for virtual - reality interaction of a naked - eye 3D large screen. Background Art

[0002] With the continuous development of display technology, naked - eye 3D large screens have been increasingly widely used in fields such as advertising display, entertainment experience, education and training, etc., because they can provide users with a 3D visual effect without wearing special glasses.

[0003] However, the current interaction methods for naked - eye 3D large screens still have the following deficiencies in the actual application process: In terms of parallax compensation and beam - splitting parameter optimization, traditional methods usually only screen parameters based on a single index of the average crosstalk rate, without considering the stability and reliability of the crosstalk rate. For example, a certain tilt angle may have a low average crosstalk rate, but the fluctuation is severe or there is occasionally a high crosstalk, and a single index cannot identify such problems, resulting in unstable display effects. No intelligent caching mechanism is established, and the beam - splitting parameters need to be calculated in real - time for each interaction, resulting in a high response delay. Especially in the scenario of multi - person high - frequency interaction, the competition for computing resources intensifies, and the delay increases significantly.

[0004] Therefore, a method and system for virtual - reality interaction of a naked - eye 3D large screen are introduced. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for virtual - reality interaction of a naked - eye 3D large screen to solve the problems raised in the above - mentioned background art.

[0006] The object of the present invention can be achieved through the following technical solutions: A method for virtual - reality interaction of a naked - eye 3D large screen, including: User position tracking: Using a camera array pre - deployed around the large screen to collect user visual images in real - time, synchronously activating a TOF depth sensor to obtain a human body depth image and an RGB image, and calculating the six - degree - of - freedom parameters of the user's head, that is, the three - dimensional coordinates (X, Y, Z) of the head and the head pose data; the head pose data includes pitch angle, yaw angle, and roll angle. Viewport boundary division: For the three - dimensional display effective area within the 3D interaction space in front of the large screen, it is divided into 20×15 grid units according to the set division mechanism. Real - time parallax compensation: Using a beam - splitting parameter calibration logic to determine the optimal beam - splitting parameters for different users for each grid unit, real - time identifying the grid unit corresponding to the current user's location, extracting the optimal beam - splitting parameters for each grid unit corresponding to the current user, and performing dynamic adjustment; where the beam - splitting parameters include the tilt angle of the cylindrical lens grating and the focal length of the microlens. Spectral parameter cache: Analyze the application data of spectral parameter combinations within a set time window, and establish a cache queue based on the analysis results.

[0007] In some embodiments, the set partitioning mechanism is specifically as follows: Taking the center of the large screen as the origin, establish a right-handed coordinate system, that is, the X-axis is horizontal to the right, the Y-axis is vertical upward, and the Z-axis is perpendicular to the large screen and forward. Divide the space into 20 equal parts along the X-axis and 15 equal parts along the Y-axis, forming 20×15 horizontal grid layers perpendicular to the Z-axis.

[0008] In some embodiments, the method of using the spectral parameter calibration logic to determine the best cylindrical lens grating tilt angle for each grid unit by different users is specifically as follows: For the center point of each grid unit, rotate the cylindrical lens grating; collect the left and right eye image crosstalk rates of different users k times at different tilt angles; Calculate the average value of the left and right eye image crosstalk rates of different users k times at different tilt angles to determine the average crosstalk rate of different users at different tilt angles; Eliminate the tilt angles with an average crosstalk rate higher than the set reference rate, mark the remaining tilt angles as screened tilt angles, and integrate them as the tilt angle data set of different users; Extract the average crosstalk rates of different users at different screened tilt angles from the tilt angle data set, calculate the difference between each and the set reference rate respectively, and take the absolute value as the crosstalk distance value tra of different screened tilt angles; calculate the crosstalk discrete value trb corresponding to different screened tilt angles using the standard deviation formula for the left and right eye image crosstalk rates of different screened tilt angles k times; Identify from the left and right eye image crosstalk rates of different screened tilt angles k times, count the number higher than the set reference rate as the high crosstalk number, and calculate the ratio of the high crosstalk number to k to obtain the crosstalk deviation value trc corresponding to different screened tilt angles; Extract the crosstalk distance value tra, crosstalk discrete value trb, and crosstalk deviation value trc calculated for different users corresponding to different screened tilt angles, perform normalization processing and then substitute them into the formula Perform weighted calculation to obtain the crosstalk optimal value tq of different users corresponding to different screened tilt angles; where are the weight coefficients of the crosstalk distance value tra, crosstalk discrete value trb, and crosstalk deviation value trc respectively; For different users, select the screened tilt angle with a higher crosstalk optimal value tq as the best cylindrical lens grating tilt angle corresponding to the grid unit, marked as ; i is the grid index of the X-axis, and j is the grid index of the Y-axis.

[0009] In some embodiments, the step of using the spectroscopic parameter calibration logic to determine the optimal microlens focal length for each grid unit for different users is specifically as follows: For the center point of each grid unit, calculate the theoretical focal length according to the object distance-image distance formula, and the formula is expressed as ; where v is the distance from the display screen to the microlens, and u is the vertical distance from the user's eyes to the microlens in each grid unit; for different users, the calculated is used as the optimal microlens focal length for the corresponding grid unit.

[0010] In some embodiments, the specific steps for establishing the cache queue are as follows: Extract the usage times of each spectroscopic parameter combination from the usage data of the spectroscopic parameter combinations within the set time window; identify the usage duration of each spectroscopic parameter combination, and determine the records with a usage duration lower than the set duration as mis-triggers; eliminate the usage times of the mis-triggered spectroscopic parameters; count the usage times of each spectroscopic parameter combination after elimination within the set time window, and record it as the reference times; Sum up the reference times of each spectroscopic parameter combination to obtain the total times within the set time window; calculate the proportion of the reference times corresponding to each spectroscopic parameter combination in the total times, and use it as the usage frequency proportion of each spectroscopic parameter combination within the set time window; Extract the usage duration corresponding to each reference time of each spectroscopic parameter combination, sum up the usage durations of each group of each spectroscopic parameter combination to obtain the cumulative duration of each spectroscopic parameter combination within the set time window, and calculate the proportion of the cumulative duration of each spectroscopic parameter combination in the set time window, and use it as the usage duration proportion of each spectroscopic parameter combination within the set time window; Identify the number of users using each spectroscopic parameter combination within the set time window, and calculate the proportion of the number of users using each spectroscopic parameter combination in the total number of users, and use it as the user concentration proportion of each spectroscopic parameter combination within the set time window; Multiply the usage frequency proportion, usage duration proportion, and user concentration proportion of each spectroscopic parameter combination by the preset frequency weight coefficient, duration weight coefficient, and concentration weight coefficient respectively, and then sum them up to obtain the priority index of each spectroscopic parameter combination within the set time window; Sort each spectroscopic parameter combination in descending order according to the priority index, and extract the first H spectroscopic parameter combinations from left to right in the sorting result to construct the cache queue.

[0011] In some embodiments, it further includes: Resource elastic allocation: Real-time evaluate the activity parameters of the user, and adjust the user's GPU computing resources based on the evaluation results; where the activity parameters include the user's movement speed, interaction frequency, and fixation duration.

[0012] In some embodiments, the specific process of real-time evaluating the activity parameter of the user is as follows: Calculate the Euclidean distance between adjacent time frames using the three-dimensional coordinates (X, Y, Z) of the user's head ; The calculation formula is expressed as ; Calculate the average value of the Euclidean distances of each adjacent time frame of the user within a preset time interval as the movement speed of the user within the preset time interval; Count the number of times the user triggers an effective interaction event within a preset time interval; Denote it as the number of interactions; Calculate the ratio of the number of interactions to the preset time interval to obtain the interaction frequency of the user within the preset time interval; Determine that the user's gaze area is the large screen. When the user's head pose is stable, that is, the pitch angle and yaw angle are lower than the set angles and the duration is higher than the set time, it is determined as an effective gaze; Count the effective gaze duration of the user within a preset time interval; Convert the movement speed, interaction frequency, and effective gaze duration of the user within a preset time interval into a speed activity score, an interaction activity score, and a gaze activity score respectively; Multiply the speed activity score, interaction activity score, and gaze activity score of the user by the set speed weight coefficient, interaction weight coefficient, and gaze weight coefficient respectively, and then sum to obtain the activity evaluation index of the user.

[0013] In some embodiments, the conversion of the movement speed, interaction frequency, and effective gaze duration of the user within a preset time interval into a speed activity score, an interaction activity score, and a gaze activity score respectively is specifically as follows: Construct a speed set, a frequency set, and a gaze set corresponding to the movement speed, interaction frequency, and effective gaze duration respectively; The speed set, frequency set, and gaze set respectively contain groups of speed value ranges, groups of frequency value ranges, and groups of duration value ranges; And each group of speed value ranges corresponds to a group of speed activity scores; Each group of frequency value ranges corresponds to a group of interaction activity scores; Each group of duration value ranges corresponds to a group of gaze activity scores; Input the movement speed, interaction frequency, and effective gaze duration of the user into the speed set, frequency set, and gaze set respectively for matching the corresponding value ranges, so as to be converted into the corresponding degree activity score, interaction activity score, and gaze activity score.

[0014] In some embodiments, the adjustment of the user's GPU computing resources based on the evaluation result is specifically as follows: Obtain the current total GPU resources , and , where respectively represent the total number of threads and the total video memory; Allocating the total GPU resources based on the active scoring index of each user, that is, through the allocation logic to obtain the threads and video memory allocated to each user in the next preset time interval; where represents the active scoring index of each user, and i is the user number; is the sum of the active scoring indices of all users.

[0015] A virtual - real interaction system for a naked - eye 3D large screen, comprising: A data acquisition module: acquiring the user's visual image, human depth image, and RGB image, and calculating the six - degree - of - freedom parameters of the user's head, that is, the three - dimensional coordinates (X, Y, Z) of the head and the head pose data; A viewing area division module: dividing the three - dimensional display effective area in the 3D interaction space in front of the large screen into 20×15 grid cells; A parallax compensation module: using the beam - splitting parameter calibration logic to determine the optimal beam - splitting parameters for each grid cell for different users, real - time identifying the grid cell corresponding to the current user's position, extracting the optimal beam - splitting parameters for each grid cell corresponding to the current user, and performing dynamic adjustment; where the beam - splitting parameters include the cylindrical lens grating tilt angle and the microlens focal length; A cache setting module: analyzing the application data of the beam - splitting parameter combinations within a set time window, and establishing a cache queue based on the analysis results; A resource scheduling module: evaluating the activity parameters of the user in real - time, and adjusting the user's GPU computing resources based on the evaluation results; where the activity parameters include the user's movement speed, interaction frequency, and fixation duration.

[0016] Compared with the prior art, the beneficial effects of the present invention are: By introducing a three - dimensional evaluation system of crosstalk distance value, crosstalk dispersion value, and crosstalk deviation value, the present invention normalizes the three - dimensional indicators and then calculates the crosstalk general excellence value, selects the screening tilt angle with a higher crosstalk general excellence value as the optimal cylindrical lens grating tilt angle for the corresponding grid cell, taking into account the crosstalk rate level, fluctuation stability, and high - crosstalk occurrence frequency, and solves the problem in the prior art that only a single index of the average crosstalk rate is used to screen parameters without considering the stability and reliability of the crosstalk rate; By establishing a multi - dimensional cache queue, first, records with a usage duration lower than the set duration are determined as false triggers and filtered. The priority index is calculated by statistically analyzing the usage frequency ratio, usage duration ratio, and user concentration ratio, and the top H are extracted and pre - cached to the edge node in advance, ensuring that the cached data truly reflects the user's needs within the current set time window, capturing the temporal pattern of the user's behavior and pre - loading it into the cache, which can significantly improve the interaction delay between the user and the large screen; The present invention evaluates the user activity in real time, conducts multi-dimensional analysis from aspects such as movement speed, interaction frequency, and fixation duration, obtains the user's activity score index, and dynamically allocates GPU resources based on the activity score index, improving the utilization efficiency of hardware resources and the satisfaction of user interaction experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In the following description of exemplary embodiments in conjunction with the drawings, more details, features, and advantages of the present application are disclosed. In the drawings: Figure 1 is a flowchart of the present invention; Figure 2 is a schematic block diagram of the principle of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] The following will describe several embodiments of the present application in more detail with reference to the drawings so that those skilled in the art can implement the present application. The present application can be embodied in many different forms and for many different purposes and should not be limited to the embodiments set forth herein. These embodiments are provided so that this application will be thorough and complete, and will fully convey the scope of the application to those skilled in the art. The embodiments do not limit the present application.

[0019] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. It will be further understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the relevant art and / or the context of this specification, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.

[0020] Embodiment 1

[0021] Please refer to Figure 1 shown, a method for virtual-real interaction of a naked-eye 3D large screen, including: User position tracking: Using a camera array pre-deployed around the large screen to collect user visual images in real time, synchronously activating a TOF depth sensor to obtain a human body depth image and an RGB image, and fusing inertial navigation data through an extended Kalman filter algorithm to solve the six-degree-of-freedom parameters of the user's head, that is, the three-dimensional coordinates of the head (X, Y, Z) and the head pose data (pitch angle, yaw angle, roll angle); the positioning refresh rate reaches 100Hz; Supplementary description, 8 groups of binocular structured light cameras (horizontally spaced 45°, vertically 1.2 - 1.8m in height); user depth image (resolution 640×480, frame rate 120fps).

[0022] Viewport boundary division: For the effective area of stereoscopic display within the 3D interaction space in front of the large screen, it is divided into 20×15 grid cells according to the set division mechanism; Supplementary description, the specific division process of the grid cells, taking an example: Taking the center of the large screen as the origin, a right-handed coordinate system is established (the X-axis is horizontal to the right, the Y-axis is vertical upward, and the Z-axis is perpendicular to the large screen and forward), and the range of the interaction space is defined as , the space is divided into 20 equal parts along the X-axis and 15 equal parts along the Y-axis, forming 20×15 horizontal grid layers perpendicular to the Z-axis.

[0023] Real-time parallax compensation: Using the calibration logic of the beam splitting parameters to determine the optimal beam splitting parameters for each grid cell for different users, real-time identifying the grid cell corresponding to the current user's location, extracting the optimal beam splitting parameters for each grid cell corresponding to the current user, and performing dynamic adjustment; among them, the beam splitting parameters include the tilt angle of the lenticular grating and the focal length of the microlens; The specific calculation process of the calibration logic of the beam splitting parameters is as follows: For the center point of each grid cell, by rotating the lenticular grating; adjustable from 0° to 30°, with a step of 0.1°; collecting the left and right eye image crosstalk rates of different users k times at different tilt angles, and k > 12, the specific number of times is initially set by the technical staff; For the left and right eye image crosstalk rates of different users k times at different tilt angles, calculate the average value to determine the average crosstalk rate of different users at different tilt angles; Eliminate the tilt angles with an average crosstalk rate higher than the set reference rate, mark the remaining tilt angles as the selected tilt angles, and integrate them as the tilt angle data set of different users; Extract the average crosstalk rates of different users at different selected tilt angles from the tilt angle data set, calculate the difference between each and the set reference rate respectively, and take the absolute value as the crosstalk distance value tra of different selected tilt angles; Measure the degree to which the crosstalk rate of different selected tilt angles is lower than the set reference rate; Use the standard deviation formula to calculate the left and right eye image crosstalk rates of different selected tilt angles k times, and obtain the crosstalk discrete value trb corresponding to different selected tilt angles; Measure the stability of the crosstalk rate of different selected tilt angles; Identify from the left and right eye image crosstalk rates of different selected tilt angles k times, count the number higher than the set reference rate as the high crosstalk number, calculate the ratio of the high crosstalk number to k, and obtain the crosstalk deviation value trc corresponding to different selected tilt angles; Extract the crosstalk distance value tra, crosstalk discrete value trb, and crosstalk deviation value trc calculated for different users corresponding to different selected tilt angles, perform normalization processing and then substitute them into the formula Perform weighted calculation to obtain the crosstalk superiority value tq of different users corresponding to different screening tilt angles; are the weight coefficients of the crosstalk distance value tra, the crosstalk discrete value trb and the crosstalk deviation value trc respectively; For different users, the screening tilt angle with a higher crosstalk general merit value tq is selected as the optimal cylindrical mirror grating tilt angle of the corresponding grid unit, which is marked as ; i is the X-axis grid index, j is the Y-axis grid index; In addition, traditional methods usually screen parameters based on a single indicator, the average crosstalk rate, without considering the stability (such as the fluctuation range) and reliability (such as the frequency of high crosstalk) of the crosstalk rate. For example, a certain tilt angle may have a low average crosstalk rate, but occasionally high crosstalk (such as flickering or ghosting) may occur. A single indicator cannot identify such problems.

[0024] For the center point of each grid unit, the theoretical focal length is calculated according to the object distance-image distance formula, which is expressed as ; Where v is the distance from the display screen to the microlens, and u is the vertical distance from the user's eyes to the microlens in each grid unit; obtained through the depth sensor; Supplementary explanation: if If the focal length of the microlens is not within the adjustable range, the boundary value of the adjustable range is directly taken. For example, if the adjustable range of the focal length of the microlens is 2mm-5mm, if <2, then directly take 2mm, if >5, then directly take 5mm.

[0025] For different users, the calculated as the optimal microlens focal length for the corresponding grid cell; Spectroscopic parameter cache: Analyze the application data of the spectral parameter combination within the set time window and establish a cache queue based on the analysis results; Specifically: Extract the usage times of each spectral parameter combination from the usage data of the spectral parameter combination within the set time window; identify the usage time of each spectral parameter combination, and determine the records with usage time shorter than the set time as false triggers; the set time can be set to 50ms; eliminate the usage times of the spectral parameters that are falsely triggered; count the usage times of each spectral parameter combination after elimination within the set time window and record them as reference times; Sum the reference times of each spectral parameter combination to obtain the total times within the set time window; calculate the proportion of the reference times corresponding to each spectral parameter combination in the total times as the usage frequency proportion of each spectral parameter combination in the set time window; Extract the usage duration corresponding to each combination of spectral parameters for each reference count, sum up the usage durations of each group of each combination of spectral parameters, obtain the cumulative duration of each combination of spectral parameters within the set time window, and calculate the proportion of the cumulative duration of each combination of spectral parameters within the set time window as the usage duration proportion of each combination of spectral parameters within the set time window; Identify the number of users using each combination of spectral parameters within the set time window, calculate the proportion of the number of users using each combination of spectral parameters in the total number of users as the user concentration proportion of each combination of spectral parameters within the set time window; Multiply the usage frequency proportion, usage duration proportion, and user concentration proportion of each combination of spectral parameters by the preset frequency weight coefficient, duration weight coefficient, and concentration weight coefficient respectively, and then sum to obtain the priority index of each combination of spectral parameters within the set time window; Sort each combination of spectral parameters in descending order according to the priority index, and extract the first H combinations of spectral parameters from left to right in the sorting result to construct a cache queue; H is initially set to 10; Supplementary note, through duration filtering and multi-dimensional statistics, it is ensured that the cached data truly reflects the user needs within the current set time window. Capturing the temporal pattern of user behavior and pre-loading it into the cache can significantly improve the interaction delay between the user and the large screen.

[0026] Resource elastic allocation: Real-time evaluate the activity parameters of users, and adjust the GPU computing resources of users based on the evaluation results; where the activity parameters include the movement speed, interaction frequency, and fixation duration of users; Specifically: Calculate the Euclidean distance between adjacent time frames using the three-dimensional coordinates (X, Y, Z) of the user's head ; with an interval of 10 ms; the calculation formula is expressed as ; Calculate the average value of the Euclidean distances of each adjacent time frame of the user within the preset time interval as the movement speed of the user within the preset time interval; Count the number of times the user triggers valid interaction events within the preset time interval. The interaction events include gesture operations, voice commands, and eye movement selections; record it as the number of interactions; calculate the ratio of the number of interactions to the preset time interval to obtain the interaction frequency of the user within the preset time interval; Determine that the user's fixation area is the large screen through eye movement tracking. When the user's head posture is stable, that is, the pitch angle and yaw angle are lower than the set angles and the duration is higher than the set time, it is determined as a valid fixation; count the valid fixation duration of the user within the preset time interval; Convert the movement speed, interaction frequency, and valid fixation duration of the user within the preset time interval into speed activity scores, interaction activity scores, and fixation activity scores respectively; For the user's speed activity score, interaction activity score, and fixation activity score, they are respectively multiplied by the set speed weight coefficient, interaction weight coefficient, and fixation weight coefficient, and then summed to obtain the user's activity evaluation index; Supplementary note, high-active users and low-active users can be distinguished through the activity evaluation index. High-activity users (such as users who move quickly and interact frequently) can preferentially obtain more GPU resources to meet scenarios with extremely high real-time requirements, improving the utilization efficiency of hardware resources and the satisfaction of user interaction experience.

[0027] Construct a speed set, a frequency set, and a fixation set corresponding to the movement speed, interaction frequency, and effective fixation duration respectively; The speed set, frequency set, and fixation set respectively contain the value ranges of each group of speeds, the value ranges of each group of frequencies, and the value ranges of each group of durations; and each value range of speeds corresponds to a speed activity score; each value range of frequencies corresponds to an interaction activity score; each value range of durations corresponds to a fixation activity score; the ranges of the speed activity score, interaction activity score, and fixation activity score are all set between 1 and 10, and the higher the movement speed, interaction frequency, and effective fixation duration, the higher the corresponding speed activity score, interaction activity score, and fixation activity score; Input the user's movement speed, interaction frequency, and effective fixation duration into the speed set, frequency set, and fixation set respectively for matching the corresponding value ranges, so as to be converted into the corresponding speed activity score, interaction activity score, and fixation activity score; Supplementary note, the value ranges of each group of speeds, the value ranges of each group of frequencies, and the value ranges of each group of durations in the speed set, frequency set, and fixation set are initially set by technical personnel.

[0028] Obtain the current total GPU resources , and , where respectively represent the total number of threads and the total video memory; Allocate the total GPU resources based on the activity score index of each user, that is, through the allocation logic obtain the threads allocated to each user in the next preset time interval and video memory ; where represents the activity score index of each user, and i is the user number; is the sum of the activity score indexes of each user.

[0029] Embodiment 2

[0030] Please refer to Figure 2As shown in the figure, based on an augmented reality interaction method for a naked-eye 3D large screen provided in Embodiment 1 of the present application, Embodiment 2 of the present application proposes an augmented reality interaction system for a naked-eye 3D large screen. Embodiment 2 is merely a preferred mode of Embodiment 1, and the implementation of Embodiment 2 will not affect the independent implementation of Embodiment 1.

[0031] Specifically, the augmented reality interaction system for a naked-eye 3D large screen provided in Embodiment 2 of the present application is different in that it includes a data acquisition module, a viewing area division module, a parallax compensation module, a cache setting module, and a resource scheduling module; The data acquisition module is used to obtain a user visual image, a human body depth image, and an RGB image, and calculate six degrees of freedom parameters of the user's head, namely the three-dimensional coordinates (X, Y, Z) of the head and the head pose data; The viewing area division module is used to divide the stereoscopic display effective area in the 3D interaction space in front of the large screen into 20×15 grid units; The parallax compensation module is used to determine the optimal beam splitting parameters for different users for each grid unit by using the beam splitting parameter calibration logic, real-time identify the grid unit corresponding to the current user's location, extract the optimal beam splitting parameters for each grid unit corresponding to the current user, and perform dynamic adjustment; where the beam splitting parameters include the cylindrical lens grating tilt angle and the microlens focal length; The cache setting module is used to analyze the application data of the beam splitting parameter combination within a set time window, and establish a cache queue based on the analysis result; The resource scheduling module is used to evaluate the activity parameters of the user in real time, and adjust the GPU computing resources of the user based on the evaluation result; where the activity parameters include the user's movement speed, interaction frequency, and fixation duration; The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the present invention to only the specific implementation manners. Obviously, many modifications and variations can be made according to the content of this specification. These embodiments are selected and specifically described in this specification to better explain the principle and practical application of the present invention, so that those skilled in the art in the technical field can understand and utilize the present invention well. The present invention is only limited by the claims and their full scope and equivalents.

Claims

1. A virtual-real interaction method for a naked-eye 3D large screen, characterized in that, Including: User location tracking: Use a camera array pre - deployed around the large screen to collect user visual images in real - time, synchronously activate the TOF depth sensor to obtain the human body depth image and RGB image, and calculate the six - degree - of - freedom parameters of the user's head, that is, the three - dimensional coordinates of the head (X, Y, Z) and the head pose data; the head pose data includes pitch angle, yaw angle, and roll angle. Viewport boundary division: For the three - dimensional display effective area within the 3D interaction space in front of the large screen, divide it into 20×15 grid cells according to the set division mechanism. Real - time parallax compensation: Use the spectral splitting parameter calibration logic to determine the optimal spectral splitting parameters for each grid cell for different users, identify the grid cell corresponding to the current user's location in real - time, extract the optimal spectral splitting parameters for each grid cell corresponding to the current user, and perform dynamic adjustment; where the spectral splitting parameters include the cylindrical lens grating tilt angle and the microlens focal length. Spectral splitting parameter caching: Analyze the application data of the spectral splitting parameter combinations within the set time window, and establish a cache queue based on the analysis results.

2. The virtual-real interaction method of a naked-eye 3D large screen according to claim 1, characterized in that, The specific set division mechanism is as follows: Taking the center of the large screen as the origin, establish a right - hand coordinate system, that is, the X - axis is horizontal to the right, the Y - axis is vertical upward, and the Z - axis is perpendicular to the large screen and forward. Divide the space into 20 equal parts along the X - axis and 15 equal parts along the Y - axis, forming 20×15 horizontal grid layers perpendicular to the Z - axis.

3. A method for virtual-real interaction of a naked-eye 3D large screen according to claim 2, characterized in that, The method of using the spectral splitting parameter calibration logic to determine the optimal cylindrical lens grating tilt angle for each grid cell for different users is specifically as follows: For the center point of each grid cell, rotate the cylindrical lens grating; collect the left - and right - eye image crosstalk rates of different users k times at different tilt angles. Calculate the average value of the left - and right - eye image crosstalk rates of different users k times at different tilt angles to determine the average crosstalk rate of different users at different tilt angles. Eliminate the tilt angles with an average crosstalk rate higher than the set reference rate, mark the remaining tilt angles as the selected tilt angles, and integrate them as the tilt angle data set for different users. Extract the average crosstalk rates of different users at different selected tilt angles from the tilt angle data set, calculate the differences from the set reference rate respectively, and take the absolute values as the crosstalk distance values tra of different selected tilt angles; use the standard deviation formula to calculate the crosstalk discrete values trb corresponding to different selected tilt angles for the left - and right - eye image crosstalk rates k times. Identify from the left - and right - eye image crosstalk rates k times of different selected tilt angles, count the number higher than the set reference rate as the high - crosstalk number, and calculate the ratio of the high - crosstalk number to k to obtain the crosstalk deviation value trc corresponding to different selected tilt angles. Extract the crosstalk distance values tra, crosstalk discrete values trb, and crosstalk deviation values trc calculated for different users corresponding to different screening tilt angles. After normalization, substitute them into the formula Perform weighted calculation to obtain the crosstalk preference value tq for different users corresponding to different screening tilt angles; where are the weight coefficients of the crosstalk distance value tra, the crosstalk discrete value trb, and the crosstalk deviation value trc respectively; For different users, a screening tilt angle with a relatively high crosstalk merit value tq is selected as the best cylindrical lens grating tilt angle for the corresponding grid unit, denoted as ; i is the grid index on the X-axis and j is the grid index on the Y-axis.

4. A virtual-real interaction method for a naked-eye 3D large screen according to claim 3, characterized in that, The method of using the spectral splitting parameter calibration logic to determine the optimal microlens focal length for each grid cell for different users is specifically as follows: For the center point of each grid cell, calculate the theoretical focal length according to the object distance-image distance formula, which is expressed as ; where v is the distance from the display screen to the microlens, and u is the vertical distance from the user's eye to the microlens in each grid cell; for different users, the calculated is used as the optimal microlens focal length for the corresponding grid cell.

5. A method for virtual-real interaction of a naked-eye 3D large screen according to claim 4, characterized in that, The specific steps for establishing the cache queue are as follows: Extract the usage times of each spectral splitting parameter combination from the usage data of the spectral splitting parameter combinations within the set time window; identify the usage duration of each spectral splitting parameter combination, and determine the records with a usage duration lower than the set duration as mis - triggers; eliminate the usage times of mis - triggered spectral splitting parameters; count the usage times of each spectral splitting parameter combination after elimination within the set time window, and record it as the reference number. Sum the reference times of each spectral parameter combination to obtain the total times within the set time window; calculate the proportion of the reference times corresponding to each spectral parameter combination in the total times as the usage frequency proportion of each spectral parameter combination in the set time window; Extract the usage time of each reference number of each spectral parameter combination, sum the usage time of each group of each spectral parameter combination, obtain the cumulative time of each spectral parameter combination in the set time window, calculate the proportion of the cumulative time of each spectral parameter combination in the set time window, and use it as the usage time proportion of each spectral parameter combination in the set time window; Identify the number of users using each spectral parameter combination within a set time window, and calculate the proportion of the number of users using each spectral parameter combination to the total number of users as the user concentration proportion of each spectral parameter combination within the set time window; The usage frequency ratio, usage duration ratio and user concentration ratio of each spectral parameter combination are multiplied by the preset frequency weight coefficient, duration weight coefficient and concentration weight coefficient respectively, and then the sum is calculated to obtain the priority index of each spectral parameter combination within the set time window; The optical splitting parameter combinations are sorted from large to small according to the priority index, and the first H optical splitting parameter combinations are extracted from left to right from the sorting result to build a cache queue.

6. A virtual-real interaction method for a naked-eye 3D large screen according to claim 5, characterized in that Also includes: Flexible resource allocation: Evaluate user activity parameters in real time and adjust the user's GPU computing resources based on the evaluation results; The activity parameters include the user's movement speed, interaction frequency, and gaze duration.

7. A virtual-real interaction method for a naked-eye 3D large screen according to claim 6, characterized in that, The specific process of real-time evaluation of user activity parameters is as follows: Calculate the Euclidean distance between adjacent time frames using the three-dimensional coordinates (X, Y, Z) of the user's head ; The calculation formula is expressed as ; Calculate the average value of the Euclidean distances of adjacent time frames of the user within the preset time interval as the movement speed of the user within the preset time interval; Count the number of times the user triggers a valid interaction event within a preset time interval; record it as the number of interactions; calculate the ratio of the number of interactions to the preset time interval to obtain the user's interaction frequency within the preset time interval; Determine that the user's gaze area is the large screen. When the user's head posture is stable, that is, the pitch angle and yaw angle are lower than the set angle, and the duration is longer than the set time, it is determined to be effective gaze; count the user's effective gaze time within the preset time interval; Convert the user's movement speed, interaction frequency and effective gaze duration within a preset time interval into a speed activity score, an interaction activity score and a gaze activity score respectively; The user's speed activity score, interaction activity score, and gaze activity score are multiplied by the set speed weight coefficient, interaction weight coefficient, and gaze weight coefficient respectively, and then the sum is calculated to obtain the user's activity evaluation index.

8. A virtual-real interaction method for a naked-eye 3D large screen according to claim 7, characterized in that The user's movement speed, interaction frequency and effective gaze duration within a preset time interval are converted into a speed activity score, an interaction activity score and a gaze activity score, respectively, as follows: Construct the speed set, frequency set and fixation set corresponding to the movement speed, interaction frequency and effective fixation duration respectively; The speed set, the frequency set, and the fixation set respectively contain the value ranges of each group of speeds, the value ranges of each group of frequencies, and the value ranges of each group of durations; and each value range of speeds corresponds to a set of speed activity scores; each value range of frequencies corresponds to a set of interaction activity scores; each value range of durations corresponds to a set of fixation activity scores; The movement speed, interaction frequency, and effective fixation duration of the user are respectively input into the speed set, the frequency set, and the fixation set to match the corresponding value ranges, so as to be converted into the corresponding speed activity scores, interaction activity scores, and fixation activity scores.

9. A virtual-real interaction method for a naked-eye 3D large screen according to claim 8, characterized in that Adjust the GPU computing resources of the user based on the evaluation results, specifically: Obtain the total resources of the current GPU and where represent the total number of threads and the total video memory respectively; Allocate the total GPU resources based on the active scoring index of each user, that is, through the allocation logic Obtain the threads allocated to each user in the next preset time interval And video memory ; Among them Represents the active scoring index of each user, and i is the user number; Is the sum of the active scoring indices of each user.

10. A virtual-real interaction system for a naked-eye 3D large screen, which is applied to the virtual-real interaction method for a naked-eye 3D large screen described in any one of the above claims 1-9, and is characterized in that, Including: Data acquisition module: Obtain the user's visual image, human depth image, and RGB image, and calculate the six-degree-of-freedom parameters of the user's head, that is, the three-dimensional coordinates of the head (X, Y, Z) and the head pose data; View area division module: Divide the stereoscopic display effective area in the 3D interaction space in front of the large screen into 20×15 grid units; Parallax compensation module: Use the calibration logic of the beam splitting parameters to determine the optimal beam splitting parameters for each grid unit for different users, identify the grid unit corresponding to the current user's location in real time, extract the optimal beam splitting parameters for each grid unit corresponding to the current user, and perform dynamic adjustment; where the beam splitting parameters include the cylindrical lens grating tilt angle and the microlens focal length; Cache setting module: Analyze the application data of the beam splitting parameter combinations within the set time window, and establish a cache queue based on the analysis results; Resource scheduling module: Evaluate the activity parameters of the user in real time, and adjust the GPU computing resources of the user based on the evaluation results; Among them, the activity parameters include the movement speed, interaction frequency, and fixation duration of the user.

Images