VR multi-terminal collaborative interaction method, system, computer device and storage medium

Abstract

The present invention is applicable to the field of VR interactive technology, and provides a VR multi-terminal collaborative interaction method, system, computer equipment and storage medium, including: displaying VR scene images through a VR holographic imaging screen, when the VR scene image appears an interactive scene after a set time value, displaying several interactive selection pop-up windows linearly in the lower edge area; receiving interactive commands input by several interactive control terminals, and installing interactive control terminals on the front seats of the VR holographic theater, and each interactive selection pop-up window corresponds to a front seat; fusing the interactive commands to obtain fusion data, and determining the plot scene of the corresponding interactive scene according to the fusion data; when the VR scene image is played and displayed, generating a VR program list selection pop-up window, receiving the VR program selection instruction input by the interactive control terminal, and determining the next VR program to be played. In this way, more connections are generated between the plot and the audience, and the audience's sense of participation will be stronger.

Description

Technical Field

[0001] The present invention relates to the field of VR interaction technology, and in particular to a VR multi-terminal collaborative interaction method, system, computer device and storage medium. Background Art

[0002] VR holographic theater is an innovative theater form that combines holographic projection technology with virtual reality technology. It uses advanced projection equipment, lighting, sound and other elements, combined with virtual reality technology, to project three-dimensional images onto the screen in a realistic way, bringing an immersive viewing experience to the audience. In the VR holographic theater, the audience can experience the scenes and plots in the film in an immersive way, as if they were in the world of the movie. This immersive viewing experience greatly enhances the audience's sense of participation and substitution, allowing people to truly feel the emotions and themes that the movie wants to express. However, at present, when watching performances in VR holographic theaters, the audience cannot interact in depth, and the audience's sense of participation and substitution needs to be strengthened. Therefore, it is necessary to provide a VR multi-terminal collaborative interaction method, system, computer equipment and storage medium to solve the above problems. Summary of the invention

[0003] In view of the deficiencies in the prior art, the purpose of the present invention is to provide a VR multi-terminal collaborative interaction method, system, computer device and storage medium to solve the problems existing in the above-mentioned background technology.

[0004] The present invention is implemented as follows: a VR multi-terminal collaborative interaction method, the method comprising the following steps:

[0005] The VR scene image is displayed through the VR holographic imaging screen. When the VR scene image shows an interactive scene after the set time value, several interactive selection pop-up windows are displayed linearly in the lower edge area;

[0006] Receive interactive commands input by several interactive control terminals, where the interactive control terminals are installed on the front seats of the VR holographic theater, and each interactive selection pop-up window corresponds to a front seat;

[0007] A plurality of interactive commands are fused to obtain fused data, and a plot scene of a corresponding interactive scenario is determined according to the fused data;

[0008] When a VR scene image is played and displayed, a VR program list selection pop-up window is generated to receive VR program selection instructions input by several interactive control terminals, and the next VR program to be played and displayed is determined according to the VR program selection instructions.

[0009] As a further solution of the present invention: the step of receiving the interactive commands input by a plurality of interactive control terminals specifically includes:

[0010] receiving interactive commands inputted by a plurality of interactive control terminals before the interactive selection pop-up window disappears, wherein the interactive selection pop-up window is provided with a display duration;

[0011] Determine an interactive selection pop-up window corresponding to each interactive command, wherein the interactive control terminal corresponds to the interactive selection pop-up window one by one;

[0012] Before the interactive selection pop-up window disappears, if no corresponding interactive command is received, the content in the interactive selection pop-up window is automatically selected.

[0013] As a further solution of the present invention: the step of fusing multiple interactive commands to obtain fused data specifically includes:

[0014] Determine an interactive selection content according to an interactive command and a corresponding interactive selection pop-up window;

[0015] All interactive selection contents are fused to obtain fused data.

[0016] As a further solution of the present invention: the step of determining the plot scene of the corresponding interactive scenario according to the fusion data specifically includes:

[0017] Retrieving a plot scene trend library corresponding to the interactive scenario, wherein the interactive scenario corresponds to the plot scene trend library one by one, and the plot scene trend library is established in advance by the staff;

[0018] Inputting the fused data into the plot scene trend library, the plot scene trend library includes a plurality of selection information sets, each selection information set corresponds to a plot scene;

[0019] The fused data is matched with the selection information set, and the plot scene corresponding to the matched selection information set is output.

[0020] As a further solution of the present invention: the step of receiving VR program selection instructions input by a plurality of interactive control terminals and determining the next VR program to be played and displayed according to the VR program selection instructions specifically includes:

[0021] receiving VR program selection instructions input by a plurality of interactive control terminals before a program selection deadline;

[0022] The VR program corresponding to each VR program selection instruction is determined, and the program selected the most times is determined as the next VR program to be played and displayed.

[0023] Another object of the present invention is to provide a VR multi-terminal collaborative interaction system, the system comprising:

[0024] An interactive selection pop-up window module is used to display a VR scene image through a VR holographic imaging screen. When an interactive scene appears in the VR scene image after a set time value, a number of interactive selection pop-up windows are linearly displayed in the lower edge area;

[0025] An interactive command receiving module is used to receive interactive commands input by a number of interactive control terminals. Interactive control terminals are installed on the front seats of the VR holographic theater, and each interactive selection pop-up window corresponds to a front seat;

[0026] A plot scene determination module is used to fuse multiple interactive commands to obtain fused data, and determine the plot scene of the corresponding interactive scenario according to the fused data;

[0027] The VR program interactive selection module is used to generate a VR program list selection pop-up window after a VR scene image is played and displayed, receive VR program selection instructions input by a number of interactive control terminals, and determine the next VR program to be played and displayed according to the VR program selection instructions.

[0028] As a further solution of the present invention: the interactive command receiving module includes:

[0029] An interactive command receiving unit, used to receive interactive commands input by a plurality of interactive control terminals before the interactive selection pop-up window disappears, wherein the interactive selection pop-up window is set with a display duration;

[0030] A selection pop-up window corresponding unit is used to determine an interactive selection pop-up window corresponding to each interactive command, and the interactive control terminal corresponds to the interactive selection pop-up window one by one;

[0031] The content automatic selection unit is used to automatically select the content in the interactive selection pop-up window when no corresponding interactive command is received before the interactive selection pop-up window disappears.

[0032] As a further solution of the present invention: the plot scene determination module includes:

[0033] A scene trend library retrieval unit is used to retrieve a plot scene trend library corresponding to the interactive scenario, wherein the interactive scenario corresponds to the plot scene trend library one by one, and the plot scene trend library is established in advance by the staff;

[0034] A fusion data input unit, used for inputting the fusion data into the plot scene trend library, the plot scene trend library includes a plurality of selection information sets, each selection information set corresponds to a plot scene;

[0035] The plot scene output unit is used to match the fused data with the selection information set and output the plot scene corresponding to the matched selection information set.

[0036] The present invention also aims to provide a computer device, comprising a memory, a processor, and a computer device program stored in the memory and executable on the processor, wherein the processor implements the VR multi-terminal collaborative interaction method when executing the computer device program.

[0037] The present invention also aims to provide a storage medium, which includes a stored program, and when the program is running, controls the device where the storage medium is located to execute the VR multi-terminal collaborative interaction method.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention sets an interactive scenario. When an interactive scenario appears in the VR scene image after a set time value, several interactive selection pop-up windows are displayed linearly in the lower edge area; interactive commands input by several interactive control terminals are received, and interactive control terminals are installed on the front seats of the VR holographic theater; multiple interactive commands are fused to obtain fused data, and the plot scene of the corresponding interactive scenario is determined according to the fused data. In this way, more connections are generated between the plot and the audience, the audience's sense of participation will be stronger, and the interactivity will be better. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 The figure is a flow chart of a VR multi-terminal collaborative interaction method.

[0041] Figure 2 The present invention is a flow chart of receiving interaction commands input by a plurality of interaction control terminals in a VR multi-terminal collaborative interaction method.

[0042] Figure 3 The present invention is a flow chart of fusing multiple interactive commands to obtain fused data in a VR multi-terminal collaborative interaction method.

[0043] Figure 4 The present invention is a flowchart of determining a plot scenario of a corresponding interaction scenario based on fusion data in a VR multi-terminal collaborative interaction method.

[0044] Figure 5 The present invention is a flow chart of determining the next VR program to be played and displayed according to a VR program selection instruction in a VR multi-terminal collaborative interaction method.

[0045] Figure 6 This is a structural diagram of a VR multi-terminal collaborative interaction system. DETAILED DESCRIPTION

[0046] In order to make the purpose, technical solution and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0047] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0048] like Figure 1 As shown, an embodiment of the present invention provides a VR multi-terminal collaborative interaction method, the method comprising the following steps:

[0049] S100, displaying a VR scene image through a VR holographic imaging screen, and when an interactive scene appears in the VR scene image after a set time value, displaying a plurality of interactive selection pop-up windows linearly in the lower edge area;

[0050] S200, receiving interactive commands input by a plurality of interactive control terminals, where the interactive control terminals are installed on the front seats of the VR holographic theater, and each interactive selection pop-up window corresponds to a front seat;

[0051] S300, fusing multiple interactive commands to obtain fused data, and determining a plot scene of a corresponding interactive scenario according to the fused data;

[0052] S400, when a VR scene image is played and displayed, a VR program list selection pop-up window is generated, VR program selection instructions input by a plurality of interactive control terminals are received, and the next VR program to be played and displayed is determined according to the VR program selection instructions.

[0053] It should be noted that in a VR holographic theater, the audience can experience the scenes and plots in the film as if they were in the world of the film. This immersive viewing experience greatly enhances the audience's sense of participation and substitution, allowing people to truly feel the emotions and themes that the film wants to express. However, at present, when the audience watches a performance in a VR holographic theater, they cannot interact in depth, and the audience's sense of participation and substitution needs to be strengthened. The embodiments of the present invention are intended to solve the above problems.

[0054] In the embodiment of the present invention, the VR scene image will first be displayed through the VR holographic imaging screen. In order to improve the audience's sense of participation and interactivity, an interactive scenario is designed in the VR scene image. When the interactive scenario appears in the VR scene image after the set time value, for example, the interactive scenario will appear after 3 minutes. At this time, several interactive selection pop-ups will be displayed linearly in the lower edge area, that is, a row of interactive selection pop-ups will be displayed at the bottom. The contents of the interactive selection pop-ups can be different. An interactive control terminal is installed on the front row seats of the VR holographic theater. Each interactive selection pop-up window corresponds to a front row seat. When customers buy tickets to watch a movie, they need to be reminded that the front row seats are exclusive interactive seats. In this way, the front row audience can input interactive commands through the interactive control terminal. The embodiment of the present invention will automatically fuse multiple interactive commands to obtain fused data, and determine the plot scene of the corresponding interactive scenario based on the fused data. In this way, more connections are generated between the plot and the audience, and the audience's sense of participation and substitution will be stronger. In addition, when a VR scene image is finished playing and displayed, a VR program list selection pop-up window will be generated. The front-row audience can also input VR program selection instructions through the interactive control terminal, and finally determine the next VR program to be played and displayed based on the VR program selection instructions. This way, the interactivity is better.

[0055] like Figure 2 As shown, as a preferred embodiment of the present invention, the step of receiving the interactive commands input by a plurality of interactive control terminals specifically includes:

[0056] S201, receiving interaction commands inputted by a plurality of interaction control terminals before an interaction selection pop-up window disappears, wherein the interaction selection pop-up window is set with a display duration;

[0057] The process of receiving the interactive commands input by the interactive control terminals before the interactive selection pop-up window disappears includes the following steps:

[0058] Based on the reaction time data of users to interactive commands in different situations, the user reaction time distribution function of each interactive command is obtained through statistical analysis;

[0059] Assign a weight value to each interactive command and set the time step of the pop-up window display;

[0060] Combined with the user reaction time distribution function and weight value of each interactive command, the comprehensive probability of the user successfully receiving the interactive command within the time step of the pop-up window display is calculated by cumulative calculation;

[0061] The received interactive command is determined using the comprehensive probability.

[0062] S202, determining an interactive selection pop-up window corresponding to each interactive command, wherein the interactive control terminal corresponds to the interactive selection pop-up window one-to-one;

[0063] S203: before the interactive selection pop-up window disappears, if no corresponding interactive command is received, the content in the interactive selection pop-up window is automatically selected.

[0064] In the embodiment of the present invention, it is easy to understand that in order to ensure the smooth projection of the VR scene image, it is necessary to receive several interactive commands input by the interactive control terminals before the interactive selection pop-up window disappears. The interactive selection pop-up window is provided with a display duration. For example, the display time of the interactive selection pop-up window in the embodiment of the present invention is 2 minutes; then the interactive selection pop-up window corresponding to each interactive command will be determined automatically, and the interactive control terminal corresponds to the interactive selection pop-up window one by one; when the corresponding interactive command is not received before the interactive selection pop-up window disappears, it means that the corresponding front-row audience has not made a selection. At this time, the content in the interactive selection pop-up window will be automatically selected to ensure the smoothness of the subsequent process.

[0065] like Figure 3 As shown, as a preferred embodiment of the present invention, the step of fusing multiple interactive commands to obtain fused data specifically includes:

[0066] S301, determining an interactive selection content according to an interactive command and a corresponding interactive selection pop-up window;

[0067] S302, fusing all interactive selection contents to obtain fused data;

[0068] In step S302, fusing all interactive selection contents to obtain fused data specifically includes the following steps:

[0069] Get all interactive commands;

[0070] Convert each interactive command into a query vector, a key vector, and a value vector;

[0071] The dot product is calculated for each query vector and key vector, and then divided by the square root of the dimension of the key vector for scaling and then calculated by the Softmax function to obtain the attention weight;

[0072] Based on the attention weight, the value vector is weighted summed to obtain the fused interaction command data;

[0073] The fused interactive command data is input into a deep neural network, and nonlinear transformation is performed on the fused interactive command data to obtain fused data;

[0074] Specifically, this complication process captures the complex relationships between interactive commands through a multi-head attention mechanism, and uses deep neural networks to further extract and fuse useful information, thereby improving the accuracy and flexibility of data fusion.

[0075] In an embodiment of the present invention, each interactive command corresponds to an interactive selection pop-up window, and an interactive selection content can be determined based on an interactive command and the corresponding interactive selection pop-up window. Each interactive selection pop-up window contains two or more interactive selection contents for the audience to choose from. Finally, all the interactive selection contents are fused to obtain fused data, and the fused data contains several selected interactive selection contents.

[0076] like Figure 4 As shown, as a preferred embodiment of the present invention, the step of determining the plot scene of the corresponding interactive scenario according to the fusion data specifically includes:

[0077] S303, calling up a plot scene trend library corresponding to the interactive scenario, wherein the interactive scenario corresponds to the plot scene trend library one by one, and the plot scene trend library is established in advance by the staff;

[0078] S304, inputting the fused data into the plot scene trend library, the plot scene trend library includes a plurality of selection information sets, each selection information set corresponds to a plot scene;

[0079] S305, matching the fused data with the selection information set, and outputting the plot scene corresponding to the matched selection information set;

[0080] In step S305, matching the fused data with the selection information set and outputting the plot scene corresponding to the matched selection information set specifically comprises the following steps:

[0081] Introduce latent variables and use Gaussian distribution to set the initial distribution of latent variables;

[0082] The fused data and the selected information set are used as input to the initialized deep neural network to generate the sample distribution of the latent variables;

[0083] Use KL divergence as a metric to calculate the difference between the variational distribution and the true posterior distribution;

[0084] The back-propagation algorithm is used to minimize the KL divergence and optimize the parameters of the deep neural network; this step is used to make the generated variational distribution as close as possible to the true posterior distribution.

[0085] Sampling multiple latent variables from the variational distribution, calculating the probability of the plot scene under the given fusion data and selection information set for each latent variable sample, and calculating the expected value of the latent variable by accumulating all the plot scene probabilities and taking the average;

[0086] The fused data, the selected information set, and the expected value of the latent variable are input into the deep conditional random field model, and for each predefined scenario, the probability under the current input conditions is calculated;

[0087] Sort all probabilities and output the plot scenario with the highest probability.

[0088] Specifically, through this step, complex plot scene matching can be achieved, making full use of fused data and selection information sets to improve the system's decision accuracy and scene matching flexibility.

[0089] In the embodiment of the present invention, each interactive scenario corresponds to a plot scene trend library, and the plot scene trend library is established in advance by the staff. The plot scene trend library contains a number of selection information sets, and each selection information set corresponds to a plot scene. In order to determine the plot scene selected by the audience, it is first necessary to call the plot scene trend library corresponding to the interactive scenario that appears after the set time value, and then input the fused data into the plot scene trend library, and then match the fused data with the selection information set, and output the plot scene corresponding to the matched selection information set, which is what the audience needs.

[0090] like Figure 5 As shown, as a preferred embodiment of the present invention, the step of receiving VR program selection instructions input by a plurality of interactive control terminals and determining the next VR program to be played and displayed according to the VR program selection instructions specifically includes:

[0091] S401, receiving VR program selection instructions input by a plurality of interactive control terminals before a program selection deadline;

[0092] S402, determining the VR program corresponding to each VR program selection instruction, and determining the program with the most selection times as the next VR program to be played and displayed;

[0093] In step S402, determining the VR program corresponding to each VR program selection instruction, and determining the program with the most selection times as the next VR program to be played and displayed specifically includes the following steps:

[0094] Collect a historical data set containing the number of times each VR program was selected and played as a training data set;

[0095] Use the training data set to initialize the Gaussian process regression model and obtain a trained Gaussian process regression model;

[0096] Using the trained Gaussian process regression model, input new sample data to calculate the posterior distribution of the number of selections for each program, and sample the posterior distribution of each program to obtain the number of selections under different circumstances;

[0097] Define the membership function of the fuzzy inference system and set the parameters of the fuzzy inference system;

[0098] By using Bayesian optimization of the posterior distribution and performing expected calculation, the expected value of the number of selections of each program under different possible situations is obtained, and the expected value is mapped to the membership quantification range to obtain the membership degree of the expected number of selections of each program.

[0099] The VR programs are sorted from high to low according to the membership of the expected number of selections, and the program with the highest membership is selected as the next VR program to be played and displayed.

[0100] Specifically, by utilizing Bayesian optimization and fuzzy inference systems, the most popular programs can be effectively selected as the next VR program to be played and displayed, thereby improving user experience and satisfaction.

[0101] In the embodiment of the present invention, the display duration of the VR program list selection pop-up window is also fixed, and the disappearance time of the VR program list selection pop-up window is the program selection deadline. Therefore, it is necessary to receive VR program selection instructions input by a plurality of interactive control terminals before the program selection deadline, and then automatically determine the VR program corresponding to each VR program selection instruction, and further determine that the program with the most selection times is the next VR program to be played and displayed.

[0102] like Figure 6 As shown, an embodiment of the present invention further provides a VR multi-terminal collaborative interaction system, the system comprising:

[0103] The interactive selection pop-up window module 100 is used to display the VR scene image through the VR holographic imaging screen. When the VR scene image shows an interactive scene after a set time value, a plurality of interactive selection pop-up windows are linearly displayed in the lower edge area;

[0104] The interactive command receiving module 200 is used to receive interactive commands input by a plurality of interactive control terminals. The interactive control terminals are installed on the front seats of the VR holographic theater, and each interactive selection pop-up window corresponds to a front seat;

[0105] The plot scene determination module 300 is used to fuse multiple interactive commands to obtain fused data, and determine the plot scene of the corresponding interactive scenario according to the fused data;

[0106] The VR program interactive selection module 400 is used to generate a VR program list selection pop-up window after a VR scene image is played and displayed, receive VR program selection instructions input by a plurality of interactive control terminals, and determine the next VR program to be played and displayed according to the VR program selection instructions.

[0107] In the embodiment of the present invention, the VR scene image will be displayed through the VR holographic imaging screen. In order to improve the audience's sense of participation and interactivity, an interactive scenario is designed in the VR scene image. When the interactive scenario appears in the VR scene image after the set time value, for example, the interactive scenario will appear after 3 minutes. At this time, several interactive selection pop-ups will be displayed linearly in the lower edge area, that is, a row of interactive selection pop-ups will be displayed at the bottom. The contents of the interactive selection pop-ups can be different. An interactive control terminal is installed on the front row seats of the VR holographic theater. Each interactive selection pop-up window corresponds to a front row seat. When customers buy tickets to watch a movie, they need to be reminded that the front row seats are exclusive interactive seats. In this way, the front row audience can input interactive commands through the interactive control terminal. The embodiment of the present invention will automatically fuse multiple interactive commands to obtain fused data, and determine the plot scene of the corresponding interactive scenario based on the fused data. In this way, more connections are generated between the plot and the audience, and the audience's sense of participation and substitution will be stronger. In addition, when a VR scene image is finished playing and displayed, a VR program list selection pop-up window will be generated. The front-row audience can also input VR program selection instructions through the interactive control terminal, and finally determine the next VR program to be played and displayed based on the VR program selection instructions, which is more interactive.

[0108] As a preferred embodiment of the present invention, the interactive command receiving module 200 includes:

[0109] An interactive command receiving unit, used to receive interactive commands input by a plurality of interactive control terminals before the interactive selection pop-up window disappears, wherein the interactive selection pop-up window is set with a display duration;

[0110] A selection pop-up window corresponding unit is used to determine an interactive selection pop-up window corresponding to each interactive command, and the interactive control terminal corresponds to the interactive selection pop-up window one by one;

[0111] The content automatic selection unit is used to automatically select the content in the interactive selection pop-up window when no corresponding interactive command is received before the interactive selection pop-up window disappears.

[0112] As a preferred embodiment of the present invention, the plot scene determination module 300 includes:

[0113] A scene trend library retrieval unit is used to retrieve a plot scene trend library corresponding to the interactive scenario, wherein the interactive scenario corresponds to the plot scene trend library one by one, and the plot scene trend library is established in advance by the staff;

[0114] A fusion data input unit, used for inputting the fusion data into the plot scene trend library, the plot scene trend library includes a plurality of selection information sets, each selection information set corresponds to a plot scene;

[0115] The plot scene output unit is used to match the fused data with the selection information set and output the plot scene corresponding to the matched selection information set.

[0116] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer device program stored in the memory and executable on the processor, wherein the processor implements the VR multi-terminal collaborative interaction method when executing the computer device program.

[0117] An embodiment of the present invention further provides a storage medium, which includes a stored program, and when the program is running, controls the device where the storage medium is located to execute the VR multi-terminal collaborative interaction method.

[0118] The above only describes in detail the preferred embodiments of the present invention, which is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0119] It should be understood that, although each step in the flow chart of each embodiment of the present invention is shown in sequence according to the indication of the arrow, these steps are not necessarily performed in sequence according to the order indicated by the arrow. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be performed in other orders. Moreover, at least a portion of the steps in each embodiment may include a plurality of sub-steps or a plurality of stages, and these sub-steps or stages are not necessarily performed at the same time, but can be performed at different times, and the execution order of these sub-steps or stages is not necessarily performed in sequence, but can be performed in turn or alternately with at least a portion of other steps or sub-steps or stages of other steps.

[0120] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the program can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0121] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the disclosure in the specification and examples. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the claims.

Claims

1. A VR multi-terminal collaborative interaction method, characterized in that: The method comprises the following steps: The VR scene image is displayed through the VR holographic imaging screen. When the VR scene image shows an interactive scene after the set time value, several interactive selection pop-up windows are displayed linearly in the lower edge area; Receive interactive commands input by several interactive control terminals, where the interactive control terminals are installed on the front seats of the VR holographic theater, and each interactive selection pop-up window corresponds to a front seat; A plurality of interactive commands are fused to obtain fused data, and a plot scene of a corresponding interactive scenario is determined according to the fused data; When a VR scene image is played and displayed, a VR program list selection pop-up window is generated to receive VR program selection instructions input by a number of interactive control terminals, and the next VR program to be played and displayed is determined according to the VR program selection instructions; The step of receiving the interactive commands input by the plurality of interactive control terminals specifically includes: receiving interactive commands inputted by a plurality of interactive control terminals before the interactive selection pop-up window disappears, wherein the interactive selection pop-up window is provided with a display duration; Determine an interactive selection pop-up window corresponding to each interactive command, wherein the interactive control terminal corresponds to the interactive selection pop-up window one by one; Before the interactive selection pop-up window disappears, if no corresponding interactive command is received, the content in the interactive selection pop-up window is automatically selected; The process of receiving the interactive commands input by the interactive control terminals before the interactive selection pop-up window disappears includes the following steps: Based on the reaction time data of users to interactive commands in different situations, the user reaction time distribution function of each interactive command is obtained through statistical analysis; Assign a weight value to each interactive command and set the time step of the pop-up window display; Combined with the user reaction time distribution function and weight value of each interactive command, the comprehensive probability of the user successfully receiving the interactive command within the time step of the pop-up window display is calculated by cumulative calculation; Determine the received interactive command using the comprehensive probability; The step of fusing the multiple interactive commands to obtain fused data specifically includes: Determine an interactive selection content according to an interactive command and a corresponding interactive selection pop-up window; All interactive selection contents are fused to obtain fused data; In the process of determining an interactive selection content according to an interactive command and a corresponding interactive selection pop-up window, and fusing all the interactive selection contents to obtain fused data, the following steps are included: Get all interactive commands; Convert each interactive command into a query vector, a key vector, and a value vector; The dot product is calculated for each query vector and key vector, and then divided by the square root of the dimension of the key vector for scaling, and then calculated by the Softmax function to obtain the attention weight; Based on the attention weight, the value vector is weighted summed to obtain the fused interaction command data; The fused interactive command data is input into a deep neural network, and nonlinear transformation is performed on the fused interactive command data to obtain fused data; The step of determining the plot scene of the corresponding interactive scenario according to the fusion data specifically includes: Retrieving a plot scene trend library corresponding to the interactive scenario, wherein the interactive scenario corresponds to the plot scene trend library one by one, and the plot scene trend library is established in advance by the staff; Inputting the fused data into the plot scene trend library, the plot scene trend library includes a plurality of selection information sets, each selection information set corresponds to a plot scene; Matching the fused data with the selection information set, and outputting the plot scene corresponding to the matched selection information set; The process of matching the fused data with the selection information set and outputting the plot scene corresponding to the matched selection information set includes the following steps: Introduce latent variables and use Gaussian distribution to set the initial distribution of latent variables; The fused data and the selected information set are used as input to the initialized deep neural network to generate the sample distribution of the latent variables; Use KL divergence as a metric to calculate the difference between the variational distribution and the true posterior distribution; Optimize the parameters of deep neural networks by minimizing KL divergence through the back-propagation algorithm; Sampling multiple latent variables from the variational distribution, calculating the probability of the plot scene under the given fusion data and selection information set for each latent variable sample, and calculating the expected value of the latent variable by accumulating all the plot scene probabilities and taking the average; The fused data, the selected information set, and the expected value of the latent variable are input into the deep conditional random field model, and for each predefined scenario, the probability under the current input conditions is calculated; Sort all probabilities and output the plot scenario with the highest probability; The step of receiving VR program selection instructions input by a plurality of interactive control terminals and determining the next VR program to be played and displayed according to the VR program selection instructions specifically includes: receiving VR program selection instructions input by a plurality of interactive control terminals before a program selection deadline; Determine the VR program corresponding to each VR program selection instruction, and determine the program with the most selection times as the next VR program to be played and displayed; In the process of determining the VR program corresponding to each VR program selection instruction and determining the program with the most selection times as the next VR program to be played and displayed, the following steps are included: Collect a historical data set containing the number of times each VR program was selected and played as a training data set; Use the training data set to initialize the Gaussian process regression model and obtain a trained Gaussian process regression model; Using the trained Gaussian process regression model, input new sample data to calculate the posterior distribution of the number of selections for each program, and sample the posterior distribution of each program to obtain the number of selections under different circumstances; Define the membership function of the fuzzy inference system and set the parameters of the fuzzy inference system; By using Bayesian optimization of the posterior distribution and performing expected calculation, the expected value of the number of selections of each program under different possible situations is obtained, and the expected value is mapped to the membership quantification range to obtain the membership degree of the expected number of selections of each program. The VR programs are sorted from high to low according to the membership of the expected number of selections, and the program with the highest membership is selected as the next VR program to be played and displayed.

2. A VR multi-terminal collaborative interaction system, characterized in that: The VR multi-terminal collaborative interaction method according to claim 1 is applied, and the system comprises: An interactive selection pop-up window module is used to display a VR scene image through a VR holographic imaging screen. When an interactive scene appears in the VR scene image after a set time value, a number of interactive selection pop-up windows are linearly displayed in the lower edge area; An interactive command receiving module is used to receive interactive commands input by a number of interactive control terminals. Interactive control terminals are installed on the front seats of the VR holographic theater, and each interactive selection pop-up window corresponds to a front seat; A plot scene determination module is used to fuse multiple interactive commands to obtain fused data, and determine the plot scene of the corresponding interactive scenario according to the fused data; The VR program interactive selection module is used to generate a VR program list selection pop-up window after a VR scene image is played and displayed, receive VR program selection instructions input by a number of interactive control terminals, and determine the next VR program to be played and displayed according to the VR program selection instructions.

3. The VR multi-terminal collaborative interaction system according to claim 2, characterized in that: The interactive command receiving module comprises: An interactive command receiving unit, used to receive interactive commands input by a plurality of interactive control terminals before the interactive selection pop-up window disappears, wherein the interactive selection pop-up window is set with a display duration; A selection pop-up window corresponding unit is used to determine an interactive selection pop-up window corresponding to each interactive command, and the interactive control terminal corresponds to the interactive selection pop-up window one by one; The content automatic selection unit is used to automatically select the content in the interactive selection pop-up window when no corresponding interactive command is received before the interactive selection pop-up window disappears.

Images