Voice-driven virtual scene generation and switching method and implementation system

Abstract

The invention discloses a voice-driven virtual scene generation and switching method and an implementation system, which are applied to meta universe VR (virtual reality) glasses equipment, and comprise the steps of recognizing an instruction of a user through voice recognition and natural language processing technologies, generating or switching a virtual scene in real time, and realizing personalized scene creation and object generation. The edge computing node is adopted to process a high-calculation-amount task, the delay of scene generation and switching is remarkably reduced, the data transmission efficiency is improved by dynamically selecting the optimal node, the privacy security of a user is guaranteed, and through frame rate smooth adjustment and a scene fade-out mechanism, the experience of the user for exiting the virtual scene is optimized, the dizziness is reduced, and the user experience is improved. And the interaction fluency and comfort are improved. The method has the characteristics of quick response, convenient operation and strong privacy protection, and is widely applied to the virtual reality application fields of games, education, medical treatment and the like.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality (VR) technology, and in particular to a voice-driven virtual scene generation and switching method and implementation system, which is suitable for VR equipment and related metaverse environment interaction systems. Background Art

[0002] With the rapid development of virtual reality (VR) technology and voice recognition technology, VR devices are gradually being used in games, education, medical care and other fields. However, existing VR applications usually rely on manual operation or use traditional remote control devices to achieve interaction, making it difficult for users to operate conveniently when wearing VR devices. The limitation of this operation mode is that when the wearer performs operations such as scene switching and object generation, the overall experience may be affected due to inconvenient operation or response delays. In addition, traditional manual operation methods are difficult to achieve efficient interaction when using VR headsets, especially in metaverse applications, where it becomes particularly important to quickly generate and switch virtual scenes.

[0003] Some voice-controlled VR devices have appeared in current technology, but there are still problems such as high response delay and insufficient operational stability, which are mainly manifested in the unsatisfactory speed of scene generation and switching. In complex scenes, the accuracy and parsing speed of voice recognition affect the consistency of user experience to a certain extent. In particular, when users need to generate specific virtual items or adjust their appearance parameters, the actual experience is often insufficient due to the insufficient computing power and response speed of existing technologies. At the same time, current technologies also have obvious deficiencies in user privacy protection. For example, users' voice data and interaction information often need to be transmitted to remote servers for processing, and this process may not only generate privacy leakage risks, but also further increase the response time of scene generation and switching due to network delays. In addition, users often feel a slight sense of dizziness when exiting virtual scenes, and existing VR devices fail to effectively deal with this problem, resulting in a decline in user experience.

[0004] Therefore, the market urgently needs a voice-driven VR system that can overcome the above-mentioned shortcomings, so as to achieve high efficiency and stability in scene generation and switching, and further protect user privacy and security and improve the smoothness of the exit experience. Summary of the invention

[0005] In view of the problems of delay, inconvenient operation, insufficient privacy protection and dizziness when exiting in existing VR devices, the present invention provides a voice-driven virtual scene generation and switching method and implementation system. The present invention quickly analyzes user instructions through voice recognition to achieve virtual scene generation, switching and object creation; uses edge computing to reduce delays and protect user privacy; and optimizes the exit experience through smooth frame rate adjustment and scene fading, thereby improving the fluency and comfort of VR interaction.

[0006] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0007] A voice-driven virtual scene generation and switching method, applied to a metaverse VR virtual reality glasses device, comprises the following steps:

[0008] Step S1: The wearer puts the Metaverse VR glasses on his head, faces the front of the glasses with his eyes, and turns on the voice recognizer;

[0009] Step S2: Acquire the wearer's voice command, parse the command through voice recognition technology, and identify the command of scene generation, object creation or scene switching;

[0010] Step S3: Generate corresponding virtual scenes or virtual objects using virtual reality technology according to the recognized voice commands, allowing users to build a personalized metaverse in the virtual world;

[0011] Step S4: responding to the user's voice command in real time, switching or updating the virtual scene, or creating a new virtual item according to the user's needs;

[0012] Step S5: When the user inputs an exit command, the system executes the exit operation, or the user manually closes the virtual reality environment by pressing a button in the middle of the glasses;

[0013] Step S6: Use optimized computing processing technology to reduce delays in scene generation and switching, ensure privacy protection of user data, and reduce the slight dizziness that accompanies the user during the exit process.

[0014] Preferably, in step S1, when the wearer wears the Metaverse VR virtual reality glasses on the head, the device automatically detects the wearing position and strength through the built-in inertial measurement unit IMU and pressure sensor. The sensor determines whether the wearing is correct based on the set threshold, and prompts the wearer to adjust the wearing status through voice or vibration feedback.

[0015] Preferably, in step S2, the speech recognition technology adopts a speech recognition model based on a deep neural network DNN, which specifically includes the following steps:

[0016] The speech recognition model performs multi-layer neural network reasoning and calculation on the user's speech input to generate output text; the speech recognition model is trained using the following formula:

[0017]

[0018] Where P(y|x) represents the probability of outputting the corresponding text y given the input speech signal x; y t represents the output text or text fragment corresponding to time step t; x t represents the input speech signal at time step t; is a multiplication symbol, indicating the probability P(y t |x t ,h t-1 ) to perform cumulative multiplication; h t-1 is the hidden state at the previous moment; T represents the time step number of the speech signal sequence; the model generates output text according to the hidden state at the previous moment in the feedforward neural network for subsequent scene generation or switching operations.

[0019] Preferably, the parsing of the voice instruction comprises the following steps:

[0020] Convert user voice input into text through an end-to-end speech-to-text ASR model;

[0021] The generated text is classified using the natural language processing (NLP) model, and the user intent is determined using the following formula:

[0022] C(y|x)=softmax(W*h+b)

[0023] Wherein, W is the classification weight matrix, h is the feature vector of the text input, b is the bias term, and the output generates a classification result through the softmax function to determine the scene generation, object creation or scene switching operation corresponding to the text.

[0024] Preferably, in step S3, the virtual reality technology uses a generative adversarial network (GAN) and a neural radiation field (NeRF) technology to generate a virtual scene, and the specific steps include:

[0025] Using the GAN composed of the generator G and the discriminator D, the image elements in the virtual scene are adversarially generated and optimized, where the generator receives the random noise input z and generates the scene. The calculation formula is:

[0026] G(z)=f(W g *z+b g )

[0027] Where G(z) is the output image of the generator; f is the activation function, which is used to introduce nonlinear factors to make the generated image more diverse and complex; z is the random noise input used to generate new images; W g is the weight matrix in the generator, which controls the generator's transformation of input noise; b g is the bias vector of the generator, used to adjust the output result;

[0028] The discriminator judges the authenticity of the generated image through adversarial training, and the expression is:

[0029] D(x)=σ(W d *x+b d )

[0030] Where D(x) is the output of the discriminator, which represents the authenticity score of the generated image; σ is the activation function; x is the generated image or the real image, which is input to the discriminator for distinguishing true from false; W d is the weight matrix of the discriminator, which determines the way the discriminator extracts features of the input image; b d is the bias vector of the discriminator, which is used to adjust the discriminant output;

[0031] High-precision 3D model rendering is achieved through 3D scene reconstruction and ray tracing based on NeRF technology. The calculation formula is:

[0032]

[0033] Among them, C(r) represents the color value along ray r, which is the color information of the final generated three-dimensional image; tn and tf represent the starting and ending positions of the light, respectively, and determine the range of ray tracing; T(t) is the light transmission function; σ(t) is the volume density function, which is used to describe the density of the object at position t and determines the degree of light absorption; c(t) is the color vector, which represents the color information of the object at position t; dt represents the small distance between each point and the next point;

[0034] The generation process is used to achieve realistic and adjustable scenes and items in a virtual reality environment.

[0035] Preferably, the size, shape, color and position attributes of the generated corresponding virtual scene or virtual object are adjusted in real time according to specific parameters input by the user's voice, and the adjustment process includes:

[0036] Use the generator G to dynamically generate items and modify the three-dimensional coordinates (x, y, z) of the items in real time. The expression for controlling the size adjustment is:

[0037] S=α*S0

[0038] Among them, S is the adjusted item size, S0 is the initial size, and α is the adjustment ratio parameter input by the user.

[0039] Preferably, in step S4, the switching or updating of the virtual scene is optimized by reinforcement learning technology, and the system learns the user's scene switching history and preferences by deep Q network DQN algorithm, and the specific steps include:

[0040] The state s in the scene switching process is the user's historical operation record, and the action a is the user's scene switching instruction;

[0041] The recommended scenario is calculated using the following Q function:

[0042] Q(s,a)=r+γm a a ′ xQ(s′,a′)

[0043] Among them, Q(s,a) is the Q value of taking action a in the current state s; r is the immediate reward, γ is the discount factor, s′ and a′ represent the next state and the possible action in this state respectively.

[0044] Preferably, in step S5, when the user exits by pressing the button in the middle of the glasses, the system pops up an exit confirmation interface, and the specific steps include:

[0045] After pressing the button, the system detects the pressing signal and displays the exit confirmation interface;

[0046] The user confirms the exit by voice or by pressing the button again, and after confirmation, the system closes the virtual reality environment.

[0047] Preferably, in step S6, the optimized computing and processing technology is used to reduce the delay in scene generation and switching, and the specific steps include:

[0048] Parse the user's voice commands on the local device to extract scene generation or switching requirements;

[0049] Transmitting the parsed instructions to edge nodes, which perform high-computation tasks such as virtual scene generation, three-dimensional model rendering, or object creation;

[0050] After the edge node completes the calculation, it transmits the generated scene or object data back to the local device to achieve real-time update and fast presentation;

[0051] The system dynamically selects the best edge node based on network latency and edge node load, optimizing data transmission speed and scene generation efficiency.

[0052] Preferably, the optimized computing and processing technology also includes alleviating the slight dizziness of the user when exiting the virtual reality scene through scene transition and frame rate adjustment, which specifically includes the following steps:

[0053] After receiving the user's exit command, the visual elements in the virtual reality scene are gradually reduced through the progressive scene fade-out technology to avoid abrupt scene switching;

[0054] The system uses frame rate adjustment to smoothly reduce the frame rate in the VR environment during the exit process;

[0055] The system dynamically adjusts the scene exit animation based on the user's head movement monitored in real time by the sensor, so that the picture changes during the scene exit process are synchronized with the user's head movement, reducing the dizziness that may occur during the exit process.

[0056] Preferably, the privacy protection of the user data comprises the following specific steps:

[0057] The local device performs preliminary training on the user behavior data, and the model parameters generated by the training are transmitted to the federated learning server for global model update;

[0058] The updated model parameters of the global model are transmitted back to the local device to ensure that the user's behavior data does not leave the local device.

[0059] A virtual scene generation and switching system based on voice drive, comprising:

[0060] The speech recognition module is used to obtain and analyze the voice commands input by the user and identify the needs of scene generation, object creation or scene switching;

[0061] An edge computing module, which is used to process the instructions parsed by the speech recognition module, perform high-computation tasks such as virtual scene generation, three-dimensional model rendering or object creation, and reduce delays in scene generation and switching;

[0062] Dynamic node selection module, which is used to monitor network latency and edge node load, dynamically select the best edge node, and further optimize data transmission speed and scene generation efficiency;

[0063] A frame rate adjustment module, which is used to smoothly reduce the frame rate of the virtual reality after receiving the user's exit instruction. The frame rate adjustment module combines the real-time data of the motion monitoring and synchronization module to reduce the slight dizziness that may accompany the user when exiting the virtual reality scene;

[0064] The scene fade-out module is used to gradually reduce the visual elements in the virtual reality scene through the progressive scene fade-out technology during the exit process to avoid the abrupt feeling of scene switching;

[0065] The motion monitoring and synchronization module includes a sensor for monitoring the user's head movement in real time and dynamically adjusting the exit animation of the scene according to the movement, so that the scene changes during the exit process are synchronized with the user's head movement, thereby reducing the feeling of dizziness.

[0066] Compared with the prior art, the beneficial effects of the present invention are as follows: the present invention realizes efficient and convenient interaction of users in the metaverse VR device through a voice-driven virtual scene generation and switching method. Users can directly enter the virtual scene by wearing VR devices and enabling the voice recognition function, which greatly improves the convenience of operation and the immersive experience. The deep neural network and natural language processing technology are used to accurately parse the user's voice commands, which can quickly identify the user's scene generation, item creation or scene switching intentions, realize instant and flexible virtual content construction, and enable users to intuitively customize the personalized virtual world through voice. The present invention has advantages in real-time response to user voice commands, ensuring that scene switching and updating are not delayed, improving the smoothness of interaction, and enhancing the user's immersion in the virtual world. In order to increase the safety of operation, the system is designed with a variety of exit methods, including voice and manual button press exit methods, so that users can quickly and safely exit the virtual environment according to their needs, improving the autonomy and flexibility of user experience. By optimizing the computing and processing technology, the present invention not only reduces the delay in the scene generation and switching process, but also effectively protects the privacy and security of user data. At the same time, it reduces the slight dizziness of the user when exiting through visual buffering processing, thereby significantly improving the overall virtual reality experience, making it more comfortable, safe and highly customized. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0068] in:

[0069] Figure 1 A schematic diagram of a method flow chart of an embodiment of the present invention;

[0070] Figure 2 This is a schematic diagram of the wearing detection and voice recognition startup process in an embodiment of the present invention;

[0071] Figure 3 A schematic diagram of virtual scene generation and object creation in an embodiment of the present invention;

[0072] Figure 4 A schematic diagram of the scene switching and updating operation flow in an embodiment of the present invention;

[0073] Figure 5 This is a schematic diagram of the exit process in an embodiment of the present invention;

[0074] Figure 6 It is a schematic diagram of the overall structure in an embodiment of the present invention. DETAILED DESCRIPTION

[0075] In order to make the purpose, technical solution and advantages of the embodiment of the present invention clearer, the technical solution of the embodiment of the present invention will be clearly and completely described below in conjunction with the drawings of the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all of the embodiments. Based on the described embodiment of the present invention, all other embodiments obtained by ordinary technicians in this field belong to the scope of protection of the present invention.

[0076] Example 1

[0077] like Figure 1 , Figure 2 As shown in FIG. 1 , an embodiment of the present invention is provided, which provides a method for generating and switching a virtual scene driven by voice, including:

[0078] Step S1: The wearer puts the Metaverse VR glasses on his head, faces the front of the glasses with his eyes, and turns on the voice recognizer;

[0079] Among them, in step S1, when the wearer wears the Metaverse VR virtual reality glasses on his head, the device automatically detects the wearing position and strength through the built-in inertial measurement unit IMU and pressure sensor. The sensor determines whether the wearing is correct according to the set threshold, and prompts the wearer to adjust the wearing status through voice or vibration feedback.

[0080] In this embodiment, when the wearer wears the virtual reality (VR) glasses on the head, the inertial measurement unit (IMU) and pressure sensor in the glasses will automatically detect the wearing angle and wearing pressure of the glasses. The inertial measurement unit is a sensor used to detect the movement of the device, which can sense the wearer's head movement and the spatial position of the device; the pressure sensor monitors the pressure of the device on the skin to ensure the comfort of wearing. The system determines whether the wearing state is correct based on the set wearing parameter standards, such as wearing position, pressure, etc. If it is detected that the wearing does not meet the requirements, the user will be prompted to make adjustments through voice or vibration feedback. When the wearing state meets the standards, the system will automatically turn on the voice recognizer and enter the standby state. This design realizes automated wearing detection, helps to optimize the user experience, and allows users to turn on the voice-driven function without additional operations.

[0081] Further, such as Figure 3As shown, step S2: obtaining the wearer's voice command, parsing the command through voice recognition technology, and identifying the command of scene generation, object creation or scene switching;

[0082] Specifically, in step S2, the speech recognition technology adopts a speech recognition model based on a deep neural network DNN, which specifically includes the following steps:

[0083] The speech recognition model performs multi-layer neural network reasoning and calculation on the user's speech input to generate output text; the speech recognition model is trained using the following formula:

[0084]

[0085] Where P(y|x) represents the probability of outputting the corresponding text y given the input speech signal x; y t represents the output text or text fragment corresponding to time step t; x t represents the input speech signal at time step t; is a multiplication symbol, indicating the probability P(y t |x t ,h t-1 ) to perform cumulative multiplication; h t-1 is the hidden state at the previous moment; T represents the time step number of the speech signal sequence; the model generates output text according to the hidden state at the previous moment in the feedforward neural network for subsequent scene generation or switching operations.

[0086] Specifically, the parsing of the voice command includes the following steps:

[0087] Convert user voice input into text through an end-to-end speech-to-text ASR model;

[0088] The generated text is classified using the natural language processing (NLP) model, and the user intent is determined using the following formula:

[0089] C(y|x)=softmax(W*h+b)

[0090] Wherein, W is the classification weight matrix, h is the feature vector of the text input, b is the bias term, and the output generates a classification result through the softmax function to determine the scene generation, object creation or scene switching operation corresponding to the text.

[0091] After the wearer puts on the device, the system will start to listen to the user's voice commands. The system uses a deep neural network DNN as a speech recognition model, and processes voice signals through a multi-layer neural network. A deep neural network is a machine learning technology that can process complex data patterns. In this case, it converts the user's voice command into text, and then uses a natural language processing (NLP) model to classify the intent of the generated text. The natural language processing model is an artificial intelligence technology that is specifically used to recognize and understand human language. Here, the NLP model will recognize the intent of the text of the command and divide the command into operational requirements such as "scene generation", "item creation" or "scene switching". This dual analysis method improves the accuracy of speech recognition, can effectively reduce the risk of misrecognition, and ensure that the system accurately executes user commands.

[0092] Further, such as Figure 4 As shown, step S3: according to the recognized voice command, the corresponding virtual scene or virtual object is generated by using virtual reality technology, allowing the user to build a personalized metaverse in the virtual world;

[0093] Among them, in step S3, the virtual reality technology uses the generative adversarial network GAN and neural radiation field NeRF technology to generate a virtual scene, and the specific steps include:

[0094] Using the GAN composed of the generator G and the discriminator D, the image elements in the virtual scene are adversarially generated and optimized, where the generator receives the random noise input z and generates the scene. The calculation formula is:

[0095] G(z)=f(W g *z+b g )

[0096] Where G(z) is the output image of the generator; f is the activation function, which is used to introduce nonlinear factors to make the generated image more diverse and complex; z is the random noise input used to generate new images; W g is the weight matrix in the generator, which controls the generator's transformation of input noise; b g is the bias vector of the generator, used to adjust the output result;

[0097] The discriminator judges the authenticity of the generated image through adversarial training, and the expression is:

[0098] D(x)=σ(W d *x+b d )

[0099] Where D(x) is the output of the discriminator, which represents the authenticity score of the generated image; σ is the activation function; x is the generated image or the real image, which is input to the discriminator for distinguishing true from false; W dis the weight matrix of the discriminator, which determines the way the discriminator extracts features of the input image; b d is the bias vector of the discriminator, which is used to adjust the discriminant output;

[0100] High-precision 3D model rendering is achieved through 3D scene reconstruction and ray tracing based on NeRF technology. The calculation formula is:

[0101]

[0102] Among them, C(r) represents the color value along ray r, which is the color information of the final generated three-dimensional image; tn and tf represent the starting and ending positions of the light, respectively, and determine the range of ray tracing; T(t) is the light transmission function; σ(t) is the volume density function, which is used to describe the density of the object at position t and determines the degree of light absorption; c(t) is the color vector, which represents the color information of the object at position t; dt represents the small distance between each point and the next point;

[0103] The generation process is used to achieve realistic and adjustable scenes and items in a virtual reality environment.

[0104] Specifically, the size, shape, color and position attributes of the generated corresponding virtual scene or virtual object are adjusted in real time according to the specific parameters input by the user's voice, and the adjustment process includes:

[0105] Use the generator G to dynamically generate items and modify the three-dimensional coordinates (x, y, z) of the items in real time. The expression for controlling the size adjustment is:

[0106] S=α*S0

[0107] Among them, S is the adjusted item size, S0 is the initial size, and α is the adjustment ratio parameter input by the user.

[0108] After identifying the specific instructions, the system generates corresponding virtual content according to user needs. This process uses generative adversarial networks (GAN) and neural radiation fields (NeRF) technology. Generative adversarial networks (GAN) are a machine learning method that includes two subnetworks: a generator and a discriminator. The generator is responsible for creating realistic images, while the discriminator determines the authenticity of the image. GAN is used in the present invention to generate details and textures of virtual objects, making the objects appear more realistic. Neural radiation field (NeRF) technology uses ray tracing to render three-dimensional structures and depth details in virtual scenes, making the scenes more realistic and deep. Through the combination of GAN and NeRF technology, the system can generate highly realistic virtual scenes and objects, providing users with a personalized construction experience in the virtual world.

[0109] Further, step S4: responding to the user's voice command in real time, switching or updating the virtual scene, or creating a new virtual item according to the user's needs;

[0110] Among them, in step S4, the switching or updating of the virtual scene is optimized by reinforcement learning technology, and the system learns the user's scene switching history and preferences through the deep Q network DQN algorithm. The specific steps include:

[0111] The state s in the scene switching process is the user's historical operation record, and the action a is the user's scene switching instruction;

[0112] The recommended scenario is calculated using the following Q function:

[0113] Q(s,a)=r+γm a a ′ xQ(s′,a′)

[0114] Among them, Q(s,a) is the Q value of taking action a in the current state s; r is the immediate reward, γ is the discount factor, s′ and a′ represent the next state and the possible action in this state respectively.

[0115] After the virtual scene is generated, the system will continue to listen to the user's voice commands and respond in real time. Real-time performance is the key feature of this step: the system uses edge computing technology to complete most computing tasks on computing devices near the user (such as VR devices or edge servers), thereby reducing data transmission delays. Edge computing is a distributed computing method that can complete complex calculations on devices close to the data source, reducing delays caused by remote server processing. Therefore, every time the user issues a switch or update command, the system can respond quickly and display the results in the virtual scene immediately, making operations such as scene switching and item updates smoother. This design significantly enhances the interactivity of virtual reality, allowing users to have a seamless immersive experience.

[0116] Further, such as Figure 5 As shown, step S5: when the user inputs an exit command, the system executes the exit operation, or the user manually closes the virtual reality environment by pressing a button in the middle of the glasses;

[0117] Among them, in step S5, when the user exits by pressing the button in the middle of the glasses, the system pops up an exit confirmation interface, and the specific steps include:

[0118] After pressing the button, the system detects the pressing signal and displays the exit confirmation interface;

[0119] The user confirms the exit by voice or by pressing the button again, and after confirmation, the system closes the virtual reality environment.

[0120] In order to facilitate users to quickly exit the virtual reality environment when needed, the system provides two ways to exit: voice exit and physical button exit. Users can directly input the "exit" command through voice, and the system will immediately respond and end the current virtual scene; at the same time, users can also exit manually by pressing the physical button in the middle of the glasses. After pressing the button, the system will detect the pressing signal and pop up the exit confirmation interface. Users can choose to confirm by voice or press the button again to confirm the exit. This design ensures that users can quickly and conveniently exit the virtual reality environment in an emergency, providing users with better operational safety and convenience.

[0121] Further, step S6: using optimized computing processing technology to reduce delays in scene generation and switching, and ensure privacy protection of user data, while reducing the slight dizziness that accompanies the user during the exit process.

[0122] Among them, in step S6, the optimized computing processing technology is used to reduce the delay in scene generation and switching, and the specific steps include:

[0123] Parse the user's voice commands on the local device to extract scene generation or switching requirements;

[0124] Transmitting the parsed instructions to edge nodes, which perform high-computation tasks such as virtual scene generation, three-dimensional model rendering, or object creation;

[0125] After the edge node completes the calculation, it transmits the generated scene or object data back to the local device to achieve real-time update and fast presentation;

[0126] The system dynamically selects the best edge node based on network latency and edge node load, optimizing data transmission speed and scene generation efficiency.

[0127] Specifically, the optimized computing processing technology also includes reducing the slight dizziness of the user when exiting the virtual reality scene through scene transition and frame rate adjustment, which specifically includes the following steps:

[0128] After receiving the user's exit command, the visual elements in the virtual reality scene are gradually reduced through the progressive scene fade-out technology to avoid abrupt scene switching;

[0129] The system uses frame rate adjustment to smoothly reduce the frame rate in the VR environment during the exit process;

[0130] The system dynamically adjusts the scene exit animation based on the user's head movement monitored in real time by the sensor, so that the picture changes during the scene exit process are synchronized with the user's head movement, reducing the dizziness that may occur during the exit process.

[0131] Specifically, the privacy protection of user data includes the following steps:

[0132] The local device performs preliminary training on the user behavior data, and the model parameters generated by the training are transmitted to the federated learning server for global model update;

[0133] The updated model parameters of the global model are transmitted back to the local device to ensure that the user's behavior data does not leave the local device.

[0134] In the process of scene generation and switching, the system combines the optimization strategies of edge computing and distributed computing to improve the processing speed, thereby achieving instant response to user commands. To protect the privacy and security of user data, the system uses federated learning technology. Federated learning is a distributed data processing method that allows user data to be calculated on local devices, and then the results are uploaded to the central server without uploading the original data, thereby avoiding privacy leakage. During the exit process, in order to reduce the possible dizziness of users, the system uses progressive scene fade-out technology, that is, the visual content of the virtual scene will gradually weaken, combined with frame rate adjustment and synchronous control of head motion sensors, so that the user's visual changes and physical perceptions are consistent, thereby effectively alleviating the discomfort caused by scene switching.

[0135] like Figure 6 As shown, a virtual scene generation and switching system based on voice drive includes:

[0136] The speech recognition module is used to obtain and analyze the voice commands input by the user and identify the needs of scene generation, object creation or scene switching;

[0137] An edge computing module, which is used to process the instructions parsed by the speech recognition module, perform high-computation tasks such as virtual scene generation, three-dimensional model rendering or object creation, and reduce delays in scene generation and switching;

[0138] Dynamic node selection module, which is used to monitor network latency and edge node load, dynamically select the best edge node, and further optimize data transmission speed and scene generation efficiency;

[0139] A frame rate adjustment module, which is used to smoothly reduce the frame rate of the virtual reality after receiving the user's exit instruction. The frame rate adjustment module combines the real-time data of the motion monitoring and synchronization module to reduce the slight dizziness that may accompany the user when exiting the virtual reality scene;

[0140] The scene fade-out module is used to gradually reduce the visual elements in the virtual reality scene through the progressive scene fade-out technology during the exit process to avoid the abrupt feeling of scene switching;

[0141] The motion monitoring and synchronization module includes a sensor for monitoring the user's head movement in real time and dynamically adjusting the exit animation of the scene according to the movement, so that the scene changes during the exit process are synchronized with the user's head movement, thereby reducing the feeling of dizziness.

[0142] Example 2

[0143] The present invention is applicable to multiple virtual reality application fields, including games, education and medical scenarios. The system uses voice recognition technology and edge computing to achieve real-time response to user commands, so as to efficiently generate and switch virtual scenes, while also having advantages in data security and privacy protection.

[0144] In game applications, users use VR headsets to perform immersive game operations. The system can generate or switch scenes based on voice commands, such as "generate a battlefield" or "switch to a forest", and quickly render the required virtual environment through edge computing nodes. At the same time, users can generate specific virtual objects (such as "generate a shield" or "create an obstacle") through voice, and dynamically adjust their size, position and other attributes, which improves the smoothness of the game interaction experience and the smoothness of the screen transition, thereby effectively reducing the dizziness during scene switching.

[0145] In educational applications, teachers or students can enter the virtual teaching environment through voice commands and perform operations such as "generate a chemical laboratory" or "show the layered structure of the earth". The system quickly generates the required scenes based on voice commands, enhancing the interactivity and immersion of teaching. Teachers can further adjust the properties of virtual objects (such as atomic models and stars) through voice, allowing students to observe and understand complex concepts more intuitively. The edge computing function of this system ensures local processing of interactive data, thereby protecting student privacy and meeting security requirements in the teaching environment.

[0146] In medical applications, the system facilitates medical training and rehabilitation treatment. Users can generate specific scenes through voice control, such as "generate an operating room" or "switch to an emergency simulation environment", which is convenient for doctors or trainees to conduct operation training. At the same time, users can generate human organ models or medical tools through voice, and adjust the position and size for detailed observation and interaction. In addition, in rehabilitation training, the system generates virtual objects such as stairs and grip trainers according to patient instructions for cognitive and motor training, and locally processes and stores patient operation data through a privacy protection mechanism.

[0147] In summary, the present invention successfully solves the problems of large response delay, inconvenient operation and weak privacy protection in the scene switching and generation process of existing VR devices through a voice-driven virtual scene generation and switching method and implementation system. The present invention can accurately parse the user's scene generation and switching instructions through advanced voice recognition and natural language processing technology, and uses edge computing to accelerate task processing, reduce delays, and ensure the local privacy security of user data. At the same time, the present invention adopts frame rate smoothing adjustment and progressive scene fade-out technology to ensure that the screen switches smoothly when the user exits the virtual scene, greatly reducing the feeling of dizziness and improving the comfort of the exit process. Overall, the present invention provides a highly intelligent VR interaction solution, which not only improves the fluency of the user experience, but also provides safer and more efficient technical support for metaverse and virtual reality applications, and has broad application and promotion prospects.

[0148] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0149] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0150] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A voice-driven virtual scene generation and switching method, applied to a metaverse VR virtual reality glasses device, characterized in that: The following steps are involved: Step S1: The wearer puts the Metaverse VR glasses on his head, faces the front of the glasses with his eyes, and turns on the voice recognizer; Step S2: Acquire the wearer's voice command, parse the command through voice recognition technology, and identify the command of scene generation, object creation or scene switching; Step S3: Generate corresponding virtual scenes or virtual objects using virtual reality technology according to the recognized voice commands, allowing users to build a personalized metaverse in the virtual world; Step S4: responding to the user's voice command in real time, switching or updating the virtual scene, or creating a new virtual item according to the user's needs; Step S5: When the user inputs an exit command, the system executes the exit operation, or the user manually closes the virtual reality environment by pressing a button in the middle of the glasses; Step S6: Use optimized computing processing technology to reduce delays in scene generation and switching, ensure privacy protection of user data, and reduce the slight dizziness that accompanies the user during the exit process.

2. The method for generating and switching a virtual scene driven by voice according to claim 1, characterized in that: In step S1, when the wearer wears the Metaverse VR glasses on his head, the device automatically detects the wearing position and strength through the built-in inertial measurement unit IMU and pressure sensor. The sensor determines whether the wearing is correct based on the set threshold, and prompts the wearer to adjust the wearing status through voice or vibration feedback.

3. The method for generating and switching a virtual scene driven by voice according to claim 1, characterized in that: In step S2, the speech recognition technology adopts a speech recognition model based on a deep neural network DNN, which specifically includes the following steps: Perform multi-layer neural network reasoning and calculation on the user's voice input through the voice recognition model to generate output text; The speech recognition model is trained using the following formula: Where P(y|x) represents the probability of outputting the corresponding text y given the input speech signal x; y t represents the output text or text fragment corresponding to time step t; x t represents the input speech signal at time step t; is a multiplication symbol, indicating the probability P(y t |x t ,h t-1 ) to perform cumulative multiplication; h t-1 is the hidden state at the previous moment; T represents the time step number of the speech signal sequence; the model generates output text according to the hidden state at the previous moment in the feedforward neural network for subsequent scene generation or switching operations.

4. The method for generating and switching a virtual scene driven by voice according to claim 1, characterized in that: The analysis of the voice command comprises the following steps: Convert user voice input into text through an end-to-end speech-to-text ASR model; The generated text is classified using the natural language processing (NLP) model, and the user intent is determined using the following formula: C(y|x)=softmax(W*h+b) Wherein, W is the classification weight matrix, h is the feature vector of the text input, b is the bias term, and the output generates the classification result through the softmax function to determine the scene generation, object creation or scene switching operation corresponding to the text.

5. The method for generating and switching a virtual scene driven by voice according to claim 1, characterized in that: In step S3, the virtual reality technology uses the generative adversarial network GAN and neural radiation field NeRF technology to generate a virtual scene. The specific steps include: Using the GAN composed of the generator G and the discriminator D, the image elements in the virtual scene are adversarially generated and optimized, where the generator receives the random noise input z and generates the scene. The calculation formula is: G(z)=f(W g *z+b g ) Where G(z) is the output image of the generator; f is the activation function, which is used to introduce nonlinear factors to make the generated image more diverse and complex; z is the random noise input used to generate new images; W g is the weight matrix in the generator, which controls the generator's transformation of input noise; b g is the bias vector of the generator, used to adjust the output result; The discriminator judges the authenticity of the generated image through adversarial training, and the expression is: D(x)=σ(W d *x+b d ) Where D(x) is the output of the discriminator, which represents the authenticity score of the generated image; σ is the activation function; x is the generated image or the real image, which is input to the discriminator for distinguishing true from false; W d is the weight matrix of the discriminator, which determines the way the discriminator extracts features of the input image; b d is the bias vector of the discriminator, used to adjust the discriminant output; Three-dimensional scene reconstruction and ray tracing based on NeRF technology achieve high-precision three-dimensional model rendering. The calculation formula is: Among them, C(r) represents the color value along ray r, which is the color information of the final generated three-dimensional image; tn and tf represent the starting and ending positions of the light, respectively, and determine the range of ray tracing; T(t) is the light transmission function; σ(t) is the volume density function, which is used to describe the density of the object at position t and determines the degree of light absorption; c(t) is the color vector, which represents the color information of the object at position t; dt represents the small distance between each point and the next point; The generation process is used to achieve realistic and adjustable scenes and items in a virtual reality environment.

6. The method for generating and switching a virtual scene driven by voice according to claim 1, characterized in that: The size, shape, color and position attributes of the generated corresponding virtual scene or virtual object are adjusted in real time according to the specific parameters input by the user's voice, and the adjustment process includes: Use the generator G to dynamically generate items and modify the three-dimensional coordinates (x, y, z) of the items in real time. The expression for controlling the size adjustment is: S=α*S0 Among them, S is the adjusted item size, S0 is the initial size, and α is the adjustment ratio parameter input by the user.

7. The method for generating and switching a virtual scene driven by voice according to claim 1, characterized in that: In step S4, the switching or updating of the virtual scene is optimized by reinforcement learning technology, and the scene switching history and preferences of the user are learned by the deep Q network DQN algorithm. The specific steps include: The state s in the scene switching process is the user's historical operation record, and the action a is the user's scene switching instruction; The recommended scenario is calculated using the following Q function: Q(s,a)=r+γm a a ′ xQ(s′,a′) Among them, Q(s,a) is the Q value of taking action a in the current state s; r is the immediate reward, γ is the discount factor, s′ and a′ represent the next state and the possible action in this state respectively.

8. The method for generating and switching a virtual scene driven by voice according to claim 1, characterized in that: In step S5, when the user exits by pressing the button in the middle of the glasses, the system pops up an exit confirmation interface, and the specific steps include: After pressing the button, the system detects the pressing signal and displays the exit confirmation interface; The user confirms the exit by voice or by pressing the button again, and after confirmation, the system closes the virtual reality environment.

9. The method for generating and switching a virtual scene driven by voice according to claim 1, characterized in that: In step S6, the optimized computing and processing technology is used to reduce the delay in scene generation and switching, and the specific steps include: Parse the user's voice commands on the local device to extract scene generation or switching requirements; Transmitting the parsed instructions to edge nodes, which perform high-computation tasks such as virtual scene generation, three-dimensional model rendering, or object creation; After the edge node completes the calculation, it transmits the generated scene or object data back to the local device to achieve real-time update and fast presentation; The system dynamically selects the best edge node based on network latency and edge node load, optimizing data transmission speed and scene generation efficiency.

10. The method for generating and switching a virtual scene driven by voice according to claim 9, characterized in that: The optimized computing and processing technology also includes alleviating the slight dizziness of the user when exiting the virtual reality scene through scene transition and frame rate adjustment, which specifically includes the following steps: After receiving the user's exit command, the visual elements in the virtual reality scene are gradually reduced through the progressive scene fade-out technology to avoid abrupt scene switching; The system uses frame rate adjustment to smoothly reduce the frame rate in the VR environment during the exit process; The system dynamically adjusts the scene exit animation based on the user's head movement monitored in real time by the sensor, so that the picture changes during the scene exit process are synchronized with the user's head movement, reducing the dizziness that may occur during the exit process.

11. The method for generating and switching a virtual scene driven by voice according to claim 1, characterized in that: The privacy protection of the user data includes the following specific steps: The local device performs preliminary training on the user behavior data, and the model parameters generated by the training are transmitted to the federated learning server for global model update; The updated model parameters of the global model are transmitted back to the local device to ensure that the user's behavior data does not leave the local device.

12. A virtual scene generation and switching system based on voice drive as claimed in claim 1, characterized in that: include: The speech recognition module is used to obtain and analyze the voice commands input by the user and identify the needs of scene generation, object creation or scene switching; An edge computing module is used to process the instructions parsed by the speech recognition module, perform high-computation tasks such as virtual scene generation, three-dimensional model rendering or object creation, and reduce delays in scene generation and switching; Dynamic node selection module, which is used to monitor network latency and edge node load, dynamically select the best edge node, and further optimize data transmission speed and scene generation efficiency; A frame rate adjustment module, which is used to smoothly reduce the frame rate of the virtual reality after receiving the user's exit instruction. The frame rate adjustment module combines the real-time data of the motion monitoring and synchronization module to reduce the slight dizziness that may accompany the user when exiting the virtual reality scene; The scene fade-out module is used to gradually reduce the visual elements in the virtual reality scene through the progressive scene fade-out technology during the exit process to avoid the abrupt feeling of scene switching; The motion monitoring and synchronization module includes a sensor for monitoring the user's head movement in real time and dynamically adjusting the exit animation of the scene according to the movement, so that the scene changes during the exit process are synchronized with the user's head movement, thereby reducing the feeling of dizziness.

Images