An intelligent interactive system for game room mode

Abstract

The present invention relates to the field of virtual reality technology, and in particular to an intelligent interactive system of a game room mode, comprising an interactive perception module for capturing composite data of user voice commands and body movements; an instruction conversion module for standardizing the composite data of user voice commands and body movements, and converting the standardized user voice commands and standardized body movement composite data into virtual reality operation commands; a difference correction module for performing difference correction on the instruction conversion module according to the use of dialects in the user voice commands and the use of the user's virtual reality device; and an engine optimization module for optimizing the difference adjustment process according to the user's real-time status. The present invention improves the accuracy and sensitivity of intelligent interaction of virtual reality devices.

Description

Technical Field

[0001] The present invention relates to the field of virtual reality technology, and in particular to an intelligent interactive system in a game room mode. Background Art

[0002] Virtual reality is a virtual environment generated through computer technology. Users can interact with the virtual environment through multiple senses such as vision, hearing, and touch, providing an immersive experience and making users feel like they are in a virtual world. The functions of virtual reality are matched with a specific physical space, allowing users to physically move within the space, thereby replicating real-world actions in the virtual world and making the virtual environment look more real. However, most current virtual reality related products still rely on handles for interaction. Even if gesture interaction is used, users are required to remember specific gestures to complete simple operations, making it difficult for virtual reality devices to achieve the desired effects.

[0003] Chinese Patent Publication No.: CN115657859A discloses an intelligent interactive system based on virtual reality, including: an eye image acquisition module, a face image acquisition module, a processing module and a control module; the eye image acquisition module is arranged in the virtual reality device, and is used to acquire the eye image of the user wearing the virtual reality device, and send the acquired eye image to the processing module; the face image acquisition module is used to acquire the face image of the user wearing the virtual reality device through an external camera arranged outside the virtual reality device, and send the acquired face image to the processing module; the processing module is used to perform user expression recognition based on the acquired eye image and face image, obtain the expression recognition result, and transmit the obtained expression recognition result to the control module; the control module is used to control the virtual reality device based on the obtained expression recognition result. However, the invention only performs intelligent interactive recognition on the user's face image, and does not perform difference correction on the user itself, and cannot improve the accuracy and sensitivity of intelligent interaction of the virtual reality device. Summary of the invention

[0004] To this end, the present invention provides an intelligent interaction system in a game room mode, so as to overcome the problem of low accuracy and sensitivity of intelligent interaction of virtual reality devices in the prior art.

[0005] To achieve the above object, the present invention provides an intelligent interactive system in a game room mode, characterized in that it includes:

[0006] Interaction perception module, used to capture composite data of user voice commands and body movements;

[0007] A command conversion module, used to perform standardized processing on the composite data of user voice commands and body movements, and convert the standardized user voice commands and standardized body movement composite data into virtual reality operation commands;

[0008] A difference correction module is used to make difference corrections to the instruction conversion module according to the dialect usage in the user's voice instruction and the user's virtual reality device usage;

[0009] The engine optimization module is used to optimize the differential adjustment process according to the real-time status of users.

[0010] Furthermore, the instruction conversion module includes:

[0011] A user voice command recognition unit, used to perform standardization processing on the user voice command to obtain a standardized user voice command, and also used to convert the standardized user voice command into a virtual reality operation command;

[0012] The body movement composite data recognition unit is used to perform standardization processing on the body movement composite data to obtain standardized body movement composite data, and is also used to convert the standardized body movement composite data into virtual reality operation instructions.

[0013] Furthermore, when the user voice instruction recognition unit performs standardization processing on the user voice instruction, the user voice instruction is converted into text information through a voice recognition algorithm, and noise processing is performed on the converted user voice instruction to obtain a standardized user voice instruction.

[0014] Furthermore, when converting standardized user voice commands into virtual reality operation commands, the user voice command recognition unit trains the long short-term memory network model by using a preset standardized user voice command data set, outputs the long short-term memory network model that meets a preset accuracy rate as a user voice command recognition model, and converts the standardized user voice commands into virtual reality operation commands through the user voice command recognition model.

[0015] Furthermore, the body movement composite data recognition unit removes noise from the body movement composite data through Kalman filtering technology, and converts the body movement composite data after the noise is removed into standardized body movement composite data, and compares the standardized body movement composite data with the standard action-virtual reality operation instruction template, and outputs the virtual reality operation instruction according to the comparison result, wherein:

[0016] When there is preset standardized limb action composite data consistent with the standardized limb action composite data in the standard action-virtual reality operation instruction template, the preset virtual reality operation instruction corresponding to the preset standardized limb action composite data is output as the virtual reality operation instruction;

[0017] When there is no preset standardized limb movement compound data consistent with the standardized limb movement compound data in the standard action-virtual reality operation instruction template, the virtual reality operation instruction is not output.

[0018] Furthermore, the difference correction module includes:

[0019] A voice correction unit, used to correct the virtual reality operation instructions according to the dialect usage in the user's voice instructions;

[0020] The action correction unit is used to correct the virtual reality operation instructions according to the user's virtual reality device usage.

[0021] Furthermore, when the voice correction unit corrects the virtual reality operation instruction according to the dialect usage in the user voice instruction, the user uses the dialect number K and the preset dialect usage number K0 are compared, and the effectiveness of the process of converting the standardized user voice instruction into the virtual reality operation instruction is judged according to the comparison result, and the process of converting the standardized user voice instruction into the virtual reality operation instruction is corrected according to the judgment result, wherein:

[0022] When K≥K0, it is determined that the process of converting the standardized user voice command into the virtual reality operation command is effective, the voice correction unit determines to correct the process of converting the user voice command into the virtual reality operation command, and obtains the dialect voice data used by the user, adds the dialect voice data to the preset standardized user voice command data set, obtains the adjusted preset standardized user voice command data set, and trains the long short-term memory network model according to the adjusted preset standardized user voice command data set;

[0023] When K<K0, the voice correction unit determines not to correct the process of converting the user voice command into the virtual reality operation command.

[0024] Furthermore, when the action correction unit corrects the virtual reality operation instruction according to the user's virtual reality device usage, it obtains the user's login status, judges the user's use of the virtual reality device according to the user's login status, and corrects the standard action-virtual reality operation instruction template according to the judgment result, wherein:

[0025] When the user login status is visitor login, the action correction unit determines that the user is using the virtual device for the first time and does not correct the standard action-virtual reality operation instruction template;

[0026] When the user login status is to use an account to log in, obtain the registration time B of the account used by the user, compare the registration time B of the account used by the user with the preset registration time B0, judge the user's use of the virtual reality device according to the comparison result, and modify the standard action-virtual reality operation instruction template according to the judgment result, wherein:

[0027] When B<B0, the action correction unit determines that the user is using the virtual device for the first time, and does not correct the standard action-virtual reality operation instruction template;

[0028] When B≥B0, the action correction unit determines that the user is not using the virtual device for the first time, and corrects the standard action-virtual reality operation instruction template.

[0029] Furthermore, when the action correction unit corrects the virtual reality operation instruction, it obtains the number of times N of use of the standardized limb action composite data, and compares the number of times N of use of each standardized limb action composite data with the preset number of times N0, judges the use frequency of the standardized limb action composite data according to the comparison result, and corrects the standard action-virtual reality operation instruction template according to the judgment result, wherein:

[0030] When N≥N0, the standardized body movement composite data is determined to be high-frequency standardized body movement composite data, and the standard action-virtual reality operation instruction template is not corrected;

[0031] When 0<N<N0, it is determined that the standardized body movement composite data is low-frequency standardized body movement composite data, and a standard movement-virtual reality operation instruction template correction window is pushed to the user, and the user fills in the correspondence between the preset standardized body movement and the preset virtual reality operation instruction, and the movement correction unit obtains the data of the correspondence between the preset standardized body movement and the preset virtual reality operation instruction in the correction window, and adds the data of the correspondence between the preset standardized body movement and the preset virtual reality operation instruction in the correction window to the standard movement-virtual reality operation instruction template;

[0032] When N=0, the limb movement compound data is determined to be invalid standardized limb movement compound data, and the preset standardized limb movement compound data and its corresponding preset virtual reality operation instruction consistent with the invalid limb movement compound data in the standard action-virtual reality operation instruction template are deleted.

[0033] Furthermore, when the engine optimization module optimizes the difference correction module according to the real-time state of the user, the virtual reality operation instruction trigger success rate T is collected in real time, and the virtual reality operation instruction trigger success rate T is compared with the preset virtual reality operation instruction trigger success rate T0, the real-time state of the user is judged according to the comparison result, and the difference correction module is optimized according to the judgment result, wherein:

[0034] When T≥T0, the engine optimization module determines that the user's real-time status is normal and does not optimize the difference correction module;

[0035] When T<T0, the engine optimization module determines that the user's real-time state is abnormal, obtains the user's facial expression image, classifies the user's facial expression image through the facial expression classification model, obtains the user's facial expression classification result, sets the user's facial expression classification result to be excitement and calmness, and optimizes the preset number of times K0 and the preset number of times N0 used in the difference correction module according to the user's facial expression classification result, wherein:

[0036] When the user's facial expression classification result is calm, the preset number of times the dialect is used K0 and the preset number of times N0 in the difference correction module are not optimized;

[0037] When the user's facial expression classification result is excitement, the preset dialect usage times K0 and the preset usage times N0 are optimized, and an optimization coefficient U is set, wherein U=Z+1, and Z is the number of times the preset dialect usage times K0 and the preset usage times N0 in the difference correction module are optimized during the use cycle of the virtual reality device. According to the optimization coefficient U, the preset usage times N0 and the preset dialect usage times K0 are optimized, and the optimized preset usage times are set to Nu0, and the optimized preset dialect usage times are set to Ku0, wherein Nu0=N0-U, Ku0=K0-U, Nu0≥1, and Ku0≥1.

[0038] Compared with the prior art, the beneficial effects of the present invention are that the system captures the composite data of user voice instructions and body movements through the interactive perception module, so as to improve the accuracy of subsequent data processing and data analysis; the system performs standardization processing on the composite data of user voice instructions and body movements through the instruction conversion module, and converts the standardized user voice instructions and the standardized body movement composite data into virtual reality operation instructions, so as to enhance the intelligent interaction efficiency of the virtual reality device, thereby improving the response speed of the intelligent interaction of the virtual reality device; the system performs difference correction on the instruction conversion module according to the dialect usage in the user voice instructions and the user's virtual reality device usage through the difference correction module, so as to improve the accuracy of the virtual reality operation instructions output in the instruction conversion module, thereby improving the accuracy of the intelligent interaction of the virtual reality device; the engine optimization module optimizes the difference adjustment process according to the real-time status of the user, further improves the accuracy of the virtual reality operation instructions output in the instruction conversion module, and at the same time improves the sensitivity of the intelligent interaction of the virtual reality device.

[0039] In particular, the user voice instruction recognition unit converts the user voice instruction into text information and performs noise processing on the converted user voice instruction to improve the accuracy of the user voice instruction, thereby improving the accuracy of the intelligent interaction of the virtual reality device.

[0040] In particular, the user voice command recognition unit obtains a user voice command recognition model by training the long short-term memory network model, and converts standardized user voice commands into virtual reality operation commands through the user voice command recognition model, thereby improving the accuracy of voice command recognition of the virtual reality device and further improving the accuracy of intelligent interaction of the virtual reality device.

[0041] In particular, the limb movement composite data recognition unit realizes the standardization of limb movement composite data through Kalman filtering technology, and recognizes virtual reality operation instructions according to the standard action-virtual reality operation instruction template, thereby improving the accuracy of the limb movement composite data of the virtual reality device and further improving the accuracy of the intelligent interaction of the virtual reality device.

[0042] In particular, the voice correction unit corrects the virtual reality operation instructions according to the use of dialects in the user voice instructions. When the user uses dialects as user voice instructions for multiple times, the long short-term memory network model is trained according to the type of dialect used by the user to obtain a user voice instruction recognition model that can recognize the dialect used by the user, thereby further improving the accuracy of voice command recognition of the virtual reality device.

[0043] In particular, the action correction unit judges the user's use of the virtual reality device and corrects the standard action-virtual reality operation instruction template according to the judgment result. When the user is not logging in for the first time and the number of times the same standardized body movement composite data is used is low, the standard action-virtual reality operation instruction template is corrected according to the user's free setting method. If the number of times the standardized body movement composite data is used is 0, it means that the standardized body movement composite data is invalid for the intelligent interaction of the virtual reality device and is noise data. The preset standardized body movement composite data consistent with the invalid body movement composite data and the corresponding preset virtual reality operation instruction in the standard action-virtual reality operation instruction template are deleted to improve the accuracy of the standard action-virtual reality operation instruction template, thereby further improving the accuracy of the virtual reality device voice command recognition.

[0044] In particular, the engine optimization module optimizes the difference correction module according to the real-time status of the user. When the virtual reality operation instruction triggering success rate is less than the preset virtual reality operation instruction triggering success rate, and the user's facial expression classification result is excitement, the optimization coefficient is set, and increases with the increase in the number of times the preset number of dialect usage times K0 and the preset number of usage times N0 in the difference correction module are optimized during the use cycle of the virtual reality device, thereby improving the accuracy and sensitivity of the intelligent interaction of the virtual reality device. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 This is a schematic diagram of the structure of the intelligent interactive system in the game room mode of this embodiment;

[0046] Figure 2 Schematic diagram of the structure of the instruction conversion module of this embodiment;

[0047] Figure 3 Schematic diagram of the structure of the difference correction module in this embodiment. DETAILED DESCRIPTION

[0048] In order to make the objects and advantages of the present invention more clearly understood, the present invention is further described below in conjunction with embodiments; it should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood by those skilled in the art that these embodiments are only used to explain the technical principles of the present invention and are not intended to limit the protection scope of the present invention.

[0050] It should be noted that in the description of the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be the internal communication of two components. For those skilled in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0051] See also Figure 1 As shown, it is a structural diagram of the intelligent interactive system of the game room mode of this example, and the system includes:

[0052] Interaction perception module, used to capture composite data of user voice commands and body movements;

[0053] A command conversion module is used to perform standardized processing on the composite data of user voice commands and body movements, and convert the standardized user voice commands and standardized body movement composite data into virtual reality operation commands, and the command conversion module is connected to the interactive perception module;

[0054] A difference correction module is used to perform difference correction on the instruction conversion module according to the dialect usage in the user's voice instruction and the user's virtual reality device usage, and the difference correction module is connected to the instruction conversion module;

[0055] The engine optimization module is used to optimize the difference adjustment process according to the real-time status of the user, and the engine optimization module is connected to the difference correction module.

[0056] Specifically, the system captures user voice commands and body movement composite data through an interactive perception module to improve the accuracy of subsequent data processing and data analysis; the system performs standardized processing on the user voice commands and body movement composite data through an instruction conversion module, and converts the standardized user voice commands and standardized body movement composite data into virtual reality operation commands to enhance the intelligent interaction efficiency of the virtual reality device, thereby improving the response speed of the intelligent interaction of the virtual reality device; the system performs difference correction on the instruction conversion module according to the dialect usage in the user voice commands and the user's virtual reality device usage through a difference correction module to improve the accuracy of the virtual reality operation commands output in the instruction conversion module, thereby improving the accuracy of the intelligent interaction of the virtual reality device; the engine optimization module optimizes the difference adjustment process according to the real-time status of the user, further improves the accuracy of the virtual reality operation commands output in the instruction conversion module, and at the same time improves the sensitivity of the intelligent interaction of the virtual reality device.

[0057] Specifically, when capturing user voice commands and body movement composite data, the interaction perception module captures the user voice commands through a microphone matrix in a virtual reality device worn by the user, and captures the user body movement composite data through a body movement sensor.

[0058] Specifically, the user voice command refers to the command and request issued by the user through voice, and the body movement composite data refers to a comprehensive data packet used to describe the user's actions and postures at a certain moment. This embodiment does not limit the content of the body movement composite data. Relevant technical personnel in this field can freely set it according to actual needs, as long as the needs of subsequent processing and analysis are met. For example, the body movement composite data can be set to include sensor raw data, position information, timestamp, sensor ID, motion trajectory and action recognition results. The body movement composite data refers to a comprehensive data packet generated by combining multiple sensor data and algorithm processing results, which is used to describe the user's real-time actions and real-time postures. The sensor raw data refers to the original measurement data of the sensor, such as acceleration, angular velocity and magnetic field strength. The position information refers to the position coordinates of various parts of the user's body in three-dimensional space. The timestamp refers to the time when the body movement composite data is collected. The sensor ID refers to a unique identifier used to identify different sensors. The motion trajectory refers to the motion path of the user within the collection period. The action recognition result refers to the action or posture identified based on deep learning and rule algorithms, such as gestures, walking and running. The microphone matrix refers to a system composed of multiple microphones arranged in a certain layout. The body movement sensor refers to a sensor used to capture and measure human body movements and postures.

[0059] See also Figure 2 As shown, it is a schematic diagram of the structure of the instruction conversion module of this embodiment, and the instruction conversion module includes:

[0060] A user voice command recognition unit, used to perform standardization processing on the user voice command to obtain a standardized user voice command, and also used to convert the standardized user voice command into a virtual reality operation command;

[0061] The body movement composite data recognition unit is used to perform standardization processing on the body movement composite data to obtain standardized body movement composite data, and is also used to convert the standardized body movement composite data into virtual reality operation instructions. The body movement composite data recognition unit is connected to the user voice instruction recognition unit.

[0062] Specifically, when the user voice instruction recognition unit performs standardization processing on the user voice instruction, the user voice instruction is converted into text information through a voice recognition algorithm, and noise processing is performed on the converted user voice instruction to obtain a standardized user voice instruction.

[0063] Specifically, the speech recognition algorithm refers to the technology and tools for converting user voice commands into text, such as Microsoft Azure voice service technology. The noise processing refers to the process of identifying, reducing or eliminating useless signal components in audio signals, i.e., noise. This embodiment does not limit the specific method of noise processing. Relevant technical personnel in this field can freely set it according to actual needs, and only need to meet the requirements of standardized processing of user voice commands. For example, the noise processing method can be set to perform noise processing on the converted user voice commands through an audio editing system. The audio editing tool refers to a tool used for audio editing, mixing and repair, such as the Aizetop audio repair tool.

[0064] Specifically, when converting standardized user voice commands into virtual reality operation commands, the user voice command recognition unit trains the long short-term memory network model by using a preset standardized user voice command data set, outputs the long short-term memory network model that meets the preset accuracy rate as a user voice command recognition model, and converts the standardized user voice commands into virtual reality operation commands through the user voice command recognition model.

[0065] Specifically, the user voice command recognition model refers to a model that meets the preset accuracy rate obtained by training the long short-term memory network model according to a preset standardized user voice command data set. The long short-term memory network model refers to a time recurrent neural network used to solve the gradient vanishing or gradient exploding problems encountered when processing long sequences. The preset standardized user voice command data set refers to a standardized user voice command data set preset for training the long short-term memory network model in the form of standardized user voice commands-virtual reality operation commands. The virtual reality operation commands refer to commands used by users when interacting in a virtual environment. The recurrent neural network refers to an artificial neural network used to process and predict sequence data. This embodiment does not limit the way in which the long short-term memory network model is trained. Those skilled in the art can set it freely, as long as the user voice commands are converted into virtual reality operation commands. The user voice instruction recognition model can be used as the recognition model of the user voice instruction recognition system, for example, 75% of the preset standardized user voice instruction data set can be divided into a user voice instruction training set, and 25% can be divided into a user voice instruction test set. The user voice instruction training set is input into the long short-term memory network model for training, and the user voice instruction test set is input into the trained long short-term memory network model. The parameters in the long short-term memory network model are optimized and iterated until the accuracy of the output result of the user voice instruction test set of the long short-term memory network model reaches 95%. The long short-term memory network model is output as a user voice instruction recognition model. The preset accuracy refers to a preset value of the accuracy reflecting the training status of the long short-term memory network model. This embodiment does not limit the value of the preset accuracy, and relevant technical personnel in the field can set it freely, as long as the requirement of reflecting the training status of the long short-term memory network model is met. For example, the preset accuracy can be set to 95%.

[0066] Specifically, the body movement composite data recognition unit removes noise from the body movement composite data through Kalman filtering technology, and converts the noise-removed body movement composite data into standardized body movement composite data, and compares the standardized body movement composite data with the standard action-virtual reality operation instruction template, and outputs the virtual reality operation instruction according to the comparison result, wherein:

[0067] When there is preset standardized limb action composite data consistent with the standardized limb action composite data in the standard action-virtual reality operation instruction template, the preset virtual reality operation instruction corresponding to the preset standardized limb action composite data is output as the virtual reality operation instruction;

[0068] When there is no preset standardized limb movement compound data consistent with the standardized limb movement compound data in the standard action-virtual reality operation instruction template, the virtual reality operation instruction is not output.

[0069] Specifically, the Kalman filtering technology refers to a recursive estimation technology used to infer the state of a dynamic system in noisy observation data, the noise refers to signals and data that interfere with data processing, the standard action-virtual reality operation instruction template refers to a predefined data template for comparison with real-time standardized limb action composite data to identify the corresponding virtual reality operation instruction, including preset standardized limb action composite data and preset virtual reality operation instructions, the preset standardized limb action composite data refers to preset standardized limb action composite data for identifying virtual reality operation instructions, the preset virtual reality operation instruction refers to preset virtual reality operation instructions corresponding to the preset standardized limb action composite data, this embodiment does not limit the conversion method of converting the limb action composite data after noise removal into the standardized limb action composite data, relevant technical personnel in this field can freely set it according to actual needs, and only need to meet the need of converting the limb action composite data after noise removal into the standardized limb action composite data, such as setting the conversion method to a physical conversion method, the physical conversion method refers to a method of converting a physical quantity that is difficult to directly measure into another physical quantity that is easy to measure and calculate through a mathematical formula.

[0070] See also Figure 3 As shown, it is a schematic diagram of the structure of the difference correction module of this embodiment, and the difference correction module includes:

[0071] A voice correction unit, used to correct the virtual reality operation instructions according to the dialect usage in the user's voice instructions;

[0072] The action correction unit is used to correct the virtual reality operation instructions according to the user's virtual reality device usage, and the action correction unit is connected to the voice correction unit.

[0073] Specifically, when the voice correction unit corrects the virtual reality operation instruction according to the dialect usage in the user voice instruction, the user uses the dialect number K and the preset dialect usage number K0 are compared, and the effectiveness of the process of converting the standardized user voice instruction into the virtual reality operation instruction is judged according to the comparison result, and the process of converting the standardized user voice instruction into the virtual reality operation instruction is corrected according to the judgment result, wherein:

[0074] When K≥K0, it is determined that the process of converting the standardized user voice command into the virtual reality operation command is effective, the voice correction unit determines to correct the process of converting the user voice command into the virtual reality operation command, and obtains the dialect voice data used by the user, adds the dialect voice data to the preset standardized user voice command data set, obtains the adjusted preset standardized user voice command data set, and trains the long short-term memory network model according to the adjusted preset standardized user voice command data set;

[0075] When K<K0, the voice correction unit determines not to correct the process of converting the user voice command into the virtual reality operation command.

[0076] Specifically, the number of times the user uses the dialect K refers to the number of times the user uses the dialect as the user voice command within the collection period. This embodiment does not limit the collection method of the number of times the user uses the dialect. Relevant technical personnel in this field can freely set it according to actual needs, and only need to meet the need to judge the effectiveness of the process of converting the standardized user voice command into the virtual reality operation command. For example, it can be set to recognize the user's voice command through voice recognition technology, obtain the number of times the dialect appears in the user's voice command, and output the number of times the dialect appears as the number of times the user uses the dialect K. The preset number of times the dialect is used K0 refers to the preset number of times the user uses the dialect to reflect the situation of the user using the dialect. This embodiment does not set the preset number of times the dialect is used. Assuming that the value of the number of times the dialect is used K0 is limited, relevant technical personnel in the field can freely set it according to actual needs, and it only needs to meet the needs of reflecting the situation of users using dialects. For example, the preset number of times the dialect is used K0=3 times can be set. The dialect voice data refers to the voice recording and collected data set using various dialects, such as dialects from various parts of China, various languages ​​​​in India, and regional languages ​​​​in Europe. This embodiment does not limit the collection method of dialect voice data. Relevant technical personnel in the field can freely set it according to actual needs, and it only needs to meet the needs of adjusting the process of converting user voice commands into virtual reality operation commands. For example, the collection method can be set to collect dialect voice data through big data.

[0077] Specifically, when the action correction unit corrects the virtual reality operation instruction according to the user's virtual reality device usage, it obtains the user's login status, judges the user's use of the virtual reality device according to the user's login status, and corrects the standard action-virtual reality operation instruction template according to the judgment result, wherein:

[0078] When the user login status is visitor login, the action correction unit determines that the user is using the virtual device for the first time and does not correct the standard action-virtual reality operation instruction template;

[0079] When the user login status is to use an account to log in, obtain the registration time B of the account used by the user, compare the registration time B of the account used by the user with the preset registration time B0, judge the user's use of the virtual reality device according to the comparison result, and modify the standard action-virtual reality operation instruction template according to the judgment result, wherein:

[0080] When B<B0, the action correction unit determines that the user is using the virtual device for the first time, and does not correct the standard action-virtual reality operation instruction template;

[0081] When B≥B0, the action correction unit determines that the user is not using the virtual device for the first time, and corrects the standard action-virtual reality operation instruction template.

[0082] Specifically, the visitor login refers to the user using the virtual reality device without providing any personal identity information and registering an account. The account login refers to the user using the virtual reality device by entering a registered account and password. The registration time refers to the time interval from the time when the user creates an account in the virtual reality device to the current use of the virtual reality device using the account and password. For example, the user creates account 1 in the virtual reality device at 12:00 on June 20, 2024, and uses account 1 to use the virtual reality device at 12:00 on June 28, 2024, then the registration time B=8. The preset registration time refers to a preset time interval used to reflect the user's use of the virtual reality device. This embodiment does not limit the method for obtaining the user's login status. Relevant technicians in this field can freely set it according to actual needs, as long as the need to judge the user's use of the virtual reality device is met. For example, it can be set to obtain the user's login status through the virtual reality device log record. The virtual reality device log record refers to a data set of events recorded by the system during the operation of the virtual reality device and the user's use.

[0083] Specifically, when the action correction unit corrects the virtual reality operation instruction, it obtains the number of times N of use of the standardized limb action composite data, and compares the number of times N of use of each standardized limb action composite data with the preset number of times N0, judges the use frequency of the standardized limb action composite data according to the comparison result, and corrects the standard action-virtual reality operation instruction template according to the judgment result, wherein:

[0084] When N≥N0, the standardized body movement composite data is determined to be high-frequency standardized body movement composite data, and the standard action-virtual reality operation instruction template is not corrected;

[0085] When 0<N<N0, it is determined that the standardized body movement composite data is low-frequency standardized body movement composite data, and a standard movement-virtual reality operation instruction template correction window is pushed to the user, and the user fills in the correspondence between the preset standardized body movement and the preset virtual reality operation instruction, and the movement correction unit obtains the data of the correspondence between the preset standardized body movement and the preset virtual reality operation instruction in the correction window, and adds the data of the correspondence between the preset standardized body movement and the preset virtual reality operation instruction in the correction window to the standard movement-virtual reality operation instruction template;

[0086] When N=0, the limb movement compound data is determined to be invalid standardized limb movement compound data, and the preset standardized limb movement compound data and its corresponding preset virtual reality operation instruction consistent with the invalid historical limb movement compound data in the standard action-virtual reality operation instruction template are deleted.

[0087] Specifically, the number of times N that the standardized limb movement composite data is used refers to the number of times the user uses the same standardized limb movement composite data within the collection period, and the preset number of times N0 refers to the preset number of times used to reflect the frequency of use of the standardized limb movement composite data. This embodiment does not limit the value of the preset number of times N0, and relevant technical personnel in the field can freely set it according to actual needs, as long as the requirement of reflecting the frequency of use of the standardized limb movement composite data is met. For example, the preset number of times N0 can be set to 10 times. The collection period refers to the preset time interval for obtaining the frequency of use of the standardized limb movement composite data and the use of the dialect in the user's voice instructions when adjusting the virtual reality operation instructions. For example, if the collection period is set to one week, the limb movement composite data one week ago and the use of the dialect in the user's voice instructions are obtained.

[0088] Specifically, when the engine optimization module optimizes the difference correction module according to the real-time status of the user, the virtual reality operation instruction trigger success rate T is collected in real time, and the virtual reality operation instruction trigger success rate T is compared with the preset virtual reality operation instruction trigger success rate T0, the real-time status of the user is judged according to the comparison result, and the difference correction module is optimized according to the judgment result, wherein:

[0089] When T≥T0, the engine optimization module determines that the user's real-time status is normal and does not optimize the difference correction module;

[0090] When T<T0, the engine optimization module determines that the user's real-time state is abnormal, obtains the user's facial expression image, classifies the user's facial expression image through the facial expression classification model, obtains the user's facial expression classification result, sets the user's facial expression classification result to be excitement and calmness, and optimizes the preset number of times K0 and the preset number of times N0 used in the difference correction module according to the user's facial expression classification result, wherein:

[0091] When the user's facial expression classification result is calm, the preset number of times the dialect is used K0 and the preset number of times N0 in the difference correction module are not optimized;

[0092] When the user's facial expression classification result is excitement, the preset dialect usage times K0 and the preset usage times N0 are optimized, and an optimization coefficient U is set, wherein U=Z+1, and Z is the number of times the preset dialect usage times K0 and the preset usage times N0 in the difference correction module are optimized during the use cycle of the virtual reality device. According to the optimization coefficient U, the preset usage times N0 and the preset dialect usage times K0 are optimized, and the optimized preset usage times are set to Nu0, and the optimized preset dialect usage times are set to Ku0, wherein Nu0=N0-U, Ku0=K0-U, Nu0≥1, and Ku0≥1.

[0093] Specifically, the virtual reality operation instruction trigger success rate T refers to the ratio of the number of times E that a user attempts to trigger an operation instruction and successfully executes the instruction to the total number of attempts E0 in the virtual reality environment, that is, T=E / E0. The preset virtual reality operation instruction trigger success rate T0 refers to a preset value of the virtual reality operation instruction trigger success rate that reflects the actual operation of the virtual reality device by the user. This embodiment does not limit the value of the preset virtual reality operation instruction trigger success rate T0. Relevant technical personnel in this field can freely set it according to actual needs, and only need to meet the needs of reflecting the actual operation of the virtual reality device by the user. For example, the preset virtual reality operation instruction trigger success rate T0 can be set to 0.85. The user facial expression image refers to a facial image that reflects the changes in the facial expression of the user under different emotional states. The facial expression classification model refers to a support vector machine model that is trained according to a preset user facial expression image-user facial expression classification result data set and meets a preset classification accuracy. The support vector machine model refers to a supervised learning model used for classification and regression analysis. This embodiment does not limit the training method of the support vector machine model. Relevant technical personnel in this field can freely set it according to actual needs. Actual needs can be freely set, and only the need to obtain the user facial expression classification result needs to be met. For example, the training method of the support vector machine model can be set to divide the preset user facial expression image-user facial expression classification result data set into 80% training set and 20% verification set, input the training set into the support vector machine model for training, and input the verification set into the trained support vector machine model, optimize the structure and parameters of the support vector machine, until the accuracy of the output result of the support vector machine model test set reaches 95%, and then output the support vector machine model as a facial expression classification model. The preset user facial expression image-user facial expression classification result data set refers to a preset data set used to reflect the corresponding relationship between the user facial expression image and the user facial expression classification result, for example, user facial expression image 1 corresponds to excitement in the user facial expression classification result, user facial expression image 2 corresponds to excitement in the user facial expression classification result... user facial expression image n corresponds to calmness in the user facial expression classification result, and the virtual reality device usage cycle refers to the time interval from the first time the user uses the virtual reality device to the current time when the user uses the virtual reality device.

[0094] So far, the technical solutions of the present invention have been described in conjunction with the preferred embodiments shown in the accompanying drawings. However, it is easy for those skilled in the art to understand that the protection scope of the present invention is obviously not limited to these specific embodiments. Without departing from the principle of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present invention.

Claims

1. An intelligent interactive system in game room mode, characterized in that: include: Interaction perception module, used to capture composite data of user voice commands and body movements; A command conversion module, used to perform standardized processing on the composite data of user voice commands and body movements, and convert the standardized user voice commands and standardized body movement composite data into virtual reality operation commands; A difference correction module is used to make difference corrections to the instruction conversion module according to the dialect usage in the user's voice instruction and the user's virtual reality device usage; Engine optimization module, used to optimize the differential adjustment process according to the real-time status of users; The difference correction module includes: A voice correction unit, used to correct the virtual reality operation instructions according to the dialect usage in the user's voice instructions; An action correction unit, used to correct the virtual reality operation instructions according to the user's virtual reality device usage; When the voice correction unit corrects the virtual reality operation instruction according to the dialect usage in the user voice instruction, the user uses the dialect times K and the preset dialect usage times K0 are compared, and the validity of the process of converting the standardized user voice instruction into the virtual reality operation instruction is judged according to the comparison result, and the process of converting the standardized user voice instruction into the virtual reality operation instruction is corrected according to the judgment result, wherein: When K≥K0, it is determined that the process of converting the standardized user voice command into the virtual reality operation command is effective, the voice correction unit determines to correct the process of converting the user voice command into the virtual reality operation command, and obtains the dialect voice data used by the user, adds the dialect voice data to the preset standardized user voice command data set, obtains the adjusted preset standardized user voice command data set, and trains the long short-term memory network model according to the adjusted preset standardized user voice command data set; When K<K0, the voice correction unit determines not to correct the process of converting the user voice command into the virtual reality operation command.

2. The intelligent interactive system of the game room mode according to claim 1, characterized in that: The instruction conversion module comprises: A user voice command recognition unit, used to perform standardization processing on the user voice command to obtain a standardized user voice command, and also used to convert the standardized user voice command into a virtual reality operation command; The body movement composite data recognition unit is used to perform standardization processing on the body movement composite data to obtain standardized body movement composite data, and is also used to convert the standardized body movement composite data into virtual reality operation instructions.

3. The intelligent interactive system of the game room mode according to claim 2, characterized in that: When the user voice instruction recognition unit performs standardization processing on the user voice instruction, the user voice instruction is converted into text information through a voice recognition algorithm, and noise processing is performed on the converted user voice instruction to obtain a standardized user voice instruction.

4. The intelligent interactive system of the game room mode according to claim 2, characterized in that: When converting the standardized user voice commands into virtual reality operation commands, the user voice command recognition unit trains the long short-term memory network model by using a preset standardized user voice command data set, outputs the long short-term memory network model that meets the preset accuracy as a user voice command recognition model, and converts the standardized user voice commands into virtual reality operation commands through the user voice command recognition model.

5. The intelligent interactive system of the game room mode according to claim 2, characterized in that: The body movement composite data recognition unit removes noise from the body movement composite data by using Kalman filtering technology, and converts the noise-removed body movement composite data into standardized body movement composite data, and compares the standardized body movement composite data with the standard action-virtual reality operation instruction template, and outputs the virtual reality operation instruction according to the comparison result, wherein: When there is preset standardized limb action composite data consistent with the standardized limb action composite data in the standard action-virtual reality operation instruction template, the preset virtual reality operation instruction corresponding to the preset standardized limb action composite data is output as the virtual reality operation instruction; When there is no preset standardized limb movement compound data consistent with the standardized limb movement compound data in the standard action-virtual reality operation instruction template, the virtual reality operation instruction is not output.

6. The intelligent interactive system of the game room mode according to claim 1, characterized in that: When the action correction unit corrects the virtual reality operation instruction according to the user's virtual reality device usage, it obtains the user's login status, judges the user's use of the virtual reality device according to the user's login status, and corrects the standard action-virtual reality operation instruction template according to the judgment result, wherein: When the user login status is visitor login, the action correction unit determines that the user is using the virtual device for the first time and does not correct the standard action-virtual reality operation instruction template; When the user login status is to use an account to log in, obtain the registration time B of the account used by the user, compare the registration time B of the account used by the user with the preset registration time B0, judge the user's use of the virtual reality device according to the comparison result, and modify the standard action-virtual reality operation instruction template according to the judgment result, wherein: When B<B0, the action correction unit determines that the user is using the virtual device for the first time, and does not correct the standard action-virtual reality operation instruction template; When B≥B0, the action correction unit determines that the user is not using the virtual device for the first time, and corrects the standard action-virtual reality operation instruction template.

7. The intelligent interactive system of the game room mode according to claim 6, characterized in that: When the action correction unit corrects the virtual reality operation instruction, it obtains the number of times N of using the standardized body movement composite data, and compares the number of times N of using each standardized body movement composite data with the preset number of times N0, judges the use frequency of the standardized body movement composite data according to the comparison result, and corrects the standard action-virtual reality operation instruction template according to the judgment result, wherein: When N≥N0, the standardized body movement composite data is determined to be high-frequency standardized body movement composite data, and the standard action-virtual reality operation instruction template is not corrected; When 0<N<N0, it is determined that the standardized body movement composite data is low-frequency standardized body movement composite data, and a standard movement-virtual reality operation instruction template correction window is pushed to the user, and the user fills in the correspondence between the preset standardized body movement and the preset virtual reality operation instruction, and the movement correction unit obtains the data of the correspondence between the preset standardized body movement and the preset virtual reality operation instruction in the correction window, and adds the data of the correspondence between the preset standardized body movement and the preset virtual reality operation instruction in the correction window to the standard movement-virtual reality operation instruction template; When N=0, the standardized limb movement compound data is determined to be invalid standardized limb movement compound data, and the preset standardized limb movement compound data and its corresponding preset virtual reality operation instruction that are consistent with the standardized limb movement compound data in the standard action-virtual reality operation instruction template are deleted.

8. The intelligent interactive system of the game room mode according to claim 1, characterized in that: When the engine optimization module optimizes the difference correction module according to the real-time status of the user, the virtual reality operation instruction trigger success rate T is collected in real time, and the virtual reality operation instruction trigger success rate T is compared with the preset virtual reality operation instruction trigger success rate T0, the real-time status of the user is judged according to the comparison result, and the difference correction module is optimized according to the judgment result, wherein: When T≥T0, the engine optimization module determines that the user's real-time status is normal and does not optimize the difference correction module; When T<T0, the engine optimization module determines that the user's real-time state is abnormal, obtains the user's facial expression image, classifies the user's facial expression image through the facial expression classification model, obtains the user's facial expression classification result, sets the user's facial expression classification result to be excitement and calmness, and optimizes the preset number of times K0 and the preset number of times N0 used in the difference correction module according to the user's facial expression classification result, wherein: When the user's facial expression classification result is calm, the preset number of times the dialect is used K0 and the preset number of times N0 in the difference correction module are not optimized; When the classification result of the user's facial expression is excitement, the preset number of times the dialect is used K0 and the preset number of times N0 are optimized, and an optimization coefficient U is set, where U=Z+1, and Z is the number of times the preset number of times the dialect is used K0 and the preset number of times N0 are optimized in the difference correction module during the use cycle of the virtual reality device. According to the optimization coefficient U, the preset number of times N0 and the preset number of times the dialect is used K0 are optimized, and the optimized preset number of times is set to Nu0, and the optimized preset number of times the dialect is used is Ku0, where Nu0=N0-U, Ku0=K0-U, Nu0≥1, and Ku0≥1.

Images