327results about How to "Improve naturalness" patented technology

A high-expressive-force audio generation method based on a natural language description text comprises the steps that S1, a pre-training multi-language text encoder is used for encoding an input natural language description instruction, and semantic information is extracted; s2, with text coding representation as a condition, semantic unit sequences of the audio are sequentially predicted through an autoregression model (AR), and the semantic unit sequences represent content event information appearing in the audio; s3, taking the text coding representation and the audio semantic unit sequence output in the S2 as a prefix condition, predicting a coarse-grained acoustic unit sequence of the audio through AR, and determining the overall duration and basic acoustic information of the audio; s4, predicting a fine-grained acoustic unit sequence of the audio by using a non-autoregressive model (NAR), and modeling audio detail representation and high-frequency information; and S5, inputting the multi-layer acoustic unit sequence into a vocoder to generate a high-quality audio waveform. The method directly supports the generation of the audio by a sentence of natural language description instruction, and improves the naturalness, expressive force and controllability of the generated audio.

The invention provides a natural language control method for a humanoid robot. By constructing the simple skill library and the complex skill library of the robot, the dependence on a large amount of operation data in a specific field is effectively reduced. According to the method, a large language model is used as a base of a task planner, and the capabilities of scene learning, semantic understanding and common sense reasoning of the task planner are brought into full play. By introducing a visual model in the field of computer vision, high-level semantic information and low-level spatial information of the environment can be obtained, and an accurate scene graph is constructed to serve as environment state representation. According to the voice interaction interface constructed by the invention, the naturalness of man-machine interaction is improved, so that a user can more conveniently communicate with the robot and input instructions, the user experience is effectively improved, and popularization of the humanoid robot in an actual application scene is facilitated.

The invention relates to a character-driven lip sound synchronous digital human generation method and device, equipment and a medium, and the method comprises the steps: obtaining an input text and a single object video, and carrying out the preprocessing of the input text, and obtaining a phoneme sequence and semantic features; the method comprises the following steps: preprocessing a single-object video to obtain a circularly played video paragraph, a mask face image and a reference face image; generating voice features based on the phoneme sequence and the semantic features; generating an audio sequence based on the voice features; acquiring face deformation parameters according to the mask face image, the reference face image and the voice features, and deforming the reference face image to generate a target face image; and embedding the target face image into a video frame of the video paragraph to generate a video sequence, and splicing the video sequence with the audio sequence to obtain a target digital human video. The digital human video is generated based on text driving, so that the voice and lip-shaped video matched with the text can be generated in time, the time delay for waiting for text to be converted into voice processing is eliminated, and the digital human video production efficiency is greatly improved.

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses an emotional speech conversion method based on rhythm prediction, which comprises the following steps: extracting phoneme sequences, acoustic features and emotional dimension representations in input text data and source audio data, and extracting language information by using a text encoder; frame-level rhythm changes and textual rhythm representations are generated based on the language information and the sentiment dimension representations. The method comprises the following steps: generating acoustic rhythm representation in combination with speaker characteristics and emotional dimension representation, calculating alignment loss by aligning text rhythm representation and acoustic rhythm representation, jointly training a rhythm prediction module and an acoustic modeling module to generate comprehensive rhythm representation, and finally generating converted voice waveforms and evaluating the converted voice waveforms through a discriminator. And optimizing each module parameter based on the comprehensive loss, and generating a target emotional voice. According to the method, accurate control of rhythm fine granularity is realized through implicit modeling, and the problem of mismatching during training and operation is avoided, so that the naturalness and emotion richness of voice are greatly improved.

The invention relates to the technical field of computer television image communication, in particular to a computer television all-in-one machine control method, system and device. The method comprises the following steps: through multi-view image acquisition and processing, generating a three-dimensional scene grid and carrying out scene element analysis to form a scene element set, then chasing limbs of a user, generating a user limb image set, restoring an environment basic tone, obtaining display content of the all-in-one machine and carrying out content semantic analysis, and finally obtaining a user limb image set. Calculating interaction intensity to obtain content interaction intensity data, speculating a user body language and performing user intention prediction in combination with the interaction intensity data, adjusting display content of the all-in-one machine based on the predicted user operation intention, generating simulation adjustment content, performing composition optimization through a scene element set, and generating updated display content. The method for controlling the computer and television all-in-one machine is more accurate and more efficient.

The invention relates to the technical field of artificial intelligence, and discloses a speech synthesis method based on hierarchical emotional distribution, which comprises the following steps: acquiring audio data and text transcription, segmenting audio into phoneme, word and sentence-level speech units, extracting acoustic characteristics such as pitch, speed and rhythm, generating an emotional distribution vector by using a pre-training model; the method comprises the following steps: processing text transcription to generate text embedded information, combining an emotion distribution vector, the text embedded information and acoustic features, generating rhythm features through a variable converter, generating a Mel-frequency spectrogram by a decoder based on the rhythm features, and converting the Mel-frequency spectrogram into a time domain audio signal by a vocoder and outputting the time domain audio signal. According to the method, the audio data are hierarchically segmented into phonemes, words and sentences, the acoustic features, the emotion distribution vectors and the text embedding information are combined, and the rhythm features are generated through the variable converter, so that fine control over emotion expression is achieved, and the naturalness and emotion expressive force of speech synthesis are remarkably improved.

The invention relates to the technical field of television image communication, in particular to a user information input method and system in an intelligent television all-in-one machine. The method comprises the following steps: collecting a user image, carrying out white balance correction and focusing processing, generating a moving object focusing image, carrying out identification and attention tendency analysis on a character in the focusing image, obtaining character tendency data, analyzing a historical interaction image, identifying an interaction mode and constructing a user intention prediction model, and obtaining a user intention prediction result. And when the tendency data shows that the user preferences the television content, intention drilling and auxiliary adjustment are performed on the current image based on the prediction model, predicted user input information is generated, logic conflicts in the predicted information are detected, feedback optimization is performed, and final user input information is output. According to the invention, a more accurate and more flexible user information input method is realized.

The embodiment of the invention provides a character animation generation method and device based on voice driving, equipment and a medium, and belongs to the technical field of financial science and technology. The method comprises the following steps: acquiring an audio signal input by a user, a target human portrait picture and a target scene reference video; encoding the audio signal to obtain an audio code; generating a three-dimensional face model motion coefficient according to the portrait picture of the target person and the audio code; extracting blink information and head posture information from the target scene reference video; correcting the motion coefficient of the three-dimensional face model according to the blinking information and the head posture information to obtain a corrected motion coefficient of the three-dimensional face model; and taking a three-dimensional deformable head model as a potential vector, mapping the three-dimensional deformable head model to the corrected three-dimensional face model motion coefficient frame by frame, converting the three-dimensional face model motion coefficient into a video frame, and generating a head speech video of the target person. According to the embodiment of the invention, the quality and reality of character animation generation driven by voice can be improved.

The invention provides a digital human control method and device based on multiple modes and electronic equipment, and relates to the technical field of data processing. In the method, voice data, facial expression data and body movement data of a user are acquired, and the user interacts with a digital human; performing semantic analysis on the voice data by adopting a Transform model to obtain an emotional state and a language habit corresponding to the user; performing feature extraction on the facial expression data by adopting a convolutional neural network model to obtain a facial expression type corresponding to the user; performing action analysis on the limb action data by adopting a hidden Markov model to obtain effective action features corresponding to the user; generating a fusion result based on the emotional state, the language habit, the facial expression type and the effective action features; and adjusting the multi-modal output corresponding to the digital human according to the fusion result. By implementing the technical scheme provided by the invention, the interaction experience feeling of the user can be improved.

The invention provides an intention recognition method and device, electronic equipment, a storage medium and a program product, and relates to the technical field of artificial intelligence. The method comprises the steps that a question text of a user is input into a small model, the confidence degree of at least one first user intention output by the small model is determined, and if the confidence degree of a target user intention is larger than a set confidence degree threshold value, the target user intention is recognized; if yes, determining an intention transfer score for transferring the second user intention of the last round of dialogue to the target user intention; judging whether the intention transfer score is greater than a set intention transfer score threshold, and if yes, outputting a target user intention; and if not, inputting the question text into the large model, and outputting a third user intention predicted by the large model. According to the method, secondary prediction is performed by introducing a large model, so that the intention recognition accuracy is improved; and through an intention transfer score threshold judgment mechanism, the context understanding capability of the system is enhanced, the naturalness and consistency of dialogues are improved, and the system can process multiple rounds of interaction of the user more intelligently.

The invention relates to a digital human interaction method and system based on multi-modal understanding, and belongs to the technical field of digital human intelligent interaction. The method comprises the following steps: acquiring multi-modal data, and obtaining multi-modal features through a feature extraction model according to the multi-modal data; obtaining enhanced modal features through a knowledge enhancement model according to the multi-modal features; according to the enhanced modal features, a modal fusion feature vector is obtained through a graph representation learning model; and the big language model learns modal fusion feature vectors through a multi-head attention mechanism to obtain natural language output. The digital human understanding of multi-modal data is realized, and the smoothness, naturalness and intellectualization in the digital human interaction process are improved.

The embodiment of the invention provides a video generation method and device based on a diffusion model, equipment and a medium, and relates to the technical field of video generation, and the method comprises the steps: firstly, splitting a training video containing a face into a plurality of video frames according to the frame number, selecting a first video frame containing the face as a condition picture, and carrying out the segmentation of the condition picture; inputting the plurality of video frames into a variational auto-encoder, adding Gaussian noise to obtain noisy feature representation, encoding a condition picture by using a CLIP picture encoder to obtain picture embedding representation, and inputting the noisy feature representation and the picture embedding representation into a de-noising U-Net for splicing to obtain a first video, and inputting the first video into the diffusion model to obtain the second video, and inputting the picture into the diffusion model to realize the purpose of generating a high-quality and long-period video from a single picture, and improve the naturalness of face generation and the fluency of video actions.

The invention relates to the technical field of artificial intelligence digital persons, and particularly discloses an interactive digital person generation method and system based on artificial intelligence, and the method comprises the steps: collecting behavior data, voice data and image data of a real person; constructing a model for generating a digital human according to the collected data; a deep learning algorithm is adopted to analyze the model, initial digital human behavior characteristics, voice characteristics and appearance characteristics are generated, the initial digital human behavior characteristics, voice characteristics and appearance characteristics are input into a generative adversarial network (GAN), the behavior logic, voice logic and appearance logic of the digital human are optimized through the iterative training and optimization process of a generator and a discriminator, and the digital human behavior logic, voice logic and appearance logic are obtained. And finally, the interactive digital human with high authenticity is generated, the high authenticity of the digital human in behavior, appearance and sound is ensured, a personalized digital human image conforming to the expectation of the user is created, the user experience is further improved, the digital human image with high authenticity is generated, and the naturalness and interactivity of communication are enhanced.

The invention relates to voice processing, and provides a voice conversion method and device, equipment and a storage medium, and the method comprises the steps: carrying out the data enhancement of a to-be-converted voice, and obtaining a sample voice; inputting the sample voice into a rhythm encoder for processing to obtain phoneme-level rhythm features and spectrum features of the sample voice; inputting the sample voice into a content encoder for processing to obtain content features of the sample voice; inputting the rhythm features and the content features into an attention-based alignment model for alignment processing to obtain aligned rhythm features; inputting the aligned rhythm features, content features, frequency spectrum features and speaker vectors with set timbres into a decoder for synthesis to obtain synthesized voice; and intercepting part of content corresponding to the voice to be converted from the synthesized voice as target voice. The invention also relates to a block chain, and aims to improve the precision and naturalness of voice conversion.

The invention discloses a facial expression recognition method based on an intelligent accompanying robot, relates to the technical field of intelligent recognition, and aims to recognize mask-type expressions (namely expressions inconsistent with real emotions) by analyzing emotional contradiction degree Qmd, effectively avoid the problem of emotion misjudgment, and improve the recognition efficiency especially when facing complex or camouflage emotions. The robot can distinguish the real state of the emotion and adjust the feedback intensity according to the information, so that the emotion demand of the accompanying object can be accurately perceived and properly responded. According to the dynamic feedback adjustment mechanism, a feedback response intensity value Fqdz is constructed, the feedback intensity is dynamically adjusted according to the emotional contradiction degree, and the interaction intensity of the robot can be accurately adjusted according to emotional states in different situations. By combining initial setting and historical data, the intelligent accompanying robot can optimize the behavior mode of the intelligent accompanying robot according to long-term interaction feedback.

The invention provides a mouth shape generation method and device for voice driving, and relates to the technical field of natural language processing. The method comprises the following steps: based on an ffmpeg tool, carrying out image processing according to original video data to obtain spliced frame image data and facial feature points; according to the original audio data, performing feature extraction through a deep audio feature extractor to obtain audio features; performing feature fusion through an audio-video sequence feature fusion device to obtain fusion features; performing video generation through a lip action generator according to the facial feature points and the fusion features to obtain synthesized video data; optimizing a lip action generator according to the original video data and the synthesized video data; and performing video generation based on the depth audio feature extractor, the audio-video sequence feature fusion device and the optimized lip action generator to obtain target synthesized video data. The invention relates to a mouth shape generation method aiming at voice driving, which is high in video resolution and sufficient in facial texture detail reservation.

The invention relates to the technical field of artificial intelligence and the field of financial science and technology, and discloses a multi-module collaborative voice generation method, which comprises the following steps: acquiring a reference text and reference voice, converting the reference text into a phoneme sequence and generating potential features, converting the reference voice into a Mel spectrogram and extracting style features, and extracting fundamental frequency information. A phoneme sequence and a Mel spectrogram are input into a text alignment module to generate alignment information, voice duration is predicted based on the alignment information and potential features, fundamental frequency of voice is predicted through fundamental frequency information, the predicted duration, fundamental frequency and style features are input into a decoder to generate a reconstructed Mel spectrogram, and finally the reconstructed Mel spectrogram is converted into voice audio data through a vocoder. According to the method, the generated voice can better reflect the natural rhythm and tone change of the text content by combining accurate control on the voice duration, the fundamental frequency and the style characteristics, and the rhythm and expressive force of the voice are enhanced, so that the naturalness and the fidelity of voice synthesis are improved.

The invention discloses a virtual digital human interactive question and answer method, system and device and a storage medium, and relates to the field of virtual reality. The method comprises the following steps: carrying out first processing on dialogue content of a user, and carrying out second processing on a facial image of the user to obtain a first emotional state; performing third processing on the text after the first processing to obtain a second emotional state of the user, and determining a target emotional state in combination with the first emotional state and the second emotional state; performing fourth processing on the text after the first processing based on a deep learning model to generate a first answer, and adjusting an expression mode of the first answer according to the target emotional state and the dialogue context to generate a second answer; and constructing a virtual digital person by using a virtual reality technology, and determining the facial expression, the body movement and the voice intonation of the virtual digital person according to the emotional tendency in the second answer to display the second answer. By implementing the technical scheme provided by the invention, the emotional state of the user can be identified more accurately, and then the answer more conforming to the emotion of the user is generated.

The invention discloses a diffusion model self-guiding-based image or video generation method and system. The method comprises the steps that the input content is a noise potential variable and a future offset step length of a current time step; the diffusion model predicts the score network twice based on the input content to obtain a conditional score network of the current time step and the future offset time step; carrying out linear combination on the two conditional score networks to generate a combined conditional score network; denoising the noise potential variable of the current time step by using a combined condition score network; calculating and updating a potential variable of the previous time step according to a denoising result; in each time step, the self-guiding strategy is adopted to carry out denoising processing on the noise, and noise prediction is optimized; and after de-noising processing of all time steps, a de-noising potential variable is finally output as a final model generation result, and an image or a video is obtained. According to the invention, the quality of the generated image or video can be obviously improved.

The invention provides an intelligent government affair question-answering method and an intelligent government affair question-answering device for enhancing a large model by combining speech recognition with a database, and relates to the technical field of intelligent government affair question-answering, comprising the following steps: S1, a speech recognition system is responsible for accurately converting a user speech instruction into a written text as a primary link of information conversion; and S2, the user intention analysis is used for accurately converting a user voice instruction into a written text, and then the text input is input into the large language model. According to the scheme, the convenience of user interaction is finally improved through the voice recognition technology, the user is allowed to rapidly interact with the system through the natural language, manual input is not needed, the information acquisition speed is increased, services are easier to access, and secondly, the natural language processing capacity of a large model is improved, and the user experience is improved. Therefore, the system can understand complex query intentions and provide accurate and humanized answers, and the naturalness and accuracy of query are enhanced.

The invention discloses a 2D government affair digital human real-time interaction method and system based on an AI painting technology, and the method comprises the steps: making a digital human image based on a modeling mode combining AI painting and a Live2D technology; acquiring an audio input by a user in real time, and identifying the audio input by the user to obtain an audio identification text and a reply text; according to the reply text, synthesizing and outputting a result audio based on a half-stream speech synthesis method of dependency syntactic analysis; and performing mouth shape synchronous real-time driving and audio broadcasting of the digital human image on the manufactured digital human image based on the result audio. According to the invention, the low-cost construction and deployment of the digital human are realized based on the combination of AI drawing and Live2D technology; when system deployment is carried out on the digital people, a semi-streaming speech synthesis scheme based on dependency syntactic analysis is adopted, and the interaction real-time performance and fluency of government affair digital people belonging to service digital people are improved.

The invention discloses an intelligent coal gangue sorting method and system based on image deep learning, and belongs to the technical field of coal rock recognition. Global images and local images of coal gangue are obtained; preprocessing the global image and the local image to obtain a preprocessed global image and a preprocessed local image; fusing the preprocessed local image to the global image to obtain a fused image; and inputting the fused image into a target detection model based on a color feature attention mechanism, and carrying out detection and identification on coal and gangue to obtain an identification result. An image fusion technology is adopted to fuse a global image and a local image of coal gangue, so that coal gangue image information with rich details can be provided, a deep learning target detection model based on a color feature attention mechanism is adopted, coal and gangue can be accurately recognized, the recognition accuracy is improved by using color features, and the recognition efficiency is improved. And it is ensured that the sorting process is more accurate.

The invention discloses a low-age face aging method based on age and key point guide generation, and mainly solves the problems that in the prior art, a high-quality aging image cannot be generated for a low-age face, and the high efficiency of a processing flow and the accuracy of a target cannot be kept at the same time. According to the implementation scheme, the method comprises the following steps: constructing a training set and a test set of 0-20-year-old face images, and endowing each image with a corresponding age label; key point detection is carried out on the training set, and a detection result is converted into a Gaussian distribution diagram; constructing a face image generative adversarial network generated based on key point information guidance and loop conditions; designing continuous age conversion loss and constructing network overall loss; training the network through the training set and the Gaussian distribution diagram until the total loss is converged; and loading the test set into the trained network to generate face images of all age stages. According to the method, the aged image with higher quality can be generated for the low-age face, and the naturalness of the generated image is improved.

The invention provides a robot control method, computer equipment and a computer readable storage medium, and the method comprises the steps: obtaining two-arm tail end posture data of a user, and estimating each joint posture data of the user, the two-arm tail end posture data including a first timestamp, and each joint posture data including a second timestamp; according to the posture data of each joint and the robot configuration data, mapping the posture data of each joint to each joint of the robot, and determining reference angle information, including a second timestamp and a reference angle, of each joint of the robot; according to the first timestamp and the second timestamp, performing data synchronization processing on the two-arm tail end attitude data and the reference angle of each joint of the robot to obtain synchronized two-arm tail end attitude data and the reference angle; and according to the synchronized posture data of the tail ends of the two arms and the reference angle, fused posture data are obtained through data fusion processing, and the robot is controlled to move according to the fused posture data so as to flexibly and accurately control the robot to move.

The invention belongs to the field of artificial intelligence, and relates to a voice generation model training method comprising the following steps: obtaining a reference tone frequency spectrum, phoneme information and a voice frequency spectrum of a target object; based on the reference tone frequency spectrum, the phoneme information and the voice frequency spectrum, training a preset initial voice generation model to obtain model parameters; and based on the model parameters, respectively adjusting parameters of a multi-timbre feature extraction network, a phoneme feature extraction network, a rhythm feature discretization network, a time sequence alignment module, an attention fusion module and a voice reconstruction decoding network of the initial voice generation model so as to construct the voice generation model. The invention further provides a device, equipment and a medium. In addition, the invention also relates to a block chain technology, and voice training data and model parameters can be stored in a block chain. According to the method and the device, the tone and the rhythm information can be decoupled, and the tone and the rhythm information can be flexibly adjusted so as to generate synthetic speech with diversity and flexibility.

The invention discloses a ghost imaging target reconstruction method based on FSD-GIRA, and relates to the technical field of digital image processing. According to the technical scheme, a fuzzy clustering algorithm is used for classifying feature values of an optical field intensity measurement matrix of an optical path detector, the feature values are reconstructed through singular value decomposition, dimensionality reduction processing of measurement data of a ghost imaging system is achieved, and the ghost imaging target reconstruction method based on FSD-GIRA is obtained. A WE-FOMP-CG algorithm is proposed, and recovery of sparse signals is optimized by combining generalized orthogonal matching pursuit, a conjugate gradient method and a multi-threshold wavelet noise reduction method; and selecting the most relevant sensing matrix array in each iteration, solving a linear equation set by using PCG, carrying out wavelet preprocessing and multi-threshold de-noising on a result, and updating and recovering estimation. The calculation efficiency and the convergence speed are improved, and a more accurate recovery result is provided in a high-noise environment. The invention also provides a WBDH algorithm, and the algorithm employs a dual denoising mechanism, including wavelet transform and a BM3D method, so as to guarantee that the restored image details and structures are reserved, and further improve the reconstruction quality of the ghost imaging target.

The invention provides a speech synthesis method, device and equipment based on an AI large model in a multi-language scene, and the method comprises the steps: carrying out the text feature extraction of an input multi-language to-be-synthesized text, and obtaining a to-be-synthesized text feature; performing speech synthesis parameter generation on the to-be-synthesized text features based on the fusion feature model; wherein the pre-trained fusion feature model has a mapping relationship between speech synthesis parameters and a text, and the speech synthesis parameters comprise a fundamental frequency parameter, a duration parameter and a frequency spectrum parameter; inputting the speech synthesis parameters into a vocoder which is trained in advance, processing a plurality of parameters in the speech synthesis parameters into intermediate feature vectors based on the vocoder, converting the intermediate feature vectors into discrete speech sample points, converting the discrete speech sample points into continuous speech waveforms, and outputting the continuous speech waveforms. And obtaining synthetic speech corresponding to the multi-language to-be-synthesized text. According to the invention, the coherence and naturalness of the generated voice waveform are improved.

A method and system for enhancing speech in a mask, a storage medium and a program product relate to the field of speech analysis or synthesis, and the method comprises the following steps: collecting first audio samples of a plurality of target objects in a mask wearing state and second audio samples of a plurality of target objects in a mask non-wearing state; according to the first audio sample and the second audio sample, determining spectrum difference characteristics under the mask effect; performing model training based on the spectrum difference features to obtain a speech enhancement model; acquiring a real-time audio signal of a target user in the working scene, and performing noise reduction preprocessing on the real-time audio signal to obtain a preprocessed audio signal; inputting the preprocessed audio signal into a speech enhancement model to obtain an enhanced audio signal; and adjusting the enhanced audio signal according to preset user configuration to obtain an output audio signal, and outputting the target voice according to the output audio signal. According to the invention, the voice quality of sound production in the mask can be improved, and smooth communication during cooperative work of multiple persons is ensured.

The invention discloses an intelligent dialogue management method of an outbound system, and relates to the technical field of intelligent management.In the system operation process, multi-dimensional data of clients are collected in real time, a client voice feature file is established through the voice recognition technology, cleaning, noise reduction, feature extraction and standardization are conducted on the data, and a preprocessed feature matrix is generated; according to an analysis result, a deep learning algorithm is applied, and after calculation, a timbre adaptation coefficient Zys and an intonation matching coefficient Zyt are obtained; a timbre and intonation model highly matched with customer features is dynamically generated; voice is synthesized in real time and dynamically adjusted and output is performed; a current scene is judged and an outbound content and an intonation style are optimized; selecting an intonation style according to a scene demand, collecting real-time feedback in an outbound process, continuously optimizing a tone and intonation matching model through deep learning, calculating to obtain a comprehensive adaptation index Zsa, comparing the comprehensive adaptation index Zsa with a preset adaptation threshold A, and judging whether adaptation succeeds or not.

The invention discloses an animation motion capture device for a virtual reality technology, and particularly relates to the technical field of virtual reality, which comprises a data acquisition module for capturing and collecting user motion and environment information, and a data processing module for analyzing, converting and optimizing the acquired data. Through cooperative work of the data acquisition module, the data processing module, the rendering feedback module and the like, accurate capture and real-time feedback of user actions are achieved, it is ensured that physical attributes of a virtual environment and a real environment are highly consistent, the data acquisition module can obtain multi-dimensional information such as somatosensory, expressions and environments of a user at high precision, and the user experience is improved. And the data processing module accurately identifies and optimizes user actions through a deep learning algorithm and a skeleton mapping technology, and the rendering and feedback module generates vivid visual, tactile and sound effect feedback, so that high immersive experience is brought, and fluency and consistency of actions and feedback are guaranteed.