An immersive eloquence expression training system based on brain-computer interface technology

Abstract

The present invention provides an immersive eloquence expression training system based on brain-computer interface technology, which relates to the field of eloquence expression training, including: a multi-mode game mode creation module for creating different types of speech training games; a comprehensive evaluation and feedback module for monitoring the performance data of users in different types of speech training games, generating personalized evaluation and feedback results, and sending them to a brain science guidance and optimization module; the performance data includes voice, speech speed, intonation, body language and emotional expression; the brain science guidance and optimization module is used to monitor the user's electroencephalogram (EEG) signals, emotional state data and physiological indicators in real time through an EEG interface and an emotional sensor, and based on the personalized evaluation and feedback results, analyze the user's eloquence expression process according to the EEG signals, the emotional state data and the physiological indicators, and conduct long-term progress tracking. The present invention can effectively improve the user's eloquence expression ability.

Description

Technical Field

[0001] The present invention relates to the field of eloquence expression training, and in particular to an immersive eloquence expression training system based on brain-computer interface technology. Background Art

[0002] Traditional eloquence training methods mainly rely on manual guidance and feedback, usually conducted by eloquence coaches or trainers, which have the following problems:

[0003] Reliance on manual evaluation: Traditional methods mainly rely on manual evaluation of the user's eloquence and expression level. The eloquence coach will provide feedback and suggestions based on experience and subjective judgment.

[0004] Limited objective data: Traditional methods lack objective data support and are difficult to quantify users’ eloquence performance, making it difficult to provide users with accurate evaluations and improvement suggestions.

[0005] Lack of personalization: Traditional methods usually fail to provide personalized training because it is difficult for eloquence coaches to meet the personalized needs of multiple users at the same time.

[0006] Limited real-time feedback: Feedback in traditional methods is usually intermittent, and users must wait for evaluation and feedback from the eloquence coach and cannot get real-time guidance. Summary of the invention

[0007] The purpose of the present invention is to provide an immersive eloquence expression training system based on brain-computer interface technology to solve the problems existing in traditional eloquence expression training methods.

[0008] To achieve the above object, the present invention provides the following solutions:

[0009] An immersive eloquence expression training system based on brain-computer interface technology, comprising:

[0010] A multi-mode game mode creation module for creating different types of speech training games;

[0011] A comprehensive evaluation and feedback module is used to monitor the user's performance data in different types of speech training games, generate personalized evaluation and feedback results, and send them to the brain science guidance and optimization module; the performance data includes voice, speech speed, intonation, body language and emotional expression;

[0012] The brain science guidance and optimization module is used to monitor the user's EEG signals, emotional state data and physiological indicators in real time through the EEG interface and emotional sensors, and based on the personalized evaluation and feedback results, analyze the user's eloquence expression process according to the EEG signals, the emotional state data and the physiological indicators, and conduct long-term progress tracking.

[0013] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects: the embodiments of the present invention generate personalized evaluation and feedback results by creating different types of speech training games, and monitoring the performance data of users in different types of speech training games in real time, and monitor the user's EEG signals, emotional state data and physiological indicators in real time through EEG interfaces and emotional sensors, and based on the personalized evaluation and feedback results, analyze the user's eloquence expression process according to the EEG signals, the emotional state data and the physiological indicators, and conduct long-term progress tracking. The entire eloquence expression training process does not require manual evaluation, is more objective, and can provide real-time personalized feedback based on the user's performance data, and conduct long-term progress tracking to improve eloquence expression ability. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0015] Figure 1 This is a schematic diagram of the immersive eloquence expression training system based on brain-computer interface technology provided by the present invention. DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0017] The purpose of the present invention is to provide an immersive eloquence expression training system based on brain-computer interface technology to effectively improve the user's eloquence expression ability.

[0018] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0019] Embodiment 1

[0020] like Figure 1 As shown, the present invention provides an immersive eloquence expression training system based on brain-computer interface technology, comprising:

[0021] A multi-mode game mode creation module for creating different types of speech training games.

[0022] In actual application, the multi-mode game mode mainly includes the following game types.

[0023] 1. Competitive games

[0024] Structure and Functionality: Competitive games are designed to improve the user's speaking and eloquence skills through the following means:

[0025] Users can choose to participate in virtual debates, speech contests and other competitive activities.

[0026] The system provides a variety of debate topics and themes to meet the interests and needs of different users.

[0027] Users debate or give speeches with other virtual contestants, and the system monitors and records their speeches, debate skills, pronunciation accuracy and fluency in real time.

[0028] Based on performance, the system provides users with personalized suggestions and improvement directions, including improving speaking speed, intonation, logic and clarity.

[0029] Users can view competition results and transcripts to understand their progress and growth.

[0030] Implementation details: To achieve this function, the system uses advanced speech recognition technology, natural language processing algorithms and brain-computer interface technology. Specific implementation methods include:

[0031] Real-time speech recognition: The system can instantly recognize the user's speech content and convert it into text for further analysis.

[0032] The specific process of speech recognition involves the following steps and technologies:

[0033] 1. Audio collection: First, the system will collect the audio data generated by the user during speech or eloquence training. This can be done through a microphone, headset or other audio collection equipment. The audio data includes the content of the user's speech, including information such as speech, speech speed, and intonation.

[0034] 2. Sound preprocessing: After the audio data is collected, the system will preprocess the sound. This includes removing noise and adjusting the audio quality to ensure that the subsequent speech recognition process is more accurate.

[0035] 3. Speech signal analysis: Next, the system will send the pre-processed audio data to the speech signal analysis engine. This engine uses acoustic models and language models to parse the audio data and convert it into text data. This process uses natural language processing and machine learning technology.

[0036] 4. Model training: In the system, the speech recognition engine needs to be trained to adapt to different speaking styles, languages ​​and accents. This means that the system needs to use a large amount of voice data during the training phase to adjust the model to make it more suitable for different users.

[0037] 5. Speech Recognition and Text Generation: Once the system’s engine is trained, it can start recognizing the user’s speech in real time. The engine analyzes the audio signal and converts it into text so that the system can understand what the user said in the speech.

[0038] 6. Real-time evaluation and feedback: Once text data is generated, it can be used to provide real-time evaluation and feedback. The system can evaluate factors such as speech fluency, speaking speed, intonation, clarity, etc., and provide instant suggestions based on these evaluations. This helps users understand their speech performance and find directions for improvement.

[0039] The entire speech recognition process covers multiple steps including audio collection, sound preprocessing, speech signal analysis, model training, text generation, etc. These steps work together to enable the system to convert the user's speech content into text in real time and provide them with feedback and suggestions to help them improve their eloquence and expression skills, which helps to improve the effectiveness of eloquence training.

[0040] Sentiment Analysis: The system analyzes the user's speech to detect emotions and emotional changes, which helps improve emotional expression.

[0041] The specific process of sentiment analysis includes the following steps and techniques:

[0042] 1. Emotional data collection: First, the system collects audio data generated by users during speeches or eloquence training. These data contain the user's voice, including speech content, speech speed, intonation, etc. In addition, the system also collects physiological sensor data, such as heart rate, skin conductance, etc., which can be used to identify the user's emotional state.

[0043] 2. Sound preprocessing: Audio data needs to be preprocessed to remove noise or other factors that interfere with sentiment analysis and ensure the accuracy of sentiment analysis.

[0044] 3. Emotional feature extraction: Once the audio data is ready, the system will extract emotional features. This includes identifying emotional clues in the sound, such as the pitch, speed, emotion-related vocabulary, emotional expression, etc.

[0045] 4. Emotion Recognition Model: Sentiment analysis relies on emotion recognition models that can identify emotions in audio. These models are usually trained using machine learning, deep learning, or natural language processing techniques with training data with known emotion labels. The model associates the extracted emotion features with the known emotion labels to identify the emotions in the audio.

[0046] 5. Emotion score generation: The emotion recognition model generates emotion scores that reflect the different emotions contained in the audio, such as happiness, nervousness, excitement, anger, etc. These scores are used to quantify the emotional state of the user in the speech or eloquence.

[0047] 6. Real-time feedback: Once the system has obtained the emotional scores, it can use them to provide real-time feedback. The system will display the user's emotional state and suggest how to improve emotional expression. For example, if the system detects that the user is nervous during a speech, it can provide suggestions and techniques to relax.

[0048] The process of sentiment analysis helps users understand their emotional state during speeches, helps them better manage their emotions, improves the quality of emotional expression, and helps improve the effectiveness of eloquence expression training.

[0049] Real-time feedback: The system provides real-time performance evaluation and suggestions through feedback from virtual judges and audiences.

[0050] (2) Simulation games

[0051] Structure and function:

[0052] Users can choose to participate in various scenarios such as simulated press conferences, press conferences, business meetings, etc.

[0053] The system provides situational context, virtual audience and dialogue scripts.

[0054] Users need to give speeches or participate in conversations in a virtual situation, and the system records their performance in real time.

[0055] The system provides detailed feedback on speech content, communication skills and emotional expression to help users continuously improve.

[0056] Implementation details: To achieve this function, the system applies virtual reality technology and brain-computer interface technology. The specific implementation methods include:

[0057] Virtual reality simulation: The system uses a virtual reality helmet and controller to provide users with an immersive situational experience.

[0058] Virtual reality helmets are usually stand-alone devices that can provide users with an immersive virtual environment experience through built-in displays and sensors. These helmets usually have panoramic vision, 3D sound effects, and head tracking functions to simulate various situations. Virtual reality controllers are usually control devices used in conjunction with helmets, and users can interact and manipulate the virtual environment through the controllers.

[0059] Speech recognition and sentiment analysis: The system recognizes the user’s speech and analyzes their facial expressions and vocal emotions to provide feedback.

[0060] Voice Recognition:

[0061] Users give speeches or participate in conversations in virtual situations.

[0062] The system captures the user's voice input, typically via a microphone or similar device.

[0063] Speech recognition technology is used to convert the user's oral expression into text. This process involves steps such as digitization of sound signals, extraction of acoustic features, and model training.

[0064] After conversion to text, the system can further analyze factors such as the content, grammar, clarity, fluency, etc. of the speech.

[0065] The analyzed text data can be used to provide feedback, including suggestions on language quality and fluency.

[0066] Sentiment Analysis:

[0067] At the same time, the system also captures the user's voice emotion, which can be identified by the pitch, volume, speaking speed and emotional content of the voice.

[0068] The sentiment analysis module uses advanced voice emotion recognition technologies, such as natural language processing and acoustic feature analysis, to detect and identify the user's emotional state.

[0069] Sentiment analysis can cover multiple emotional dimensions, such as excitement, nervousness, confidence, etc.

[0070] The system can provide feedback based on the user's emotional expression, encouraging them to better manage their emotions during speech or conversation to enhance their expressiveness.

[0071] The synergy of speech recognition and sentiment analysis enables the system to more comprehensively evaluate the user's eloquence skills, including not only language quality but also emotional expression, thereby providing more comprehensive feedback and suggestions. This helps users continuously improve their eloquence skills and enhance their performance in various situations.

[0072] Virtual audience interaction: The system simulates real-life scenarios through the responses and interactions of virtual audiences, which helps improve users’ confidence in expression.

[0073] (III) Interactive Games

[0074] Structure and function: Interactive games help users improve their eloquence and social skills by interacting with virtual characters.

[0075] Users can choose to engage in conversations, debates, or collaborative missions with virtual characters.

[0076] Virtual characters have various personalities and characteristics, simulating different social situations.

[0077] Users' conversations and interactions are recorded by the system, including language, body language, and emotional expressions.

[0078] The system provides feedback to users, including evaluation of the conversation, accuracy of emotional expression, and suggestions for improvement of social skills.

[0079] Implementation details: To realize interactive games, it is necessary to combine virtual reality technology, natural language processing and brain-computer interface technology. The specific implementation methods include:

[0080] Natural language understanding: The system analyzes the conversation between the user and the virtual character to understand the user's expression and emotional state.

[0081] Natural language understanding includes text analysis, intent recognition, sentiment analysis, and context understanding.

[0082] 1. Text analysis: The system uses natural language processing technology, including text analysis, syntactic analysis, semantic analysis, etc., to understand the conversation between users and virtual characters. These technologies can help the system understand the structure of sentences, the meaning of words, and contextual relationships.

[0083] 2. Intent recognition: The system identifies the intentions and needs of users by analyzing their conversations. This includes understanding the purpose expressed by users in interactive games, such as asking questions, expressing opinions, or requesting help.

[0084] 3. Emotional analysis: In addition to understanding the literal meaning of language, the system also analyzes the user's emotional state to ensure that the virtual character can respond appropriately to emotions and emotional changes. This can be recognized in a variety of ways, such as voice emotions, text emotions, and facial expressions.

[0085] 4. Contextual understanding: The system is able to track the context of the conversation to ensure that the virtual character's response is coherent and relevant. This helps provide a more realistic interactive experience.

[0086] Working together, these technologies enable the system to more fully understand the user's conversation, including not only the surface meaning of the language, but also the emotion, intent, and context, thereby providing a more intelligent and realistic virtual interaction experience. This helps improve eloquence and social skills.

[0087] Speech synthesis: The virtual character responds to the user's conversation through voice, providing a realistic interactive experience.

[0088] Speech synthesis technology: The voice response of the virtual character is achieved through speech synthesis technology. This technology is also known as text-to-speech (TTS) technology, which can automatically convert text information into human-audible speech. This is a key component that enables the virtual character to respond to the user's dialogue in a vocal manner.

[0089] 2. Text-to-speech processing: When a user interacts with a virtual character, the user's text input (questions, comments, or any conversation content) is passed to the text-to-speech processing engine. The engine converts the text information into a speech signal so that it can be played to the user through an audio output device.

[0090] 3. Personalized voice modeling: The system also includes personalized voice modeling of virtual characters. This allows the virtual characters to have different voices, tones, and emotional expressions to simulate different virtual character personalities and characteristics.

[0091] 4. Emotional Response: The voice response of the virtual character can also be adjusted according to the emotional needs of the conversation content and situation. The system can make the response of the virtual character more emotionally expressive according to the emotional state or emotional expression of the user. This can be achieved through emotion recognition technology, which adjusts the voice response of the virtual character according to the emotional state of the user to provide a more realistic and personalized interactive experience.

[0092] In summary, the virtual character responds to the user's dialogue by converting text information into speech, using speech synthesis technology, and can be adjusted in terms of emotion, personalization, etc. to provide a realistic and personalized interactive experience. This helps users improve their eloquence and social skills, and practice eloquence in different social situations through interaction with virtual characters.

[0093] Body language simulation: Virtual characters can mimic body language, including gestures, expressions, and postures, to enhance the realism of interactions.

[0094] Body language simulation specifically includes:

[0095] 1. Motion capture technology: Motion capture technology is used to track the user's body movements and postures. Users wear sensor devices or use sensors such as cameras to capture their body movements and postures.

[0096] 2. Virtual character modeling: The virtual characters in the system need to be modeled in advance, including their appearance, body structure and movements. This modeling can be a three-dimensional model that can correspond to the user's body movements.

[0097] 3. Real-time motion matching: When the user interacts with the virtual character, motion capture technology is used to capture the user's body movements. The system matches these real-time body movements with the model of the virtual character to simulate the body language of the virtual character. For example, if the user makes a gesture, the system will move the virtual character's hand model accordingly in real time.

[0098] 4. Expression and gesture simulation: The model of the virtual character also includes data for simulating different expressions and gestures. According to the user's interaction and dialogue situation, the system can adjust the expression, gesture and posture of the virtual character to make the interaction more realistic.

[0099] 5. Contextual awareness: The system uses natural language processing technology to understand the content of the conversation and emotion recognition technology to detect the emotional state of the user and the virtual character, so as to adjust the virtual character's body language simulation according to the context.

[0100] In summary, the simulation of virtual character's body language is achieved through motion capture technology, virtual character modeling, real-time motion matching, expression and posture simulation, etc. These technologies work together to enable virtual characters to imitate real body language, including gestures, expressions and postures, to enhance the realism of interaction and help users improve their eloquence and social skills.

[0101] 4. Role Playing Games

[0102] Structure and Function: Role-playing games allow users to assume different roles, from historical figures to fictional characters, and participate in eloquence activities while learning about history, culture, and society.

[0103] Users can choose to play a variety of different roles, such as famous orators.

[0104] The system provides historical and background information about the characters played.

[0105] Users are required to give speeches, debates or conversations in a virtual situation, and the system records their performance.

[0106] The system provides users with feedback on character expression, historical context, and cultural connotations.

[0107] Implementation details: The role-playing game combines virtual reality technology and eloquence training. The implementation methods include:

[0108] Virtual reality simulation: The system creates virtual situations, including historical scenes and virtual characters, to provide an immersive experience.

[0109] The creation of historical scenes and virtual characters is achieved through the following specific processes and technologies:

[0110] 1. Virtual scene modeling: First, the system uses virtual reality technology to create historical scenes. This includes reconstructing elements such as buildings, scenery, roads, etc. in historical periods. This process usually requires the use of 3D modeling tools such as Unity or UnrealEngine to create a realistic virtual environment.

[0111] 2. Virtual Character Modeling: The system also needs to create models for virtual characters in role-playing games. These virtual characters can be historical figures. This involves designing the character's appearance, clothing, facial expressions, and movements. Usually, the modeling and animation of virtual characters are also done using 3D modeling tools and animation software.

[0112] 3. Cultural and historical data integration: In order to provide historical and background information about the characters being played, the system needs to integrate cultural and historical data. This includes information on historical events, cultural details, celebrity biographies, etc. This data can be obtained from various historical and cultural databases and presented to the user by the system.

[0113] 4. Scene interaction design: In the virtual scene, the system also needs to design interactive elements so that users can interact with the virtual environment. This includes dialogues with virtual characters, participation in historical events, speeches and other interactive elements.

[0114] 5. Speech recognition and sentiment analysis: During the role-playing process, the system recognizes the user’s speech and dialogue through speech recognition technology, and detects their emotional state through sentiment analysis technology. This helps provide feedback on speech and expression, as well as understand the user’s emotional experience during the interaction.

[0115] In summary, the establishment of historical scenes and virtual characters is achieved through a series of processes such as virtual reality modeling, virtual character modeling, cultural and historical data integration, and scene interaction design. These processes work together to provide an immersive role-playing experience and help users better understand and express historical, cultural, and social connotations.

[0116] Cultural learning support: The system provides multimedia information about the character's background, history, and culture to help users better understand and express themselves.

[0117] Speech Recognition and Sentiment Analysis: The system analyzes the user's speech and expressions, providing feedback on voice, sentiment, and expression.

[0118] The specific process of speech recognition and sentiment analysis includes the following steps:

[0119] 1. Voice Recognition:

[0120] Audio recording: First, the system records the audio of the user's speech, conversation, or expression through the user's microphone or voice input device.

[0121] Audio preprocessing: The recorded audio needs to be preprocessed, including removing noise and adjusting the audio quality to ensure clarity.

[0122] Feature extraction: Extracting sound features from the audio, such as pitch, speaking rate, intonation, pronunciation accuracy, etc. This can be achieved using audio processing and signal processing techniques.

[0123] Speech-to-text: Using natural language processing techniques, the system converts the user's speech into text. This typically includes a speech recognition engine, such as Google Speech Recognition or Baidu Speech Recognition, to convert the audio into analyzable text.

[0124] 2. Sentiment Analysis:

[0125] Emotional feature extraction: Extract emotional features from the user's speech and text, such as speaking speed, pitch changes, use of emotional vocabulary, etc.

[0126] Sentiment classification: The system uses machine learning algorithms or deep learning models to classify the extracted sentiment features. Sentiment classification includes the recognition of emotional states, such as happiness, frustration, tension, etc.

[0127] Emotional feedback: Based on the results of the sentiment analysis, the system provides users with feedback on their emotional expression. This can include suggestions on how to improve emotional expression, the appropriateness of emotions, and the consistency of emotions with the content of the speech.

[0128] In summary, the process of speech recognition and sentiment analysis involves multiple steps such as audio recording, preprocessing, feature extraction, text conversion, sentiment feature extraction, sentiment classification, and sentiment feedback. These steps work together to provide users with detailed feedback on their speech and expression, including voice accuracy, emotional expression, and suggestions for expression. This helps users continuously improve in their eloquence.

[0129] 5. Music Games

[0130] Structure and function: Music games allow users to participate in music-related eloquence activities, such as singing competitions or music speeches, to improve their musical expression skills.

[0131] Users can choose to participate in music speech, singing competitions and other music and eloquence expression activities.

[0132] The system provides different types of music, songs and speech topics.

[0133] Users are required to express music in a virtual situation, and the system records and evaluates their performance.

[0134] Feedback includes suggestions for musical expression techniques, vocal control, and emotional communication.

[0135] Implementation details: The music game combines audio processing technology, speech recognition and sentiment analysis. The implementation includes:

[0136] Audio Analysis: The system analyzes the user's voice, musical expression, and pitch accuracy.

[0137] Sentiment Analysis: The system evaluates the emotions conveyed in the user's musical expression and provides suggestions for improvement.

[0138] The specific process of audio analysis and sentiment analysis is as follows:

[0139] 1. Audio Analysis:

[0140] Audio recording: The system first records the user's voice through the user's microphone or audio input device, including singing, music speech, etc.

[0141] Audio feature extraction: The system extracts music-related sound features from the recorded audio, including information about pitch, tone, music rhythm, beat, etc. The extraction of these features can be achieved using audio signal processing technology.

[0142] Musical skills assessment: The system analyzes audio features to assess the user's musical skills, including performance in terms of musical accuracy, sense of rhythm, etc. This involves algorithms related to music education and audio processing.

[0143] 2. Sentiment Analysis:

[0144] Emotional feature extraction: The system extracts emotional features from the audio, such as the emotional communication in musical expression, changes in musical emotions, etc.

[0145] Emotion classification: The system uses machine learning or deep learning models to classify the extracted emotional features. This helps the system identify the emotional states in the musical expression, such as happiness, sadness, excitement, etc.

[0146] Emotional feedback: Based on the results of the sentiment analysis, the system provides feedback to the user on the emotional communication in their musical performance. This can include the appropriateness of the emotion, suggestions on how to better express the emotion in the music, etc.

[0147] In summary, audio analysis includes audio recording, audio feature extraction, and music skills evaluation, which is used to evaluate the user's music skills. Emotion analysis involves emotion feature extraction, emotion classification, and emotion feedback, which is used to evaluate the emotional communication in the user's music expression. These analysis processes help the system provide users with detailed feedback on their music expression to help them improve their music eloquence expression skills.

[0148] Real-time evaluation: The system provides immediate feedback to help users continuously improve in their musical expression.

[0149] For the first part, the core structure + function + implementation method are as follows:

[0150] Structure (I):

[0151] 1. Game engine: responsible for the operation and control of the game, including the generation of game scenes, the control of game characters, the definition of game rules, etc.

[0152] 2. Game content: including game scenes, characters, props, tasks, etc.

[0153] 3. Game evaluation: responsible for evaluating the user's game performance, including voice, speaking speed, tone, body language, emotional expression and other aspects.

[0154] 4. Game feedback: Provide users with detailed feedback and improvement suggestions based on the game evaluation results.

[0155] Function (two):

[0156] 1. Improve users' verbal expression skills: Through different types of games, including competitive, simulation, interactive, role-playing and music games, users can exercise their verbal expression skills in real situations and improve the fluency, accuracy and appeal of language expression.

[0157] 2. Improve users' thinking ability: Through logical reasoning, problem solving and other tasks in the game, cultivate users' thinking ability and improve the logic and orderliness of oral expression.

[0158] 3. Cultivate users’ self-confidence: Cultivate users’ self-confidence through competitions, challenges and other situations in the game, and improve the confidence and appeal of oral expression.

[0159] Implementation method (three):

[0160] 1. Virtual reality technology: To provide an immersive experience, using virtual reality headsets and controllers, users can immerse themselves in virtual situations, interact with virtual characters, and participate in various activities.

[0161] 2. Natural language processing technology: Through speech recognition technology, the system can instantly recognize the user's speech content and convert it into text for further analysis. At the same time, sentiment analysis can be used to understand the user's voice emotions and emotional changes, which helps to improve emotional expression.

[0162] Natural Language Processing Technology:

[0163] Speech Recognition: The system uses advanced speech recognition technologies, such as natural language processing and speech recognition algorithms, to recognize the user's speech content. This includes converting the user's spoken expression into text information.

[0164] Text Analysis: The recognized text information is analyzed to understand the user’s expression, grammatical accuracy, and other key factors. Natural language processing algorithms help detect grammatical errors, vocabulary usage, and syntactic structure, etc.

[0165] Sentiment analysis: Through natural language processing technology, the system can also analyze the user's speech to detect emotions and emotional changes. This helps improve emotional expression and provide feedback on emotional expression.

[0166] 3. Speech synthesis technology: Virtual characters communicate with users through speech synthesis technology, providing a realistic interactive experience and enhancing the practicality of eloquence.

[0167] Speech Synthesis Technology:

[0168] Virtual Character Speech Generation: Speech synthesis technology is used to generate voice responses for virtual characters. This can include using text-to-speech (TTS) technology to synthesize realistic speech based on system-generated text responses.

[0169] Emotional communication: Speech synthesis technology can adjust the voice expression according to the emotional state of the virtual character to provide a realistic interactive experience and enhance the practicality of eloquence. Emotion-driven speech synthesis technology can ensure that the character's voice matches the situation and interaction method.

[0170] 4. Body language simulation: Through gesture sensing devices, virtual characters can imitate body language, including gestures, expressions and postures, to enhance the realism of interaction, which is particularly suitable for interactive games.

[0171] Body language simulation:

[0172] Gesture and expression simulation: Body language simulation involves using gesture-sensing devices in VR devices, such as controllers and head trackers, to mimic the body language of virtual characters. These devices can capture the user's gestures, expressions, and postures and reflect them on the virtual character, thereby enhancing the realism of the interaction.

[0173] 5. Big Data Analytics: Collect and analyze user performance data in various game modes to provide personalized suggestions and track progress, helping educators better understand user progress and needs.

[0174] Big Data Analysis:

[0175] Data Collection: The system collects user performance data in various game modes, including speech content, audio data, interaction data, timestamps, etc.

[0176] Data analysis: Through big data analysis techniques, the system conducts in-depth analysis of the collected data to identify the strengths and needs of users. This includes applying data mining, machine learning, and statistical analysis methods.

[0177] Personalized recommendations: Based on the analyzed data, the system provides personalized recommendations for each user, including improvement points, course recommendations, and learning paths. This helps educators better understand the user's progress and needs to better support their development.

[0178] In summary, these technical means and implementation processes help the system provide highly personalized eloquence training, including recognizing the user's voice content and emotional state, generating realistic virtual character responses, simulating body language, and providing personalized feedback and support to users through big data analysis. These technologies work together to enable users to improve their eloquence skills in an immersive virtual situation.

[0179] A comprehensive evaluation and feedback module is used to monitor the user's performance data in different types of speech training games, generate personalized evaluation and feedback results, and send them to the brain science guidance and optimization module; the performance data includes voice, speaking speed, intonation, body language and emotional expression.

[0180] In practical applications, the comprehensive evaluation and feedback module specifically includes:

[0181] (I) Structure:

[0182] Evaluation engine: responsible for collecting and analyzing the performance data of users in eloquence training, including voice, speaking speed, intonation, body language, emotional expression, etc. The evaluation engine will combine speech recognition technology, emotion analysis algorithm and brain-computer interface technology to achieve multi-dimensional evaluation.

[0183] Data processing and analysis: After collecting the user's performance data, the system will use advanced data processing and analysis technologies, including voice signal processing, sentiment analysis, body language recognition, etc., to deeply analyze the user's eloquence and expression level.

[0184] Personalized evaluation: The evaluation engine will generate a personalized evaluation of the user's eloquence based on the user's performance data. This evaluation will take into account the performance in different game modes, including competitive, simulation, interactive, role-playing, music, etc.

[0185] Detailed Feedback: Users will receive a detailed feedback report that includes their strengths and areas for improvement in their eloquence. This feedback will be presented in a visual way to help users better understand.

[0186] Improvement suggestions: The system will provide personalized improvement suggestions, including suggestions on language skills, emotional expression, logical thinking, body language, etc., to help users improve their eloquence and expression skills.

[0187] (II) Function:

[0188] Speech recognition: The system can accurately recognize the user's speech content and convert it into text for analysis.

[0189] Sentiment analysis: By analyzing the user's voice, intonation, and emotional expressions, the system can detect emotions and emotional changes, and gain insight into emotional communication.

[0190] Body language analysis: The system can analyze the user's body language, including gestures, facial expressions, and posture, to supplement verbal expression assessment.

[0191] Evaluation dimensions: The system can comprehensively evaluate the user's performance based on multiple evaluation dimensions, including verbal fluency, logic, emotional communication, body language, etc.

[0192] Personalized evaluation: The evaluation engine will integrate performance data from different modes to generate personalized evaluations, taking into account the user's unique needs and improvement points.

[0193] Detailed Feedback: The system provides detailed feedback reports that visualize the user's performance for better understanding.

[0194] Improvement suggestions: The system provides targeted improvement suggestions to help users continuously improve in all aspects of eloquence and expression.

[0195] Implementation method (three):

[0196] Speech recognition technology: Using advanced speech recognition technology, the user's speech content is converted into text, providing a basis for subsequent analysis.

[0197] Sentiment analysis algorithm: Use sentiment analysis algorithm to analyze the user's voice, tone, and emotional state to gain a deeper understanding of emotional expression.

[0198] Body language recognition: Analyzes the user’s body language, including gestures, facial expressions, and posture, using image processing and computer vision techniques.

[0199] Multidimensional evaluation model: Establish a multidimensional evaluation model, including verbal fluency, logic, emotional communication, etc., to comprehensively evaluate the ability of eloquence and expression.

[0200] Personalized algorithm: Use machine learning and personalization algorithms to generate personalized evaluations and feedback based on user performance data.

[0201] Visualization tools: Use visualization tools to present assessment results and feedback to help users better understand and analyze.

[0202] Improvement suggestion generation: Combined with natural language processing technology, it generates targeted improvement suggestions to help users improve their eloquence and expression skills.

[0203] The brain science guidance and optimization module is used to monitor the user's EEG signals, emotional state data and physiological indicators in real time through the EEG interface and emotional sensors, and based on the personalized evaluation and feedback results, analyze the user's eloquence expression process according to the EEG signals, the emotional state data and the physiological indicators, and conduct long-term progress tracking.

[0204] In practical applications, the brain science guidance and optimization module specifically includes:

[0205] (I) Structure:

[0206] Brain-computer interface and emotion sensors: These sensors are responsible for monitoring the user's EEG signals and emotional state, as well as other physiological indicators. This includes EEG caps, skin conductance sensors, heart rate sensors, etc.

[0207] Data processing and analysis: The monitored data is processed and analyzed to extract useful information. This includes interpretation of EEG signals, analysis of emotional states, processing of physiological data, etc.

[0208] Personalized optimization algorithm: The system uses a personalized algorithm to generate personalized suggestions and improvement recommendations based on multi-dimensional data, including EEG signals, emotional states, physiological indicators, etc.

[0209] Real-time feedback interface: Users can monitor their EEG signals, emotional state and physiological indicators in real time through the real-time feedback interface. This interface shows the user's current status and provides instant suggestions to help them optimize during the speech.

[0210] Data storage and analysis tools: The system will store the user's multi-dimensional data for further analysis and long-term progress tracking. This data helps to understand the user's progress in speaking skills.

[0211] (II) Function:

[0212] Brain activity monitoring: Through brain-computer interface technology, the system can monitor the user's EEG signals in real time, including the frequency and intensity of brain activity, as well as other brain characteristics.

[0213] Emotional state detection: The system uses emotional sensors and physiological sensors to detect the user's emotional state, including nervousness, excitement, etc. This helps understand the user's emotional experience during the speech.

[0214] Personalized suggestion generation: Based on multi-dimensional data such as EEG signals, emotional state, and physiological indicators, the system generates personalized speech suggestions and improvement suggestions. These suggestions are customized according to the user's unique needs and performance.

[0215] 1. EEG signal monitoring: The system monitors the user's EEG signals in real time through brain-computer interface technology. These signals include the frequency, intensity and other brain characteristics of brain activity. The monitoring process usually involves wearing an electroencephalogram (EEG) headset that can record data on brain activity.

[0216] EEG signal monitoring is an important part of the brain science guidance and optimization system. It monitors the user's EEG signals in real time through brain-computer interface technology to obtain useful information about brain activity. The following are the steps and methods of the EEG signal monitoring process in detail:

[0217] 1) Wearing an EEG headset:

[0218] The user needs to wear an electroencephalogram (EEG) headset, usually a cap or helmet with multiple electrodes attached. These electrodes are placed on the scalp to cover different areas of the brain. The position of the electrodes is usually arranged according to the international standard 10-20 system to ensure that the activity of various parts of the brain is accurately monitored.

[0219] 2) Signal acquisition:

[0220] Once a user puts on the EEG headset, the device begins to collect brain electrical signals, which are weak electrical currents recorded by electrodes and reflect the activity of neurons in different areas of the brain.

[0221] Signal acquisition is typically performed at a high sampling rate, often collecting data hundreds of times per second, to capture subtle changes in brain activity.

[0222] 3) Signal amplification and filtering:

[0223] The collected EEG signals are very weak and need to be amplified for subsequent analysis. Signal amplification can improve the signal-to-noise ratio and make brain activity easier to detect.

[0224] The signal may also be filtered to remove high- or low-frequency noise and thereby focus on a frequency range of interest, such as a specific frequency band of EEG activity.

[0225] 4) Data transmission and storage:

[0226] The amplified and filtered EEG signals are usually transmitted via a wired or wireless connection to the system's data processing unit, which can be a computer or a specialized brain-computer interface device.

[0227] The collected data will be stored for later analysis and real-time monitoring. This can be used for long-term progress tracking and more in-depth research.

[0228] 5) Real-time analysis and feedback:

[0229] EEG signals can be used to analyze the user's brain activity status in real-time monitoring. This may include detecting specific frequency components, spectrum analysis, event-related potential (ERP) analysis, etc.

[0230] The results of real-time analysis can be used to generate real-time feedback about the user's brain state. For example, during a musical speech, the system can detect the user's anxious or excited EEG patterns and provide the user with corresponding emotional adjustment suggestions.

[0231] EEG signal monitoring involves multiple steps such as wearing an EEG headset, signal acquisition, signal processing, data transmission and storage to capture and analyze the user's brain activity. These signals provide valuable information for understanding the user's emotional state, mood, concentration, etc., and can be used to generate personalized suggestions to help them improve their performance in speeches or other eloquence activities.

[0232] 2. Emotional state detection: The system uses emotional sensors and physiological sensors to detect the user's emotional state. These sensors can monitor physiological indicators such as heart rate, electrodermal activity (EDA), muscle tension, etc. to help understand the user's emotional experience during the speech. Emotional sensors can detect emotional states such as tension and excitement.

[0233] 3. Data integration: The system integrates EEG signals, emotional sensor data, and physiological sensor data to create a multidimensional data file of the user. These data files include information about brain activity, emotional state, and physiological indicators.

[0234] Data integration is the process of integrating multiple types of data from different sensors into a comprehensive data file or data model to provide a more comprehensive understanding of the user's state during the speech. In the Brain Science Guidance and Optimization module, this step involves integrating EEG signals, emotional sensor data, and physiological sensor data to create a user data file containing multi-dimensional information. The following is a detailed description of this process:

[0235] 1) Data collection: First, different sensors collect multimodal data. These sensors include brain-computer interfaces for monitoring EEG signals, emotional sensors for monitoring emotional states (such as heart rate, skin electrical activity, etc.), and physiological sensors for monitoring physiological indicators (such as blood pressure, respiratory rate, etc.). Each sensor records a specific type of data during the user's speech.

[0236] 2) Data synchronization: Ensure that data from different sensors are synchronized on the same timeline. This is to associate information from different data sources with events at a specific point in time to better understand the user's state. For example, understand when a user experiences emotional changes and the relationship between speech activities.

[0237] 3) Data preprocessing: The collected data is preprocessed, including denoising, filtering, calibration, and normalization. These preprocessing steps help ensure the quality and consistency of the data so that subsequent data analysis is more accurate.

[0238] 4) Feature extraction: Extract meaningful features from the raw data of each sensor. For EEG signals, this can include extracting spectral features or event-related potentials. For emotional state data, feature extraction may involve the intensity of emotional expression and the type of emotion. For physiological indicator data, feature extraction includes extracting physiological responses related to emotional experience.

[0239] 5) Multimodal fusion: In this step, data from different sensors are integrated into a comprehensive data file or data model. This integration can be achieved using a variety of methods, including machine learning techniques such as neural networks or fusion algorithms. The goal is to integrate information from different sensors into a comprehensive description of the user's state to obtain more comprehensive information.

[0240] 6) Data Analysis: The integrated data is further analyzed to understand the user's state. This includes exploring the data to find patterns, trends, or correlations to better understand the user's experience during the speech. Data analysis helps reveal the potential relationship between the user's EEG signals, emotional state, and physiological indicators, providing deeper insights.

[0241] 7) Personalized algorithm application: The integrated multimodal data is used to generate personalized recommendations and improve recommendations. This usually requires machine learning algorithms to customize recommendations based on the user's multimodal state information to meet their unique needs and performance. The results of data integration can help personalized algorithms better understand the user's state and needs.

[0242] 8) Real-time feedback interface: The final integrated data and personalized suggestions are presented in a user-friendly way on the real-time feedback interface. This can be a visual interface or text prompts so that users can understand and take corresponding actions. The real-time feedback interface is an important interface for users to interact with the system, helping them understand their status and make adjustments and optimizations when necessary.

[0243] Data integration is a key step in the brain science guidance and optimization system, which integrates multimodal data obtained by different sensors and provides a basis for subsequent analysis and personalized recommendation generation. This helps the system to understand the user's status more comprehensively and provide personalized support and suggestions.

[0244] 4. Data Analysis: By analyzing the integrated data, the system can identify patterns and trends related to the user's speech performance. For example, the system can determine the correlation between a certain emotional state and speech quality, or that a specific EEG signal pattern is related to fluency.

[0245] 5. Suggestion generation: Based on the results of data analysis, the system can generate personalized speech suggestions and improvement suggestions. These suggestions include providing tips for improving emotional expression, ways to reduce nervousness, suggestions for improving speech fluency, etc. The generated suggestions are customized according to the user's unique needs and performance to meet their personalized requirements.

[0246] 6. Real-time feedback: Users can view their EEG signals, emotional states, and physiological indicators through the system’s real-time feedback interface. The system will provide instant suggestions to help users adjust and optimize during the presentation.

[0247] 7. Long-term progress tracking: The system also stores the user's multi-dimensional data for long-term progress tracking. This allows users and educators to understand the long-term improvement of presentation skills and adjust training plans and goals.

[0248] Long-term progress tracking is an important step in the Brain Science Coaching and Optimization module, which is designed to help users and educators understand the long-term improvement of speech skills so as to better adjust training plans and goals. This process involves the following key aspects:

[0249] 1) Data storage: First, the system needs to ensure long-term storage of multi-dimensional data. These data include information such as EEG signals, emotional states, and physiological indicators. They are recorded and saved for subsequent analysis and tracking. Data storage usually needs to be highly reliable, secure, and scalable to ensure data integrity and privacy protection.

[0250] 2) Data Archiving and Tagging: For long-term progress tracking, data needs to be archived and tagged. This means organizing the data by time and user ID so that data for a specific period can be easily retrieved and analyzed. Tagging data also helps track the user's speech skill progress and history.

[0251] 3) Data Analysis and Trend Identification: A key part of long-term progress tracking is the analysis of stored data. By analyzing historical data, the system can identify long-term trends and improvement patterns in speech skills. This may involve detecting trends in speech quality, changes in emotional states, changes in physiological indicators, etc. Data analysis can be performed using techniques such as statistical methods and machine learning algorithms to help users and educators better understand the development of speech skills.

[0252] 4) Adjustment of recommendations and improvement suggestions: Based on the results of long-term progress tracking, the system can adjust personalized recommendations and improvement suggestions to suit the user's long-term development needs. If the system finds that the user has made significant progress in a specific area or needs a specific type of support, the recommendations can be optimized based on these findings. This helps ensure that training plans and goals are aligned with the user's actual progress.

[0253] 5) Set long-term goals: Long-term progress tracking can also help users and educators set more challenging and achievable long-term goals. By understanding the user's improvement rate and trend, future speech skills goals can be set more intelligently and ensure that training programs are moving toward those goals.

[0254] 6) User feedback and engagement: Long-term progress tracking can be a useful communication tool between users and educators. Users can view their long-term improvements, understand that their efforts are recognized, and gain more motivation. Educators can provide more precise guidance and support based on the data to meet the needs of users.

[0255] In summary, long-term progress tracking is an important part of the brain science guidance and optimization system, which provides users with more effective personalized support and guidance by storing, analyzing and understanding the long-term development of speech skills. This helps to ensure that training plans and goals are consistent with the actual progress of users, thereby improving the long-term effects of eloquence and speech skills. At the same time, this also promotes active interaction and feedback between users and educators to jointly achieve continuous improvement of speech skills.

[0256] Through these steps, the system is able to generate personalized speech suggestions and improvement suggestions specific to each user based on data such as EEG signals, emotional states, and physiological indicators to help them improve their eloquence. These suggestions are based on real-time monitoring and data analysis to meet the unique needs of each user.

[0257] Real-time feedback: Users can view their EEG signals, emotional states, and physiological indicators in real time through the real-time feedback interface. The system will provide instant suggestions to help users adjust and optimize during the speech.

[0258] Long-term progress tracking: The system stores multi-dimensional data of users for long-term progress tracking. This helps users and educators understand the long-term improvement of presentation skills.

[0259] Implementation method (three):

[0260] Multi-sensor fusion technology: Multi-sensor fusion technology, including brain-computer interface, emotional sensors and physiological sensors, is used to integrate multi-dimensional data into the system.

[0261] Advanced data processing and analysis algorithms: Leverage advanced data processing and analysis algorithms, such as machine learning and deep learning, to interpret and understand data obtained from a variety of sensors.

[0262] Multimodal data fusion: Fusion of multimodal data such as EEG signals, emotional states, and physiological indicators to obtain more comprehensive user status information.

[0263] Multimodal data fusion is to integrate multiple types of data from different sensors into a comprehensive data model to obtain more comprehensive user status information. This usually involves multiple technical means and data processing steps. The specific process is as follows:

[0264] 1. Data collection: First, multimodal data is collected from different sensors. This includes EEG signals from the brain-computer interface, emotional state data from emotional sensors (such as heart rate, skin galvanic response, etc.), and physiological indicator data from physiological sensors (such as blood pressure, respiratory rate, etc.).

[0265] 2. Data synchronization: Ensure that data from different sensors are synchronized on the same timeline. This is to associate information from different data sources with events at a specific point in time. For example, understand when a user experiences an emotional change or the relationship between an emotional surge and speech activity.

[0266] 3. Data preprocessing: Preprocess the collected data, including denoising, filtering, calibration and normalization, etc. This helps ensure the quality and consistency of the data.

[0267] 4. Feature extraction: Extract meaningful features from the raw data of each sensor. For EEG signals, this involves extracting spectral features or event-related potentials. For emotional state data, this involves extracting the intensity of emotional expression and the type of emotion. For physiological indicator data, this includes extracting physiological responses associated with emotional experience.

[0268] 5. Multimodal fusion: In this step, multimodal data is fused into a comprehensive data model. This can be achieved through a variety of methods, including machine learning techniques such as neural networks or fusion algorithms. The goal of fusion is to integrate the information of different sensors into a comprehensive description of the user's state to obtain more comprehensive information.

[0269] 6. Data Analysis: The fused data is analyzed to understand the user’s state. This includes exploring the data to find patterns, trends, or correlations to better understand the user’s experience during the presentation.

[0270] 7. Personalized algorithm application: Use fused multimodal data to generate personalized recommendations and improve recommendations. This usually requires machine learning algorithms to customize recommendations based on the user's multimodal state information to meet their unique needs and performance.

[0271] 8. Real-time feedback interface: Create a user-friendly interface to provide users with visualization and real-time suggestions of multimodal data. This can help users understand their status and make adjustments and optimizations when necessary.

[0272] The whole process involves multi-sensor data integration, data processing, feature extraction, multi-modal fusion and data analysis, which helps to more comprehensively understand the user's status during the speech process in order to provide personalized support and improvement suggestions.

[0273] Personalized algorithms: Develop personalized algorithms to generate customized recommendations and improvement suggestions based on multi-dimensional data.

[0274] Generating customized recommendations and improvements based on multidimensional data typically involves the use of machine learning and data analytics techniques. The following are the technical means and steps to achieve this goal:

[0275] 1. Data preprocessing: Before multimodal data is used to generate recommendations, the data needs to be preprocessed. This includes denoising, data cleaning, outlier detection, etc. Data preprocessing helps ensure the quality and consistency of the data.

[0276] 2. Feature extraction: Extract meaningful features from multidimensional data. For EEG signals, emotional states, and physiological indicators, this includes extracting features related to the user's state, such as spectral characteristics, emotional intensity, physiological response, etc.

[0277] 3. Data integration: Integrate the data from different sensors to obtain a complete multimodal description of the user state. This can be achieved by fusing the data into a comprehensive data representation.

[0278] 4. Recommendation model training: Use machine learning algorithms, such as supervised learning or reinforcement learning, to train a personalized recommendation model based on the integrated multimodal data. The model’s task is to understand the user’s state, including emotions, physiological reactions, and EEG activity, and provide corresponding recommendations.

[0279] 5. Personalized suggestion generation: Based on the trained model, generate personalized suggestions for the user. This can include suggestions on emotion adjustment, speech skill improvement, emotion management, etc. The way to generate suggestions can vary depending on the specific problem and system design.

[0280] 6. Improvement suggestion generation: Similar to personalized suggestion generation, improvement suggestions are designed to help users improve their speech or eloquence. This includes providing more specific techniques or methods to improve speech quality and expression ability.

[0281] 7. Real-time feedback interface: The generated suggestions are presented in a user-friendly way on the real-time feedback interface. This can be a visual interface or text prompts so that users can understand and take corresponding actions.

[0282] 8. Algorithm tuning: Continuously improve and optimize personalization algorithms to provide more accurate and effective recommendations. This includes periodic model retraining and performance evaluation.

[0283] Personalized recommendations and improved recommendation generation involve multiple steps such as data preprocessing, feature extraction, model training, and recommendation generation. Machine learning and data analysis techniques are used to process multimodal data and generate personalized recommendations related to user status and performance.

[0284] Real-time feedback interface: Create a user-friendly real-time feedback interface to display multi-dimensional data and provide instant suggestions.

[0285] Data storage and analysis tools: Build systems for storing and analyzing multi-dimensional data to enable long-term progress tracking and deeper research.

[0286] In practical applications, the present invention further includes: a user management module; the user management module is used to store the personalized evaluation and feedback results, the user's EEG signal and emotional state data.

[0287] The user management module specifically includes:

[0288] (I) Structure:

[0289] 1. User Authentication:

[0290] Registration and Login: Users can register a new account or log into the system using an existing account.

[0291] Social Media Integration: Provide social media login option so that users can quickly log in using their social media accounts like Google, Facebook, LinkedIn, etc.

[0292] Password Management: Allows users to change passwords, retrieve passwords, and manage account security.

[0293] 2. Study plan management:

[0294] Goal setting: Users can set learning goals, such as improving presentation skills, enhancing oral skills, etc.

[0295] Course Selection: Users can browse available courses, modules, and resources and select content that fits their learning goals.

[0296] Study Schedule: Users can create, edit, and view a study schedule to schedule study activities and tasks.

[0297] 3. Learning progress tracking:

[0298] Learning History: Provides learning history, showing the courses, modules, and tasks that the user has completed.

[0299] Learning Report: Generate learning reports including grades, progress, and suggestions to help users understand their learning performance.

[0300] 4. Resource access:

[0301] Resource Library: Users can access a resource library including textbooks, videos, audios, exercises, and lecture samples to meet their learning needs.

[0302] Personalized recommendations: The system recommends relevant resources based on the user’s learning goals and historical progress.

[0303] 5. Notifications and reminders:

[0304] Notification Center: Users can view system-generated notifications, reminders, and suggestions, including task reminders, new course notifications, etc.

[0305] Personalized reminders: Users can customize notification settings and choose how to receive notifications (email, SMS, push notifications) and how often.

[0306] 6. User Support:

[0307] Online chat support: Provide online chat support so that users can get help, answers to questions and support at any time.

[0308] Help Center: Create a help center, including FAQs, tutorials, and user guides, for users to query and learn by themselves.

[0309] (II) Function:

[0310] 1. User Authentication: Data Security: Ensure that users’ personal information and learning data are protected by strong data security measures, including data encryption and privacy protection.

[0311] 2. Learning plan management: Goal management: Users can update their learning goals at any time to adapt to changing needs.

[0312] Learning Plan: Allows users to create, edit, and delete learning plans to organize learning activities effectively.

[0313] 3. Learning progress tracking: Learning history: The system saves complete learning history, including completed tasks, courses and modules.

[0314] Study Report: Generate detailed study reports, including performance analysis, progress tracking and study suggestions.

[0315] 4. Resource Access: Resource Search and Filtering: Users can easily search and filter resources to find the learning materials they need.

[0316] Resource evaluation: Users can provide evaluation and feedback for resources to help other users choose appropriate learning resources.

[0317] 5. Notifications and Reminders: Notification Personalization: Users can customize notification settings to ensure they receive notifications according to their preferences.

[0318] Reminder Management: Users can set and manage study reminders to avoid missing important tasks and deadlines.

[0319] 6. User Support: Live Chat Support: Provide fast-response online chat support to resolve user questions and concerns.

[0320] Help Center Maintenance: The Help Center is regularly updated and maintained to provide the latest support information and tutorials.

[0321] Implementation method (three):

[0322] 1. Authentication and Security: Use secure authentication protocols and tools, such as OAuth2.0, to ensure the security of user authentication.

[0323] Implement data encryption to ensure that users' personal information and learning data are protected.

[0324] 2. Learning plan management: Develop a custom learning plan management tool that allows users to create, edit, and delete learning plans.

[0325] Integrate goal-setting tools so users can clarify their learning goals.

[0326] 3. Learning progress tracking: Integrate learning record database to record users’ learning history and progress.

[0327] Develop a learning report generation system to automatically generate learning reports.

[0328] 4. Resource access: Build a resource library and search engine to enable users to easily find and access learning resources.

[0329] Integrate user rating and feedback systems so that users can provide reviews and suggestions for resources.

[0330] 5. Notifications and Reminders: Integrated notification system allows the system to generate personalized notifications and reminders.

[0331] Develop reminder management tools to allow users to customize reminder settings.

[0332] 6. User Support: Integrate a live online chat support system so that users can communicate with support representatives in real time.

[0333] Create and maintain a help center including FAQs, tutorials, and user guides.

[0334] The process of each module working together is as follows:

[0335] 1. The voice recognition and sentiment analysis module works in conjunction with the comprehensive evaluation and feedback module: When a user starts eloquence training, the voice recognition and sentiment analysis module is responsible for real-time monitoring of the user's speech content, speech speed and tone, as well as emotional expression. These data are transmitted to the comprehensive evaluation and feedback module, which analyzes the user's performance through an evaluation engine, including language fluency, emotional communication, etc.

[0336] 2. The comprehensive evaluation and feedback module works in collaboration with the brain science guidance and optimization module: The comprehensive evaluation and feedback module generates personalized evaluation and feedback, pointing out the user's performance strengths and improvement points. These evaluation and feedback information are passed to the brain science guidance and optimization module, which analyzes the user's EEG signals and emotional state. At the same time, improvement suggestions can also be conveyed through this process to help users better optimize.

[0337] 3. Collaboration between the Brain Science Guidance and Optimization Module and the Comprehensive Evaluation and Feedback Module: The Brain Science Guidance and Optimization Module generates personalized guidance and optimization suggestions based on EEG signals and emotional state data. These suggestions can be passed back to the Comprehensive Evaluation and Feedback Module to enrich the user's feedback report and help the user better understand their performance.

[0338] 4. Data support for the user management module: The user management module is responsible for managing the user's personal data, including eloquence training history, EEG signals and emotional state records, etc. These data are very important for the work of other modules, and they support personalized evaluation, guidance and optimization.

[0339] 5. The user interface module works in coordination with all other modules: The user interface module is the window for users to interact with the system. It delivers information to users through a real-time feedback interface, including speech recognition results, sentiment analysis results, eloquence expression evaluation, EEG signal charts, etc. This helps users understand their performance and optimize it during the speech.

[0340] In summary, the modules work together in the "Immersive Eloquence Training System Based on Brain-Computer Interface Technology" to achieve objective, personalized, and real-time eloquence training. The close cooperation between speech recognition, sentiment analysis, comprehensive evaluation and feedback, brain science guidance and optimization, user management, and user interface modules enables the system to provide users with a better eloquence training experience.

[0341] The present invention can bring about many beneficial effects:

[0342] 1. Improved objectivity and accuracy: By monitoring the user's EEG signals and emotional state, the system is able to provide a more objective and accurate evaluation of eloquence. The subjective evaluation and feedback problems in traditional methods are solved, and users can understand their performance level more accurately.

[0343] 2. Personalized training: The present invention generates personalized eloquence evaluation and improvement suggestions based on each user's EEG signals and emotional state data. This personalized training can better meet the unique needs of users and help them improve their eloquence more quickly.

[0344] 3. Real-time feedback and optimization: The present invention can provide real-time feedback and suggestions during the speech process, helping users to adjust their speech strategies instantly. This helps users to make timely optimizations in actual expression and improves the efficiency of training.

[0345] 4. Long-term progress tracking: In addition to real-time feedback, the system also stores the user's EEG signals and emotional state data, allowing users and educators to track long-term progress. This helps understand the long-term development trend of speech skills and develop a more comprehensive training plan.

[0346] 5. Improve eloquence: In general, the present invention helps to improve the eloquence of users. Through objective, personalized, and real-time evaluation and feedback, users can improve their speech skills and emotional expression in a more targeted manner, thereby improving their eloquence.

[0347] 6. Promote innovation in the eloquence training industry: This invention represents the innovative trend in the field of eloquence training. It combines brain-computer interface technology, emotion recognition technology and speech training, and is expected to promote the eloquence training industry to develop in a more scientific and intelligent direction.

[0348] In summary, the present invention can improve objectivity, personalized training, real-time feedback, long-term progress tracking, enhancement of eloquence and expression ability, and promote innovation in the eloquence training industry.

Claims

1. An immersive eloquence expression training system based on brain-computer interface technology, characterized in that: include: A multi-mode game mode creation module for creating different types of speech training games; The multi-mode game mode creation module specifically includes: A competitive game creation unit, used to create competitive games; the competitive games include virtual debates and speech contests; A speech recognition unit, used to recognize the speech content of the user participating in the competitive game and convert the speech content into text; An emotion analysis unit, used for analyzing the emotion state and emotion change of the users participating in the competitive game according to the speech content; A real-time feedback module for providing real-time performance evaluation and suggestions on the speech content through feedback from virtual judges and audiences; The multi-mode game mode creation module specifically includes: A simulation game creation unit, used to create simulation games; the simulation games include press conferences and press conferences; A speech recognition and emotion analysis unit, used to recognize the speech of the user participating in the simulation game, and analyze the expression and voice emotion of the user participating in the simulation game according to the speech, and provide feedback; The multi-mode game mode creation module specifically includes: An interactive game creation unit, used to create interactive games; the interactive games include dialogues, debates or cooperative tasks with virtual characters; A speech synthesis unit, used to enable the virtual character to respond to the dialogue of the users participating in the interactive game through speech; A body language simulation unit, used to make the virtual character imitate body language; the body language includes gestures, expressions and postures; Among them, body language simulation specifically includes: Motion capture technology: Motion capture technology is used to track the user's body movements and postures; users wear sensor devices to capture their body movements and postures; Virtual character modeling: Virtual characters need to be modeled in advance, including their appearance, body structure and movements; Real-time motion matching: When users interact with virtual characters, motion capture technology is used to capture the user's body movements and match these real-time body movements with the model of the virtual character to simulate the body language of the virtual character; Expression and gesture simulation: The virtual character model also includes data for simulating different expressions and gestures; according to the user's interaction and dialogue situation, the virtual character's expression, gesture and posture are adjusted to make the interaction more realistic; Contextual awareness: Natural language processing technology is used to understand the content of the conversation, and emotion recognition technology is used to detect the emotional state of the user and the virtual character to adjust the virtual character's body language simulation according to the situation; A natural language understanding unit, for analyzing the conversation between the user participating in the interactive game and the virtual character based on the imitated body language, and understanding the user's expression and emotional state; The multi-mode game mode creation module specifically includes: A role-playing game creation unit, used to create a role-playing game; the role-playing game includes playing the role of a famous orator; A virtual reality simulation unit, used to create a virtual situation in the role-playing game; the virtual situation includes a historical scene and a virtual character; a cultural learning support unit for providing multimedia information on character background, history, and culture in said role-playing game; A speech recognition and emotion analysis unit, for analyzing the speech and expression of the user participating in the role-playing game based on the virtual situation, the character background, the historical and cultural multimedia information, and providing feedback on speech, emotion and expression; The multi-mode game mode creation module specifically includes: A music game creation unit, used to create a music game; the music game includes a music speech and a singing competition; an audio analysis unit, for analyzing the voice, musical expression and tone of a user participating in the music game, and generating an analysis result; An emotion analysis unit, used to evaluate the emotion expression of the users participating in the music game and generate an evaluation result; A real-time evaluation unit, used for providing instant feedback based on the analysis results and the evaluation results; A comprehensive evaluation and feedback module is used to monitor the performance data of users in different types of speech training games, generate personalized evaluation and feedback results, and send them to the brain science guidance and optimization module; the performance data includes voice, speech speed, intonation, body language and emotional expression; the performance data is obtained by combining speech recognition technology, emotion analysis algorithm and brain-computer interface technology; A brain science guidance and optimization module is used to monitor the user's EEG signals, emotional state data and physiological indicators in real time through an EEG interface and an emotional sensor, and based on the personalized evaluation and feedback results, analyze the user's eloquence expression process according to the EEG signals, the emotional state data and the physiological indicators, and conduct long-term progress tracking; the brain science guidance and optimization module specifically includes: an EEG interface and an emotional sensor, which are used to monitor the user's EEG signals, emotional state data and physiological indicators; a data processing and analysis unit, which is used to extract effective information from the EEG signals, the emotional state data and the physiological indicators; a personalized optimization unit, which is used to generate personalized suggestions and improvement suggestions based on the effective information; a real-time feedback unit, which is used to display the monitored EEG signals, emotional state data, physiological indicators and personalized suggestions and improvement suggestions in real time; a storage unit, which is used to store the monitored EEG signals, the emotional state data and the physiological indicators; Multimodal data fusion of EEG signals, emotional states and physiological indicators to obtain more comprehensive user status information; The immersive eloquence expression training system based on brain-computer interface technology performs big data analysis, and its process includes: data collection: collecting user performance data in various game modes, including speech content, audio data, interaction data, and timestamps; data analysis: through big data analysis technology, in-depth analysis of the collected data is performed to identify the user's strengths and needs; including the application of data mining, machine learning and statistical analysis methods.

2. The immersive eloquence expression training system based on brain-computer interface technology according to claim 1 is characterized in that: Also includes: User management module; The user management module is used to store the personalized evaluation and feedback results, the user's EEG signal and emotional state data.

3. The immersive eloquence expression training system based on brain-computer interface technology according to claim 1 is characterized in that: The comprehensive evaluation and feedback module specifically includes: An evaluation engine unit, which is used to monitor the performance data of users who participate in different types of speech training games in eloquence training by combining speech recognition technology, sentiment analysis algorithms, and brain-computer interface technology; A personalized evaluation unit, configured to generate personalized evaluation and feedback results based on the performance data; The feedback and improvement unit is used to generate a feedback report and personalized improvement suggestions based on the personalized evaluation and feedback results.

4. The immersive eloquence expression training system based on brain-computer interface technology according to claim 2 is characterized in that: The user management module specifically includes: User identity verification unit, used to verify user information; Learning plan management unit, used to set learning goals, select learning courses and create learning reports; Learning progress tracking unit, used to provide learning history and generate learning reports; A resource access unit, used to access the resource library; Notification and reminder unit, used to view notifications, reminders and suggestions through the user center, and customize personalized notification methods; User support unit for providing online chat support and creating help centers.

Images