A multi-dimensional eloquence improvement and collaborative creation method, system, device and medium

By combining deep learning and natural language processing technologies with real-time interaction with virtual objects, the problem of insufficient interactivity and creativity in traditional public speaking training has been solved, enabling personalized public speaking improvement and creative content generation, thereby enhancing the user's presentation performance.

CN118070777BActive Publication Date: 2024-11-26CHINA NEW LINE EDUCATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410407063.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-07
Publication Date
2024-11-26
Estimated Expiration
2044-04-07

AI Technical Summary

Technical Problem

Traditional methods of public speaking training and speech preparation are limited by offline interactivity and the lack of real-time interaction in online resources, making it difficult to stimulate users' creative thinking. The speech content lacks systematic support, resulting in limited improvement in public speaking skills and a lack of diverse viewpoints.

Method used

It uses deep learning and natural language processing technologies to analyze speech content, interacts with users in real time through virtual objects, provides personalized feedback and generates creative content, comprehensively evaluates eloquence skills, and builds an online collaborative creation platform.

Benefits of technology

It enhances the interactivity and creativity of learning, provides customized feedback, comprehensively improves speaking skills, stimulates users' creative thinking, and enriches speech content.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118070777B_ABST
    Figure CN118070777B_ABST
Patent Text Reader

Abstract

The present invention proposes a multi-dimensional eloquence improvement and collaborative creation method, system, device and medium, the method comprising: obtaining speech content, analyzing the speech content, and obtaining specified fields; determining the key creative points of the speech based on the specified fields based on a pre-built deep learning model; triggering a virtual object to simulate the dialogue and feedback in the speech scene based on a preset strategy, and controlling the virtual object to output creative content associated with the key creative points in a specified manner; obtaining real-time interactive information, which is user feedback collected when the user interacts with the virtual object, and editing the speech content according to the real-time interactive information to generate a speech manuscript. The present invention interacts with users in real time through virtual objects and inspires users to generate new creative thinking based on unique creative highlights, helping users improve their eloquence and speech skills.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of speech training, and in particular to a multi-dimensional eloquence improvement and collaborative creation method, system, equipment and medium. Background Art

[0002] In the traditional eloquence training and speech preparation process, users usually rely on offline courses, personal practice or use relatively static online resources. However, since the interactivity of offline courses is limited by time and place, and online resources often lack real-time interaction, it is difficult to stimulate users' active participation and creative thinking; in addition, creative stimulation is related to personal knowledge reserves, and it is difficult for users to write speech ideas beyond their own knowledge reserves. In addition, in the absence of systematic and scientific support, speech ideas are relatively single, unable to reflect eye-catching creative ideas, and unable to find new speech ideas, resulting in monotonous speech content, which limits the improvement of users' eloquence skills. Summary of the invention

[0003] The embodiment of the present invention provides a multi-dimensional eloquence improvement and collaborative creation method, system, device and medium to solve the problems existing in the related technologies. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present invention provides a multi-dimensional method for improving eloquence and collaborative creation, including:

[0005] Get the speech content, analyze the speech content, and get the specified fields;

[0006] Determine the key creative points of the speech based on the specified fields based on the pre-built deep learning model;

[0007] Triggering virtual objects to simulate dialogue and feedback in a speech scene based on preset strategies, and controlling virtual objects to output creative content associated with key creative points in a specified manner;

[0008] Acquire real-time interactive information. Real-time interactive information is user feedback collected when users interact with virtual objects. Edit the speech content based on the real-time interactive information to generate a speech manuscript.

[0009] In one implementation, identifying a specified field in the speech content includes:

[0010] Extract words and sentences from the speech content based on natural language processing algorithms to obtain key words and sentences;

[0011] The occurrence frequency of key words and phrases in the key text sentences corresponding to the speech content is calculated, and the key words and phrases with an occurrence frequency higher than a preset frequency value are marked as designated fields.

[0012] In one embodiment, determining the key creative points of a speech according to a specified field based on a pre-built deep learning model includes:

[0013] Analyze the semantic similarity between the specified field and the text part outside the specified field based on the BERT model;

[0014] The specified fields whose semantic similarity is lower than the preset similarity value are marked as key creative points.

[0015] In one embodiment, triggering a virtual object to simulate a conversation and feedback in a speech scene based on a preset strategy includes:

[0016] Conduct eloquence skill analysis on the speech content to obtain eloquence skill analysis results;

[0017] Obtain speech goals and historical conversation data, and import speech goals, eloquence skill analysis results, and historical conversation data into a pre-built deep learning model to extract comprehensive features;

[0018] Based on the preset interaction strategy and eloquence guidance strategy, the interaction mode between the virtual object and the user is determined according to the comprehensive characteristics, and eloquence improvement suggestions associated with the comprehensive characteristics are generated, so that the virtual object outputs the eloquence improvement suggestions associated with the comprehensive characteristics in a specified interaction mode.

[0019] In one embodiment, the method for generating creative content includes:

[0020] Extract eloquence characteristics based on speech content; eloquence characteristics include emotional diversity, expression clarity and language diversity;

[0021] Based on the conditional encoder receiving key creative points and eloquence features, outputting a latent space;

[0022] Generate creative contents matching eloquence characteristics from latent space based on conditional generator.

[0023] In one embodiment, it further includes:

[0024] Extract features from speech content to obtain eloquence features;

[0025] The scores of various eloquence skill indicators are calculated according to various eloquence characteristics. The eloquence skill indicators include language richness index, rhetoric skill index, emotional intensity index and logical coherence index.

[0026] Based on the weighted fusion algorithm, a comprehensive scoring result is calculated according to the scores of each eloquence skill indicator, and instant feedback suggestions are generated based on the comprehensive scoring result.

[0027] In one embodiment, it further includes:

[0028] Receiving response information generated by the user in response to the instant feedback suggestion;

[0029] The corresponding weights in the weighted fusion algorithm are adjusted according to the response information to optimize the generation results of instant feedback suggestions.

[0030] In a second aspect, an embodiment of the present invention provides a multi-dimensional eloquence improvement and collaborative creation system, which executes the multi-dimensional eloquence improvement and collaborative creation method as described above.

[0031] In a third aspect, an embodiment of the present invention provides an electronic device, the device comprising: a memory and a processor. The memory and the processor communicate with each other through an internal connection path, the memory is used to store instructions, the processor is used to execute the instructions stored in the memory, and when the processor executes the instructions stored in the memory, the processor executes the method in any one of the above-mentioned embodiments.

[0032] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer, the method in any one of the above-mentioned embodiments is executed.

[0033] The advantages or beneficial effects of the above technical solution include at least:

[0034] The present invention can determine unique creative highlights according to the content of the speech, and stimulate users to generate new creative thinking based on the unique creative highlights during real-time interaction with users through virtual objects. Users can improve the content of the speech from different angles based on their creative thinking, and can modify the speech manuscript in time during the interaction to enhance the interaction and creativity of learning; in addition, the user's speech skills can be analyzed according to the user's speech content, and targeted suggestions can be made on the user's eloquence skills from multiple dimensions, providing users with a more efficient, interactive and personalized eloquence improvement platform.

[0035] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present invention will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present invention and should not be regarded as limiting the scope of the present invention.

[0037] Figure 1It is a flowchart of the multi-dimensional eloquence improvement and collaborative creation method of the present invention;

[0038] Figure 2 The structure block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present invention. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0040] In traditional eloquence training and speech preparation, participants usually rely on offline courses, personal practice, or the use of relatively static online resources. Although these methods are effective to a certain extent, they have several significant limitations:

[0041] (1) When it comes to modifying speech manuscripts and improving their eloquence skills, participants can only rely on their own intuition or the audience's personal experience to make improvements. However, this improvement method is often not comprehensive enough. Since it only focuses on one or several aspects of eloquence, such as language expression or emotional communication, it ignores the multi-dimensional characteristics of eloquence improvement, which limits the extent to which participants can improve their eloquence.

[0042] (2) The interactivity of offline courses is limited by time and location, while online resources often lack real-time interaction and are difficult to stimulate users' active participation and creative thinking;

[0043] (3) Creative stimulation is also related to personal knowledge reserves. Even if users participate in creative stimulation discussions, they may not necessarily get better creative thinking. As a result, creative ideas are relatively single when knowledge reserves are limited, and the content of users’ speeches cannot be enriched.

[0044] (4) The lack of a mechanism to evaluate user performance in real time and provide immediate improvement suggestions makes real-time evaluation and optimization difficult, making it difficult for users to significantly improve their eloquence in a short period of time.

[0045] In view of the above problems, this embodiment provides a multi-dimensional eloquence improvement and collaborative creation method, which can effectively overcome the limitations of traditional eloquence training methods and provide users with a more efficient, interactive and personalized eloquence improvement platform:

[0046] (1) Through deep learning and natural language processing technology, the user's speech content and performance can be analyzed, and customized feedback based on individual characteristics can be provided, thereby effectively improving the effect of eloquence training;

[0047] (2) Build an online collaborative creation platform that allows users to interact with virtual objects in real time, jointly edit speech manuscripts, and provide instant improvement suggestions to enhance the interactivity and creativity of learning;

[0048] (3) Comprehensively consider multiple dimensions of eloquence, such as language style, voice characteristics, and emotional expression, and conduct a comprehensive assessment of the user's eloquence skills to comprehensively improve the user's eloquence level.

[0049] In order to better understand the user's speech intentions, this embodiment can pre-identify multi-dimensional eloquence characteristics such as language style and emotions of the speech content, thereby providing users with more personalized eloquence improvement suggestions to help them improve their speech skills and emotional expression, thereby improving the overall speech effect.

[0050] The speech content can be input text data or collected audio data. The speech content is preprocessed. For text data, preprocessing steps such as word segmentation, stop word removal, and part-of-speech tagging can be performed to convert the text into a format that can be processed by the feature extraction model; for audio data, feature extraction can be performed, such as extracting Mel-frequency cepstral coefficients (MFCCs), fundamental frequency (F0), energy and other features, and converting the audio signal into a numerical representation that can be processed by the feature extraction model.

[0051] Among them, the feature extraction model can use the embedding layer in the deep learning model, such as Word2Vec, GloVe, or the embedding layer of the pre-trained language model (such as BERT), to convert the text data into a high-dimensional vector representation and capture the semantic information of the vocabulary. For audio data, an acoustic model, such as acoustic RNN or CNN, can be applied to process the extracted audio features to capture the language style related information in the audio.

[0052] Use models such as recurrent neural networks (RNN), long short-term memory networks (LSTM) or gated recurrent units (GRU) to process the above-mentioned serialized text vectors and audio feature vectors, and extract language style features from the output of models such as RNN. These features may include but are not limited to sentence complexity, vocabulary diversity, emotional tendency, intonation fluctuations, etc., and then combine the results of text and audio analysis to comprehensively obtain the user's language style feature vector.

[0053] The extracted language style feature vector is input into a classifier, such as a support vector machine (SVM), a multi-layer perceptron (MLP) or other suitable machine learning model, to classify and identify the user's language style, and identify the uniqueness and category of the user's language style; the classifier can be trained based on predefined language style categories (such as formal, informal, didactic, narrative, etc.) to achieve accurate style recognition.

[0054] Finally, based on the results of the language style analysis, the system can provide users with specific feedback and suggestions on how to improve their speaking style, such as adjusting vocabulary selection, improving sentence structure, or adjusting voice intonation to better suit the occasion and target audience of the speech.

[0055] In some embodiments, the user's speech content can be analyzed to identify emotions and moods, such as excitement, nervousness, confidence or uncertainty. It uses natural language processing and audio analysis technology to deeply understand the user's emotional state.

[0056] Emotion recognition mainly analyzes the speech content of audio type, analyzing multiple dimensions such as pitch variation, expressiveness index, language fluency, etc. Specifically, pitch variation is mainly used to reflect the diversity and complexity of the tone in the speech, and its calculation formula is:

[0057] ;

[0058] T i represents the average pitch of the i-th speech segment;

[0059] Represents the average value of the pitch of all speech segments;

[0060] N t Indicates the total number of speech segments.

[0061] The expressiveness index is used to quantify the dynamic range and emotional intensity of a speech. The calculation formula is:

[0062] ;

[0063] N e is the total number of words or phrases that are emphasized;

[0064] E i is the energy array of the i-th emphasized word or phrase; since the emphasized part usually has a higher energy value and / or a change in fundamental frequency, the emphasized word or phrase can be identified by the change in sound intensity (energy); for example, when the user wants to emphasize a certain point, they may increase the volume of the voice (increase energy) and change the tone of voice. For each recognized emphasized word or phrase, calculate its energy array, which can be achieved by extracting the sound intensity (energy) value of the corresponding part of the word or phrase in the audio, which may involve averaging or integrating the energy value of the audio segment to obtain a representative energy value; and the energy values ​​of different words or phrases constitute the energy array of the emphasized word.

[0065] The language flow index is used to assess the fluency of speech, taking into account vocabulary diversity and grammatical correctness. Its calculation formula is:

[0066] ;in,

[0067] U w the number of unique words assigned to the speech;

[0068] N w is the total number of words in the speech;

[0069] S e is the number of grammatical errors;

[0070] N s is the total number of sentences.

[0071] The extracted emotional features, audio features and the pitch change degree T calculated above are vi , expressiveness index E fi and language procedural index F li Weighted fusion is performed according to certain weight coefficients λ, μ, and ν to obtain a comprehensive feature vector; the fused comprehensive feature vector is input into the deep learning model, which can simultaneously conduct a comprehensive analysis of the user's emotional state and eloquence performance, and output emotion classification and comprehensive scoring results. Finally, based on the emotion classification and comprehensive scoring results, the model provides users with personalized instant feedback suggestions to help them improve their speech skills and emotional expression, thereby improving the overall speech effect.

[0072] The comprehensive score results can help identify which eloquence indicators have a greater impact on overall speech performance, thereby determining the priority and key areas for improvement, which is beneficial for users with limited resources and time to quickly get useful suggestions and information to improve their skills. The comprehensive score can also serve as an indicator of user progress, allowing users to see their overall score improve over time and practice, which can greatly motivate users to continue to improve and practice.

[0073] In addition, the model parameters can be optimized and adjusted according to the response information generated by users responding to instant feedback suggestions in actual applications, including the adjustment of weight coefficients λ, μ, ν, and the optimization of the deep learning model structure, so as to improve the recognition accuracy and adaptability.

[0074] In some embodiments, the match between the speech goal and the speech content can also be calculated according to the speech goal, and appropriate eloquence improvement suggestions can be generated based on the match. Among them, the speech goal is used to describe the purpose that the user wants to achieve in this speech. For example, the speech goal that the user can enter in the current interaction is "How can I convey my point of view more effectively?" The speech goal entered by the user can be converted into an embedding vector through an NLP model (such as BERT), that is, the speech goal in text form is input into the pre-trained BERT model. The BERT model understands the context and meaning of the text through its multi-layer Transformer structure, and converts the natural language text into a point in a high-dimensional space, that is, the embedding vector V target ,The embedding vector contains the semantic information of multiple aspects of the speech goal. Through this process, the complex and abstract speech goal is converted into a mathematical representation that can be understood and processed by the subsequent models and algorithms to evaluate the matching degree between the speech content and the speech goal.

[0075] Afterwards, the features of the speech content are extracted to generate the corresponding feature vector V content , and use cosine similarity to calculate the matching degree between the target vector and the content vector. The matching degree calculation formula is:

[0076] ;

[0077] Assume that the calculated matching degree is higher than the preset value, which indicates that the content of the speech is more consistent with the set goals.

[0078] The matching degree can also be combined with the eloquence skill index to jointly evaluate the user's speech performance. The eloquence skill index includes logic and emotional expression; and the corresponding weights are set to calculate the comprehensive score of eloquence skills.

[0079] Among them, the logic score can be obtained by analyzing the sentence coherence and structural logic of the speech content, assuming the score is 0.9; and the emotional expression score can be obtained by evaluating the emotional diversity and intensity of the speech content through the sentiment analysis model, assuming the score is 0.85; assuming that the weights of logic and emotional expression are 0.6 and 0.4 respectively, the comprehensive score of eloquence skills is 0.6×0.9+0.4×0.85=0.88.

[0080] Finally, the above evaluation scores are integrated to obtain the final evaluation score. The final evaluation score integrates the matching degree between the speech goal and the speech content, as well as the eloquence skill score, and weighs the importance of the two, that is, the matching degree and the comprehensive score of eloquence skills are integrated. Assuming that the weights of the matching degree and the comprehensive score of eloquence skills in the final evaluation are 0.5 and 0.5 respectively (assuming that the two are equally important), the final evaluation score is: 0.5×0.8+0.5×0.88=0.84.

[0081] Through the above calculation process, we can get a final evaluation score that comprehensively considers the match degree of speech goals and the performance of eloquence skills; assuming that the final evaluation score is 0.84, a higher score indicates that the speech content not only has a high match degree with the set speech goals, but also performs well in logic and emotional expression.

[0082] Based on the comprehensive scoring system, specific feedback can be generated to help users identify strengths and areas for improvement in their speeches. In addition, the system can adaptively adjust weight parameters based on user feedback and historical data to optimize the accuracy and applicability of the evaluation model.

[0083] In addition, in addition to providing personalized feedback to users in the above manner, this embodiment can also use natural language processing technology to detect innovative ideas, highlights and eye-catching points in the speech, and inspire users to think innovatively based on the creative highlights to enrich the content of the speech. Figure 1 As shown, the specific steps include:

[0084] Step S1: Acquire speech content, analyze the speech content, and obtain designated fields.

[0085] After preprocessing steps such as word segmentation, stop word removal, and part-of-speech tagging of the speech manuscripts submitted by users, natural language processing technologies such as TF-IDF, TextRank and other algorithms are used to extract key words and phrases that can represent the main content and viewpoints of the text from the processed text.

[0086] Each sentence in the text is scored to assess its importance, and sentences with importance above a threshold are marked as key text sentences; this can be achieved through a graph-based ranking algorithm such as TextRank, which takes into account the interrelationships between sentences and the frequency of occurrence of keywords in the sentences.

[0087] The occurrence frequency of key words and phrases in the key text sentences corresponding to the speech content is calculated, and the key words and phrases with a frequency higher than a preset frequency value are marked as designated fields, that is, the designated fields are key words and phrases with a higher frequency of occurrence.

[0088] Step S2: Determine the key creative points of the speech according to the specified fields based on the pre-built deep learning model.

[0089] In order to further identify unique and eye-catching content in the speech content and avoid the creative points being too common and resulting in the sameness of the speech content, the semantic similarity between the identified designated fields and other parts of the text is further analyzed to identify unique innovative ideas. Using semantic similarity analysis technology, such as sentence embedding comparison based on BERT, the designated fields with semantic similarity lower than the preset similarity value are marked as key creative points, which is equivalent to key creative points being words and sentences that appear frequently in the speech text but are relatively unique and relatively novel in semantics; generating related creative content based on key creative points to inspire users to think creatively. The creative content can propose key ideas, the most influential viewpoints, and possible improvement suggestions in the speech content, and develop unique and eye-catching speech ideas.

[0090] Step S3: triggering the virtual object to simulate the dialogue and feedback in the speech scene based on the preset strategy, and controlling the virtual object to output creative content associated with the key creative points in a specified manner.

[0091] This embodiment creates a virtual object through natural language processing technology and a conversational AI engine. The virtual object can be a tutor or a virtual character imagined by the user. The virtual object can interact with the user in real time and simulate the dialogue and feedback in a speech scene.

[0092] The personalized feedback and suggestions generated above can be output to users in the form of natural language dialogue through virtual objects, allowing users to understand their own speech problems and receive targeted suggestions for improving their eloquence during the interaction with virtual objects.

[0093] Furthermore, the eloquence skills can be deeply analyzed to quantify and reflect the user's unique needs and status through multiple factors such as the speech goals input by the user, the eloquence skill analysis results, the current conversation context and / or the previous conversation history, providing a basis for personalized conversation strategies and eloquence guidance when the virtual object interacts with the user in real time.

[0094] Specifically, the results of the eloquence skill analysis are obtained by analyzing the speech content, including voice tone, speaking speed stability, and argument clarity.

[0095] 1) Voice pitch can be calculated by analyzing the average fundamental frequency and variation range of the user's voice through voice signal processing technology. The calculation formula is:

[0096] ;

[0097] Among them, f n represents the fundamental frequency of the nth syllable, σ represents the standard deviation, and N represents the total number of syllables.

[0098] The average fundamental frequency and variation range of the user's voice can be used to assess whether the user's voice pitch has enough variation to attract the audience's attention and convey different emotions and emphasis points. For example, a large fundamental frequency variation range may indicate that the user effectively uses different pitches in his speech to increase the dynamics of his expression, while a small fundamental frequency variation range may suggest that the user needs to practice increasing the diversity of his voice tonality.

[0099] 2) Speech rate stability can be evaluated by measuring the change in speech rate within a time window:

[0100] ;

[0101] Among them, V w represents the speaking rate in the wth time window. The speaking rate usually refers to the number of words or syllables spoken in a certain period of time. is the average speaking speed.

[0102] 3) The clarity of the argument can be analyzed through natural language processing technology to determine the logical coherence and clarity of expression of the argument:

[0103] ;

[0104] Among them, arg j represents the j-th argument, Coherence and Clarity represent the scoring functions of coherence and clarity respectively, and the scoring functions of coherence and clarity have been disclosed in the prior art and will not be described in detail here.

[0105] The current conversation context is information about the current conversation stage, which may include the current conversation topic, the type of question raised by the user, etc.

[0106] Conversation history, which is the accumulated information of previous conversations, can help understand the user's progress, previous questions, and feedback from virtual objects.

[0107] The current conversation context and conversation history can be processed by a deep learning model (such as LSTM or Transformer) to obtain context feature vectors and historical feature vectors.

[0108] An advanced deep learning model (such as deep Transformer) is used to fuse all feature vectors such as the above speech goals, eloquence skill analysis results, current conversation context, and previous conversation history to generate a comprehensive feature vector. The comprehensive feature vector can quantitatively capture the user's personal needs and status, and based on the comprehensive feature vector, a personalized interaction strategy and eloquence guidance strategy generation module is used to determine the most suitable interaction method and eloquence improvement suggestions for the user, and guide the virtual object to generate more personalized and targeted eloquence improvement strategies and feedback. The interaction method can be in the form of a dialogue between the virtual object and text prompts for speech practice.

[0109] The comprehensive feature vector will directly affect the subsequent interaction methods and eloquence improvement suggestions proposed by the system. The determination of personalized dialogue strategies and eloquence guidance strategies can be achieved through the multi-layer perceptron (MLP) model, which takes the comprehensive feature vector as input, obtains the user's current status and needs based on the comprehensive feature vector, determines the next action in the dialogue (such as asking questions, giving feedback, and encouraging), and provides specific eloquence improvement suggestions and exercises based on the user's eloquence skill analysis results, such as adjusting the speaking speed and improving the language structure.

[0110] The following are examples of recommendations based on different combinations of features:

[0111] Case 1: The user is asking how to improve persuasiveness, and the previous feedback shows that the speaking speed is unstable. The advanced semantic feature extraction model may fuse this information to obtain a comprehensive feature vector, which tends to suggest that the user pay attention to the practice of speaking speed control, so that the subsequent generated eloquence improvement suggestions emphasize the importance of speaking speed control practice, that is, through virtual objects, it is suggested that the user perform breathing and relaxation exercises to control nervousness, and improve the stability of speaking speed by practicing slow reading and conscious pauses.

[0112] Case 2: The user asks how to use emotional expression more effectively in a speech, and the conversation history shows that the user is interested in emotional expression but lacks practice. The comprehensive features may reflect the need for guidance on emotional expression skills, so that the subsequent eloquence improvement suggestions generated will strengthen the improvement of the user's emotional expression skills. That is, the eloquence improvement suggestions proposed by the virtual object will provide suggestions on how to incorporate personal stories into the speech or use vivid language to stimulate emotional resonance, or provide cases that may include simulated emotionally rich scene narration or analysis of moving speeches for user reference.

[0113] In this way, the most appropriate suggestions and exercises can be provided according to the user's specific needs and current ability level, helping the user to improve his or her eloquence.

[0114] Furthermore, in this embodiment, when the virtual object interacts with the user in a specified interactive manner, creative content associated with key creative points will be output to the user. The creative content is generated by the creative stimulator. The creative stimulator uses a generative adversarial network (GAN) or other advanced algorithms to simulate different creative concepts and perspectives to encourage the user's creative thinking.

[0115] Among them, the method for generating creative content includes: extracting eloquence characteristics based on speech content; eloquence characteristics include emotional diversity, expression clarity and language diversity; receiving key creative points and eloquence characteristics based on a conditional encoder, and outputting a latent space, thereby encoding the speech content and eloquence characteristics into the latent space; generating creative content matching the eloquence characteristics from the latent space based on a conditional generator.

[0116] The emotional diversity in eloquence characteristics can be analyzed by NLP tools (such as NLTK, TextBlob, or sentiment analysis models based on deep learning) to analyze the speech content, obtain the emotional labels of each sentence or paragraph, and then calculate diversity indicators such as emotional entropy based on these labels. The clarity of expression can be analyzed by language models (such as GPT, BERT, etc.) to analyze the grammatical and contextual structure of sentences, determine the grammatical error rate and logical fluency, and evaluate grammatical correctness and logical coherence. Language diversity can be calculated by counting the types and numbers of words and calculating vocabulary richness indicators such as Type-Token Ratio (TTR).

[0117] The quantitative results of the above sentiment diversity, expression clarity, and language diversity are combined into a feature vector. Assume that a user's speech contains a variety of emotional expressions, from positive to negative to neutral, and the sentiment diversity score obtained after sentiment analysis is 0.85. The speech content is grammatically accurate and logically clear, and the expression clarity score is 0.9. The user uses a rich vocabulary and diverse sentence structures, and the language diversity score is 0.8. Therefore, the eloquence feature vector of this user can be expressed as [0.85, 0.9, 0.8].

[0118] In this way, a quantitative representation of the user's eloquence dimension can be provided, and the necessary input can be provided for subsequent model training and creative content generation.

[0119] The conditional generator (CVAE part) is designed so that it can generate creative content based on eloquence characteristics. This process takes into account the eloquence characteristics of users in terms of emotional diversity, clarity of expression, and language diversity, and generates matching creative content from the latent space based on these characteristics. The following are some possible output examples under different input parameters:

[0120] Example 1: High Sentiment Diversity

[0121] Input Parameters: If the user's eloquence characteristics show high emotional diversity, it means that the user is able to express rich emotions in his speech.

[0122] Output: The conditional generator might generate a speech draft that contains a variety of emotional expressions, such as a story that starts out sad, has a hopeful middle, and ends with a climax of joy. Such a story can better engage the audience and make the speech more vivid and moving.

[0123] Example 2: Low clarity of expression

[0124] Input parameters: If the user's clarity of expression is low, this may mean that the user is not clearly structured when expressing his thoughts or his language is confusing.

[0125] Output content: The conditional generator can generate a clearly structured and logically cohesive speech framework, providing a clear argument, strong evidence, and a clear conclusion, thereby helping users improve the clarity of their expression.

[0126] Example 3: Medium language diversity

[0127] Input Parameters: If a user's language diversity score is medium, this indicates that the user may not fully utilize a rich vocabulary and diverse sentence structures to enhance the appeal of their speech.

[0128] Output content: The conditional generator may generate a speech example that contains diverse vocabulary and innovative sentence structures, encouraging users to try new ways of expression, such as using metaphors, personification and other rhetorical devices to enrich the content of the speech.

[0129] Example 4: High clarity of expression but single emotion

[0130] Input parameters: Users may express themselves clearly in logic, but their emotions are monotonous and lack variation.

[0131] Output: The conditional generator can create a draft of a story or speech interspersed with elements that evoke different emotional responses, such as adding emotional breadth and depth by introducing inspiring victories, touching sacrifices, and humorous anecdotes.

[0132] In this way, the creative content generated by the conditional generator based on the user's eloquence characteristics is designed to stimulate users' creative thinking and help them explore new speaking styles and techniques, thereby improving their eloquence and expression skills.

[0133] Furthermore, the authenticity of the generated creative content can be evaluated through the discriminator. At the same time, by minimizing the loss function of the discriminator and the conditional generator, the system alternately trains the discriminator and the conditional generator, and continuously optimizes the model parameters to improve the quality and diversity of the generated content.

[0134] Step S4: Acquire real-time interaction information. The real-time interaction information is user feedback collected when the user interacts with the virtual object. Edit the speech content according to the real-time interaction information to generate a speech manuscript.

[0135] This embodiment provides an online collaborative creation platform, in which virtual objects interact with users, providing a shared, real-time, synchronized editing environment for users and virtual objects. The platform provides functional components such as a text editor, a real-time chat window, and a version control system, so that users and virtual objects can jointly edit speech manuscripts and view and modify content in real time.

[0136] The implementation of a real-time collaborative authoring environment can use collaborative editing tools, such as Google Docs or similar tools, to create a real-time online editing environment to allow users and virtual objects to edit speech manuscripts at the same time and synchronize changes in real time. Users and virtual objects work together in a shared editing environment and can simultaneously view, edit and improve speech content, and their changes will be reflected in the edited manuscript immediately.

[0137] In some embodiments, user satisfaction feedback and speech effect data are collected to help optimize the performance of virtual objects. Specifically, the corresponding data can be collected through a satisfaction survey, that is, a series of questionnaires or satisfaction rating systems are designed to collect user satisfaction with the guidance, feedback and overall collaborative creation experience provided by the virtual object, which includes but is not limited to evaluation of teaching content, interactive experience, and system usability.

[0138] At the same time, data on the effectiveness of users' speech practice and actual speeches under the guidance of virtual objects are collected, which may include speech scores, audience feedback, self-assessment and other forms of data.

[0139] Satisfaction feedback submitted by users is collected and classified by an automated system. For quantitative data (such as ratings), statistical analysis is performed directly; for qualitative feedback (such as text reviews), natural language processing technology is used to perform sentiment analysis and keyword extraction.

[0140] Conduct in-depth analysis of the collected speech performance data to evaluate the user’s improvement in different eloquence dimensions and the teaching effectiveness of the virtual object.

[0141] At the same time, satisfaction feedback and speech effect data can be comprehensively analyzed to identify the advantages and potential improvement points in the virtual object system; statistical analysis, data visualization and other methods can be used to extract valuable insights. Based on the comprehensive analysis results, a detailed performance evaluation report is generated, including user satisfaction overview, teaching effect evaluation, user behavior analysis and other aspects.

[0142] Feedback the performance evaluation report to system developers and virtual object AI algorithms as a basis for optimization and updating. Adjust teaching strategies, content, interaction design, etc. based on the recommendations in the report and user needs.

[0143] In addition, users are informed of the results of optimization and updating, invited to experience the improved system, and again conduct satisfaction and effectiveness evaluations, forming a continuous improvement and evaluation cycle.

[0144] In some embodiments, a multi-dimensional eloquence improvement and collaborative creation system is also provided, which executes the multi-dimensional eloquence improvement and collaborative creation method as described in Example 1. The functions implemented by the system can be found in the corresponding description of the above method, which will not be repeated here.

[0145] In some embodiments, an electronic device is also provided. Figure 2 As shown in the structural block diagram, the electronic device includes: a memory 100 and a processor 200, and the memory 100 stores a computer program that can be run on the processor 200. When the processor 200 executes the computer program, the multi-dimensional eloquence improvement and collaborative creation method in the above embodiment is implemented. The number of the memory 100 and the processor 200 can be one or more.

[0146] The electronic device also includes:

[0147] The communication interface 300 is used to communicate with external devices and perform data exchange transmission.

[0148] If the memory 100, the processor 200 and the communication interface 300 are implemented independently, the memory 100, the processor 200 and the communication interface 300 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0149] Optionally, in a specific implementation, if the memory 100, the processor 200 and the communication interface 300 are integrated on a chip, the memory 100, the processor 200 and the communication interface 300 can communicate with each other through an internal interface.

[0150] An embodiment of the present invention provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present invention when executed by a processor.

[0151] An embodiment of the present invention further provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present invention.

[0152] An embodiment of the present invention also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided by the embodiment of the invention.

[0153] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the advanced RISC machines (ARM) architecture.

[0154] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0155] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0156] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0157] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0158] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of various changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A multi-dimensional method for improving eloquence and collaborative creation, characterized in that: include: Acquire speech content, and analyze the speech content to obtain specified fields; Determine the key creative points of the speech according to the specified fields based on a pre-built deep learning model; Performing an eloquence skill analysis on the speech content to obtain an eloquence skill analysis result; Acquire speech goals and historical conversation data, and import the speech goals, the eloquence skill analysis results, and the historical conversation data into a pre-built deep learning model to extract comprehensive features; Determine the interaction mode between the virtual object and the user according to the comprehensive features based on the preset interaction strategy and eloquence guidance strategy, generate eloquence improvement suggestions associated with the comprehensive features, and make the virtual object output the eloquence improvement suggestions associated with the comprehensive features in a specified interaction mode; Extracting eloquence features according to the speech content; the eloquence features include emotional diversity, expression clarity and language diversity; receiving the key creative points and the eloquence features based on a conditional encoder, and outputting a latent space; generating creative content matching the eloquence feature from the latent space based on a condition generator, and controlling the virtual object to output the creative content associated with the key creative point in a specified manner; Real-time interaction information is obtained, where the real-time interaction information is user feedback collected when the user interacts with the virtual object, and the speech content is edited according to the real-time interaction information to generate a speech manuscript.

2. The multi-dimensional eloquence improvement and collaborative creation method according to claim 1 is characterized in that: The step of analyzing the speech content to obtain a specified field includes: Extracting words and sentences from the speech content based on a natural language processing algorithm to obtain key words and sentences; The occurrence frequency of the key words and phrases in the key text sentences corresponding to the speech content is calculated, and the key words and phrases whose occurrence frequency is higher than a preset frequency value are marked as designated fields.

3. The multi-dimensional eloquence improvement and collaborative creation method according to claim 1 is characterized in that: The determining the key creative points of the speech according to the specified field based on the pre-built deep learning model includes: Analyze the semantic similarity between the specified field and the text part other than the specified field based on the BERT model; The designated field whose semantic similarity is lower than a preset similarity value is marked as the key creative point.

4. The multi-dimensional eloquence improvement and collaborative creation method according to claim 1 is characterized in that: Also includes: Extracting features from the speech content to obtain eloquence features; Calculating the scores of various eloquence skill indicators according to the eloquence characteristics, wherein the eloquence skill indicators include a language richness indicator, a rhetorical skill indicator, an emotional intensity indicator, and a logical coherence indicator; A comprehensive scoring result is calculated according to the scores of each of the eloquence skill indicators based on a weighted fusion algorithm, and instant feedback suggestions are generated according to the comprehensive scoring result.

5. The multi-dimensional eloquence improvement and collaborative creation method according to claim 4 is characterized in that: Also includes: Receiving response information generated by the user in response to the instant feedback suggestion; The corresponding weights in the weighted fusion algorithm are adjusted according to the response information to optimize the generation result of the instant feedback suggestion.

6. A multi-dimensional eloquence improvement and collaborative creation system, characterized by: Execute the multi-dimensional eloquence improvement and collaborative creation method as described in any one of claims 1 to 5.

7. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the multi-dimensional eloquence improvement and collaborative creation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the multi-dimensional eloquence improvement and collaborative creation method as described in any one of claims 1 to 5 is implemented.