A public speaking auxiliary training method, system, device and storage medium

Abstract

The present application proposes a method, system, device and storage medium for auxiliary training of public speaking, the method comprising: obtaining multi-dimensional speech data of the speaker during the current speech; obtaining preset multi-dimensional eloquence indicators; based on the multi-dimensional eloquence indicators, extracting corresponding features from the multi-dimensional speech data respectively to obtain a variety of key features; based on the multiple key features, evaluating and analyzing the speaker's performance in the current speech to obtain a personalized evaluation result of the speaker. The present application can provide real-time feedback on the speaker's performance in the current speech, and can accurately identify the speaker's speaking skills and potential areas for improvement in public speaking, generate customized speech improvement training content for the speaker that matches his or her speaking skill level and needs, help the speaker improve his or her speaking skills in a targeted manner, and thus effectively improve the speaker's learning efficiency and learning effect in public speaking.

Description

Technical Field

[0001] The present application relates to the technical field of speech and eloquence, and in particular to a method, system, device and storage medium for auxiliary training of public speaking. Background Art

[0002] In today’s society, public speaking skills are essential for personal career development. However, traditional speech training methods usually rely on face-to-face teaching or static learning materials such as books and videos. These methods often lack personalized feedback and guidance, and cannot provide customized training content for the specific needs of individual users. In addition, traditional training methods have limitations in real-time feedback, which affects learning efficiency and effectiveness. Summary of the invention

[0003] The embodiments of the present application provide a public speaking auxiliary training method, system, device and storage medium to solve the problems existing in the related technologies. The technical solutions are as follows:

[0004] In a first aspect, an embodiment of the present application provides a public speaking auxiliary training method, comprising:

[0005] Acquire multi-dimensional speech data of a speaker's current speech, wherein the multi-dimensional speech data includes audio data, video data, and text data;

[0006] Obtaining preset multi-dimensional eloquence indicators, wherein the multi-dimensional eloquence indicators include language richness, argument logic, emotional communication and audience interaction;

[0007] Based on the multi-dimensional eloquence index, corresponding features are extracted from the multi-dimensional speech data to obtain multiple key features, wherein the multiple key features include audio features, video features and text features;

[0008] Based on the multiple key features, the speaker's performance in the current speech is evaluated and analyzed to obtain a personalized evaluation result of the speaker. The personalized evaluation result includes the speaker's multi-dimensional performance score, overall performance score and customized speech improvement training content. The multi-dimensional performance score includes a language richness score, an argument logic score, an emotional communication score and an audience interaction score.

[0009] In one implementation, based on the multi-dimensional eloquence index, corresponding features are extracted from the multi-dimensional speech data to obtain a variety of key features including:

[0010] Calling fast Fourier transform and Mel frequency cepstrum coefficients to extract features related to the multi-dimensional eloquence index from the audio data of the multi-dimensional speech data to obtain the audio features;

[0011] Invoking a computer vision algorithm to extract features related to the multi-dimensional eloquence indicator from the video data of the multi-dimensional speech data to obtain the video features;

[0012] A natural language processing algorithm is called to extract features related to the multi-dimensional eloquence index from the text data of the multi-dimensional speech data to obtain the text features.

[0013] In one embodiment, based on the multiple key features, the speaker's performance in the current speech is evaluated and analyzed to obtain a personalized evaluation result of the speaker, including:

[0014] Based on the text features, evaluating and analyzing the speaker's language performance to obtain the language richness score;

[0015] Based on the text features and the audio features, analyzing the logic of the speaker's argument expression to obtain a logic score of the argument;

[0016] Analyzing the speaker's emotional communication performance based on the audio features and the video features to obtain the emotional communication score;

[0017] Analyzing the speaker's interactive performance based on the audio features and the video features to obtain the audience interaction score;

[0018] Obtaining the overall performance score based on the language richness score, the argument logic score, the emotional communication score, and the audience interaction score;

[0019] Generating the customized speech improvement training content for the speaker based on the language richness score, the argument logic score, the emotional communication score, the audience interaction score and the overall performance score;

[0020] The personalized evaluation result is obtained based on the language richness score, the argument logic score, the emotional communication score, the audience interaction score, the overall performance score and the customized speech improvement training content.

[0021] In one embodiment, the text features include keywords, syntactic structure, and logical coherence, and the audio features include pause frequency;

[0022] Based on the text features, evaluating and analyzing the speaker's language performance to obtain the language richness score includes: calling a high-level language model, evaluating and analyzing the speaker's language performance based on the keywords and syntactic structures in the text features, and obtaining the lexical diversity index of each sentence in the speaker's speech text content and the information density of each sentence; obtaining the language richness score based on the lexical diversity index of each sentence and the information density of each sentence;

[0023] Based on the text features and the audio features, the logical expression of the speaker's argument is analyzed to obtain the logic score of the speaker's argument, including: calling a probabilistic statistical model to evaluate the structure and coherence of the arguments used by the speaker in the current speech based on the logical coherence in the text features and the pause frequency in the audio features, so as to analyze the logical expression of the speaker's argument and obtain the coherence index of each argument used by the speaker in the current speech; based on the coherence index of each argument, the argument logic score is obtained.

[0024] In one embodiment, the audio features include speech speed and pitch, and the video features include facial expressions; based on the audio features and the video features, analyzing the speaker's emotional communication performance to obtain the emotional communication score includes:

[0025] Calling a speech emotion analysis algorithm and a facial expression recognition algorithm to analyze the speaker's emotional expression based on the speech speed and pitch in the audio features and the facial expressions in the video features, and obtain the speaker's emotional intensity index and emotional diversity index;

[0026] The emotion communication score is obtained based on the emotion intensity index and the emotion diversity index.

[0027] In one embodiment, the audio features include keywords, and the video features include facial expressions and body language; based on the audio features and the video features, the interactive performance of the speaker is analyzed to obtain the audience interaction score, which includes:

[0028] Invoking a deep learning model and a video analysis algorithm to analyze the interactive performance of the speaker based on the keywords in the audio features and the facial expressions and body language in the video features, and obtaining a stage control index corresponding to each interactive event, wherein each interactive event is an interactive event used by the speaker in the current speech;

[0029] The audience interaction score is obtained based on the stage control index corresponding to each interactive event.

[0030] In one embodiment, generating the customized speech improvement training content for the speaker based on the language richness score, the argument logic score, the emotional communication score, the audience interaction score, and the overall performance score includes:

[0031] Based on the language richness score, the argument logic score, the emotional communication score, the audience interaction score and the overall performance score, a training suggestion list suitable for the speaker is generated according to predefined logical rules;

[0032] Invoking a content recommendation algorithm to select target improved training content that matches the training suggestion list from a preset speech material library, wherein the preset speech material library includes improved training content corresponding to different training suggestions;

[0033] Based on the target improvement training content, the customized speech improvement training content is generated for the speaker.

[0034] In a second aspect, the present application also provides a public speaking auxiliary training system, including:

[0035] A receiving unit is used to obtain multi-dimensional speech data of the speaker during the current speech, wherein the multi-dimensional speech data includes audio data, video data and text data; and obtain preset multi-dimensional eloquence indicators, wherein the multi-dimensional eloquence indicators include language richness, argument logic, emotional communication and audience interaction;

[0036] A processing unit, configured to extract corresponding features from the multi-dimensional speech data based on the multi-dimensional eloquence index to obtain a plurality of key features, wherein the plurality of key features include audio features, video features and text features;

[0037] An evaluation unit is used to evaluate and analyze the speaker's performance in the current speech based on the multiple key features to obtain a personalized evaluation result of the speaker, wherein the personalized evaluation result includes the speaker's multi-dimensional performance score, overall performance score and customized speech improvement training content, wherein the multi-dimensional performance score includes a language richness score, an argument logic score, an emotional communication score and an audience interaction score.

[0038] In a third aspect, an embodiment of the present application further provides a computer device, comprising: a memory and a processor, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement a method in any one of the above-mentioned aspects, wherein the memory and the processor communicate with each other through an internal connection path.

[0039] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the method in any one of the above-mentioned embodiments is implemented.

[0040] The advantages or beneficial effects of the above technical solution include at least:

[0041] In the present application, the speaker's performance in the current speech is evaluated and analyzed through preset multi-dimensional eloquence index analysis and multiple key features extracted from the speaker's multi-dimensional speech data during the current speech. Real-time feedback on the speaker's performance in the current speech can be provided, and the speaker's speaking skills and potential areas for improvement in public speaking can be accurately identified. Customized speech improvement training content that matches the speaker's speaking skill level and needs is generated for the speaker, helping the speaker to improve his or her speaking skills in a targeted manner, thereby effectively improving the speaker's learning efficiency and learning effect in public speaking.

[0042] The above summary is for illustrative purposes only and is not intended to be limiting in any way. In addition to the illustrative aspects, embodiments and features described above, further aspects, embodiments and features of the present application will be readily apparent by reference to the accompanying drawings and the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments disclosed in the present application and should not be regarded as limiting the scope of the present application.

[0044] Figure 1 A flowchart of a public speaking auxiliary training method provided in an embodiment of the present application;

[0045] Figure 2 A structural block diagram of a public speaking auxiliary training system provided in an embodiment of the present application;

[0046] Figure 3 A structural block diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the spirit or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary and non-restrictive in nature.

[0048] In the related art, traditional speech training methods usually rely on face-to-face teaching or static learning materials. In this case, the following problems are common in public speaking training:

[0049] Lack of personalization: Training content is usually one-size-fits-all and difficult to adjust according to different users’ skill levels, speaking styles, and progress rates. It lacks personalized feedback and guidance, and therefore cannot provide customized training content for the specific needs of individual users.

[0050] Feedback delay: Feedback is delayed, so speakers are often unable to correct their mistakes or improve their skills immediately, which may lead to the formation of wrong habits and affect the speaker's learning efficiency.

[0051] Based on this, the present application proposes a solution that can provide customized speech training and real-time feedback to solve the above problems.

[0052] The technical solution of the present application will be described in detail below with reference to the accompanying drawings.

[0053] Figure 1 FIG. 1 is a flow chart showing a method for assisting public speaking training according to an embodiment of the present application. Figure 1 As shown, the method may include the following steps:

[0054] S110. Obtain multi-dimensional speech data of the speaker during the current speech.

[0055] In one embodiment, the multi-dimensional speech data may include but is not limited to: audio data, video data, and text data.

[0056] In one implementation, the current speech may be a speech practice or a live speech, which is not limited in this embodiment.

[0057] As an example, the audio data and video data in the multi-dimensional speech data can be obtained from an online speech platform or a live speech event recording. In this case, the audio data and video data are real-time data.

[0058] In specific implementation, it can also support the speech practice video uploaded by the speaker, and then obtain the audio data and video data of the speaker's current speech practice from the speech practice video.

[0059] In one embodiment, the obtained audio data and video data can be preliminarily processed, such as standardization of data formats (such as unifying the video format to MP4 and the audio format to MP3), video resolution adjustment, and audio noise filtering, so as to ensure the accuracy and efficiency of subsequent analysis, and also reduce unnecessary resource consumption for data storage. Afterwards, the preliminarily processed audio data and video data can be preprocessed, such as frame rate adjustment of video data, marking of key frames, and audio time code synchronization, so that useful information can be accurately extracted from the audio data and video data, such as the speaker's voice characteristics, facial expressions, body language, and other information.

[0060] As an example, the pre-processed audio data and video data can be stored and backed up for further use or historical data analysis, and to prevent the loss or damage of the audio data and video data, thereby ensuring the reliability and durability of the audio data and video data.

[0061] As an example, the text data in the multi-dimensional speech data can be obtained from the speech manuscript uploaded by the speaker before the speech or from the audio data obtained in real time. Wherein, when the text data is obtained from the audio data obtained in real time, the text data can be the text obtained by transcribing the audio data, for example, it can be implemented using speech-to-text (STT) technology.

[0062] In a specific implementation, the text data may also be obtained from a written speech uploaded by the speaker after the speech.

[0063] In one embodiment, the text data may be preprocessed, such as removing stop words, correcting spelling errors, and standardizing the text format, etc. This can improve the accuracy and efficiency of subsequent analysis and prepare clean and standardized text data for subsequent deep text analysis.

[0064] In one embodiment, during the transmission and storage of multidimensional speech data, an encryption algorithm combined with anonymization processing can be used to securely protect the multidimensional speech data, such as using a relevant encryption algorithm to encrypt the multidimensional speech data, using corresponding anonymization processing to anonymize the speaker's privacy information (such as name, address, telephone number, etc.) in the multidimensional speech data, and setting access rights for the multidimensional speech data.

[0065] In the present application, by executing step S110, relevant data of the speaker during the current speech can be collected from multiple dimensions such as sound, video, and text, so as to facilitate a subsequent comprehensive evaluation of the speaker's performance during the current speech, such as tone, speaking speed, body language, facial expressions, etc.

[0066] S120: Obtain preset multi-dimensional eloquence indicators.

[0067] In one embodiment, the multi-dimensional eloquence index may include, but is not limited to: language richness, argument logic, emotional communication, and audience interaction.

[0068] In the specific implementation, considering the need to provide speakers with personalized speech evaluation and improvement research, in this application, a set of multi-dimensional eloquence indicators is pre-defined to facilitate the subsequent accurate evaluation of the speaker's speaking skills and the skill points that need to be strengthened, which helps to provide customized improvement training content in a timely manner according to the specific needs of the speaker, so as to improve the speaker's learning efficiency and learning effect in public speaking.

[0069] S130. Based on the multi-dimensional eloquence index, corresponding features are extracted from the multi-dimensional speech data to obtain a variety of key features.

[0070] In one embodiment, the multiple key features may include, but are not limited to, audio features, video features, and text features. Among them, audio features may include, but are not limited to, speech speed, pitch, and pause frequency. Video features may include, but are not limited to, facial expressions and body language (such as gestures), and text features may include, but are not limited to, keywords, syntactic structure, and logical coherence.

[0071] In one implementation, step S130 may include the following steps:

[0072] S131, calling Fast Fourier Transform (FFT) and Mel-Frequency Cepstral Coefficients (MFCCs) to extract features related to the multi-dimensional eloquence index from the audio data of the multi-dimensional speech data to obtain audio features.

[0073] As an example, FFT and MFCCs may be called to extract features such as speech rate, pitch, pause frequency, etc., which are related to the emotional communication and argument logic in the multi-dimensional eloquence index, from the audio data to obtain the audio features.

[0074] S132 calls a computer vision algorithm to extract features related to the multi-dimensional eloquence index from the video data of the multi-dimensional speech data to obtain the video features.

[0075] As an example, the optical flow algorithm and facial recognition algorithm in the computer vision algorithm can be called to extract facial expressions, body language and other features related to the emotional communication and audience interaction in the multi-dimensional eloquence indicators from the video data to obtain the video features.

[0076] S133, calling a natural language processing (NLP) algorithm to extract features related to the multi-dimensional eloquence index from the text data of the multi-dimensional speech data to obtain the text features.

[0077] As an example, semantic analysis and syntactic analysis in the NLP algorithm can be called to extract keywords, syntactic structures, logical coherence and other features related to the language richness, argument logic and audience interaction in the multidimensional eloquence indicators from the text data to obtain the text features.

[0078] In the present application, by executing step S130, it is possible to subsequently conduct a comprehensive evaluation of the speaker's current speech based on these multiple key features, and to accurately evaluate the speaker's speaking skills and skill points that need to be strengthened. At the same time, it is also possible to provide the speaker with personalized speech improvement training content to assist the speaker in improving his or her eloquence and expression skills in public speaking, thereby improving the speaker's learning efficiency and learning effect in public speaking.

[0079] S140. Based on multiple key features, the speaker's performance in the current speech is evaluated and analyzed to obtain a personalized evaluation result of the speaker.

[0080] In one embodiment, the personalized evaluation results may include, but are not limited to: the speaker's multi-dimensional performance score, overall performance score, and customized speech improvement training content. The multi-dimensional performance score may include, but is not limited to: language richness score, argument logic score, emotional communication score, and audience interaction score.

[0081] In one implementation, step S140 may include the following steps:

[0082] S141. Based on the text features, the speaker's language performance is evaluated and analyzed to obtain a language richness score.

[0083] In specific implementation, an advanced language model (such as the BERT model or the GPT model) can be called to evaluate and analyze the speaker's language performance based on the keywords and syntactic structures in the text features, and obtain the lexical diversity index of each sentence in the speaker's speech text content and the information density of each sentence; then based on the lexical diversity index of each sentence and the information density of each sentence, the language richness score is obtained.

[0084] Among them, BERT refers to Bidirectional Encoder Representations from Transformer, and GPT refers to Generative Pre-Trained.

[0085] As an example, the specific implementation process of step S141 may be as follows:

[0086] S1411. Calling a high-level language model to evaluate and analyze the speaker's language performance based on the keywords and syntactic structures in the text features, and obtain the number of unique words contained in each sentence in the speech text content, the information density of each sentence and the total number of words in the speech text content.

[0087] S1412. Obtain a lexical diversity index for each sentence based on the number of unique words contained in each sentence and the total number of words in the speech text content.

[0088] For example, the lexical diversity index of each sentence can be obtained according to the following formula (1) based on the number of unique words contained in each sentence and the total number of words in the speech text content.

[0089]

[0090] Among them, V ti It is represented as the vocabulary diversity index of the ith sentence in the speech text content, |unique words| is represented as the number of unique words contained in the ith sentence in the speech text content, and |total words| is represented as the total number of words in the speech text content. The unique words can be obtained by analyzing the topics in the speech text content.

[0091] In practical application, if the speaker uses a wide range of professional terms and rich descriptions in his discussion of the topic in the current speech, so that each sentence contains rich information and diverse vocabulary, then the lexical diversity index of each sentence in the speech text content and the information density of each sentence will get higher values.

[0092] S1413. Based on the lexical diversity index of each sentence and the information density of each sentence, a language richness score is obtained.

[0093] For example, the language richness score can be obtained based on the lexical diversity index of each sentence and the information density of each sentence according to the following formula (2).

[0094]

[0095] Among them, S lang is the language richness score, N is the total number of sentences in the speech text, N is an integer greater than 0, and I di It is represented as the information density of the i-th sentence in the speech text content.

[0096] In the present application, by executing step S141, the vocabulary diversity, sentence structure complexity and topic coverage of the speaker's speech text content are evaluated through the keywords and syntactic structures in the text features, so that the complexity and depth of the speaker's speech text content can be deeply analyzed, so that the speaker's language performance can be accurately evaluated through the speaker's language richness score.

[0097] S142. Based on the text features and audio features, the speaker's argument expression logic is analyzed to obtain an argument logic score.

[0098] In specific implementation, a probabilistic statistical model (such as a logistic regression model) can be called to evaluate the structure and coherence of the arguments used by the speaker in the current speech based on the logical coherence in the text features and the pause frequency in the audio features, so as to analyze the logical expression of the speaker's argument and obtain the coherence index of each argument used by the speaker in the current speech; and then, based on the coherence index of each argument, a logical score of the argument is obtained.

[0099] As an example, the logic score of each argument can be obtained based on the coherence index of the argument according to the following formula (3).

[0100]

[0101] Among them, S logic is the logical score of the argument, M is the total number of arguments used by the speaker in the current speech, and C l Denote as the coherence index of the lth argument, ω l Expressed as the weight coefficient of the lth argument.

[0102] That is, in this embodiment, the argument logic score is measured by evaluating the coherence of the various arguments used by the speaker in the current speech. The argument logic score measures the clarity and effectiveness of the logical connection between the various arguments.

[0103] In practical application, if a speaker can clearly and fluently describe how point A leads to point B and then logically derive conclusion C when discussing a certain topic, and each step is closely connected and easy to understand, then the speaker's argument logic score in this speech will be higher.

[0104] In the present application, by executing step S142, the structure and coherence of the arguments used by the speaker in the current speech are evaluated through the logical coherence in the text features and the pause frequency in the audio features, so that the fluency, coherence and persuasiveness of the speaker's argument can be deeply analyzed, so as to accurately evaluate the logical expression of the speaker's argument through the logic score of the speaker's argument.

[0105] S143. Analyze the speaker's emotional communication performance based on audio features and video features to obtain an emotional communication score.

[0106] In specific implementation, the speech emotion analysis algorithm and the facial expression recognition algorithm can be called to analyze the speaker's emotional communication performance based on the speaking speed and pitch in the audio features and the facial expressions in the video features to obtain the speaker's emotional intensity index and emotional diversity index; then based on the emotional intensity index and emotional diversity index, the emotional communication score is obtained.

[0107] Among them, the speech emotion analysis algorithm and the facial expression recognition algorithm can adopt the existing ones, and this embodiment does not limit this.

[0108] As an example, the specific implementation process of step S143 may be as follows:

[0109] S1431. Calling a speech emotion analysis algorithm to analyze the speaker's emotional expression based on the speech speed and pitch in the audio features, and obtaining the speaker's emotional intensity corresponding to each time period in the current speech.

[0110] S1432. Obtain the emotion intensity index based on the emotion intensity corresponding to each time period.

[0111] For example, the emotion intensity index may be obtained based on the emotion intensity corresponding to each time period according to the following formula (4).

[0112]

[0113] Among them, E intens It is expressed as the emotion intensity index, e t Expressed as the intensity of emotion in time period t, α t It is represented as the weight factor of the emotion intensity at time period t, and T is represented as the length of the time series of the current speech.

[0114] S1433, calling a facial expression recognition algorithm, analyzing the speaker's emotional expression based on the facial expressions in the video features, and obtaining the speaker's emotional state corresponding to each period in the current speech.

[0115] Among them, the above-mentioned step S1431 and step S1433 can be executed simultaneously, or can be executed in a corresponding order, such as executing step S1431 first and then step S1433, or executing step S1433 first and then step S1431, and this embodiment does not limit this.

[0116] S1434. Based on the emotional states corresponding to each time period, the probability of each emotional state of the speaker appearing in the current speech is counted.

[0117] For example, we can first count the emotional states corresponding to each time period, count the number of each emotional state, and the total number of all emotional states, and then use the ratio between the number of each emotional state and the total number of all emotional states as the numerical value corresponding to the probability of each emotional state.

[0118] S1435. Based on the probability of each emotional state, obtain the emotion diversity index.

[0119] Exemplarily, the emotion diversity index may be obtained based on the probability of each emotion state according to the following formula (5).

[0120]

[0121] Among them, p k It is represented as the probability of the kth emotional state appearing in the current speech of the speaker, and K is represented as the total number of categories of emotional states appearing in the current speech of the speaker.

[0122] S1436. Based on the emotion intensity index and the emotion diversity index, obtain the emotion communication score.

[0123] Exemplarily, the emotion communication score can be obtained based on the emotion intensity index and the emotion diversity index according to the following formula (6).

[0124] S emo =α·E vari +β·E intens (6)

[0125] Among them, S emo It is represented as the emotion communication score, α is represented as the adjustment parameter of the emotion diversity index, and β is represented as the adjustment parameter of the emotion intensity index.

[0126] In practical application, if the speaker can effectively express high emotions by using appropriate tone, speed and body language when discussing exciting topics, and the emotional intensity is obvious at key points that need to be emphasized, such as showing obvious enthusiasm and strength when motivating or mobilizing the audience, then the speaker's emotional communication score in this speech will be high.

[0127] In the present application, by executing step S143, the variability and intensity of the speaker's emotional expression in the current speech can be evaluated through the speech speed and pitch in the audio features and the facial expressions in the video features, which can reflect the diversity of the speaker's emotions in the current speech and the consistency of emotions with content, so as to accurately reflect how the speaker uses emotions to enhance the appeal of the speech in the current speech, thereby accurately evaluating the speaker's emotional communication performance through the speaker's emotional communication score.

[0128] S144. Based on the video features, analyze the speaker's interactive performance and obtain the audience interaction score.

[0129] In specific implementation, deep learning models and video analysis algorithms can be used to analyze the speaker's interactive performance based on keywords in audio features and facial expressions and body language in video features, and obtain the stage control index corresponding to each interactive event; then based on the stage control index corresponding to each interactive event, the audience interaction score is obtained. Among them, each interactive event is an interactive event used by the speaker in the current speech.

[0130] Among them, the framework of the deep learning model can adopt the existing one, such as the existing CNN model framework, and the video analysis algorithm can adopt the existing one, which is not limited in this embodiment.

[0131] As an example, the specific implementation process of step S144 may be as follows:

[0132] S1441. Calling a deep learning model and a video analysis algorithm to analyze the speaker's interactive performance based on keywords in the audio features and facial expressions and body language in the video features, to obtain the speaker's eye contact usage index in each interactive event and body language usage index in each interactive event.

[0133] For example, a deep learning model can be used to analyze the speaker's interactive performance based on the keywords in the audio features to obtain all the interactive events used by the speaker in the current speech. Then, a video analysis algorithm can be used to analyze the speaker's interactive performance based on the facial expressions in the video features to obtain the speaker's eye contact usage index in each interactive event; and a video analysis algorithm can be used to analyze the speaker's interactive performance based on the body language in the video features to obtain the speaker's body language usage index in each interactive event.

[0134] S1442. The speaker's eye contact usage index in each interactive event and the speaker's body language usage index in each interactive event are used to obtain the stage control index corresponding to each interactive event.

[0135] Exemplarily, based on the eye contact usage index and the body language usage index, the stage control index corresponding to each interactive event is obtained according to the following formula (7).

[0136] r j =γ j1 P eye +γ j2 P gesture (7)

[0137] Among them, r j It is represented as the stage control index corresponding to the jth interactive event used by the speaker in the current speech, P eye It is represented as the speaker's eye contact usage index in the jth interaction event, P gesture Denoted as the speaker’s body language usage index in the jth interaction event, γ j1 It is expressed as the adjustment weight of the eye contact usage index, γ j2 Represents the adjustment weight of the body language usage index.

[0138] S1443. Based on the stage control index corresponding to each interactive event, obtain the audience interaction score.

[0139] For example, the audience interaction score can be obtained based on the stage control index corresponding to each interactive event according to the following formula (8).

[0140]

[0141] Among them, E inter represents the audience interaction score, λ j It is represented as the weight factor of the jth interaction event, and m is represented as the total number of interaction events used by the speaker in the current speech.

[0142] In the present application, by executing step S144, the speaker's stage interactivity in the current speech can be evaluated through keywords in the audio features and facial expressions and body language in the video features, so as to accurately reflect the speaker's ability to handle audience interaction in the current speech, thereby accurately evaluating the speaker's interactive performance through the speaker's audience interaction score.

[0143] S145. Based on the language richness score, argument logic score, emotional communication score and audience interaction score, the overall performance score is obtained.

[0144] In specific implementation, the overall performance score can be obtained according to the following formula (9) based on the language richness score, argument logic score, emotional communication score and audience interaction score.

[0145] S total =λ lang ·S lang +λ logic ·S logic +λ emo ·S emo +λ inter ·S inter (9)

[0146] Among them, S total Denotes the overall performance score, λ lang Denotes the weight factor of language richness score, λ logic Expressed as the weight factor of the argument logic score, λ emo Denotes the weight factor of the emotional communication score, λ inter It is the weight factor of the audience interaction score. lang , logic , emo and λ inter Can be determined by data driven methods such as regression analysis.

[0147] That is, in this embodiment, the scores of the multi-dimensional eloquence indicators can be integrated by using a weighted summation method to generate an overall performance score of the speaker.

[0148] In the present application, by executing step S145, the scores of multi-dimensional eloquence indicators can be integrated through language richness score, argument logic score, emotional communication score and audience interaction score to generate an overall performance score for the speaker, so that the overall performance score can be used to accurately evaluate the speaker's comprehensive performance in the current speech.

[0149] S146. Generate customized speech improvement training content for speakers based on language richness score, argument logic score, emotional communication score, audience interaction score and overall performance score.

[0150] In one implementation, the specific implementation process of step S146 may be as follows:

[0151] S1461. Based on the language richness score, argument logic score, emotional communication score, audience interaction score and overall performance score, a list of training suggestions suitable for the speaker is generated according to predefined logical rules.

[0152] In a specific implementation, the predefined logic rule can be expressed by the following formula (10):

[0153] R=generate_recommendations(S total , S lang , S logic, S emo , S inter ) (10)

[0154] Here, R represents a training suggestion list. The function gencrate rcommendations determines multiple training suggestions suitable for the speaker based on the input scores, merges the multiple training suggestions into the training suggestion list, and outputs the training suggestion list.

[0155] As an example, a suggestion database can be pre-set, which contains various possible training suggestions, each of which is associated with a specific scoring interval, and these intervals are defined based on historical data and expert input, aiming to provide the most appropriate suggestions for different levels of speech performance. Among them, the training suggestions in the suggestion database can be continuously optimized through iterative learning using speaker feedback and speech performance data to ensure the continuous adaptability and effectiveness of the training suggestions.

[0156] In this example, the function gencrate rcommendations can be used to determine which eloquence skill points the speaker needs to improve based on the input scores. In other words, the speaker's strong and weak eloquence skill points in public speaking can be identified. For example, when the function gencrate rcommendations determines that the value of the emotional communication score is lower than a certain threshold, it is determined that the speaker's weak eloquence skill points include emotional expression skills, and training suggestions related to improving emotional expression will be generated. Correspondingly, each scoring dimension has a corresponding set of suggestions, and the function gencrate rcommendations will select the most appropriate training suggestion based on the specific values ​​of the input scores. Among them, low scores may trigger basic-level training suggestions, while high scores may trigger more advanced or more challenging training suggestions. For example, when the function gencrate rcommendations determines that the value of the argument logic score is higher than a certain threshold, it is determined that the speaker's strong eloquence skill points include logical expression skills, and training suggestions related to strengthening logical expression will be generated.

[0157] S1462. Calling a content recommendation algorithm to select target improved training content that matches the training suggestion list from a preset speech material library.

[0158] In specific implementation, the preset speech material library may include, but is not limited to: improved training content corresponding to different training suggestions. In the preset speech material library, detailed metadata may be established for each improved training content material, including difficulty level, subject category, required skills, etc., to ensure that the improved training content material covers various speech skills and topics. In the preset speech material library, the improved training content in the preset speech material library may be continuously optimized through iterative learning using the speaker's feedback and speech performance data to ensure the continuous adaptability and effectiveness of the improved training content.

[0159] In a specific implementation, the content recommendation algorithm may be a cosine similarity, collaborative filtering, or deep matching network algorithm.

[0160] S1463. Improve the training content based on the goal and generate customized speech improvement training content for the speaker.

[0161] During specific implementation, the target improvement training content can be arranged according to the enhancement training content and the strengthening training content to obtain the customized speech improvement training content.

[0162] In the present application, by executing step S146, the speaker's scores can be converted into specific and practical implementation steps, and customized speech improvement training content can be provided according to the specific needs of the speaker, so that the speaker can obtain the improvement training content that best suits his current abilities and development goals, thereby maximizing the speaker's public speaking skills.

[0163] S147. Get personalized evaluation results based on language richness score, argument logic score, emotional communication score, audience interaction score, overall performance score and customized speech improvement training content.

[0164] As an example, a visual scoring bar can be used to display the language richness score, argument logic score, emotional communication score, audience interaction score, and overall performance score, and then the corresponding content in the customized speech improvement training content can be displayed in text form under each scoring bar to obtain the personalized evaluation result, which can enhance the readability and practicality of the personalized evaluation result.

[0165] In a specific implementation, the personalized evaluation result can be displayed on a designated display screen, wherein the designated display screen can be provided with dynamic charts and indicators that can display the eloquence indicators of various dimensions in real time, so that these eloquence indicators can be displayed through intuitive graphics and color coding, so that the speaker can quickly identify his or her performance trend.

[0166] As an example, the designated display screen may allow the speaker to customize the interface layout, notification frequency, feedback detail, etc. according to personal preferences, so that the display panel of the designated display screen is easy to access and easy to operate, allowing the speaker to optimize the user experience according to his or her needs.

[0167] As another example, the designated display screen can also provide a series of interactive tutorials and exercises related to eloquence indicators. These tutorials and exercises can be dynamically adjusted based on the speaker's performance and historical feedback to ensure the relevance of the content and the timeliness of the teaching.

[0168] In the present application, by executing step S140, specific feedback and overall performance of different eloquence skills of the speaker can be given according to the speaker's speech performance, and the speaker's improvement points and reinforcement points in public speaking can be accurately located, which can help the speaker to improve his or her speaking skills in a targeted manner. For example, the speaker can perform improvement exercises on his or her weak eloquence skill points in public speaking, and reinforcement exercises on his or her strong eloquence skill points in public speaking, thereby effectively improving the speaker's learning efficiency and learning effect in public speaking.

[0169] In summary, the public speaking auxiliary training method provided in the embodiment of the present application evaluates and analyzes the speaker's performance in the current speech through preset multi-dimensional eloquence index analysis and multiple key features extracted from the speaker's multi-dimensional speech data in the current speech. It can provide real-time feedback on the speaker's performance in the current speech, and can accurately identify the speaker's speaking skills and potential areas for improvement in public speaking, generate customized speech improvement training content for the speaker that matches his or her speaking skill level and needs, help the speaker improve his or her speaking skills in a targeted manner, and thus effectively improve the speaker's learning efficiency and learning effect in public speaking.

[0170] Furthermore, the public speaking auxiliary training method provided in the embodiment of the present application, the customized speech improvement training content and real-time feedback mechanism provided can significantly enhance the speaker's learning experience and satisfaction. The speaker can feel obvious progress and a sense of accomplishment, so that he or she can participate in the training more actively, which improves the speaker's enthusiasm and continuity in learning. At the same time, the public speaking auxiliary training method provided in the embodiment of the present application can ensure that all types of speakers can find improvement training content that suits them, whether they are novice speakers or experienced speakers, they can gain value from it and promote their personal development.

[0171] Figure 2 FIG. 2 shows a structural block diagram of a public speaking auxiliary training system according to an embodiment of the present application. Figure 2 As shown, the system may include:

[0172] The receiving unit 210 is used to obtain multi-dimensional speech data of the speaker during the current speech, the multi-dimensional speech data including audio data, video data and text data; obtain preset multi-dimensional eloquence indicators, the multi-dimensional eloquence indicators including language richness, argument logic, emotional communication and audience interaction;

[0173] The processing unit 220 is used to extract corresponding features from the multi-dimensional speech data based on the multi-dimensional eloquence index to obtain multiple key features, where the multiple key features include audio features, video features, and text features;

[0174] The evaluation unit 230 is used to evaluate and analyze the speaker's performance in the current speech based on multiple key features to obtain a personalized evaluation result of the speaker. The personalized evaluation result includes the speaker's multi-dimensional performance score, overall performance score and customized speech improvement training content. The multi-dimensional performance score includes a language richness score, an argument logic score, an emotional communication score and an audience interaction score.

[0175] The functions of each unit in the public speaking auxiliary training system of the embodiment of the present application can be referred to the corresponding description in the above method, and will not be repeated here. Among them, each unit can be integrated on the same device or distributed on different devices, as long as it can ensure that each unit can communicate effectively, and the embodiment of the present application does not limit this.

[0176] Figure 3 FIG. 2 shows a block diagram of a computer device according to an embodiment of the present application. Figure 3 As shown, the computer device includes: a memory 310 and a processor 320. The memory 310 stores instructions, and the instructions are loaded and executed by the processor 320 to implement the public speaking auxiliary training method in the above embodiment. The number of the memory 310 and the processor 320 can be one or more.

[0177] The computer device also includes:

[0178] The communication interface 330 is used to communicate with external devices and perform data exchange transmission.

[0179] If the memory 310, the processor 320 and the communication interface 330 are implemented independently, the memory 310, the processor 320 and the communication interface 330 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0180] Optionally, in a specific implementation, if the memory 310, the processor 320 and the communication interface 330 are integrated on a chip, the memory 310, the processor 320 and the communication interface 330 can communicate with each other through an internal interface.

[0181] An embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer program runs on a computer, the method provided in the embodiment of the present application is implemented.

[0182] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in a memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0183] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the application.

[0184] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0185] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory, and may also include a non-volatile random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of example but not limitation, many forms of RAM are available. For example, static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct rambus RAM (DR RAM).

[0186] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0187] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0188] In the description of the present application, “plurality” means two or more than two, unless otherwise clearly and specifically defined.

[0189] Any process or method description in the flow chart or otherwise described herein can be understood to represent a module, fragment or portion of a code including one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0190] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be embodied in any computer-readable medium for use by an instruction execution system, apparatus or device (such as a computer-based system, a system including a processor or other system that can fetch instructions from an instruction execution system, apparatus or device and execute instructions), or used in combination with these instruction execution systems, apparatuses or devices.

[0191] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0192] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0193] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A public speaking auxiliary training method, characterized in that: include: Acquire multi-dimensional speech data of a speaker's current speech, wherein the multi-dimensional speech data includes audio data, video data, and text data; Obtaining preset multi-dimensional eloquence indicators, wherein the multi-dimensional eloquence indicators include language richness, argument logic, emotional communication and audience interaction; Based on the multi-dimensional eloquence index, corresponding features are extracted from the multi-dimensional speech data to obtain multiple key features, wherein the multiple key features include audio features, video features and text features; Based on the multiple key features, the speaker's performance in the current speech is evaluated and analyzed to obtain a personalized evaluation result of the speaker, wherein the personalized evaluation result includes a multi-dimensional performance score, an overall performance score, and customized speech improvement training content of the speaker, wherein the multi-dimensional performance score includes a language richness score, an argument logic score, an emotional communication score, and an audience interaction score; The text features include keywords, syntactic structure and logical coherence, the audio features include pause frequency, speech speed, pitch and keywords, and the video features include facial expressions and body language. Based on the multiple key features, the speaker's performance in the current speech is evaluated and analyzed, and the personalized evaluation results of the speaker include: Calling a high-level language model to evaluate and analyze the speaker's language performance based on the keywords and syntactic structures in the text features, and obtaining the lexical diversity index of each sentence in the speaker's speech text content and the information density of each sentence; and obtaining the language richness score based on the lexical diversity index of each sentence and the information density of each sentence; Calling a probabilistic statistical model to evaluate the structure and coherence of the arguments used by the speaker in the current speech based on the logical coherence in the text features and the pause frequency in the audio features, so as to analyze the logical expression of the speaker's argument and obtain the coherence index of each argument used by the speaker in the current speech; and obtaining the argument logic score based on the coherence index of each argument; Calling a speech emotion analysis algorithm and a facial expression recognition algorithm to analyze the speaker's emotional communication performance based on the speech speed and pitch in the audio features and the facial expressions in the video features, and obtain the speaker's emotional intensity index and emotional diversity index; and obtaining the emotional communication score based on the emotional intensity index and the emotional diversity index; Invoke a deep learning model and a video analysis algorithm to analyze the speaker's interactive performance based on the keywords in the audio features and the facial expressions and body language in the video features, and obtain a stage control index corresponding to each interactive event, wherein each interactive event is an interactive event used by the speaker in the current speech; and obtain the audience interaction score based on the stage control index corresponding to each interactive event; Obtaining the overall performance score based on the language richness score, the argument logic score, the emotional communication score, and the audience interaction score; Generating the customized speech improvement training content for the speaker based on the language richness score, the argument logic score, the emotional communication score, the audience interaction score and the overall performance score; The personalized evaluation result is obtained based on the language richness score, the argument logic score, the emotional communication score, the audience interaction score, the overall performance score and the customized speech improvement training content.

2. The method according to claim 1, characterized in that Based on the multi-dimensional eloquence index, corresponding features are extracted from the multi-dimensional speech data to obtain a variety of key features including: Calling fast Fourier transform and Mel frequency cepstral coefficients to extract features related to the multi-dimensional eloquence index from the audio data of the multi-dimensional speech data to obtain the audio features; Invoking a computer vision algorithm to extract features related to the multi-dimensional eloquence indicator from the video data of the multi-dimensional speech data to obtain the video features; A natural language processing algorithm is called to extract features related to the multi-dimensional eloquence index from the text data of the multi-dimensional speech data to obtain the text features.

3. The method according to claim 1 or 2, characterized in that: Generating the customized speech improvement training content for the speaker based on the language richness score, the argument logic score, the emotional communication score, the audience interaction score and the overall performance score includes: Based on the language richness score, the argument logic score, the emotional communication score, the audience interaction score and the overall performance score, a training suggestion list suitable for the speaker is generated according to predefined logical rules; Invoking a content recommendation algorithm to select target improved training content that matches the training suggestion list from a preset speech material library, wherein the preset speech material library includes improved training content corresponding to different training suggestions; Based on the target improvement training content, the customized speech improvement training content is generated for the speaker.

4. A public speaking auxiliary training system, characterized in that: include: A receiving unit, used to obtain multi-dimensional speech data of a speaker during a current speech, wherein the multi-dimensional speech data includes audio data, video data and text data; Obtaining preset multi-dimensional eloquence indicators, wherein the multi-dimensional eloquence indicators include language richness, argument logic, emotional communication and audience interaction; A processing unit, configured to extract corresponding features from the multi-dimensional speech data based on the multi-dimensional eloquence index to obtain a plurality of key features, wherein the plurality of key features include audio features, video features and text features; An evaluation unit, configured to evaluate and analyze the speaker's performance in the current speech based on the multiple key features, and obtain a personalized evaluation result of the speaker, wherein the personalized evaluation result includes a multi-dimensional performance score, an overall performance score, and customized speech improvement training content of the speaker, wherein the multi-dimensional performance score includes a language richness score, an argument logic score, an emotional communication score, and an audience interaction score; The text features include keywords, syntactic structure and logical coherence, the audio features include pause frequency, speech speed, pitch and keywords, and the video features include facial expressions and body language; the evaluation unit is used to evaluate and analyze the speaker's performance in the current speech based on the multiple key features to obtain the speaker's personalized evaluation result, specifically for: Calling a high-level language model to evaluate and analyze the speaker's language performance based on the keywords and syntactic structures in the text features, and obtaining the lexical diversity index of each sentence in the speaker's speech text content and the information density of each sentence; and obtaining the language richness score based on the lexical diversity index of each sentence and the information density of each sentence; Calling a probabilistic statistical model to evaluate the structure and coherence of the arguments used by the speaker in the current speech based on the logical coherence in the text features and the pause frequency in the audio features, so as to analyze the logical expression of the speaker's argument and obtain the coherence index of each argument used by the speaker in the current speech; and obtaining the argument logic score based on the coherence index of each argument; Calling a speech emotion analysis algorithm and a facial expression recognition algorithm to analyze the speaker's emotional communication performance based on the speech speed and pitch in the audio features and the facial expressions in the video features, and obtain the speaker's emotional intensity index and emotional diversity index; and obtaining the emotional communication score based on the emotional intensity index and the emotional diversity index; Invoke a deep learning model and a video analysis algorithm to analyze the speaker's interactive performance based on the keywords in the audio features and the facial expressions and body language in the video features, and obtain a stage control index corresponding to each interactive event, wherein each interactive event is an interactive event used by the speaker in the current speech; and obtain the audience interaction score based on the stage control index corresponding to each interactive event; Obtaining the overall performance score based on the language richness score, the argument logic score, the emotional communication score, and the audience interaction score; Generating the customized speech improvement training content for the speaker based on the language richness score, the argument logic score, the emotional communication score, the audience interaction score and the overall performance score; The personalized evaluation result is obtained based on the language richness score, the argument logic score, the emotional communication score, the audience interaction score, the overall performance score and the customized speech improvement training content.

5. A computer device, characterized in that: include: A memory and a processor, wherein the memory stores instructions, and the instructions are loaded and executed by the processor to implement the method according to any one of claims 1 to 3.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a computer, the method according to any one of claims 1 to 3 is implemented.

Images