Speech assistance method, electronic device, computer storage medium, and program product

Abstract

The application provides a speech assistance method, an electronic device, a computer storage medium and a program product, which are applied to the field of artificial intelligence technology. The method comprises the following steps: receiving speech-related information sent by a plurality of clients; constructing a multi-dimensional semantic space based on the speech-related information, portrait data of a target user and conference information of a current conference; the multi-dimensional semantic space comprises one or more of the following: a conference theme vector, a real-time context vector and a user feature vector; determining a target speech type of the target user and a target fault type of speech logic of the target user based on speech-related information of the target user; generating speech prompt information based on the multi-dimensional semantic space, the target speech type and the target fault type; and sending the speech prompt information to a target client corresponding to the target user and presenting the speech prompt information through the target client.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a speech assistance method, electronic device, computer storage medium, and program product. Background Technology

[0002] Currently, with the widespread use of online meetings, participants face numerous challenges during the speaking segment. For example, many speakers often deviate from the topic due to disorganized logic, a lack of professional terminology, and insufficient content organization skills, or they may stumble and deliver empty content due to nervousness, resulting in poor speaking quality. Therefore, there is a need to provide a speaking assistance method that can improve speaking quality. Summary of the Invention

[0003] This application provides a speaking assistance method, an electronic device, a computer storage medium, and a program product.

[0004] The technical solution of this application is implemented as follows: This application provides a speech assistance method applied to a server, the method comprising: Receive speech-related information from multiple clients; Based on the speech-related information, the target user's profile data, and the current meeting information, a multi-dimensional semantic space is constructed; the multi-dimensional semantic space includes one or more of the following: meeting topic vector, real-time context vector, and user feature vector; Based on the target user's speech information, determine the target speech type of the target user and the target fault type of the target user's speech logic; Based on the multidimensional semantic space, the target speech type, and the target tomographic type, speech prompt information is generated; The speaking prompt information is sent to the target client corresponding to the target user and presented through the target client; the target client can be any one of the plurality of clients.

[0005] This application provides an electronic device, the device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the speaking assistance method provided by one or more of the foregoing technical solutions.

[0006] This application provides a computer storage medium storing a computer program; when the computer program is executed, it can implement the speech assistance method provided by one or more of the aforementioned technical solutions.

[0007] This application provides a computer program product, including a computer program that, when executed by a processor, implements the speech assistance method provided by one or more of the aforementioned technical solutions.

[0008] This application provides a speaking assistance method. After receiving speaking-related information from multiple clients, a multi-dimensional semantic space is constructed based on the speaking-related information, the target user's profile data, and the current meeting information. This enables more accurate semantic understanding and provides a reliable data foundation for generating subsequent speaking prompts. By determining the target user's target speaking type and target segment type, the user's speaking intent can be accurately identified, ensuring the accuracy of subsequent speaking prompts. Furthermore, personalized speaking prompts are generated and presented based on the multi-dimensional semantic space, target speaking type, and target segment type, which can help users reduce the difficulty of speaking and improve speaking quality. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the structure of a speech assistance system provided in an embodiment of this application; Figure 2 A flowchart illustrating a speaking assistance method provided in an embodiment of this application; Figure 3 A flowchart illustrating another speech assistance method provided in an embodiment of this application; Figure 4 This is a schematic diagram of the composition of the speaking assistance device according to an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0010] The technical solutions in this application will now be clearly and completely described with reference to the accompanying drawings.

[0011] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments provided herein are merely illustrative of the present application and are not intended to limit the present application. Furthermore, the embodiments provided below are some embodiments for implementing the present application, and not all embodiments for implementing the present application. Unless otherwise specified, the technical solutions described in the present application can be implemented in any combination.

[0012] It should be noted that, in this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a method or apparatus that includes a list of elements includes not only the elements expressly described, but also other elements not expressly listed, or elements inherent to implementing the method or apparatus. Without further limitations, an element defined by the phrase "comprising a..." does not exclude the presence of other related elements (e.g., steps in the method or units in the apparatus, such as portions of a processor, program, or software, etc.) in the method or apparatus that includes that element.

[0013] For example, the speech assistance method provided in this application includes a series of steps, but the speech assistance method provided in this application is not limited to the steps described therein. Similarly, the speech assistance device provided in this application includes a series of modules, but the speech assistance device provided in this application is not limited to the modules explicitly described, but may also include modules that need to be set up for obtaining relevant information or processing based on information.

[0014] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0015] It should be noted that the collection, use, storage, sharing and transfer of user personal information involved in the technical solution of this application all comply with the provisions of relevant laws and regulations, and require notification to users and obtaining their consent or authorization. Where applicable, user personal information has been subjected to de-identification and / or anonymization and / or encryption technical processing.

[0016] In this embodiment, the speaking assistance method can be applied to various scenarios that require providing speaking assistance to users, in order to help users reduce the difficulty of speaking and improve the quality of speaking; for example, it can be used in various meeting scenarios, without specific limitations here.

[0017] Figure 1 This is a schematic diagram of the structure of a speech assistance system provided in an embodiment of this application, as shown below. Figure 1As shown, the speech assistance system may include multiple clients 100 and a server 101; each client 100 is used to collect speech-related information and send it to the server 101; the server 101 is used to receive speech-related information sent by multiple clients; based on the speech-related information, the target user's profile data, and the current meeting information, a multi-dimensional semantic space is constructed; the multi-dimensional semantic space includes one or more of the following: meeting topic vector, real-time context vector, and user feature vector; based on the target user's speech-related information, the target speech type and the target segmentation type of the target user's speech logic are determined; based on the multi-dimensional semantic space, the target speech type, and the target segmentation type, speech prompt information is generated; the speech prompt information is sent to the target client corresponding to the target user and presented through the target client; the target client is any one of the multiple clients.

[0018] For example, each client 100 can connect to the server 101 via wired or wireless communication; the client can be, but is not limited to, various desktop computers, laptops, smartphones, tablets, etc. The server can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, without specific limitations here.

[0019] It should be noted that the speech assistance method can be executed by the client or server alone, or by both the terminal and the server; no specific limitation is made here. The following will apply this method to... Figure 1 The following explanation will be based on the server-side example.

[0020] like Figure 2 As shown in the diagram, this application provides a flowchart of a speech assistance method applied to a server. The method includes the following steps: Step 200: Receive speech-related information sent by multiple clients.

[0021] For example, during the meeting, each client can collect speaking-related information in real time; speaking-related information may include the user's spoken voice and / or draft text; spoken voice refers to the user's oral speaking data during the meeting, and draft text refers to the draft text entered by the user during the meeting to assist in their own speaking; the draft text may be entered by the user through the client's input box or through other means, without specific limitations here.

[0022] For example, the client can capture the spoken audio during the meeting in real time through audio input devices such as microphones; at the same time, it can also capture the draft text entered by the user in the input box in real time. After capturing the spoken audio and / or draft text, the client will synchronize this speech-related information to the server in real time; since there are multiple clients, the server can receive speech-related information sent by multiple clients.

[0023] Understandably, the speech-related information received by the server is raw data, which may contain noise interference, typos, non-standard expressions, and inconsistent formats. To ensure the accuracy of subsequent processing, the received speech-related information can be preprocessed to obtain preprocessed speech-related information. This will be used as an example for the following explanation.

[0024] For example, see Figure 1 The server includes a preprocessing module, which can preprocess the received speech-related information. In an optional embodiment, if the speech-related information includes speech and draft text, the corresponding preprocessing process can be: converting the speech into text to obtain the speech text; and cleaning and denoising the speech text and draft text.

[0025] For example, speech recognition technology can be used to convert spoken speech into text, and then the converted spoken text can be cleaned and denoised; draft text can also be cleaned and denoised.

[0026] Here, the purpose of cleaning and denoising is to make the text more in line with the norms of expression. For spoken speech, cleaning and denoising may include removing ambient noise, meaningless content such as interjections, correcting typos, replacing homophones, and correcting transcription errors. For draft text, cleaning and denoising may include removing meaningless symbols, correcting typos, replacing homophones, and correcting semantic logic. No specific limitations are made here.

[0027] Step 201: Construct a multidimensional semantic space based on speech-related information, target user profile data, and current meeting information.

[0028] Here, the target user profile data can include tag information such as the target user's historical speaking habits, professional fields, and job roles; the current meeting information can include information such as the meeting name, meeting documents, and participants, and the meeting documents include information such as the meeting topic; the target user profile data and the current meeting information can be pre-stored on the server.

[0029] In this embodiment, the server can preload the target user's profile data and the current meeting information before the meeting starts, which can avoid delays caused by data acquisition and loading during the meeting. Correspondingly, after the server obtains the preprocessed speech-related information, it can quickly construct a multi-dimensional semantic space based on the preprocessed speech-related information, the preloaded target user's profile data, and the preloaded current meeting information, ensuring the real-time nature of speech assistance.

[0030] Understandably, the preprocessed speech-related information, profile data, and meeting information are multi-source heterogeneous data. Directly constructing a multidimensional semantic space based on these three types of data could lead to problems such as data dimension mismatch and broken semantic relationships. To avoid these issues, the preprocessed speech-related information, pre-loaded profile data, and pre-loaded meeting information can be integrated before constructing the multidimensional semantic space to obtain an integrated data object. Then, the multidimensional semantic space can be constructed based on this integrated data object. Here, integration refers to structuring and standardizing processes; the integrated data object can include speech-related information, profile data, and meeting information in a unified format.

[0031] For example, the multidimensional semantic space may include one or more of the following: conference topic vector, real-time context vector, and user feature vector; see also Figure 1 The server includes a semantic parsing engine, which can construct a multi-dimensional semantic space based on the integrated data objects and deeply analyze the meeting context. Compared with traditional keyword matching technology, the embodiments of this application can perform semantic modeling in three dimensions: meeting topic, real-time context, and user, to achieve more accurate semantic understanding. The process is illustrated below.

[0032] For example, when the multidimensional semantic space includes the conference topic vector T, the real-time context vector C, and the user feature vector U, the multidimensional semantic space is a three-dimensional semantic space, which can be represented as [T,C,U]; this will be used as an example for further explanation.

[0033] In some embodiments, a multidimensional semantic space is constructed based on speech-related information, target user profile data, and current meeting information. This may include: extracting keywords from the meeting information to obtain a keyword set; generating a meeting topic vector based on the keyword set; using a context analysis model to transform speech-related information collected in the most recent time period to obtain a real-time context vector; and extracting features from the profile data to obtain a user feature vector.

[0034] For example, the semantic parsing engine can extract keywords from meeting information, such as meeting name and meeting documents, to obtain a set of keywords related to the current meeting; the set of keywords includes multiple keywords; then, the weight of each keyword in the set of keywords is calculated, and finally all the keywords with weights are determined as the meeting topic vector T; this vector can represent the core points of the meeting.

[0035] Here, the algorithm for calculating keyword weights is not specifically limited. For example, it can be the Term Frequency-Inverse Document Frequency (TF-IDF) algorithm, or other types of algorithms.

[0036] Understandably, by calculating keyword weights to generate topic semantic vectors, the core points of a meeting can be accurately anchored, avoiding semantic deviations caused by scattered keywords. For example, in a financial meeting, keywords such as "revenue," "profit," and "cost" can be extracted and assigned weights to construct a meeting topic vector that fits the core points of the meeting.

[0037] For example, a context analysis model refers to a model with context modeling capabilities. For example, it can be a Transformer model or other types of models. No specific limitation is made here. The Transformer model will be used as an example for illustration in the following explanation.

[0038] In this embodiment of the application, the meeting content can be collected in a fixed time period (e.g., every 5 minutes). The meeting content can include information related to the speech of each user during this period. Here, the duration of the fixed time period is not specifically limited. For example, it can be 5 minutes, that is, the meeting content is collected in a 5-minute period.

[0039] For example, to ensure that the real-time context vector always fits the current meeting process, the Transformer model can be used to transform the speech-related information collected in the most recent time period to obtain the real-time context vector C. This vector can capture the evolution of topics and the shift of focus in the real-time context. For example, when the meeting shifts from discussing "last quarter's sales data" to "next quarter's sales strategy", the real-time context vector C will be dynamically updated. In this embodiment, feature extraction can also be performed on the target user's profile data to obtain a user feature vector. As mentioned above, the profile data includes tag information such as professional field, job role, and historical speaking style. Feature extraction algorithms or models can be used to extract features from the profile data to obtain a personalized user feature vector U. This vector is a quantitative expression of the user's multi-dimensional information, which can cover the user's professional field, job role, historical speaking style, and other feature information, ensuring the personalization of subsequent speaking prompts.

[0040] Understandably, after obtaining the user feature vector U, subsequent parsing of information related to the user's speech (such as statements made in meetings, questions raised, etc.) can refer to the user feature information contained in this vector. The parsed results will be more consistent with the user's expression habits and professional background; that is, the content of their speech can be understood according to the user's habitual logical approach. At the same time, it will also be more aligned with the user's professional background, such as accurately identifying the professional terms mentioned by the user, thereby providing more personalized feedback that better meets their needs.

[0041] Step 202: Based on the target user's speech information, determine the target speech type and the target fault type of the target user's speech logic.

[0042] In this embodiment of the application, the semantic parsing engine can also determine the target speech type and the target fault type of the target user's speech logic based on the target user's speech-related information; here, the target speech type and the target fault type can reflect the core intent of the user's speech.

[0043] For example, while performing semantic parsing and modeling of the meeting context, the semantic parsing engine can rely on a three-dimensional semantic space to deeply mine the core intent of users' speeches, ensuring that the parsing results closely align with the meeting theme and the pace of real-time discussion, while also adapting to users' personalized expression habits. Speech intent parsing can include two key aspects: speech type determination and logical gap detection. Speech type determination focuses on the speech-related information of the target user. By extracting semantic features and combining them with vector parameters from the three-dimensional semantic space, the speech type can be accurately defined. Logical gap detection focuses on identifying structural gaps in the user's speech logic. By constructing a standard logical chain and comparing the user's actual logical chain with the standard logical chain, missing logical elements can be located, and the type of logical gap in the speech logic can be determined. Both aspects, through refined classification of speech functions and refined scanning of expression structures, provide crucial input for the subsequent prompting strategy generator; the process is illustrated below.

[0044] In some embodiments, determining the target speech type and the target fault type of the target user's speech logic based on speech-related information of the target user may include: semantically extracting the speech-related information of the target user to obtain a semantic vector; comparing the semantic vector with multiple speech type rules to obtain a first comparison result; determining the target speech type based on the first comparison result; performing semantic recognition on the semantic vector to generate an actual logic chain including logical elements; comparing the actual logic chain with multiple standard logic chains to obtain a second comparison result; determining the missing logical elements based on the second comparison result; and determining the target fault type based on the actual logic chain and the missing logical elements.

[0045] For example, an intent recognition model can be pre-constructed, which may include speech type rules, standard logical chains, and discontinuity features. High-frequency logical structures can be extracted from historical meeting corpora, and a certain number of standard logical chains can be generated through unsupervised clustering, such as the problem statement chain: phenomenon-cause-countermeasure. Speech type rules can be extracted, such as data argumentation containing numerical and time keywords, and solution suggestion containing action and strategy keywords. At the same time, multiple types of discontinuity feature vectors can be pre-stored, such as missing premises, insufficient evidence, logical jumps, and logical contradictions.

[0046] For example, a first preset model can be used to extract semantic information related to the target user's speech to obtain a semantic vector. Here, the type of the first preset model is not specifically limited; for example, it can be the BERT-NER model, which, after training, can output a sequence of logical element labels.

[0047] For example, the BERT-NER model can be used to extract semantic information from the statements of target users, resulting in semantic vectors and logical element labels. During the extraction process, core words related to the meeting topic vector are captured first, redundant information is filtered out by combining real-time context vectors, and the element weights are adjusted with reference to user feature vectors. For instance, based on an operations staff member's statement, "The market share of a certain product has declined," the phenomenon element of declining market share can be extracted, and the association encoding with relevant contextual elements such as "user churn" and "competitive impact" can be strengthened.

[0048] For example, after obtaining the semantic vector, the semantic vector can be compared with the various speech type rules of the intent recognition model to obtain a first comparison result; then, based on the first comparison result, the target speech type can be determined.

[0049] For example, the comparison process can be as follows: calculate the similarity between the semantic vector and each speech type rule, that is, the first comparison result includes the similarity of each of the multiple speech type rules; then, select the speech type rule with the highest similarity as the target speech type.

[0050] For example, if the semantic vector contains a high proportion of action keywords such as "optimize" and "implement", the similarity with the "solution suggestion" rule will be significantly improved; if it contains keywords such as "growth rate", the matching degree with the "data argument" rule will be significantly increased. For example, if the semantic vector corresponding to a certain statement has a similarity of 0.8 with the suggestion rule and a similarity of 0.6 with the summary rule, then the target statement type is determined to be a suggestion statement.

[0051] For example, after obtaining the semantic vector, the semantic vector can be used for logical integrity detection and fault location; correspondingly, a second preset model can be used to perform semantic recognition on the semantic vector to obtain the actual logical chain including logical elements; here, the type of the second preset model is not specifically limited, for example, it can be a BiLSTM-CRF model.

[0052] For example, a user might say: "The market share of a certain product is declining, so I suggest a price reduction and promotion." After semantic extraction and semantic recognition, an actual logical chain containing logical elements (such as phenomenon-countermeasure) can be obtained. At this point, this actual logical chain can be compared with multiple standard logical chains to obtain a second comparison result. Then, based on the second comparison result, the missing logical elements in the actual logical chain can be determined.

[0053] For example, the comparison process can be as follows: by comparing the actual logic chain (e.g., phenomenon-countermeasure) with each standard logic chain (e.g., phenomenon-cause-countermeasure), the missing logical elements can be determined (missing cause analysis). That is, by comparing with the standard logic chain, the location of the missing node can be accurately located.

[0054] In this embodiment of the application, after obtaining the missing logical element, a feature vector can be constructed based on the actual logical chain and the missing logical element. The constructed feature vector is then compared with each of the pre-stored multiple types of fault feature vectors to calculate the similarity. Next, the fault feature vector with the highest similarity is selected as the target fault type. For example, assuming that the missing logical element is a missing cause analysis, the target fault type is determined to be a logical jump.

[0055] Step 203: Generate speech prompt information based on the multidimensional semantic space, target speech type, and target fault type.

[0056] In this embodiment of the application, after obtaining the multidimensional semantic space, target speech type and target tomographic type according to the above steps, speech prompt information can be generated based on this information.

[0057] For example, see Figure 1The server includes a prompting strategy generator, which can generate prompting information based on a multi-dimensional semantic space, such as [T,C,U], the target speech type, and the target fault type.

[0058] In some embodiments, generating speech prompt information based on a multidimensional semantic space, a target speech type, and a target fault type may include: determining a target speech framework corresponding to a target speech type according to a first correspondence; determining a target supplementary suggestion corresponding to a target fault type according to a second correspondence; adding the target supplementary suggestion to the target speech framework to obtain an updated target speech framework; and generating speech prompt information based on the multidimensional semantic space and the updated target speech framework.

[0059] Here, the first correspondence can represent the correspondence between different speech types and different speech frameworks; the second correspondence can represent the correspondence between different fault types and different supplementary suggestions. In an alternative embodiment, see Figure 1 The server can also include a template library, which contains various speaking types, speaking frames, and the correspondence between different speaking types and speaking frames (i.e., the primary correspondence). It should be noted that the speaking frames in the template library are commonly used speaking frames, summarized from common meeting speaking logic and types, and can cover speaking needs in different scenarios.

[0060] For example, the prompting strategy generator can determine the target speech frame corresponding to the target speech type based on the first correspondence in the template library, that is, determine the target speech frame that matches the target speech type. For example, if the target speech type is "problem analysis", the prompting strategy generator can find the target speech frame that matches the target speech type "problem analysis" based on the first correspondence in the template library. For example, the matched target speech frame can be "propose problem - cause analysis - solution".

[0061] In some embodiments, the target speaking frame includes placeholders. After obtaining the target speaking frame, the above method may further include: extracting keywords from the conference topic vector; replacing the placeholders in the target speaking frame with the keywords to obtain the replaced target speaking frame.

[0062] Understandably, the target speaking framework is a general speaking structure that lacks key information about the meeting. To ensure the accurate generation of subsequent speaking prompts, after determining the target speaking framework, keywords can be extracted from the meeting topic vector T. Here, keywords refer to high-frequency keywords that can reflect the core issues of the meeting.

[0063] Here, there are no specific limitations on the extraction method of high-frequency keywords. For example, the extraction method can be: first, segment the conference topic vector T into words, remove non-keywords such as function words and general terms, and then count the frequency of each remaining word segment and select the most frequent words as high-frequency keywords.

[0064] For example, after extracting high-frequency keywords, these keywords can be used to replace placeholders in the target speech frame, resulting in a replaced target speech frame; for instance, replacing "ask a question" in the target speech frame with "the market share of a certain product is declining". Understandably, the replaced target speaking framework is more in line with the meeting theme. If the replaced target speaking framework is used to generate speaking prompts, it can ensure that the speaking prompts are more targeted.

[0065] In an alternative embodiment, see Figure 1 The server may also include a knowledge base, which contains multiple fault types, multiple supplementary suggestions, and the correspondence between different fault types and different supplementary suggestions, i.e., the second correspondence. For example, the prompting strategy generator can determine the target supplementary suggestions corresponding to the target fault type based on the second correspondence in the knowledge base, that is, determine the target supplementary suggestions that match the target fault type. For example, if the target fault type is "logical jump" (such as a lack of cause analysis for declining market share), the prompting strategy generator can find target supplementary suggestions that match the target fault type "logical jump" based on the second correspondence in the knowledge base to supplement the speaking framework and improve the speaking logic.

[0066] In this embodiment of the application, after obtaining the replaced target speech framework and the target supplementary suggestion, the target supplementary suggestion can be added to the replaced target speech framework to obtain the updated target speech framework; then, based on the multidimensional semantic space and the updated target speech framework, speech prompt information is generated.

[0067] In some embodiments, generating speaking prompt information based on a multidimensional semantic space and an updated target speaking framework may include: obtaining domain-specific terminology based on user feature vectors; determining key meeting information based on real-time context vectors; supplementing the updated target speaking framework with domain-specific terminology and key meeting information to obtain initial prompt information; determining the speaking style information of the target user based on user feature vectors; and adjusting the initial prompt information using the speaking style information to obtain speaking prompt information.

[0068] For example, professional domain terms refer to high-frequency, trending terms related to a specific professional field. The knowledge base also includes high-frequency, trending terms from different professional fields. The prompting strategy generator can determine the target user's professional field based on the user's feature vector U, and retrieve high-frequency, trending terms related to the professional field and the conference topic from the knowledge base; that is, professional domain terms. For instance, for conferences on different themes such as technology, finance, marketing, and sales, a set number (e.g., 100) of high-frequency, trending terms can be selected as professional domain terms. The use of these terms can improve the professionalism and accuracy of the speech.

[0069] For example, the prompt strategy generator can also locate the discussion focus of the meeting based on the real-time context vector C, and then extract key information from the meeting documents based on the discussion focus; for example, key information could be "product sales increased by 30% in the third quarter," etc. By integrating this information into the speaking prompts, specific data support can be provided for users' speeches, enhancing the credibility and persuasiveness of their speeches.

[0070] In this embodiment of the application, after obtaining the professional field terminology and key conference information, the professional field terminology and key conference information can be added to the updated target speaking framework to obtain initial prompt information.

[0071] For example, the prompt strategy generator can also determine the user's historical speaking style, such as "concise" or "data-driven", based on the user's feature vector U, and use the historical speaking style to adjust the initial prompt information to obtain the final speaking prompt information.

[0072] For example, if the target user's historical speaking style is concise, the corresponding adjustment method could be: extract the core points and delete redundant descriptions; if the target user's historical speaking style is data-driven, the corresponding adjustment method could be: strengthen data support and supplement more quantitative information.

[0073] Understandably, by using a multi-dimensional semantic space for precise data extraction and style adaptation, it is possible to ensure that the generated speaking prompts not only fit the meeting scenario and professional needs, but also match the user's expression habits, thus truly achieving personalized and precise speaking assistance.

[0074] In some embodiments, the above method may further include: concatenating the target user's speech-related information with a multidimensional semantic space to obtain concatenated information; and processing the concatenated information using a generative model to obtain extended speech information.

[0075] In this embodiment of the application, the speech-related information of the target user can also be concatenated with the multidimensional semantic space. Correspondingly, when the multidimensional semantic space is a three-dimensional vector [T,C,U], the prompting strategy generator can concatenate the three-dimensional vector [T,C,U] with the speech-related information of the target user to obtain the concatenated information. Then, the generative model is used to process the concatenated information to obtain speech extension information.

[0076] Here, there is no specific limitation on the type of generative model; for example, it can be the ChatGPT model, etc.

[0077] Understandably, extended comments refer to reasonable suggestions for improvement. The number of these suggestions can be set according to the actual situation and is not specifically limited here; for example, it could be two or three. These suggestions will concisely expand upon the target user's comment. For example, if the target user mentions "the need to improve the product's user experience," a generative model can be used to generate three related extended comments: 1) Redesign important page effects; 2) Optimize the product's interaction flow; 3) Add personalized services.

[0078] In practical applications, to enhance the reference value of extended speaking information, when multiple pieces of extended speaking information are obtained, the similarity between each piece of extended speaking information and the three-dimensional vector [T,C,U] can be calculated. These pieces are then sorted in descending order of similarity, retaining the top few pieces, for example, the first two. This ensures that the suggested content closely matches the meeting topic and the current context, providing users with more valuable references.

[0079] Step 204: Send the speaking prompt message to the target client corresponding to the target user, and display it through the target client.

[0080] In this embodiment, after generating the speaking prompt information according to the above steps, the server sends the speaking prompt information to the target client corresponding to the target user, and then displays it through the target client. Here, the target client can be any one of multiple clients.

[0081] It should be noted that after the target client receives the speaking prompt information sent by the server, it can present it to the target user in a seamless manner; for example, it can automatically pop up when the target user is typing draft text, or actively push it during the target user's speaking intervals.

[0082] For example, as described above, in addition to generating the speaking prompt message, the server also generates several extended speaking messages. Correspondingly, sending the speaking prompt message to the target client corresponding to the target user can include sending both the speaking prompt message and the extended speaking messages to the target client corresponding to the target user. Here, the way the target client presents the extended speaking messages is similar to that of the speaking prompt message, and to avoid repetition, it will not be described again here.

[0083] Understandably, presenting speaking prompts and extended speaking information in a seamless manner can both avoid affecting the user's speaking status and provide speaking assistance to improve speaking quality; thus, it can enhance the user's meeting experience.

[0084] To facilitate understanding, the following will be combined with Figure 3 The speech assistance method provided in the embodiments of this application will be further described.

[0085] Figure 3 This is a flowchart illustrating another speech assistance method provided in an embodiment of this application, applied to... Figure 1 The speech assistance system shown, such as Figure 3 As shown, the method may include: Step 300: The client performs data collection.

[0086] For example, during the meeting, each client can collect data, specifically, real-time information related to the speech of different users, such as voice recordings and / or draft text, and send the collected data to the server.

[0087] Step 301: Data preprocessing.

[0088] For example, after receiving data from multiple clients, the server can preprocess the received data.

[0089] Step 302: Construct a multidimensional semantic space.

[0090] For example, the server-side semantic parsing engine can construct a multi-dimensional semantic space based on preprocessed speech-related information, target user profile data, and current meeting information.

[0091] Step 303: Determine the speech type and fault type.

[0092] For example, the server-side semantic parsing engine can also determine the target speech type and the target segmentation type of the target user's speech logic based on the target user's speech-related information.

[0093] Step 304: Generating speaking prompts.

[0094] For example, the server-side prompt strategy generator can generate prompt information based on a multidimensional semantic space, the target speech type, and the target fragment type.

[0095] Step 305: Presentation of speaking prompts.

[0096] For example, after generating the speaking prompt information according to the above steps, the server will send the speaking prompt information to the target client corresponding to the target user, and the target client will display it.

[0097] As can be seen, in this embodiment of the application, constructing a multidimensional semantic space can achieve more accurate semantic understanding and provide a reliable data foundation for the generation of subsequent speech prompts; by determining the user's speech type and fragmentation type, the user's speech intent can be accurately identified, ensuring the accuracy of the speech prompts; furthermore, based on the multidimensional semantic space, target speech type, and target fragmentation type, generating and presenting personalized speech prompts can help users reduce the difficulty of speaking and improve the quality of speaking.

[0098] In some embodiments of this application, the speech assistance method can be implemented using a processor in the speech assistance device. The processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), controller, microcontroller, and microprocessor.

[0099] Figure 4 This is a schematic diagram of the composition structure of the speaking assistance device according to an embodiment of this application, such as... Figure 4 As shown, the device includes: a receiving module 400, a building module 401, a determining module 402, a generating module 403, and a sending module 404, wherein: The receiving module 400 is used to receive speech-related information sent by multiple clients; The construction module 401 is used to construct a multi-dimensional semantic space based on the speech-related information, the target user's profile data, and the current meeting information; the multi-dimensional semantic space includes one or more of the following: meeting topic vector, real-time context vector, and user feature vector; The determining module 402 is used to determine the target speech type of the target user and the target fault type of the speech logic of the target user based on the speech-related information of the target user; The generation module 403 is used to generate speech prompt information based on the multidimensional semantic space, the target speech type, and the target tomographic type; The sending module 404 is used to send the speaking prompt information to the target client corresponding to the target user, and to present it through the target client; the target client is any one of the plurality of clients.

[0100] In some embodiments, where the multidimensional semantic space includes meeting topic vectors, real-time context vectors, and user feature vectors, the construction module 401 is further configured to: Keyword extraction is performed on the meeting information to obtain a keyword set; Based on the keyword set, the conference topic vector is generated; The context analysis model is used to transform the speech-related information collected in the most recent time period to obtain the real-time context vector; Feature extraction is performed on the portrait data to obtain the user feature vector.

[0101] In some embodiments, the determining module 402 is further configured to: Semantic vectors are obtained by extracting semantic information from the speech of the target user. The semantic vector is compared with rules for various speech types to obtain the first comparison result; Based on the first comparison result, determine the target speech type; The semantic vector is semantically recognized to generate an actual logical chain including logical elements; The actual logic chain is compared with multiple standard logic chains to obtain a second comparison result; Based on the second comparison result, determine the missing logical element; Based on the actual logical chain and the missing logical element, the target fault type is determined.

[0102] In some embodiments, the generation module 403 is further configured to: Based on the first correspondence, the target speech frame corresponding to the target speech type is determined; the first correspondence represents the correspondence between different speech types and different speech frames; Based on the second correspondence, the target supplementary recommendations corresponding to the target fault type are determined; the second correspondence represents the correspondence between different fault types and different supplementary recommendations. The target supplementary suggestions are added to the target speech frame to obtain the updated target speech frame; Based on the multidimensional semantic space and the updated target speech framework, the speech prompt information is generated.

[0103] In some embodiments, the generation module 403 is further configured to: Extract keywords from the meeting topic vector; The placeholders in the target speech frame are replaced with the keywords to obtain the replaced target speech frame; The target supplementary suggestions are added to the replaced target speech frame to obtain the updated target speech frame.

[0104] In some embodiments, the generation module 403 is further configured to: Based on the user feature vector, obtain professional domain terminology; Based on the real-time context vector, key meeting information is determined; The professional terminology and key meeting information are added to the updated target speaking framework to obtain initial prompt information; Based on the user feature vector, the historical speaking style of the target user is determined; The initial prompt information is adjusted using the speaking style information to obtain the speaking prompt information.

[0105] In some embodiments, the generation module 403 is further configured to: The speech-related information of the target user is concatenated with the multidimensional semantic space to obtain the concatenated information; The spliced ​​information is processed using a generative model to obtain extended speech information; In some embodiments, the sending module 404 is further configured to: The message prompt and the message extension information are sent to the target client corresponding to the target user.

[0106] In practical applications, the receiving module 400, the building module 401, the determining module 402, the generating module 403, and the sending module 404 can all be implemented by a processor located in an electronic device. The processor can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor.

[0107] Furthermore, in this embodiment, the functional modules can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional module.

[0108] If the integrated unit is implemented as a software functional module and is not sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this embodiment, in essence, or the part that contributes to related technologies, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the method of this embodiment. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0109] Specifically, the computer program instructions corresponding to a speech assistance method in this embodiment can be stored on storage media such as optical discs, hard disks, and USB flash drives. When the computer program instructions corresponding to a speech assistance method in the storage media are read or executed by an electronic device, any of the speech assistance methods in the aforementioned embodiments are implemented.

[0110] Based on the same technical concept as the foregoing embodiments, see Figure 5 It illustrates an electronic device 500 provided in an embodiment of this application, which may include: a memory 501 and a processor 502; wherein, Memory 501 is used to store computer programs and data; Processor 502 is configured to execute a computer program stored in memory to implement any of the speech assistance methods described in the foregoing embodiments.

[0111] In practical applications, the aforementioned memory 501 can be volatile memory, such as RAM; or non-volatile memory, such as ROM, flash memory, hard disk drive (HDD), or solid-state drive (SSD); or a combination of the above types of memory, and provide instructions and data to the processor 502.

[0112] The processor 502 described above can be at least one of ASIC, DSP, DSPD, PLD, FPGA, CPU, controller, microcontroller, and microprocessor. It is understood that, for different model training devices, the electronic device used to implement the above processor function can also be other types, and this application embodiment does not specifically limit the specific types.

[0113] This application provides a computer program product, including a computer program that, when executed by a processor, implements any of the speech assistance methods described in the foregoing embodiments.

[0114] In some embodiments, the functions or modules of the apparatus provided in this application can be used to perform the methods described in the above method embodiments. Specific implementations can be referred to the descriptions of the above method embodiments, and for brevity, will not be repeated here. The descriptions of the various embodiments above tend to emphasize the differences between them; their similarities or commonalities can be referred to each other, and for brevity, will not be repeated here.

[0115] The methods disclosed in the various method embodiments provided in this application can be arbitrarily combined to obtain new method embodiments without conflict.

[0116] The features disclosed in the various product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0117] The features disclosed in the various method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0118] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0119] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0120] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0121] The above are merely preferred embodiments of this application and are not intended to limit the scope of protection of this application.

Claims

1. A speaking assistance method, characterized in that, Applied to the server side, the method includes: Receive speech-related information from multiple clients; Based on the speech-related information, the target user's profile data, and the current meeting information, a multidimensional semantic space is constructed; the multidimensional semantic space includes one or more of the following: meeting topic vector, real-time context vector, and user feature vector; Based on the target user's speech information, determine the target speech type of the target user and the target fault type of the target user's speech logic; Based on the multidimensional semantic space, the target speech type, and the target tomographic type, speech prompt information is generated; The speaking prompt information is sent to the target client corresponding to the target user and presented through the target client; the target client can be any one of the plurality of clients.

2. The method according to claim 1, characterized in that, In a multidimensional semantic space comprising meeting topic vectors, real-time context vectors, and user feature vectors, the construction of the multidimensional semantic space based on the speaking-related information, the target user's profile data, and the current meeting information includes: Keyword extraction is performed on the meeting information to obtain a keyword set; Based on the keyword set, the conference topic vector is generated; The context analysis model is used to transform the speech-related information collected in the most recent time period to obtain the real-time context vector; Feature extraction is performed on the portrait data to obtain the user feature vector.

3. The method according to claim 1, characterized in that, The step of determining the target user's target speech type and the target fragmentation type of the target user's speech logic based on the target user's speech-related information includes: Semantic vectors are obtained by extracting semantic information from the speech of the target user. The semantic vector is compared with rules for various speech types to obtain the first comparison result; Based on the first comparison result, determine the target speech type; The semantic vector is semantically recognized to generate an actual logical chain including logical elements; The actual logic chain is compared with multiple standard logic chains to obtain a second comparison result; Based on the second comparison result, determine the missing logical element; Based on the actual logical chain and the missing logical element, the target fault type is determined.

4. The method according to claim 1, characterized in that, The generation of speech prompt information based on the multidimensional semantic space, the target speech type, and the target tomographic type includes: Based on the first correspondence, the target speech frame corresponding to the target speech type is determined; the first correspondence represents the correspondence between different speech types and different speech frames; Based on the second correspondence, the target supplementary recommendations corresponding to the target fault type are determined; the second correspondence represents the correspondence between different fault types and different supplementary recommendations. The target supplementary suggestions are added to the target speech frame to obtain the updated target speech frame; Based on the multidimensional semantic space and the updated target speech framework, the speech prompt information is generated.

5. The method according to claim 4, characterized in that, The above methods may also include: Extract keywords from the meeting topic vector; The placeholders in the target speech frame are replaced with the keywords to obtain the replaced target speech frame; The target supplementary suggestions are added to the target speaking frame to obtain an updated target speaking frame, including: The target supplementary suggestions are added to the replaced target speech frame to obtain the updated target speech frame.

6. The method according to claim 4, characterized in that, The step of generating the speech prompt information based on the multidimensional semantic space and the updated target speech framework includes: Based on the user feature vector, obtain professional domain terminology; Based on the real-time context vector, key meeting information is determined; The professional terminology and key meeting information are added to the updated target speaking framework to obtain initial prompt information; Based on the user feature vector, the historical speaking style of the target user is determined; The initial prompt information is adjusted using the speaking style information to obtain the speaking prompt information.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: The speech-related information of the target user is concatenated with the multidimensional semantic space to obtain the concatenated information; The spliced ​​information is processed using a generative model to obtain extended speech information; Sending the speaking prompt information to the target client corresponding to the target user includes: The message prompt and the message extension information are sent to the target client corresponding to the target user.

8. An electronic device, characterized in that, The device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the method according to any one of claims 1 to 7.

9. A computer storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the method described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1 to 7.

Images