Intelligent conference record processing method, system, equipment and medium

Abstract

The invention discloses an intelligent conference record processing method, system and device and a medium, and the method specifically comprises the steps: building a causal atlas through an extraction event based on a primary text and a conference audio stream, marking decision points according to the causal atlas, and obtaining a logic chain analysis result; based on the primary text and the conference audio stream, analyzing emotional fluctuation of the speaker in combination with the acoustic features and the text to obtain an emotional voiceprint analysis result; based on a logic chain analysis result and an emotional voiceprint analysis result, adopting a large language model guided by a specific Prompt to generate an initial conference summary abstract attached with an original text sentence sequence label as a source tracing corner mark; and receiving user role configuration information, processing the initial conference summary abstract according to the user role configuration information, and generating a granularity-adjustable target conference summary abstract matched with the user role. The conference recording efficiency and quality are improved, and diversified requirements are met.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an intelligent meeting recording processing method, system, device, and medium. Background Technology

[0002] In modern office environments, meetings serve as crucial venues for information exchange and decision-making, and meeting minutes are a key means of ensuring accurate information transmission and effective implementation of decisions. Traditional meeting minutes primarily rely on manual recording, which has several drawbacks. Firstly, manual recording requires significant manpower and time; during lengthy meetings, recorders are prone to fatigue, leading to omissions or errors, compromising the completeness and accuracy of the minutes. Secondly, manual recording is inefficient, failing to provide participants with timely meeting minutes, thus hindering subsequent work.

[0003] In recent years, with the rapid development of recording equipment and speech recognition technology, the automation of meeting minutes has become possible, providing a new way to address the shortcomings of traditional manual recording. However, existing automated meeting minutes technologies still face a series of problems that urgently need to be solved in practical applications, as follows: 1. Recording Format Compatibility Issues: Many meeting recording tools on the market currently only support a limited number of audio formats. This means that if the recorded audio format is incompatible with the tool, users will need to perform additional format conversion. This process not only increases the complexity for users but may also affect the quality of subsequent speech recognition and meeting recordings due to data loss or errors during format conversion. For example, certain audio files generated by some professional recording equipment may not be directly recognized by common meeting recording software. Users need to use third-party software for format conversion, which is not only time-consuming but may also lead to a decrease in audio quality due to software compatibility issues.

[0004] 2. Text Recognition Accuracy Issues: Existing speech recognition technology suffers a significant drop in accuracy in complex meeting environments, especially in the presence of noise interference. Various background noises, such as air conditioning hum, footsteps, and interference from other electronic devices, can interfere with the speech recognition system's accurate identification of meeting audio, leading to numerous errors in the transcribed text and severely impacting the quality of meeting records. For example, in a noisy meeting room, the speech recognition system might misidentify "discuss project progress" as "discuss project longitude," a mistake that could mislead subsequent understanding and analysis of the meeting content.

[0005] 3. The Issue of Intelligent Content Summarization: Most existing meeting recording tools have relatively limited functionality, primarily focusing on simple text transcription and lacking in-depth analysis and intelligent summarization capabilities. Meeting minutes are not merely about converting spoken content into text; more importantly, they involve extracting key information from a large volume of statements, clarifying logical relationships, and analyzing speakers' emotions to generate a comprehensive, accurate, and organized meeting summary. However, existing tools fail to meet users' needs for efficient meeting recording. Users still need to spend a significant amount of time manually organizing and analyzing the transcribed text, thus failing to fully leverage the advantages of automated meeting recording technology. Summary of the Invention

[0006] The purpose of this invention is to provide an intelligent meeting recording processing method, system, device, and medium, which realizes full-process automation and intelligence of meeting recording from audio acquisition to the generation of meeting minutes summaries adapted to different user roles, improves the efficiency and quality of meeting recording, meets diverse needs, and solves at least one of the above-mentioned problems in the prior art.

[0007] In a first aspect, the present invention provides an intelligent meeting recording processing method, the method specifically comprising: Collect the conference audio stream, perform speech-to-text transcription on the conference audio stream, and simultaneously output word-level confidence scores and word-level timestamps to obtain the transcribed text; The transcribed text is segmented into sentences, and a unique sequence number is added to each sentence to form the initial text; Based on primary text and conference audio streams, a causal graph is constructed by extracting events, and decision points are marked according to the causal graph to obtain logical chain analysis results; Based on primary text and conference audio streams, and combining acoustic features with text analysis of the speaker's emotional fluctuations, emotional voiceprint analysis results are obtained. Based on the results of logical chain analysis and sentiment voiceprint analysis, a large language model guided by a specific prompt is used to generate an initial meeting minutes summary with original sentence sequence numbers as source tracing subscripts. Receive user role configuration information, process the initial meeting minutes summary according to the user role configuration information, and generate a target meeting minutes summary with adjustable granularity that matches the user role.

[0008] Secondly, the present invention provides an intelligent meeting recording processing system, the system specifically comprising: The first processing module is used to acquire the conference audio stream, perform speech-to-text transcription on the conference audio stream, and simultaneously output word-level confidence scores and word-level timestamps to obtain the transcribed text. The second processing module is used to segment the transcribed text into sentences and add a unique sequence number to each sentence to form the initial text; The third processing module is used to construct a causal graph by extracting events based on the primary text and conference audio stream, mark decision points according to the causal graph, and obtain the logical chain analysis results. The fourth processing module is used to obtain emotional voiceprint analysis results by combining acoustic features with text analysis of the speaker's emotional fluctuations based on the primary text and conference audio stream. The fifth processing module is used to generate an initial meeting minutes summary with original sentence sequence numbers as source indices based on the results of logical chain analysis and sentiment voiceprint analysis, using a large language model guided by a specific prompt. The sixth processing module is used to receive user role configuration information, process the initial meeting minutes summary according to the user role configuration information, and generate a target meeting minutes summary with adjustable granularity that matches the user role.

[0009] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, and a computer program stored in the memory, wherein when the computer program is executed on the processor, it implements the intelligent meeting recording processing method as described in any of the above methods.

[0010] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the intelligent meeting recording processing method as described in any of the above methods.

[0011] Compared with the prior art, the present invention has at least one of the following technical effects: 1. This invention automates and intelligently processes the entire process of meeting recording, from audio acquisition to generating meeting minutes summaries adapted to different user roles, thereby improving the efficiency and quality of meeting recording and meeting diverse needs.

[0012] 2. This invention improves the accuracy of speech transcription and facilitates subsequent quality assessment and correction by synchronously outputting word-level confidence scores and timestamps through an end-to-end model and marking low-confidence segments.

[0013] 3. This invention integrates multiple boundary determination methods to segment the transcribed text by sentence and add unique sequence numbers, forming a primary text with a clear structure and complete information, providing a good foundation for subsequent analysis.

[0014] 4. This invention extracts events from primary text and audio streams to construct a causal graph and marks decision points, clearly presenting the logical chain of the meeting, which is conducive to understanding the meeting decision-making process and key information.

[0015] 5. This invention combines acoustic features with text analysis to analyze the speaker's emotional fluctuations, generate an emotional fluctuation curve and associate it with the text, accurately grasp the speaker's emotional changes, and enrich the information dimensions of the meeting record.

[0016] 6. This invention integrates logical chains and emotional voiceprint analysis results, and uses a specific prompt to guide a large language model to generate an initial summary with source tagging, ensuring the accuracy and traceability of the content.

[0017] 7. This invention calls the rule base to match content mapping rules based on user role configuration information, filters and adjusts the granularity of the initial summary, and generates personalized meeting minutes that meet user needs. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating an intelligent meeting recording processing method according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an intelligent meeting recording processing system provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0020] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0021] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0022] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0024] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0025] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0026] In this application embodiment, the entity executing the process includes a terminal device. This terminal device includes, but is not limited to, devices capable of executing the methods disclosed in this application, such as servers, computers, smartphones, and tablets. Figure 1 A flowchart illustrating an embodiment of the intelligent meeting recording processing method disclosed in this invention is shown below in detail: S101 acquires the conference audio stream, performs speech-to-text transcription on the conference audio stream, and simultaneously outputs word-level confidence scores and word-level timestamps to obtain the transcribed text.

[0027] In this embodiment, the acquired conference audio stream is transcribed into speech and input into a speech recognition module. This module employs an advanced deep learning algorithm to build a speech recognition model, which has been trained on a large amount of speech data from different scenarios and accents. During the transcription process, the speech recognition model analyzes the input audio stream frame by frame, matching audio features with speech patterns stored in the model to identify the corresponding text information. To improve the accuracy and adaptability of the transcription, the speech recognition model adjusts and optimizes in real time based on the characteristics of the conference audio. For example, for technical terms and specific names that may appear in the meeting, the system pre-builds a dedicated vocabulary database and prioritizes searching for matches from this database during the recognition process to improve the accuracy of recognizing these special words. Simultaneously, the speech recognition model dynamically adjusts recognition parameters based on features such as speech rate and intonation to ensure accurate transcription of speech content under different speaking styles.

[0028] Simultaneously with speech-to-text transcription, the system outputs word-level confidence scores and word-level timestamps. For each recognized word, the system assigns a confidence score based on the certainty of the speech recognition model's recognition of that word. The confidence score typically ranges from 0 to 1; a score closer to 1 indicates greater confidence in the system's recognition of the word, while a lower score indicates greater uncertainty. The word-level timestamp records the start and end times of each word in the audio stream, accurate to the millisecond level. These timestamps accurately pinpoint the specific location of each word in the meeting audio, providing a temporal reference for subsequent text analysis and processing. For example, when it's necessary to locate and analyze a specific viewpoint in a meeting, the corresponding audio segment can be quickly found based on the word-level timestamps of keywords within that viewpoint, facilitating playback and review for the user.

[0029] The text information obtained from speech-to-text transcription, along with the synchronously output word-level confidence scores and word-level timestamps, is integrated to form the transcribed text. The transcribed text is stored in a structured manner, such as using XML or JSON format, encapsulating each word and its corresponding confidence score and timestamp information for easy subsequent processing and analysis. In the transcribed text, words are arranged sequentially according to their order of appearance in the conference audio, and each word is accompanied by its corresponding confidence score and timestamp information, forming a complete and traceable transcription result.

[0030] S102, the transcribed text is segmented into sentences, and a unique sequence number is added to each sentence to form the primary text.

[0031] In this embodiment, since the transcribed text is obtained through speech recognition technology, there may be some special characters, extra spaces, or non-standard punctuation caused by speech recognition errors. Therefore, the transcribed text needs to be cleaned and standardized first.

[0032] The system employs a combination of punctuation and semantic analysis to determine sentence boundaries. Firstly, it uses common punctuation marks such as periods, question marks, and exclamation marks as initial criteria for sentence segmentation. When these punctuation marks are identified, the system considers them potential sentence endings. However, relying solely on punctuation for segmentation can be inaccurate, as punctuation usage may not fully conform to conventional sentence segmentation rules in some contexts; for example, certain ellipses may not indicate the end of a sentence. Therefore, the system incorporates semantic analysis to further determine sentence boundaries. The system integrates a natural language processing model trained on extensive text data, capable of understanding the semantic information of text. When a potential sentence-ending punctuation mark is identified, the model performs semantic analysis on the text before and after the punctuation mark to determine whether the content before and after it is semantically complete and independent. If the preceding and following content is a complete semantic expression, the punctuation mark is designated as the sentence boundary, and sentence segmentation is performed. If the semantics are incomplete, for example, if the content immediately following the ellipsis is semantically closely related to the preceding text, the punctuation mark is not used as the sentence boundary, and the search continues to find a suitable segmentation position. By combining punctuation and semantic analysis, this method can more accurately segment transcribed text into individual sentences.

[0033] Each segmented sentence is assigned a unique sequence label. The system maintains a global sequence counter, initially set to 1. After each sentence is segmented, the system uses the current sequence counter value as the sentence's sequence label and adds it to a specified position, such as the beginning or end of the sentence. After adding the sequence label, the sequence counter value is automatically incremented by 1 to ensure that the next sentence receives a different unique sequence label. In this way, each sentence is given a unique identifier, facilitating subsequent referencing, searching, and processing of sentences.

[0034] All sentences with unique sequence numbers are combined in the order they appear in the original transcribed text to form a primary text. The primary text is stored in a structured manner, such as in a text file format, with each sentence occupying one line and its corresponding sequence number displayed at the beginning of each line. In this way, the primary text not only retains the content information of the original transcribed text but also provides a clear identifier for each sentence through the sequence numbers.

[0035] S103, based on primary text and conference audio stream, constructs a causal graph by extracting events, marks decision points according to the causal graph, and obtains logical chain analysis results.

[0036] In this embodiment, each sentence in the primary text is analyzed sentence by sentence. Natural language processing technology is used to identify verbs and related nouns, phrases, and other elements in the sentences to determine the core content of the event. For example, for the sentence "We decided to hold a project kickoff meeting this Friday," the system can identify the verb "decided" and the related action "hold a project kickoff meeting," thereby extracting the event "decided to hold a project kickoff meeting." Simultaneously, the system also incorporates acoustic features from the meeting audio stream, such as changes in tone and speed, to help determine the emphasis and importance of the event. If an event is spoken with a heavier tone and slower speed in the audio, the system considers the event relatively important and gives it more attention in subsequent processing. During the event extraction process, the system creates an event list, recording all identified events in the order they appear in the primary text and assigning each event a unique event number.

[0037] The system analyzes the semantic logic and context of sentences in the initial text to find causal clues between events. For example, if a sentence contains a conjunction like "because...therefore...", the system will directly identify a causal relationship between the two events. For cases without obvious conjunctions, the system uses semantic understanding models from natural language processing to analyze factors such as the sequence of events and conditional relationships to infer whether a causal link exists between them. For instance, in the sentence "Project progress delays lead to adjustments in subsequent plans," although there are no obvious conjunctions, semantic understanding can determine that "project progress delays" is the cause and "adjustments to subsequent plans" is the result. Furthermore, the system also considers acoustic features such as pauses and emphasis in the meeting audio stream. If a speaker pauses for a long time after describing one event before describing another, and emphasizes the latter, it may suggest a causal relationship between the two events. These acoustic clues can be used to further verify and confirm the causal relationship. Through these methods, a causal network between events can be identified.

[0038] Using events from the event list as nodes, events with causal relationships are connected by directed edges. The direction of the edge indicates the direction of the causal relationship, i.e., from the cause event to the result event. For example, if event A is the cause of event B, then there will be a directed edge in the causal graph pointing from event A to event B. During the construction of the causal graph, the system considers the importance and relevance of events, and performs reasonable layout and optimization of the graph to make the causal relationships clearer and easier to understand. At the same time, the system annotates relevant information for each event, such as event number and event description, on the corresponding nodes for easy viewing and understanding by users. Ultimately, a complete causal graph is formed, intuitively displaying the causal logical relationships between various events in the meeting.

[0039] Decision points in the causal graph are marked. Decision points are typically event nodes that play a crucial role in the causal relationship and can influence the direction of subsequent events. Decision points are identified by analyzing the scope of influence of events and the diversity of subsequent events in the causal graph. If the outcome of an event leads to multiple events in different directions, or if the outcome of the event has a significant impact on the direction of subsequent meetings, then this event is considered a decision point. For example, in a causal graph, after the event "decide to adopt option A," multiple event branches with different outcomes such as "option A was successfully implemented" and "option A failed to implement" appear. Therefore, the event "decide to adopt option A" is a decision point. The system will specially mark these decision points in the causal graph, such as using different colors, shapes, or adding specific symbols, so that users can quickly identify key decision-making steps in the meeting.

[0040] Based on the labeled causal graph, logical chain analysis results are obtained. These results are presented as a visual causal graph, and the system generates a detailed textual report. The report describes the main logical chains in the causal graph, explaining the causes, potential impacts, and subsequent development directions of each decision point. For example, the report might state, "During the meeting, the project delay (cause) led to adjustments in the subsequent plan (result). During the adjustment process, the decision to adopt Option A was made, which has multiple possible outcome branches..." This combination of visual graphs and detailed textual explanations allows users to comprehensively and deeply understand the logical relationships and decision-making processes within the meeting, providing strong support for subsequent work decisions and debriefing.

[0041] S104, based on primary text and conference audio stream, combines acoustic features with text analysis to analyze the speaker's emotional fluctuations and obtain emotional voiceprint analysis results.

[0042] In this embodiment, key acoustic features are extracted from the conference audio stream, and in-depth semantic analysis is performed on the initial text. After completing the acoustic feature extraction and text semantic analysis, the results are initially fused to make an initial judgment on the speaker's emotion. Specifically, an emotion mapping model is established, corresponding different combinations of acoustic features and text semantic features to common emotion types. For example, high pitch, loud volume, and rapid speech combined with text rich in positive vocabulary may correspond to excitement or agitation; low pitch, soft volume, and slow speech combined with text rich in negative vocabulary may correspond to sadness or frustration. For each audio segment and its corresponding text, the most matching emotion type is searched in the emotion mapping model based on the extracted acoustic and text semantic features, serving as the preliminary emotion judgment result. In this process, considering that the influence weights of acoustic features and text semantic features on emotion judgment may differ, a reasonable weight allocation is made according to the actual situation. For example, in some cases, textual semantics may be more accurate in expressing the speaker's emotions, and in such cases, the weight of textual semantic features can be appropriately increased; while in other cases, such as when the speaker deliberately uses a calm tone to express strong emotions, acoustic features may be more valuable, and the weight of acoustic features should be increased accordingly.

[0043] Because initial emotion judgments may contain some errors, they need to be verified and corrected. On one hand, verification should be conducted in conjunction with the overall context of the meeting and the speaker's personal style. Understand the topic and background of the meeting discussion, as well as the speaker's emotional characteristics in previous meetings. For example, in a serious project discussion meeting, if the initial judgment is that the speaker is overly excited, but considering the meeting topic and the speaker's consistently calm style, this judgment may be inaccurate and needs correction. On the other hand, refer to other relevant audio and text information. For example, examine the speaker's emotional performance at other times in the same meeting, or analyze other participants' reactions to the current statement. If other participants show surprise or skepticism towards the speaker's content, it may indicate a special circumstance in the speaker's emotional expression, requiring a re-examination of the initial emotion judgment. Through these verification and correction measures, the accuracy and reliability of emotion judgments can be improved.

[0044] After the aforementioned emotion analysis and correction, the final emotional voiceprint analysis results are generated. The speaker's emotional changes over different time periods are presented visually, for example, by plotting an emotion fluctuation curve, with the horizontal axis representing time and the vertical axis representing emotion type and intensity. Different colors or line styles are used in the curve to represent different emotion types, such as red for anger, blue for calm, and green for happiness, etc. The thickness of the lines or the shade of the color are adjusted according to the intensity of the emotion. In addition to the visual charts, a detailed textual report is generated, analyzing the speaker's emotional changes segment by segment and explaining the basis and reasons for the emotion judgments at each time period.

[0045] S105, based on the results of logical chain analysis and sentiment voiceprint analysis, uses a large language model guided by a specific prompt to generate an initial meeting minutes summary with original sentence sequence numbers as source tracing subscripts.

[0046] In this embodiment, the results of logical chain analysis and emotional voiceprint analysis are meticulously analyzed and integrated. The logical chain analysis results contain the causal relationships between various events in the meeting and the marking information of decision points, clearly presenting the context and direction of the meeting discussion. The emotional voiceprint analysis results reflect the speaker's emotional fluctuations at different stages, such as excitement, calmness, and hesitation. These two results are correlated and integrated, using a timeline as a benchmark, matching the logical relationships and emotional states corresponding to each time point.

[0047] Based on the requirements for generating meeting minutes summaries, a specific Prompt is carefully designed. The Prompt is a key instruction guiding a large language model to generate content that meets the requirements. When designing it, the characteristics of the logical chain analysis results and the sentiment voiceprint analysis results, as well as the objectives of the meeting minutes summary, must be fully considered. For example, the Prompt can include the following key parts: First, it clearly informs the language model of the type and content of the input data, i.e., "You will receive comprehensive data containing meeting logical chain information and speaker sentiment voiceprint information"; second, it details the requirements for generating the summary, such as "Please generate a meeting minutes summary based on this information. The summary needs to cover the main decisions, key events, and sentiment trends of each stage of the meeting"; third, it emphasizes the source tracing function, pointing out that "In the summary, for each viewpoint and decision, the original sentence sequence number should be attached as a source tracing index so that the original speech content can be quickly located." Through such a comprehensive and specific design, it ensures that the large language model can accurately understand the task requirements and generate a meeting minutes summary that meets the expectations.

[0048] The integrated data, including logical chain and sentiment voiceprint information, is input into a large language model according to a pre-designed prompt. During input, it's crucial to ensure the data is formatted correctly and completely so the language model can read and process it smoothly. For example, the integrated data should be presented in structured text, clearly separating logical chain information, sentiment voiceprint information, and corresponding time markers. Simultaneously, the specific prompt is input as guiding information, clearly instructing the language model on how to use this input data for summary generation. After input, the large language model's program is launched to begin processing and analyzing the input data.

[0049] After receiving input data and a specific prompt, the large-scale language model utilizes its powerful natural language processing capabilities and deep learning algorithms for in-depth analysis. It first parses the logical chain analysis results to understand the causal relationships and decision-making processes between various events in the meeting, grasping the core content and key directions. For example, by analyzing the logical chain, the model can identify key decision points and important turning points in the project's progress. Then, the model combines the results of sentiment analysis to understand the speaker's emotional state at different stages, thereby better understanding the atmosphere of the meeting discussion and the speaker's true intentions. For instance, when a speaker is emotionally excited while discussing a solution, the model will consider that solution to be highly feasible and attractive. Based on a comprehensive understanding of logical and sentiment information, the model begins to generate a meeting minutes summary according to the specific prompt's requirements. During the generation process, the model extracts key information from the original transcribed text and organizes it in logical order, while also appropriately adjusting the expression based on sentiment analysis results to make the summary more vivid and accurate. Furthermore, the model strictly adheres to requirements, attaching original sentence sequence numbers as source markers to each viewpoint and decision, facilitating users' subsequent review of the original speech content. The final generated initial meeting minutes summary, with the original sentence sequence numbers as source indices, is output.

[0050] S106, Receive user role configuration information, process the initial meeting minutes summary according to the user role configuration information, and generate a target meeting minutes summary with adjustable granularity that matches the user role.

[0051] In this embodiment, a user interface is constructed to present the user with intuitive and convenient information, receiving user-inputted role configuration information. The interface design can utilize common elements such as drop-down menus, radio buttons, or text input boxes. For example, common user roles, such as project manager, technical personnel, marketing personnel, and financial personnel, can be pre-set in the drop-down menu; users simply need to select their corresponding role from the list. Simultaneously, to meet specific needs, text input boxes can be provided, allowing users to customize role names. Furthermore, the interface should offer options for adjusting the granularity of meeting minutes summaries, such as different levels like "detailed," "moderate," and "brief," allowing users to select according to their needs.

[0052] Upon receiving user role configuration information, the system initiates an analysis process for user roles and requirements. For different user roles, the system has predefined requirement models. For example, a project manager's requirement model explicitly states the need to focus on meeting decisions, project schedules, resource allocation, and solutions to key issues, with a high level of detail required for meeting minutes summaries to ensure a comprehensive understanding of all aspects of the project. For marketing personnel, the requirement model emphasizes information related to market strategies, competitor analysis, and customer feedback from meetings, with a more moderate summary granularity, highlighting key market information. The system matches the received user role with the built-in requirement models to quickly determine the user's specific requirements for meeting minutes summaries, including the required content points, level of detail, and key areas of focus.

[0053] After clarifying user needs, the system begins in-depth analysis of the initial meeting minutes summary. This summary contains comprehensive meeting information, presented in structured text format with source tagging for the original sentences. The system utilizes natural language processing (NLP) techniques to perform word segmentation, part-of-speech tagging, and syntactic analysis, breaking the text down into easily processed semantic units. Simultaneously, based on the source tags, the system can establish a connection between the summary content and the original transcribed text, facilitating rapid location and retrieval of more detailed information in subsequent processing. For example, if the summary's description of a decision is insufficiently detailed, the system can use the source tags to find the detailed discussion of that decision in the original speeches.

[0054] Based on the user's role and corresponding requirement model, the system filters and adjusts the content of the parsed initial meeting minutes summary. For content of key user interest, the system retains and further refines the description; content less relevant to user needs is appropriately simplified or deleted. For example, if the meeting discussed implementation details of technical solutions or solutions to technical problems, the system will retain this information in detail in the summary, providing accurate explanations of technical terms and concepts to ensure clear understanding for technical personnel. Administrative arrangements or marketing content unrelated to technology are simplified. During the adjustment process, the system also considers the summary granularity set by the user. If the user selects "detailed" granularity, the system retains as much detail as possible; if "brief" granularity is selected, only key points and core conclusions are retained. After content filtering and adjustment, a target meeting minutes summary with adjustable granularity matching the user's role is generated.

[0055] In some embodiments, step S101 above, which involves transcribing the conference audio stream and simultaneously outputting word-level confidence scores and word-level timestamps to obtain transcribed text, specifically includes: The conference audio stream is fed into a pre-built end-to-end deep learning speech recognition model; The speech transcription and metadata generation operations are performed synchronously based on the end-to-end deep learning speech recognition model, and the transcribed text, the confidence score of each word and its corresponding timestamp information are output. The word-level confidence score is obtained by calculating the Softmax posterior probability of the corresponding word in the output layer of the end-to-end deep learning speech recognition model, and the word-level timestamp information is obtained by analyzing the alignment relationship between the attention weight matrix of the end-to-end deep learning speech recognition model and the input audio time frame.

[0056] The transcribed text is associated and encapsulated with the corresponding word-level confidence score and word-level timestamp. The word-level confidence score is judged according to the preset confidence threshold, and low-confidence word segments below the confidence threshold are automatically marked in the transcribed text.

[0057] In this embodiment, a large-scale and diverse dataset of conference audio is collected, covering audio from different accents, speaking speeds, pronunciation habits, and various conference scenarios. After the dataset is collected, it is annotated by labeling each audio segment with corresponding accurate text content, forming an annotated speech dataset.

[0058] Choose a suitable end-to-end deep learning architecture, such as a model based on Transformer or Conformer structures. These architectures have advantages in processing sequence data and can better capture temporal information and semantic features in speech. Train the selected model using a labeled speech dataset. During training, use appropriate loss functions and optimization algorithms to continuously adjust the model's parameters so that the model can learn the mapping relationship from speech features to text. Through multiple iterations of training, until the model's performance on the validation set reaches the expected standards, such as accuracy and recall, the pre-built end-to-end deep learning speech recognition model is trained and ready for use.

[0059] The acquired conference audio stream is used as input and transmitted to a pre-built end-to-end deep learning speech recognition model. This model has parallel processing capabilities and simultaneously initiates two key operations while receiving the audio stream: speech transcription and metadata generation.

[0060] In speech-to-text transcription, the model extracts features from the input audio stream, converting the continuous audio signal into a sequence of feature vectors suitable for the model's processing. Then, using the trained model parameters, these feature vectors are calculated and inferred to gradually generate the corresponding text sequence, i.e., the transcribed text.

[0061] In the metadata generation process, the model simultaneously calculates the confidence score for each word and determines its corresponding timestamp information. For word-level confidence scores, the model obtains them by calculating the Softmax posterior probability of the corresponding word in the output layer. The Softmax function transforms the model's output into a probability distribution; the posterior probability of each word reflects the model's confidence level in recognizing that word, with higher probability values ​​indicating higher confidence. Word-level timestamp information is determined by analyzing the alignment between the model's attention weight matrix and the input audio timeframes. The attention mechanism allows the model to focus on relevant parts of the audio stream when processing each word. By analyzing the attention weight matrix, the most relevant audio timeframes for each word can be found, thus determining the start and end times of that word in the audio stream and obtaining accurate timestamp information.

[0062] After the model outputs the transcribed text, the confidence score for each word, and its corresponding timestamp, this information needs to be associated and encapsulated. A data structure is established to bind each word in the transcribed text with its corresponding confidence score and timestamp, ensuring that each word carries complete metadata information. Then, the word-level confidence scores are judged based on a preset confidence threshold. This confidence threshold can be adjusted according to the actual application scenario and requirements. For example, in some meeting scenarios with extremely high accuracy requirements, the threshold can be set higher; while in some scenarios with high real-time requirements and a certain margin of error, the threshold can be relatively lower. By traversing each word in the transcribed text, its confidence score is compared with the preset confidence threshold. If the confidence score of a word is lower than the threshold, that word is determined to be a low-confidence word segment.

[0063] For identified low-confidence word segments, automatic marking is performed in the transcribed text. Special colors, font styles, or additional markers can be used for annotation, allowing users to quickly identify potentially erroneous word segments when viewing the transcribed text. This enables manual verification and correction, improving the overall quality and reliability of the transcribed text.

[0064] The final transcribed text, along with its associated word-level confidence score and word-level timestamp information, will be output together.

[0065] In some embodiments, step S102 above, which involves segmenting the transcribed text into sentences and adding a unique sequence number to each sentence to form a primary text, specifically includes: Based on the transcribed text, the duration of the silent pause between adjacent words is calculated. The pause duration is compared with a threshold that is dynamically adjusted according to the real-time speech rate of the current speaker to locate potential sentence boundaries and obtain the first sentence boundary determination result. The transcribed text is input into a pre-trained language model. By calculating the perceptual probability distribution of the insertion of delimiters at different positions in the language model, the low points of the probability distribution are identified as candidate positions of semantic boundaries, and the boundary determination results of the second sentence are obtained. Identify low-confidence word clusters in the transcribed text, adjust decision weights within these regions, prioritize boundary determination based on acoustic features, and obtain the boundary determination results for the third sentence. Based on the first sentence boundary determination results, the second sentence boundary determination results, and the third sentence boundary determination results, sentence boundary determination and segmentation are performed on the transcribed text to obtain a set of sentence units. Generate a unique sequence number associated with a start timestamp for each sentence unit in the sentence unit set; The text content, unique sequence number, start timestamp, and average confidence level of each sentence unit are linked and integrated to form a primary text.

[0066] In this embodiment, for each adjacent word in the transcribed text, its corresponding audio time point is found from the audio time information, and then the duration of the silent pause between adjacent words is calculated. Since different speakers have different speaking speeds, the real-time speaking speed will affect the normal pause duration between sentences. Therefore, it is necessary to dynamically adjust the threshold used to determine sentence boundaries based on the current speaker's real-time speaking speed. The calculated duration of the silent pause between adjacent words is compared with the dynamically adjusted threshold. If the duration of the silent pause between adjacent words exceeds the threshold, then this position is initially determined as a potential sentence boundary. Combining the determinations of all adjacent words, the final first sentence boundary determination result is obtained.

[0067] The transcribed text is input into a pre-trained language model. This pre-trained language model is trained on a large amount of text data and possesses a deep understanding of the semantic and syntactic structure of the text. During the processing of the transcribed text by the language model, the perceptual probability distribution for inserting delimiters at different positions is calculated. The perceptual probability reflects the semantic and syntactic rationality of the entire text after inserting a delimiter at that position. A higher probability indicates that the text is more fluent and logical after inserting a delimiter at that position; a lower probability indicates that this may be a semantic or syntactic turning point. The troughs in the calculated probability distribution are identified as candidate semantic boundary positions. These troughs mean that inserting a delimiter at these positions would significantly affect the coherence of the text and are likely sentence boundaries. The second sentence boundary determination result is obtained by combining the determinations of all troughs.

[0068] A comprehensive scan of the transcribed text was performed to identify clusters of low-confidence words. Low-confidence words are those for which the model's recognition results were previously considered less reliable; their clustering may affect the accurate determination of sentence boundaries. After identifying these low-confidence word clusters, the decision weights for sentence boundary determination were adjusted. Since the accuracy of speech transcription may be affected within these regions, and semantic and grammatical coherence is also difficult to guarantee, boundary determination is prioritized based on acoustic features. For example, acoustic features such as the duration of silent pauses between adjacent words and pitch variations within the region were re-analyzed, and these features were combined to determine sentence boundaries. The determination results within the low-confidence word clusters were then combined to obtain a third sentence boundary determination result.

[0069] Based on the obtained first, second, and third sentence boundary determination results, a comprehensive decision-making method is adopted to determine and segment sentence boundaries in the transcribed text. For each possible sentence boundary location, the support levels of the three determination results are comprehensively considered. For example, if a location is identified as a potential boundary in the first sentence boundary determination result, is also identified as a semantic boundary candidate location in the second sentence boundary determination result, and is also supported for segmentation in the third sentence boundary determination result, then this location is highly determined to be a sentence boundary. Conversely, if the support levels of the three determination results for this location are inconsistent, a trade-off needs to be made according to certain rules, such as prioritizing the first sentence boundary determination result based on acoustic features and the semantic language model determination result. Through this comprehensive determination method, the transcribed text is segmented into multiple sentence units, forming a set of sentence units.

[0070] Generate a unique sequence number for each sentence unit in the sentence unit set. This sequence number should be associated with the sentence unit's start timestamp, which can be obtained from the time the corresponding sentence unit begins to be pronounced in the conference audio stream.

[0071] The average confidence score for each sentence unit can be calculated by averaging the confidence scores of all words within that unit. The text content, unique sequence number, start timestamp, and average confidence score of each sentence unit are then correlated and integrated. This information can be stored in a specific data structure, such as a table with multiple fields, each corresponding to the text content, unique sequence number, start timestamp, and average confidence score, respectively. In this way, the relevant information from all sentence units is integrated to form a preliminary text, providing a foundation for subsequent text processing and analysis.

[0072] In some embodiments, step S103 above, which involves constructing a causal graph by extracting events based on primary text and conference audio streams, marking decision points according to the causal graph, and obtaining the logical chain analysis results, specifically includes: Semantic analysis is performed on each sentence unit in the primary text to extract key entities and events, and to obtain the extracted event sequence. The extracted event sequences are input into a pre-trained causal reasoning model, and implicit causal relationships are inferred by calculating the causal probability between event pairs. By using a pre-built causal phrase pattern library, we can scan and match sentence units in primary texts to establish explicit causal relationships. Construct a causal graph based on implicit and explicit causal relationships; By analyzing the node attributes of events in the causal graph and combining them with the acoustic features in the conference audio stream, key decision points in the conference are identified and marked, and logical chain analysis results are obtained.

[0073] In this embodiment, semantic parsing is performed on each sentence unit in the primary text. Semantic parsing aims to gain a deeper understanding of the meaning of sentence units, breaking them down into more easily processed semantic components. During this process, natural language processing techniques are used to identify the semantic roles of each word in the sentence within a specific context, such as subject, predicate, and object. Based on the results of semantic parsing, key entities and events are extracted. Key entities can be concrete or abstract objects involved in the meeting, such as people, organizations, projects, or products; events are summaries of the actions or states described in the sentences, such as "discussing solutions," "passing resolutions," and "making suggestions." After performing the above operations on all sentence units in the primary text, the extracted key events are arranged into an extracted event sequence according to the chronological order of the sentence units in the text. This sequence reflects the event development process of the meeting content over time, providing basic data for subsequent causal inference.

[0074] The extracted event sequences are input into a pre-trained causal inference model. This model, trained on a large amount of textual data containing causal relationships, is capable of identifying potential causal connections between events. When processing the extracted event sequences, the model analyzes the feature information of each event pair, including the semantic content of the events and their positional relationship within the sequence. By calculating the causal probability between event pairs, the model can infer implicit causal relationships that are not explicitly stated in the text but can be inferred through semantics and logic. For example, in a meeting where "propose a new solution" precedes "discuss the new solution," the model can infer, based on this common meeting flow and semantic logic, that "propose a new solution" is the implicit cause of "discuss the new solution."

[0075] A causal phrase pattern library is pre-built, containing various common phrases and sentence structures expressing causal relationships, such as "because...therefore...", "due to...leading to...", "this makes...", etc. This causal phrase pattern library is then used to perform scanning matching between sentence units in the initial text. When a matching phrase or sentence structure is found between sentence units, an explicit causal relationship is determined between the events described by those two sentence units. For example, if the initial text contains the sentence "Because time was tight, we sped up the process," scanning matching can clearly establish an explicit causal relationship between the events "time was tight" and "specified the process."

[0076] Based on the previously inferred implicit causal relationships and established explicit causal relationships, a causal graph is constructed. A causal graph is a graphical structure used to represent causal relationships between events, where nodes represent events and edges represent causal relationships between events. Each event in the extracted event sequence is used as a node in the graph, and different types of edges are used to represent implicit and explicit causal relationships, respectively. For example, solid edges can represent explicit causal relationships, and dashed edges can represent implicit causal relationships. In this way, all events and their causal relationships are integrated to form a complete causal graph, visually demonstrating the causal logic between various events in the meeting.

[0077] Analyzing the node attributes of events in a causal graph allows us to assess factors such as the event's importance, scope of influence, and correlation with other events. Importance can be evaluated through factors like the event's frequency in meeting discussions and the key figures involved; scope of influence can be determined by the event's impact on subsequent events and the overall meeting outcome; and correlation is measured by the number and strength of causal connections between the event and other events. Furthermore, we can incorporate acoustic features from the meeting's audio stream, such as rising intonation, increased speaking speed, and prolonged pauses. These acoustic features often reflect the speaker's emotional changes and decision-making moments during the discussion. For example, a significantly higher intonation and faster speaking speed when discussing a particular event may indicate that the event is a crucial decision point.

[0078] By combining the node attributes of events in the causal graph with the acoustic features of the conference audio stream, key decision points in the conference are identified and marked. Key decision points are typically event nodes that occupy important positions in the causal graph and exhibit significant changes in acoustic features within the conference audio. Marking these key decision points yields logical chain analysis results. These results help users clearly understand the decision-making process and the causal logic between events in the conference, providing strong support for subsequent decision review and summarization.

[0079] In some embodiments, in step S104 above, obtaining emotional voiceprint analysis results by combining acoustic features and text analysis of the speaker's emotional fluctuations based on primary text and conference audio streams, specifically includes: Speakerprint separation is performed on the conference audio stream to obtain speech segment data, and advanced acoustic prosodic features are extracted from the speech segment data. Based on advanced acoustic prosodic features, corresponding text sentiment features are extracted from primary text according to timestamps and sequence numbers; Advanced acoustic prosodic features and textual sentiment features are jointly input into an attention-based multimodal fusion network for collaborative analysis and weighted decision-making; Based on the output of the multimodal fusion network, the speaker's emotional state at different points in time is determined, and an emotional fluctuation curve is generated. Emotional turning points and high-intensity points are marked on the emotional fluctuation curve, and these points are associated with the corresponding timestamps and sequence numbers in the primary text to obtain emotional voiceprint analysis results.

[0080] In this embodiment, since the conference audio may contain the voices of multiple speakers and interference factors such as background noise, voiceprint separation technology can utilize the differences in the spectral and temporal characteristics of different sound sources to separate the speech of each speaker from the complex audio stream, thereby obtaining independent speech segment data. Advanced acoustic prosodic features are extracted from the separated speech segment data. Advanced acoustic prosodic features encompass multiple dimensions, including pitch, intensity, duration, and their dynamic changes. Pitch reflects the highness or lowness of the sound; pitch changes significantly under different emotional states, such as rising pitch when angry. Intensity indicates the strength of the sound; intensity is often greater when excited. Duration is related to the length of the pronunciation; duration may be prolonged when hesitant. Simultaneously, the dynamic changes of these features are considered, such as the amplitude of pitch fluctuations and the rate of change of intensity. These features can more comprehensively reflect the speaker's emotional state.

[0081] Based on the extracted advanced acoustic prosodic features, corresponding timestamps and sequence numbers are used to accurately extract the corresponding text sentiment features from the primary text. Each sentence unit in the primary text has a unique timestamp and sequence number, which allows for precise mapping between speech and text. When extracting text sentiment features, natural language processing techniques are used to perform in-depth analysis of the corresponding sentence units. By analyzing the sentiment tendencies of words, such as words with clear emotional connotations like "happy," "angry," and "sad," as well as the overall grammatical structure and semantic logic of the sentence, the emotion expressed by the sentence is determined. Simultaneously, the influence of context on emotion is considered, as the same word may express different emotions in different contexts.

[0082] The extracted high-level acoustic prosodic features and textual sentiment features are jointly input into a multimodal fusion network based on an attention mechanism. The attention mechanism enables the network to automatically focus on features that are more important for emotion judgment. The importance of features in different modalities in emotion expression may vary, and the attention mechanism can assign different weights to features based on the correlation between features and emotions.

[0083] In multimodal fusion networks, high-level acoustic prosodic features and text sentiment features are analyzed collaboratively. The network comprehensively considers information from both types of features to uncover potential correlations between them. For example, when high-level acoustic prosodic features show increased pitch and intensity, and text sentiment features contain words expressing anger, the network combines these two pieces of information to more accurately determine the speaker's emotional state. Through weighted decision-making, the network comprehensively evaluates different features to arrive at more reliable sentiment analysis results.

[0084] Based on the output of the multimodal fusion network, the speaker's emotional state at different points in time is determined. The network output can be a probability distribution of emotion categories, such as the probability values ​​of emotions like anger, happiness, calmness, and sadness. By selecting the emotion category with the highest probability, the speaker's emotional state at that particular point in time is determined.

[0085] The emotional states at each point in time are recorded and organized chronologically to generate an emotion fluctuation curve. The curve, with time on the horizontal axis and emotional state on the vertical axis, visually illustrates the speaker's emotional changes throughout the meeting. For example, the curve might show a speaker becoming increasingly agitated while discussing an important topic, and then gradually calming down after hearing suggestions from others.

[0086] The generated emotional fluctuation curve identifies emotional turning points and high-intensity points. An emotional turning point is a point where an emotional state undergoes a significant change, such as a sudden shift from happiness to anger, or from calm to excitement. High-intensity points are points where the emotional state reaches a high level, such as extreme anger or extreme happiness.

[0087] The identified emotional turning points and high-intensity points are correlated with corresponding timestamps and sequence numbers in the primary text. Since sentence units in the primary text have clear time markers, mapping them to key points on the emotional fluctuation curve allows for the accurate identification of specific content in the meeting text that triggers emotional changes. For example, if an emotional turning point is found to correspond to a sentence unit in the primary text concerning a proposed change, its impact on the speaker's emotions can be further analyzed. Through this correlation, the final emotional voiceprint analysis results are obtained, providing strong evidence for a deeper understanding of the meeting process and the speaker's psychological state.

[0088] In some embodiments, in step S105 above, the step of generating an initial meeting minutes summary with original sentence sequence numbers as source tracing labels using a large language model guided by a specific Prompt, based on the results of logical chain analysis and sentiment voiceprint analysis, specifically includes: By integrating the results of logical chain analysis, sentiment voiceprint analysis, and primary text, an enhanced semantic network containing semantic tags and source mapping relationships is constructed. A Prompt instruction set is constructed based on the enhanced semantic network. The Prompt instruction set includes role definition, task definition, instructions for injecting the enhanced semantic network as the only factual background, a rigid requirement that the generated content must be accompanied by the original text sequence number as a source label, and guidance on emotional integration. Submit the Prompt instruction set to a large language model to generate an initial meeting minutes summary.

[0089] In this embodiment, the results of logical chain analysis, sentiment voiceprint analysis, and primary text are comprehensively integrated. Logical chain analysis reveals the causal relationships and decision-making processes between events in the meeting, sentiment voiceprint analysis reflects the speaker's emotional state at different points in time, and the primary text is the original record of the meeting content. During the integration process, semantic tags are added to each sentence unit in the primary text. Semantic tags are categorized according to the sentence's content and function, such as "problem statement," "solution discussion," and "decision making," which help to accurately understand the semantic role of the sentence. Simultaneously, a source mapping relationship is established, associating key information from the logical chain analysis and sentiment voiceprint analysis results with the corresponding sentence units in the primary text. For example, if the logical chain analysis shows that a decision point is triggered by a specific event, a mapping is established between that decision point and the sentence unit containing the original event that triggered it; if the sentiment voiceprint analysis indicates that a speaker was emotionally agitated at a certain point in time, this emotional information is mapped to the sentence unit at the corresponding time point. In this way, an enhanced semantic network containing semantic tags and source mapping relationships is constructed, which comprehensively and systematically integrates various key information from the meeting.

[0090] Based on the constructed enhanced semantic network, a Prompt instruction set was carefully built. The Prompt instruction set is a key set of instructions that guides large language models to generate content that meets the requirements, and it contains several important components.

[0091] The role definition section clearly defines the role of the large language model in this task, for example, defining it as a "professional meeting minutes writing assistant," so that the model understands its responsibilities and task scope. The task definition details the specific tasks that the model needs to complete, namely, generating an initial meeting minutes summary.

[0092] The instruction to inject the enhanced semantic network as the sole factual background is one of the core components. This instruction requires the model to rely solely on the information in the enhanced semantic network when generating summaries, ensuring that the generated content accurately reflects the true situation of the meeting and avoiding the introduction of irrelevant external information or misunderstandings.

[0093] The rigid requirement that the generated content must be accompanied by the original text sequence number as a traceability label means that, through explicit instructions, the model is forced to annotate the corresponding original text sentence sequence number after each generated summary sentence, so as to trace and verify the summary content in the future and ensure the accuracy and reliability of the information.

[0094] The emotion incorporation guideline requires the model to fully consider the emotional information in the voiceprint analysis results when generating summaries. For example, if a speaker is emotionally agitated during a meeting, the model should appropriately reflect this emotional atmosphere when writing the summary, making the summary more vivid and realistically reflect the meeting scene.

[0095] The constructed Prompt instruction set is submitted completely and accurately to a large-scale language model. Upon receiving the instruction set, the large-scale language model analyzes and processes the enhanced semantic network according to the requirements of the instructions. First, the model parses the semantic tags in the enhanced semantic network to understand the core content and function of each sentence unit. Then, based on the source mapping relationship, it obtains the logical chain information and sentiment voiceprint information related to each sentence unit. During the summary generation process, the model strictly follows the task definition, selecting key information from the meeting, such as important decisions, key issues, and main viewpoints, and organizing and integrating them systematically. Simultaneously, the model adheres to rigid requirements, accurately labeling each generated summary sentence with the corresponding original sentence sequence number, achieving content traceability. Furthermore, based on sentiment integration guidelines, emotional information from the sentiment voiceprint analysis results is reasonably integrated into the summary, ensuring that the summary not only contains objective factual information but also reflects the emotional atmosphere of the meeting. Finally, the large-scale language model generates an initial meeting minutes summary with original sentence sequence numbers as source indices, providing strong support for subsequent meeting record organization and analysis.

[0096] In some embodiments, step S106 above, which involves processing the initial meeting minutes summary based on user role configuration information to generate a target meeting minutes summary with adjustable granularity that matches the user role, specifically includes: Call the pre-built role-content mapping rule library, which is used to define the information focus and detail granularity corresponding to different roles; Based on the user role configuration information, the corresponding content mapping rules are matched from the role-content mapping rule library; Based on the matched content mapping rules, the initial meeting minutes summary is filtered and its granularity is adjusted to reconstruct and generate a target meeting minutes summary that matches the user's role.

[0097] In this embodiment, the information focus and granularity of detail corresponding to different roles are defined. The information focus clarifies the scope of content that each role is most concerned with in the meeting minutes, while the granularity of detail specifies the level of detail required for different information focuses. For example, for the meeting organizer, the information focus is set to the overall meeting situation, and the granularity of detail is coarse-grained, requiring only a general description of the main process and key decisions of the meeting; while for the project executor, the information focus is the details of the project tasks, and the granularity of detail is fine-grained, requiring detailed recording of the specific requirements, schedule, responsible persons, and other information of the tasks. The defined information focus and granularity of detail corresponding to different roles are organized and summarized to construct a pre-built role-content mapping rule base. This rule base stores the mapping relationship between each role and the corresponding information focus and granularity of detail in a structured manner for quick subsequent querying and use.

[0098] Once the user role configuration information is obtained, the system will use this information as a query condition to perform an exact match in a pre-built role-content mapping rule base. The user role configuration information explicitly specifies the role type of the current user, and the system uses this information to search for the corresponding record in the rule base.

[0099] During the matching process, the system compares the user role configuration information with the role names recorded in the rule base one by one to ensure that a perfect match is found. Once a matching record is found, the system extracts the content mapping rules defined in that record. These rules contain key information such as the information focus that the corresponding role should pay attention to and the granularity of the required content. For example, if the user role configuration information shows that the current user is a department head, the system will find the record corresponding to the department head in the rule base and extract the content mapping rules for the information focus (such as department-related work, resource allocation, etc.) and granularity (such as medium granularity, which requires a brief description of key decisions and task arrangements involving the department) that the role should pay attention to.

[0100] Based on the matched content mapping rules, the system begins to perform content filtering and granularity adjustment operations on the initial meeting minutes summary.

[0101] In terms of content filtering, the system selects content related to the information focus from the initial meeting minutes summary. For example, if the information focus in the matched content mapping rule is project task details, the system will traverse the initial meeting minutes summary, extract all sentences and paragraphs related to the project task, and remove information unrelated to the project task, such as casual conversation during the meeting or discussions of other topics unrelated to the current project.

[0102] Regarding granularity adjustment, the system further processes the filtered content according to the required level of detail. If the granularity is coarse, the system will summarize and generalize the filtered content, removing unnecessary details and retaining only key information. For example, for project task scheduling, coarse granularity might only mention the start and end times of the task without detailing the specific time nodes for each stage; if the granularity is fine, the system will supplement more detailed information, making the content more comprehensive and complete. For example, regarding project task details, fine granularity will record the specific requirements, responsible person, and estimated completion time for each subtask.

[0103] After content filtering and granularity adjustment, the system reconstructs and generates target meeting minutes summaries that match the user's role. These summaries not only meet the information needs of the user's role but also satisfy their work requirements in terms of detail, providing users with more targeted and practical meeting information references.

[0104] Reference Figure 2 An embodiment of the present invention provides an intelligent meeting recording processing system 2, the system 2 specifically comprising: The first processing module 201 is used to collect the conference audio stream, perform speech-to-text transcription on the conference audio stream, and simultaneously output word-level confidence scores and word-level timestamps to obtain transcribed text. The second processing module 202 is used to segment the transcribed text into sentences and add a unique sequence number to each sentence to form a primary text; The third processing module 203 is used to construct a causal graph by extracting events based on the primary text and conference audio stream, mark decision points according to the causal graph, and obtain logical chain analysis results. The fourth processing module 204 is used to obtain emotional voiceprint analysis results by combining acoustic features with text analysis of the speaker's emotional fluctuations based on the primary text and conference audio stream. The fifth processing module 205 is used to generate an initial meeting minutes summary with original sentence sequence numbers as source indices based on the results of logical chain analysis and sentiment voiceprint analysis, using a large language model guided by a specific prompt. The sixth processing module 206 is used to receive user role configuration information, process the initial meeting minutes summary according to the user role configuration information, and generate a target meeting minutes summary with adjustable granularity that matches the user role.

[0105] It is understandable that, such as Figure 1 The content of the intelligent meeting recording processing method embodiments shown is applicable to the intelligent meeting recording processing system embodiments. The specific functions implemented by the intelligent meeting recording processing system embodiments are the same as those shown in the examples. Figure 1The intelligent meeting recording processing method shown in the embodiment is the same, and the beneficial effects achieved are the same as those described above. Figure 1 The beneficial effects achieved by the intelligent meeting recording processing method embodiment shown are also the same.

[0106] It should be noted that the information interaction and execution process between the above systems are based on the same concept as the method embodiments of the present invention. For details on their specific functions and technical effects, please refer to the method embodiments section, which will not be repeated here.

[0107] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the system can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0108] Reference Figure 3 The present invention also provides a computer device 3, including: a memory 302 and a processor 301, and a computer program 303 stored in the memory 302. When the computer program 303 is executed on the processor 301, it implements the intelligent meeting recording processing method as described in any of the above methods.

[0109] The computer device 3 may be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device 3 may include, but is not limited to, a processor 301 and a memory 302. Those skilled in the art will understand that... Figure 3 The computer device 3 is merely an example and does not constitute a limitation on the computer device 3. It may include more or fewer components than shown in the figure, or combine certain components, or different components, such as input / output devices, network access devices, etc.

[0110] The processor 301 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0111] In some embodiments, the memory 302 may be an internal storage unit of the computer device 3, such as a hard disk or memory of the computer device 3. In other embodiments, the memory 302 may be an external storage device of the computer device 3, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 3. Furthermore, the memory 302 may include both internal and external storage units of the computer device 3. The memory 302 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 302 can also be used to temporarily store data that has been output or will be output.

[0112] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the intelligent meeting recording processing method as described in any of the above methods.

[0113] In this embodiment, if the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a photographing device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0114] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0115] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0116] In the embodiments disclosed in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0117] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

Claims

1. A method for intelligent meeting recording processing, characterized in that, The method specifically includes: Collect the conference audio stream, perform speech-to-text transcription on the conference audio stream, and simultaneously output word-level confidence scores and word-level timestamps to obtain the transcribed text; The transcribed text is segmented into sentences, and a unique sequence number is added to each sentence to form the initial text; Based on primary text and conference audio streams, a causal graph is constructed by extracting events, and decision points are marked according to the causal graph to obtain logical chain analysis results; Based on primary text and conference audio streams, and combining acoustic features with text analysis of the speaker's emotional fluctuations, emotional voiceprint analysis results are obtained. Based on the results of logical chain analysis and sentiment voiceprint analysis, a large language model guided by a specific prompt is used to generate an initial meeting minutes summary with original sentence sequence numbers as source tracing subscripts. Receive user role configuration information, process the initial meeting minutes summary according to the user role configuration information, and generate a target meeting minutes summary with adjustable granularity that matches the user role.

2. The method according to claim 1, characterized in that, The process of transcribing the conference audio stream, simultaneously outputting word-level confidence scores and word-level timestamps to obtain the transcribed text, specifically includes: The conference audio stream is fed into a pre-built end-to-end deep learning speech recognition model; The speech transcription and metadata generation operations are performed synchronously based on the end-to-end deep learning speech recognition model, and the transcribed text, the confidence score of each word and its corresponding timestamp information are output. The word-level confidence score is obtained by calculating the Softmax posterior probability of the corresponding word in the output layer of the end-to-end deep learning speech recognition model, and the word-level timestamp information is obtained by analyzing the alignment relationship between the attention weight matrix of the end-to-end deep learning speech recognition model and the input audio time frame. The transcribed text is associated and encapsulated with the corresponding word-level confidence score and word-level timestamp. The word-level confidence score is judged according to the preset confidence threshold, and low-confidence word segments below the confidence threshold are automatically marked in the transcribed text.

3. The method according to claim 1, characterized in that, The process of segmenting the transcribed text into sentences and adding a unique sequence number to each sentence to form a primary text specifically includes: Based on the transcribed text, the duration of the silent pause between adjacent words is calculated. The pause duration is compared with a threshold that is dynamically adjusted according to the real-time speech rate of the current speaker to locate potential sentence boundaries and obtain the first sentence boundary determination result. The transcribed text is input into a pre-trained language model. By calculating the perceptual probability distribution of the insertion of delimiters at different positions in the language model, the low points of the probability distribution are identified as candidate positions of semantic boundaries, and the boundary determination results of the second sentence are obtained. Identify low-confidence word clusters in the transcribed text, adjust decision weights within these regions, prioritize boundary determination based on acoustic features, and obtain the boundary determination results for the third sentence. Based on the first sentence boundary determination results, the second sentence boundary determination results, and the third sentence boundary determination results, sentence boundary determination and segmentation are performed on the transcribed text to obtain a set of sentence units. Generate a unique sequence number associated with a start timestamp for each sentence unit in the sentence unit set; The text content, unique sequence number, start timestamp, and average confidence level of each sentence unit are linked and integrated to form a primary text.

4. The method according to claim 3, characterized in that, The process, based on primary text and conference audio streams, involves extracting events to construct a causal graph, marking decision points according to the causal graph, and obtaining logical chain analysis results. Specifically, this includes: Semantic analysis is performed on each sentence unit in the primary text to extract key entities and events, and to obtain the extracted event sequence. The extracted event sequences are input into a pre-trained causal reasoning model, and implicit causal relationships are inferred by calculating the causal probability between event pairs. By using a pre-built causal phrase pattern library, we can scan and match sentence units in primary texts to establish explicit causal relationships. Construct a causal graph based on implicit and explicit causal relationships; By analyzing the node attributes of events in the causal graph and combining them with the acoustic features in the conference audio stream, key decision points in the conference are identified and marked, and logical chain analysis results are obtained.

5. The method according to claim 1, characterized in that, The method of obtaining emotional voiceprint analysis results based on primary text and conference audio streams, combined with acoustic features and text analysis of the speaker's emotional fluctuations, specifically includes: Speakerprint separation is performed on the conference audio stream to obtain speech segment data, and advanced acoustic prosodic features are extracted from the speech segment data. Based on advanced acoustic prosodic features, corresponding text sentiment features are extracted from primary text according to timestamps and sequence numbers; Advanced acoustic prosodic features and textual sentiment features are jointly input into an attention-based multimodal fusion network for collaborative analysis and weighted decision-making; Based on the output of the multimodal fusion network, the speaker's emotional state at different points in time is determined, and an emotional fluctuation curve is generated. Emotional turning points and high-intensity points are marked on the emotional fluctuation curve, and these points are associated with the corresponding timestamps and sequence numbers in the primary text to obtain emotional voiceprint analysis results.

6. The method according to claim 1, characterized in that, Based on the results of logical chain analysis and sentiment voiceprint analysis, a large-scale language model guided by a specific prompt is used to generate an initial meeting minutes summary with original sentence sequence numbers as source tracing subscripts, specifically including: By integrating the results of logical chain analysis, sentiment voiceprint analysis, and primary text, an enhanced semantic network containing semantic tags and source mapping relationships is constructed. A Prompt instruction set is constructed based on the enhanced semantic network. The Prompt instruction set includes role definition, task definition, instructions for injecting the enhanced semantic network as the only factual background, a rigid requirement that the generated content must be accompanied by the original text sequence number as a source label, and guidance on emotional integration. Submit the Prompt instruction set to a large language model to generate an initial meeting minutes summary.

7. The method according to claim 1, characterized in that, The step of processing the initial meeting minutes summary based on user role configuration information to generate a target meeting minutes summary with adjustable granularity that matches the user role specifically includes: Call the pre-built role-content mapping rule library, which is used to define the information focus and detail granularity corresponding to different roles; Based on the user role configuration information, the corresponding content mapping rules are matched from the role-content mapping rule library; Based on the matched content mapping rules, the initial meeting minutes summary is filtered and its granularity is adjusted to reconstruct and generate a target meeting minutes summary that matches the user's role.

8. An intelligent meeting recording processing system, characterized in that, The system specifically includes: The first processing module is used to acquire the conference audio stream, perform speech-to-text transcription on the conference audio stream, and simultaneously output word-level confidence scores and word-level timestamps to obtain the transcribed text. The second processing module is used to segment the transcribed text into sentences and add a unique sequence number to each sentence to form the initial text; The third processing module is used to construct a causal graph by extracting events based on the primary text and conference audio stream, mark decision points according to the causal graph, and obtain the logical chain analysis results. The fourth processing module is used to obtain emotional voiceprint analysis results by combining acoustic features with text analysis of the speaker's emotional fluctuations based on the primary text and conference audio stream. The fifth processing module is used to generate an initial meeting minutes summary with original sentence sequence numbers as source indices based on the results of logical chain analysis and sentiment voiceprint analysis, using a large language model guided by a specific prompt. The sixth processing module is used to receive user role configuration information, process the initial meeting minutes summary according to the user role configuration information, and generate a target meeting minutes summary with adjustable granularity that matches the user role.

9. A computer device, characterized in that, include: The memory and processor, and the computer program stored in the memory, which, when executed on the processor, implement the intelligent meeting recording processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, It stores a computer program, which, when executed by a processor, implements the intelligent meeting recording processing method as described in any one of claims 1 to 7.

Images