Method and system for automatically generating conference summary based on voice recognition of AI (Artificial Intelligence)

Abstract

The invention relates to the technical field of voice summary automatic generation, in particular to a conference summary automatic generation method and system based on AI voice recognition, and the method comprises the steps: processing a conference audio to obtain a multi-channel voice stream, and carrying out the recognition and transliteration of a multi-speaker segment after enhancement; after the same speaker fragments which are adjacent in time are combined into logic units, subject clustering and intention recognition are carried out on the logic units to construct a semantic structure tree, and therefore the content hierarchy and the logic relation are analyzed; and according to the depth and association degree of the nodes in the semantic structure tree, adjusting the initial score based on the transcription quality to form a semantic perception quality score so as to accurately evaluate the content importance. And extracting a text from the semantic structure tree according to the key hierarchy, and outputting a summary document after recombination and retouching. According to the method, deep semantic understanding is integrated, so that the logical continuity of the summary and the accuracy of core information extraction are improved.

Description

AI-based speech recognition method and system for automatic generation of meeting minutes Technical Field

[0001] This invention relates to the field of automatic voice minutes generation technology, and in particular to a method and system for automatic generation of meeting minutes based on AI-based speech recognition. Background Technology

[0002] Currently, automatic meeting minutes technology based on speech recognition mainly employs a process combining speech transcription and text summarization. Existing methods perform noise reduction and speaker separation on the meeting audio, convert it into text, and then compress the content through keyword extraction or statistical summarization models. While this type of technology can achieve basic transcription and content simplification, it essentially separates audio processing from semantic understanding, failing to analyze the inherent logical hierarchy of the meeting content. The generated results are mostly linear lists of sentences or summaries based on surface features, unable to reflect the deeper structure of the discussion, such as the evolution of topics and supporting arguments.

[0003] The shortcomings of existing solutions lie in their reliance on superficial indicators such as transcription confidence, word frequency, or sentence length for selecting important content, failing to assess the semantic weight of content within the overall discussion. This leads to the potential for the incorrect retention of high-quality but minor details, while core arguments are often overlooked due to unclear transcription. Furthermore, existing technologies typically limit text clustering to topic division, struggling to automatically identify speaker intent and integrate it into a logically coherent document format. Consequently, the generated minutes suffer from deficiencies in logical coherence and emphasis on key points.

[0004] There is a need for a method that can deeply understand the semantic structure and logical relationships of meeting content, and select and organize content based on semantic importance rather than surface features, in order to generate logically clear and concise meeting minutes. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing an AI-based method and system for automatically generating meeting minutes using speech recognition.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: an AI-based method for automatically generating meeting minutes based on speech recognition, comprising: receiving and processing meeting audio signals containing multiple speakers to form a multi-channel speech data stream; extracting speech features and suppressing noise in the multi-channel speech data stream to generate enhanced speech data; identifying multi-speaker segments in the enhanced speech data, assigning a unique identifier to each speaker segment and recording its start and end times; transcribing the content of the multi-speaker segments and calculating an initial quality score associated with each multi-speaker segment; merging multiple multi-speaker segments that are temporally adjacent and have the same speaker identifier to form a logical discourse unit; performing topic clustering and intent recognition on the content of the logical discourse unit to generate a semantic structure tree; adjusting the initial quality score based on the depth and relevance of each node in the semantic structure tree to form a semantic perception quality score; determining the key level of the meeting content based on the distribution of the semantic perception quality score; extracting the text content corresponding to the nodes that meet the key level requirements from the semantic structure tree; integrating the text content and reorganizing and polishing it according to a preset minutes format to output the final meeting minutes document.

[0007] Preferably, the step of forming a multi-channel speech data stream specifically includes: acquiring raw audio signals from different physical locations or different pickup devices; performing pre-emphasis, framing, and windowing processing on each raw audio signal to obtain a corresponding audio frame sequence; extracting spectral envelope, fundamental frequency trajectory, and formant features from the audio frame sequence to form a multi-dimensional feature vector; aligning and fusing all the multi-dimensional feature vectors from the same time window to generate the multi-channel speech data stream, wherein each channel corresponds to an independent speech source.

[0008] Preferably, the step of generating enhanced speech data specifically includes: estimating background noise for each channel of the multi-channel speech data stream to obtain a noise spectrum; separating steady-state noise components and non-steady-state noise components from the multi-channel speech data stream; suppressing the steady-state noise components using spectral subtraction based on speech activity detection; constructing an adaptive filter bank to track and cancel the non-steady-state noise components; performing beamforming on the noise-suppressed speech data from each channel to enhance the speech energy in the direction of the main speaker, and outputting the enhanced speech data.

[0009] Preferably, the step of assigning a unique identifier to each speaker segment specifically includes: extracting acoustic fingerprint features from the enhanced speech data of the multi-speaker segments, wherein the acoustic fingerprint features include the dynamic trajectory of Mel-frequency cepstral coefficients and timbre feature vectors; based on the acoustic fingerprint features, performing preliminary separation of different speakers using a clustering algorithm; extracting the intra-cluster acoustic feature centroids for each cluster as a reference voiceprint template for the speaker; calculating the similarity between the acoustic features of the current speaker segment and all existing reference voiceprint templates; if the highest similarity exceeds the confirmation threshold, assigning the current segment to the speaker represented by the corresponding reference voiceprint template and using the existing identifier of the reference voiceprint template; if it does not exceed the threshold, creating a new reference voiceprint template for the current segment and assigning a new, unused unique identifier.

[0010] Preferably, the step of calculating the initial quality score associated with each of the multi-speaker segments specifically includes: performing automatic speech recognition on the multi-speaker segments to generate corresponding candidate texts and multiple alternative text hypotheses; calculating the language model perplexity and acoustic model score of the candidate texts; evaluating the signal-to-noise ratio, harmonicity, and speech intelligibility index of the multi-speaker segments to obtain an acoustic clarity metric; detecting whether there are non-fluency phenomena within the multi-speaker segments, including repetitions, corrections, and excessively long pauses, and generating a fluency penalty factor; and combining the language model perplexity, the acoustic model score, the acoustic clarity metric, and the fluency penalty factor to obtain the initial quality score through weighted fusion calculation.

[0011] Preferably, the step of forming a logical discourse unit specifically includes: acquiring multiple consecutive multi-speaker segments with the same unique identifier; calculating the cosine similarity and semantic coherence score of adjacent multi-speaker segments in terms of text content; determining whether the cosine similarity and semantic coherence score are both higher than a merging threshold; if both are higher, merging these multi-speaker segments into a single logical discourse unit, the content of which is a sequential concatenation of the content of each segment, and the time span being from the start time of the first segment to the end time of the last segment; if not higher, the current segment is regarded as an independent logical discourse unit, and merging judgment continues with subsequent segments.

[0012] Preferably, the step of generating a semantic structure tree specifically includes: extracting a set of keywords and named entities from the text content of each logical discourse unit; matching the set of keywords with a predefined conference topic thesaurus to assign at least one topic tag to the logical discourse unit; analyzing the language behavior of the logical discourse unit and classifying it into one of the intent types: statement, question, suggestion, resolution, or to-do item; aggregating units with the same or similar topic tags into a topic cluster based on the temporal order of the logical discourse units and the relevance of the topic tags; constructing a tree structure with the overall conference theme as the root node and each topic cluster as a child node, wherein the text content of the child node is a summary of the content of all logical discourse units belonging to the topic cluster; and marking the attributes of the logical discourse unit within the child node according to the intent type in the tree structure.

[0013] Preferably, the step of forming a semantic-aware quality score specifically includes: obtaining the initial quality score corresponding to the logical discourse unit; locating the child node to which the logical discourse unit belongs in the semantic structure tree and determining the depth level of the child node in the tree; analyzing the textual association strength between the logical discourse unit and other units within its topic cluster, as well as with nodes in other topic clusters; positively adjusting the quality scores of units within nodes with shallower depth levels and positively adjusting the quality scores of units with higher association strength; and using the adjusted quality score as the semantic-aware quality score of the logical discourse unit.

[0014] Preferably, the step of determining the key level of the meeting content specifically includes: collecting the semantic perception quality scores of all the logical discourse units and calculating their distribution histograms; dividing the scores into three intervals—high, medium, and low—based on the distribution histograms; marking the logical discourse units falling into the high interval as core-level content, those falling into the medium interval as supporting-level content, and those falling into the low interval as reference-level content; and, in conjunction with the depth of the nodes in the semantic structure tree, elevating the content located as a direct child node of the root node and marked as supporting-level content to the core level.

[0015] Preferably, the present invention also includes an AI-based speech recognition automatic meeting minutes generation system, the system including a memory, a processor, and a computer program stored in the memory and running on the processor, wherein when the processor executes the computer program, it implements the steps of the AI-based speech recognition automatic meeting minutes generation method described above.

[0016] Compared with existing technologies, the advantages and positive effects of this invention are as follows: By performing topic clustering and intent recognition on logical discourse units and constructing a semantic tree structure accordingly, a deep analysis of the semantic context and hierarchical relationships of meeting discussions is achieved. This process organizes discrete speaking units into a tree-like network structure with clear parent-child or parallel relationships based on their topic affiliation and functional intent. This structured semantic representation enables machines to understand the logical connections between different speeches. The generated minutes naturally follow the logical order and hierarchy defined by this semantic structure in terms of content organization, thereby effectively overcoming the problems of loose content and scattered focus caused by traditional methods that simply list information chronologically or group it by topic, thus enhancing the document's organization and information transmission efficiency.

[0017] Based on the depth and relevance of nodes in the semantic tree structure, the initial score based on audio quality is weighted and adjusted to generate a semantic perception quality score that integrates semantic importance. The depth index quantifies the level of abstraction of content in the discussion, while the relevance index quantifies the strength of its semantic connection with the core topic cluster. This technical mechanism transforms the criteria for determining the criticality of content from traditionally relied-upon surface signals such as acoustic confidence and lexical features to a comprehensive evaluation of its position and role in the overall semantic network. This allows content located at the root or trunk nodes of the tree structure, representing core conclusions or decisions, to receive higher final scores and be retained even if its speech transcription is partially unclear; conversely, content located at the terminal branches and leaves, which constitutes detailed explanations or repetitive statements, will have its scores effectively suppressed, even if its transcription is very clear. In this way, the accuracy of content selection is fundamentally improved, and the final output minutes can more reliably focus on the core achievements and key arguments of the meeting, effectively filtering out non-substantive redundant information. Attached Figure Description

[0018] Figure 1 is a flowchart of the AI-based speech recognition automatic meeting minutes generation method described in this invention; Figure 2 is a flowchart of multi-channel speech data stream generation; Figure 3 is a flowchart of speaker identifier allocation; Figure 4 is a bar chart of semantic perception quality score of logical discourse units; Figure 5 is a graph of meeting flow and quality score changes. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0020] In the description of this invention, it should be understood that the terms "length," "width," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," and "outer," etc., indicating orientation or positional relationships, are based on the orientation or positional relationships shown in the accompanying drawings and are only for the convenience of describing the invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the invention. Furthermore, in the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0021] The AI-based speech recognition method for automatically generating meeting minutes provided by this invention, as shown in Figure 1, has the following overall implementation scheme: Receiving and processing meeting audio signals containing multiple speakers to form a multi-channel speech data stream; extracting speech features and suppressing noise in the multi-channel speech data stream to generate enhanced speech data; identifying multi-speaker segments in the enhanced speech data, assigning a unique identifier to each speaker segment and recording its start and end times; transcribing the content of the multi-speaker segments and calculating an initial quality score associated with each multi-speaker segment; merging multiple multi-speaker segments that are temporally adjacent and have the same speaker identifier to form logical discourse units; performing topic clustering and intent recognition on the content of the logical discourse units to generate a semantic structure tree; adjusting the initial quality score based on the depth and relevance of each node in the semantic structure tree to form a semantic-perceived quality score; determining the key level of the meeting content based on the distribution of the semantic-perceived quality score; extracting the text content corresponding to nodes that meet the key level requirements from the semantic structure tree; integrating the text content and reorganizing and polishing it according to a preset minutes format to output the final meeting minutes document.

[0022] In one embodiment of the present invention, referring to Figure 2, raw audio signals from different physical locations or different pickup devices are acquired. Each raw audio signal is pre-emphasized, framed, and windowed to obtain a corresponding audio frame sequence. Spectral envelope, fundamental frequency trajectory, and formant features are extracted from the audio frame sequence to form a multi-dimensional feature vector. All multi-dimensional feature vectors from the same time window are aligned and fused to generate the multi-channel speech data stream, where each channel corresponds to an independent speech source. Background noise is estimated for each channel of the multi-channel speech data stream to obtain a noise spectrum. Steady-state noise components and non-steady-state noise components are separated from the multi-channel speech data stream. The steady-state noise components are suppressed using spectral subtraction based on speech activity detection. An adaptive filter bank is constructed to track and cancel the non-steady-state noise components. Beamforming is performed on the noise-suppressed speech data from each channel to enhance the speech energy in the direction of the main speaker, and the enhanced speech data is output.

[0023] In the implementation, a conference room scenario with three omnidirectional microphones was used for illustration. The three microphones were located in the center, left, and right of the conference table, respectively. Each microphone continuously collected the raw audio signal from its physical location. During the meeting, the continuous low-frequency hum of the air conditioner and the occasional keyboard clicks of the participants were also recorded, forming a raw audio signal containing the target speech and mixed noise. In the implementation, the raw audio signal from the central microphone was pre-emphasized to enhance the high-frequency components. Hamming windowing was used to frame and window the pre-emphasized signal, resulting in the audio frame sequence corresponding to the central microphone. From each audio frame sequence, the spectral envelope containing linear predictive coding coefficients, the fundamental frequency trajectory calculated by the autocorrelation method, and the formant features of the first three formant frequencies and their bandwidths were extracted. These features together constitute a multidimensional feature vector characterizing the frame signal. In practice, the same pre-emphasis, framing, windowing, and feature extraction process is performed on the raw audio signals from the left and right microphones to obtain the left and right audio frame sequences and their corresponding multi-dimensional feature vector sets. Within a time window of 50 milliseconds, the three multi-dimensional feature vectors from the middle, left, and right sides are time-aligned, and a multi-channel speech data stream is generated through a feature-level fusion algorithm, where each channel independently corresponds to the speech source of a microphone.

[0024] In some embodiments, the process of generating enhanced speech data begins with estimating background noise for each channel of the multi-channel speech data stream. Silent segments are identified through speech activity detection, and the average spectrum within each silent segment is calculated as the noise spectrum for that channel. From the multi-channel speech data stream, a high-pass filter separates non-stationary noise components such as keyboard clicks, while air conditioner humming is identified as a stationary noise component. In a specific implementation, spectral subtraction based on speech activity detection is used to subtract the estimated stationary noise spectrum from the signal spectrum of each channel, thereby suppressing low-frequency air conditioner humming. For non-stationary noise components, an adaptive filter bank containing multiple filtering units is constructed. This filter bank tracks the spectral characteristics of keyboard clicks in real time and cancels them in the time domain. After the aforementioned noise suppression processing, the speech data from each channel is fed into a beamforming processor. The beamforming processor calculates a set of weights based on the sound source localization results and performs a weighted sum of the signals from the three channels to enhance the speech energy from the direction of the main speaker at the center of the conference table. The signal output by the beamformer is the enhanced speech data. The formula used to calculate the weight vector in a beamformer can be expressed as:

[0025] in, This represents the complex weight vector of the beamformer; This represents the estimated noise covariance matrix; This represents the estimated inverse noise covariance matrix; Indicates the direction corresponding to the target. Array steering vector; superscript This indicates the conjugate transpose operation.

[0026] It is understandable that during data processing, by comparing the spectrum of the central microphone channel before and after noise suppression, it can be observed that the continuous low-frequency band energy representing air conditioning noise is significantly reduced, and the sudden high-frequency spikes representing keyboard typing are also effectively smoothed. Numerically, beamforming processing increases the amplitude of the speech signal from the speaker's direction by approximately 8 dB compared to lateral interference speech. In some embodiments, short-time energy analysis of the enhanced speech data shows that the signal-to-noise ratio (SNR) of the active speech segment is significantly improved compared to the average SNR of the original multi-channel speech data stream, and the harmonic structure of the speech segment is more clearly represented in the spectrum. Optionally, after generating the enhanced speech data, an overall signal quality index is calculated, which integrates the degree of SNR improvement with the directional gain after beamforming, for use as a quality assessment reference in subsequent processes. Optionally, if the conference room is equipped with a circular microphone array, when generating the multi-channel speech data stream, the signals of each microphone unit in the array are processed simultaneously, and the beamforming algorithm can adopt a minimum variance distortion-free response algorithm to maintain the waveform of the target direction signal without distortion while suppressing noise.

[0027] In one embodiment of the present invention, referring to FIG3, acoustic fingerprint features are extracted from the enhanced speech data of the multi-speaker segments. The acoustic fingerprint features include the dynamic trajectory of Mel-frequency cepstral coefficients and timbre feature vector. Based on the acoustic fingerprint features, different speakers are initially separated by a clustering algorithm. For each cluster, the intra-cluster acoustic feature centroid is extracted as a reference voiceprint template for the speaker. The similarity between the acoustic features of the current speaker segment and all existing reference voiceprint templates is calculated. If the highest similarity exceeds the confirmation threshold, the current segment is assigned to the speaker represented by the corresponding reference voiceprint template, and the existing identifier of the reference voiceprint template is used. If it does not exceed the threshold, a new reference voiceprint template is created for the current segment and a new, unused unique identifier is assigned.

[0028] In practice, the conference audio contains multiple speakers taking turns speaking. The enhanced speech data is segmented into a series of multi-speaker segments based on silence detection. Each multi-speaker segment contains a continuous speech signal from the start of speaking until a noticeable pause occurs. For each multi-speaker segment, acoustic fingerprint features are extracted from its corresponding enhanced speech data. The acoustic fingerprint features include a dynamic trajectory of Mel-frequency cepstral coefficients across a continuous frame and a timbre feature vector that integrates the spectral centroid, spectral roll-off point, and spectral flux. The Mel-frequency cepstral coefficient dynamic trajectory reflects the change pattern of the speaker's spectral envelope over time, while the timbre feature vector is calculated from the stable segment to characterize the speaker's vocal quality. In practice, for a one-hour conference, hundreds of multi-speaker segments may initially be generated. All acoustic fingerprint features extracted from these multi-speaker segments are fed into a clustering algorithm. The clustering algorithm uses a hierarchical clustering method based on Euclidean distance to initially group nearby multi-speaker segments in the feature space into several clusters, each cluster initially corresponding to a potential independent speaker. In some embodiments, for each cluster generated by the clustering algorithm, the arithmetic mean of the acoustic fingerprint features of all speaker segments within the cluster is calculated. The resulting mean vector is the intra-cluster acoustic feature centroid of the cluster. This centroid is stored as a reference voiceprint template for a speaker, and a numerical identifier, such as "speaker 001", is assigned to this reference voiceprint template.

[0029] In practical implementation, when processing a new multi-speaker segment, a unique identifier needs to be assigned to it. First, acoustic fingerprint features are extracted from this new multi-speaker segment, and then the similarity between these features and each existing reference voiceprint template in the system is calculated. The formula for calculating the similarity between acoustic features and reference voiceprint templates can be expressed as:

[0030] in, This represents the highest similarity score obtained through calculation; Represents the timbre feature vector of the new segment; This represents a timbre feature vector in a reference voiceprint template; This represents a function for calculating cosine similarity. This represents the dynamic trajectory matrix of the Mel frequency cepstral coefficients of the new segment; This represents the dynamic trajectory matrix stored in the reference voiceprint template; This represents a function for calculating the dynamic time-warped distance; This is a weighting coefficient used to balance the two contributions. It can be understood that the system presets a confirmation threshold; if the calculated highest similarity score... If the similarity score exceeds this confirmation threshold, it is determined that the current new multi-speaker segment and the corresponding reference voiceprint template belong to the same speaker, and the digital identifier of the reference voiceprint template is assigned to the current new multi-speaker segment. In some embodiments, if the highest similarity score does not exceed the confirmation threshold, it means that the current new multi-speaker segment may belong to a new, unregistered speaker. In this case, the system will create a new reference voiceprint template based on the acoustic fingerprint features of the current new multi-speaker segment, and assign a new, unused unique identifier to this new reference voiceprint template, such as "speaker 005". At the same time, the start and end times of this new speaker segment are associated with this unique identifier. It is understandable that in actual meeting scenarios, the acoustic features of the same speaker's voice may fluctuate due to differences in distance from the microphone or emotional fluctuations. By comparing the dynamic trajectory of Mel-frequency cepstral coefficients using a dynamic time warping algorithm, the matching difficulties caused by changes in speech rate can be effectively addressed. Optionally, the reference voiceprint template will be updated online. When multiple multi-speaker segments belonging to the same identifier are successfully clustered, the average value of the features of these segments can be used to update the corresponding reference voiceprint template, making it more representative. Optionally, the confirmation threshold can be adjusted based on statistical analysis of historical conference data to adapt to different acoustic environments and speaker groups.

[0031] In one embodiment of the present invention, automatic speech recognition is performed on the multi-speaker segment to generate corresponding candidate texts and multiple alternative text hypotheses. The language model perplexity and acoustic model score of the candidate texts are calculated. The signal-to-noise ratio, harmonicity, and speech intelligibility index of the multi-speaker segment are evaluated to obtain an acoustic clarity metric. Non-fluency phenomena, including repetition, corrections, and excessively long pauses, are detected within the multi-speaker segment to generate a fluency penalty factor. The initial quality score is obtained by weighted fusion calculation of the language model perplexity, the acoustic model score, the acoustic clarity metric, and the fluency penalty factor. A series of consecutive multi-speaker segments with the same unique identifier are obtained. The cosine similarity and semantic coherence scores of adjacent multi-speaker segments in terms of text content are calculated. It is determined whether the cosine similarity and semantic coherence scores are both higher than the merging threshold. If both are higher, these multi-speaker segments are merged into a logical discourse unit, the content of which is a sequential splicing of the contents of each segment, and the time span is from the start time of the first segment to the end time of the last segment. If the scores are not higher, the current segment is regarded as an independent logical discourse unit, and the merging judgment with subsequent segments continues.

[0032] In practice, a multi-speaker segment labeled "Speaker 003" enters the processing flow. This segment contains 3.2 seconds of enhanced speech data. Automatic speech recognition (ASR) is performed on this segment. The ASR engine generates an optimal candidate text and several alternative texts with slightly lower confidence based on acoustic and language models. For example, the candidate text might be "We need to reassess the third quarter budget." Simultaneously, the perplexity of the language model and the acoustic model score of this candidate text are calculated. The perplexity reflects the frequency of the text sequence under a general language model, while the acoustic model score reflects the probability of matching between speech features and phoneme sequences. In practice, evaluating the acoustic intelligibility metric of the multi-speaker segment requires calculating the signal-to-noise ratio (SNR), harmonicity, and speech intelligibility index. The SNR is obtained by comparing the energy of speech segments with that of silence segments. Harmonicity is obtained by calculating the proportion of harmonic energy in sub-bands. The speech intelligibility index is estimated based on the intelligibility index formula. These values ​​are normalized and then fused into a single acoustic intelligibility metric. Understandably, detecting non-fluency within multi-speaker segments involves temporal analysis of the identified word sequences. The algorithm flags word repetitions, mid-sentence corrections, and pauses exceeding 500 milliseconds, generating a fluency penalty factor between 0 and 1 based on the frequency and severity of non-fluency. The initial quality score is calculated by weighted fusion of language model perplexity, acoustic model score, acoustic clarity metric, and fluency penalty factor, using the following formula:

[0033] in, This represents the calculated initial quality score; Indicates the perplexity of the language model; Indicates the acoustic model score; This represents a measure of acoustic clarity. This represents the fluency penalty factor; These are pre-defined positive weighting coefficients. In some embodiments, three consecutive multi-speaker segments with the same unique identifier "Speaker 003" are obtained. The start and end times of the three multi-speaker segments are closely adjacent, and the content texts are segment A "Regarding Market Strategy", segment B "I think we need to adjust", and segment C "Promotion Plan for the North American Market", respectively. In a specific implementation, the cosine similarity and semantic coherence score of adjacent multi-speaker segments in terms of text content are calculated. The cosine similarity between the word vectors of segment A and segment B is calculated. At the same time, a trained neural network model is used to analyze the coherence of the texts of segment A and segment B in terms of contextual meaning and outputs the semantic coherence score. Understandably, the system presets a merging threshold, which includes a lower limit for cosine similarity and a lower limit for semantic coherence score. It determines whether the cosine similarity and semantic coherence scores of fragment A and fragment B are both higher than the merging threshold. If both are higher, fragment A and fragment B are merged into a single logical discourse unit. The content of the merged logical discourse unit is "Regarding market strategy, I believe we need to adjust it," with a time span from the start time of fragment A to the end time of fragment B. In some embodiments, the system continues to determine the cosine similarity and semantic coherence score between the newly merged logical discourse unit and the immediately following fragment C. If the calculated values ​​are again higher than the merging threshold, the content of fragment C is also merged, forming a final logical discourse unit containing the three original fragments, with the complete content being "Regarding market strategy, I believe we need to adjust our promotional plan for the North American market." Optionally, if a segment's cosine similarity or semantic coherence score with subsequent segments does not exceed the merging threshold (e.g., a segment exhibits a clear topic shift), then that segment will terminate as an independent logical discourse unit. The system will then use this independent logical discourse unit as the starting point to continue merging with subsequent segments in a new round. Optionally, the merging threshold is not fixed and can be dynamically adjusted based on the overall speaking speed and topic focus of the meeting. Discussions with a focused topic can be handled with a slightly lower merging threshold to promote the generation of longer discourse units.

[0034] In one embodiment of the present invention, a set of keywords and named entities are extracted from the text content of each logical discourse unit. The set of keywords is matched with a predefined conference topic thesaurus. At least one topic tag is assigned to each logical discourse unit. The language behavior of each logical discourse unit is analyzed and classified into one of the intent types: statement, question, suggestion, resolution, or to-do item. Based on the time sequence of the logical discourse units and the relevance of the topic tags, units with the same or similar topic tags are aggregated into a topic cluster. A tree structure is constructed with the overall conference theme as the root node and each topic cluster as a child node. The text content of each child node is a summary of the content of all logical discourse units belonging to the topic cluster. In the tree structure, the attributes of the logical discourse unit within the child node are marked according to the intent type.

[0035] In practical implementation, the processed sequence of logical discourse units forms the basis for generating a semantic structure tree. Assuming a meeting with "quarterly product review" as the overall theme generates five logical discourse units, natural language processing is performed on the text content of each unit to extract keyword sets and named entities. For example, if the text content of a logical discourse unit is "Sales revenue in East China last quarter was five million yuan, but market share decreased by two percent," the extracted keyword set includes "East China," "sales revenue," "market share," and "decline," while the extracted named entities include "last quarter" (time entity) and "five million yuan" (currency entity). In practice, the system maintains a predefined meeting topic thesaurus, which includes categories such as "market," "R&D," "finance," and "human resources," along with their specific entries. The keyword sets extracted from the logical discourse units are matched against the predefined meeting topic thesaurus. The matching process calculates the semantic similarity between keywords and the thesaurus entries and assigns at least one topic tag to each logical discourse unit. Taking the above logical discourse unit as an example, its keywords "sales revenue" and "market share" highly match the "market" category in the thesaurus; therefore, the topic tag "market" is assigned to this unit. Analyzing the linguistic behavior of logical discourse units involves syntactic analysis and pattern matching. For example, if a unit text contains interrogative words such as "whether" or "how" and ends with a question mark, it is classified as a question intention type; if a unit text contains imperative content introduced by verbs such as "suggest" or "propose," it is classified as a suggestion intention type; if the unit text is a description of facts, it is classified as a statement intention type; if a unit text contains words indicating conclusions such as "decide" or "pass," it is classified as a resolution intention type; if a unit text contains expressions clearly indicating responsibility and time such as "to be responsible for" or "deadline," it is classified as a to-do item intention type. See Table 1 for partial results of assigning topic tags and intention types to each logical discourse unit: Table 1: Example Information Table of Logical Discourse Units Logical Discourse Unit Number Text Content Summary Topic Tags Intent Type Unit 1 Sales revenue in East China is five million yuan, market share decreased by two percent Market Statement Unit 2 Should we consider adjusting the promotional strategy in East China? Market Question Unit 3 suggests increasing social media advertising investment by 500,000 yuan. Financial Advice Unit 4: New UI development is 80% complete. R&D Presentation Unit 5: Decision made to adopt the suggestion to increase advertising investment; the marketing department will execute the marketing plan. (Decision Items) In some embodiments, clustering is performed based on the temporal order of logical discourse units and the relevance of topic tags. This checks whether temporally adjacent logical discourse units have the same or similar topic tags. The formula for determining topic tag similarity can be expressed as:

[0036] in, This indicates the similarity score of topic tags; and These represent the sets of topic tags for logical discourse unit A and logical discourse unit B, respectively. This represents a function that calculates the cosine similarity between the overall text embedding vectors of two topic tag sets; and It is the corresponding embedding vector; It is an adjustment weight. Calculate the overall text embedding vector. The process involves first using a pre-trained word vector model to generate a set of topic tags. Each tag word in the set is mapped to a high-dimensional real vector. Then, the arithmetic mean of all the tag word vectors is taken, and the average vector is the overall text embedding vector representing the set. Cosine similarity The calculation formula is If the calculated If the value exceeds the clustering threshold, then Unit A and Unit B are aggregated into the same topic cluster. For example, Units 1, 2, 3, and 5 all contain the tag "market," and are close in time and related in content, so they are aggregated into the same topic cluster about "market performance and strategy." Unit 4, on the other hand, has the tag "R&D," so it is classified into another topic cluster about "R&D progress." It can be understood that the semantic tree structure is built with the overall conference theme "quarterly product review" as the root node, and the aforementioned aggregated topic clusters "market performance and strategy" and "R&D progress" as direct child nodes. The text content of each child node is not simply a list of the original texts of all the logical discourse units, but rather a summary description generated by a text summarization model. For example, the summary content of the "market performance and strategy" child node might be: "The conference discussed the decline in sales and market share in East China, proposed adjustments to promotional strategies and increased advertising investment, and ultimately passed a resolution to increase advertising investment." In the tree structure of the semantic tree structure, the attributes of logical discourse units within child nodes are labeled according to intent type. Attribute information is stored along with the logical discourse unit itself to identify its function in the dialogue. Optionally, topic cluster aggregation is not limited to time-adjacent units. By globally analyzing the similarity of topic tags across all units, scattered but highly topic-related units can be aggregated into the same topic cluster, forming a more complete semantic context. Optionally, the predefined conference topic thesaurus supports dynamic expansion. High-frequency new words identified during the conference, after confirmation, can be added to the topic thesaurus for real-time topic tag assignment of subsequent logical discourse units.

[0037] Referring to Figure 4, this is a bar chart showing the semantic perception quality scores of logical discourse units, derived from the analysis results of the "Quarterly Product Review Meeting." The chart visually compares the semantic perception quality scores of five logical discourse units; a higher score indicates greater semantic importance and quality of the unit's content within the meeting. The semantic perception quality score is the core basis for constructing the semantic structure tree of meeting minutes. Units with high scores are prioritized as core-level content and highlighted in the final meeting minutes. This visualization helps meeting organizers quickly locate key conclusions and high-value information. The scoring mechanism provides an objective and quantifiable evaluation standard for the importance of meeting content, avoiding subjective differences in judgment regarding "key content" and ensuring more consistent and accurate information delivery.

[0038] In one embodiment of the present invention, the initial quality score corresponding to the logical discourse unit is obtained; the child node to which the logical discourse unit belongs is located in the semantic structure tree, and the depth level of the child node in the tree is determined; the textual association strength between the logical discourse unit and other units within its topic cluster, as well as with nodes in other topic clusters, is analyzed; the quality scores of units within nodes with shallower depth levels are positively adjusted, and the quality scores of units with higher association strength are positively adjusted; the adjusted quality score is used as the semantic perception quality score of the logical discourse unit. The semantic perception quality scores of all logical discourse units are collected, and their distribution histogram is calculated. According to the distribution histogram, the scores are divided into three intervals: high, medium, and low. Logical discourse units falling into the high interval are marked as core-level content, those falling into the medium interval are marked as support-level content, and those falling into the low interval are marked as reference-level content. Combining the depth of nodes in the semantic structure tree, content located as direct child nodes of the root node and marked as support-level content is promoted to the core level.

[0039] In practice, the initial quality scores of logical discourse units are adjusted based on the constructed semantic structure tree to form a semantic perception quality score. The semantic structure tree is exemplified by a three-layer structure. The root node is the overall meeting theme, "Quarterly Product Review." The direct child nodes of the root node include the "Market Strategy" topic cluster and the "R&D Progress" topic cluster. The "Market Strategy" topic cluster further includes two child nodes: "Current Situation Analysis" and "Action Decisions." Each node contains several logical discourse units. The initial quality score corresponding to each logical discourse unit is obtained; for example, a logical discourse unit belonging to the "Market Strategy - Current Situation Analysis" node... The logical discourse unit, with an initial quality score of 0.72, is located as the child node "Current Status quo Analysis" in the semantic tree structure. The depth level of the "Current Status quo Analysis" child node in the tree is determined to be the third level. The textual association strength between this logical discourse unit and other units within the "Current Status quo Analysis" topic cluster is analyzed. At the same time, the textual association strength between this logical discourse unit and units within the "Action Decision" node under the "Market Strategy" topic cluster is also analyzed. The textual association strength is quantified by calculating the cosine similarity of the text vectors between units and the weighted score of co-occurring keywords.

[0040] The formula for adjusting the initial quality score to form the semantic-aware quality score can be expressed as:

[0041] in, This represents the semantic perception quality score to be calculated. This represents the initial quality score of the logical discourse unit; Indicates the depth adjustment factor; Indicates the maximum depth of the semantic structure tree; This indicates the depth level of the node to which the logical discourse unit belongs in the tree; This represents the correlation adjustment factor; This represents the average textual association strength between the logical discourse unit and other units within its own topic cluster, as well as units within other related topic cluster nodes. It can be understood that, according to the formula, nodes with shallower depth levels correspond to... A larger value indicates a greater positive adjustment to the logical discourse units within it, thus increasing the correlation strength. Higher-level units also receive positive adjustments, such as setting the maximum depth of the semantic structure tree. The depth adjustment coefficient is 3. The depth level of the "Current Situation Analysis" node to which the logical discourse unit belongs is 0.05. If the value is 3, then the adjustment obtained by this unit in the depth term is: Furthermore, the average text association strength between this unit and other related nodes within the cluster was calculated. The correlation adjustment coefficient is set to 0.45. If the value is 0.2, then the adjustment obtained by this unit in the relevance item is... Ultimately, the semantic perception quality score of this logical discourse unit is determined. Calculated as The adjusted score will be used as the semantic perception quality score of the logical discourse unit.

[0042] In some embodiments, determining the criticality level of meeting content requires collecting the semantic awareness quality scores of all logical discourse units. Assuming a meeting generates twenty logical discourse units with semantic awareness quality scores ranging from 0.45 to 0.95, a distribution histogram of these scores is generated. The histogram equally divides the score range into several intervals and counts the number of units in each interval. Based on the distribution histogram, the scores are divided into high, medium, and low intervals. This division can be based on percentile methods; for example, the interval corresponding to the top 30% of units with the highest scores is defined as the high interval, the middle 40% as the medium interval, and the bottom 30% as the low interval. Logical discourse units falling into the high interval are marked as core-level content, those falling into the medium interval are marked as supporting-level content, and those falling into the low interval are marked as reference-level content. It is understandable that the labeling results are corrected by combining the depth of nodes in the semantic tree structure. Content located as a direct child of the root node and labeled as supporting level is elevated to core level. For example, a logical discourse unit with a semantic perception quality score of 0.68, which was originally labeled as supporting level based on its score distribution falling into the middle interval, is elevated to core level because it directly belongs to the root node's child node "Research and Development Progress". Optionally, the threshold for dividing the score interval can be configured according to the required level of detail in the meeting minutes. For minutes requiring high conciseness, the threshold for the high interval can be increased to reduce the amount of core level content. Optionally, the level elevation rule can be extended. For example, supporting level units that are not directly children of the root node but have strong connections with multiple core level units can also be considered for level elevation.

[0043] Referring to Figure 5, this is a graph showing the changes in meeting flow and quality score. The blue line represents the average semantic perception quality score at different points in the meeting, and the red dashed line represents the smoothed overall trend. The overall trend shows a steady increase, indicating that the semantic value and importance of the discussion content gradually increase as the meeting progresses. Score fluctuations can reflect the efficiency and rhythm of the meeting. For example, a low point in the initial stage can optimize the opening process and reduce unnecessary warm-up time; the timing of the peak stage can help determine the timing of discussions on core topics. The smooth upward trend verifies the focus of the meeting discussions. As the topics progress, the semantic value of the content continues to increase, indicating that the meeting has not deviated from the topic and has high overall efficiency. Content from high-scoring periods should be the focus of the meeting minutes, while content from low-scoring periods can be simplified or omitted, thereby significantly improving the efficiency and accuracy of the minutes generation.

[0044] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments that can be applied to other fields. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for automatically generating meeting minutes based on AI-powered speech recognition, characterized in that: The operation includes the following steps: receiving and processing conference audio signals containing the voices of multiple people to form a multi-channel voice data stream; extracting voice features and suppressing noise from the multi-channel voice data stream to generate enhanced voice data; Identify multi-speaker segments in the enhanced speech data, assign a unique identifier to each speaker segment and record its start and end times; transcribe the content of the multi-speaker segments and calculate the initial quality score associated with each multi-speaker segment; merge multiple multi-speaker segments that are temporally adjacent and have the same speaker identifier to form a logical discourse unit. The content of the logical discourse units is subjected to topic clustering and intent recognition to generate a semantic structure tree; the initial quality score is adjusted according to the depth and relevance of each node in the semantic structure tree to form a semantic perception quality score; the key level of the meeting content is determined according to the distribution of the semantic perception quality score; the text content corresponding to the nodes that meet the key level requirements is extracted from the semantic structure tree; the text content is integrated and reorganized and polished according to the preset minutes format to output the final meeting minutes document.

2. The method for automatically generating meeting minutes based on AI-powered speech recognition according to claim 1, characterized in that, The steps for forming a multi-channel speech data stream specifically include: acquiring raw audio signals from different physical locations or different pickup devices; performing pre-emphasis, framing, and windowing processing on each raw audio signal to obtain a corresponding audio frame sequence; extracting spectral envelope, fundamental frequency trajectory, and formant features from the audio frame sequence to form a multi-dimensional feature vector; aligning and fusing all the multi-dimensional feature vectors from the same time window to generate the multi-channel speech data stream, wherein each channel corresponds to an independent speech source.

3. The method for automatically generating meeting minutes based on AI-powered speech recognition according to claim 1, characterized in that, The steps for generating enhanced speech data specifically include: estimating background noise for each channel of the multi-channel speech data stream to obtain a noise spectrum; separating steady-state noise components and non-steady-state noise components from the multi-channel speech data stream; suppressing the steady-state noise components using spectral subtraction based on speech activity detection; constructing an adaptive filter bank to track and cancel the non-steady-state noise components; and performing beamforming on the noise-suppressed speech data from each channel to enhance the speech energy in the direction of the main speaker, and outputting the enhanced speech data.

4. The method for automatically generating meeting minutes based on AI-powered speech recognition according to claim 1, characterized in that, The step of assigning a unique identifier to each speaker segment specifically includes: extracting acoustic fingerprint features from the enhanced speech data of the multi-speaker segments, wherein the acoustic fingerprint features include the dynamic trajectory of Mel-frequency cepstral coefficients and timbre feature vectors; based on the acoustic fingerprint features, performing preliminary separation of different speakers using a clustering algorithm; for each clustered group, extracting its intra-cluster acoustic feature centroid as a reference voiceprint template for the speaker; calculating the similarity between the acoustic features of the current speaker segment and all existing reference voiceprint templates; if the highest similarity exceeds the confirmation threshold, assigning the current segment to the speaker represented by the corresponding reference voiceprint template and using the existing identifier of the reference voiceprint template; if it does not exceed the threshold, creating a new reference voiceprint template for the current segment and assigning a new, unused unique identifier.

5. The method for automatically generating meeting minutes based on AI speech recognition according to claim 1, characterized in that, The step of calculating the initial quality score associated with each of the multi-speaker segments specifically includes: performing automatic speech recognition on the multi-speaker segments to generate corresponding candidate texts and multiple alternative text hypotheses; calculating the language model perplexity and acoustic model score of the candidate texts; evaluating the signal-to-noise ratio, harmonicity, and speech intelligibility index of the multi-speaker segments to obtain an acoustic clarity metric; detecting whether there are non-fluency phenomena within the multi-speaker segments, including repetitions, corrections, and excessively long pauses, and generating a fluency penalty factor; and combining the language model perplexity, the acoustic model score, the acoustic clarity metric, and the fluency penalty factor to obtain the initial quality score through weighted fusion calculation.

6. The method for automatically generating meeting minutes based on AI-powered speech recognition according to claim 1, characterized in that, The steps for forming a logical discourse unit specifically include: acquiring multiple consecutive multi-speaker segments with the same unique identifier; calculating the cosine similarity and semantic coherence score of adjacent multi-speaker segments in terms of text content; determining whether the cosine similarity and semantic coherence score are both higher than a merging threshold; if both are higher, merging these multi-speaker segments into a single logical discourse unit, the content of which is a sequential concatenation of the content of each segment, with a time span from the start time of the first segment to the end time of the last segment; if not higher, treating the current segment as an independent logical discourse unit and continuing to merge it with subsequent segments.

7. The method for automatically generating meeting minutes based on AI-powered speech recognition according to claim 1, characterized in that, The steps for generating the semantic structure tree specifically include: extracting a set of keywords and named entities from the text content of each logical discourse unit; matching the set of keywords with a predefined conference topic thesaurus to assign at least one topic tag to the logical discourse unit; analyzing the language behavior of the logical discourse unit and classifying it into one of the intent types: statement, question, suggestion, resolution, or to-do item; aggregating units with the same or similar topic tags into a topic cluster based on the temporal order of the logical discourse units and the relevance of the topic tags; constructing a tree structure with the overall conference theme as the root node and each topic cluster as a child node, wherein the text content of the child node is a summary of the content of all logical discourse units belonging to the topic cluster; and marking the attributes of the logical discourse unit within the child node according to the intent type in the tree structure.

8. The method for automatically generating meeting minutes based on AI-powered speech recognition according to claim 1, characterized in that, The steps for forming a semantic perception quality score specifically include: obtaining the initial quality score corresponding to the logical discourse unit; locating the child node to which the logical discourse unit belongs in the semantic structure tree and determining the depth level of the child node in the tree; analyzing the textual association strength between the logical discourse unit and other units within its topic cluster, as well as with nodes in other topic clusters; positively adjusting the quality scores of units within nodes with shallower depth levels and positively adjusting the quality scores of units with higher association strengths; and using the adjusted quality score as the semantic perception quality score of the logical discourse unit.

9. The method for automatically generating meeting minutes based on AI-powered speech recognition according to claim 1, characterized in that, The steps for determining the key levels of meeting content specifically include: collecting the semantic perception quality scores of all logical discourse units and calculating their distribution histograms; dividing the scores into high, medium, and low intervals based on the distribution histograms; marking logical discourse units falling into the high interval as core-level content, those falling into the medium interval as supporting-level content, and those falling into the low interval as reference-level content; and, based on the depth of nodes in the semantic structure tree, elevating the level of content located as direct children of the root node and marked as supporting-level content to the core level.

10. An AI-based speech recognition-based automatic meeting minutes generation system, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the AI-based speech recognition automatic meeting minutes generation method according to any one of claims 1 to 9.

Images