Meeting minutes generation method and device, equipment, storage medium and program product

Abstract

This application relates to a method, apparatus, device, storage medium, and program product for generating meeting minutes. The method includes: extracting multiple valid audio segments from meeting audio; performing speech recognition on each of the valid audio segments using a preset target speech recognition model to obtain text for each valid audio segment; wherein the target speech recognition model matches the business type to which the meeting audio belongs; generating initial text with speaker tags based on each valid audio segment and its text; extracting meeting elements from the initial text using a preset language model and generating structured meeting text based on the meeting elements; and generating meeting minutes corresponding to the meeting audio based on the meeting text and a preset business knowledge base for the business type. This ensures the accuracy of speech recognition and improves the accuracy of the meeting minutes content.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, device, storage medium, and program product for generating meeting minutes. Background Technology

[0002] With the advent of the information age, meetings have become a core part of daily operations and decision-making communication for enterprises. High-quality meeting minutes can not only help participants recall the content of the communication, but also enable information synchronization for those who did not attend, ensuring that meeting resolutions are effectively implemented and tracked.

[0003] Traditional meeting minutes are usually recorded manually, but manual recording is prone to missing key points, and post-processing is time-consuming. Although some meeting recording software can automatically generate meeting minutes, it can only convert meeting audio into text and cannot reflect the actual meeting process. It often requires manual reprocessing, and this simple speech-to-text conversion also suffers from low recognition accuracy. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, apparatus, device, storage medium, and program product for generating meeting minutes to address the aforementioned technical problems.

[0005] In a first aspect, this application provides a method for generating meeting minutes, the method comprising: extracting multiple valid audio segments from meeting audio; performing speech recognition on each of the valid audio segments using a preset target speech recognition model to obtain text for each valid audio segment; wherein the target speech recognition model is matched with the business type to which the meeting audio belongs; generating initial text with speaker tags based on each of the valid audio segments and their text; extracting meeting elements from the initial text using a preset language model, and generating structured meeting text based on the meeting elements; and generating meeting minutes corresponding to the meeting audio based on the meeting text and a preset business knowledge base for the business type.

[0006] In an optional implementation, extracting multiple valid audio segments from the conference audio includes: preprocessing the conference audio to obtain initial audio; using a preset speech detection model to perform human voice detection on each audio frame in the initial audio to obtain a detection probability for each audio frame, the detection probability representing the probability that the audio frame contains human voice; selecting multiple candidate audio frames from the multiple audio frames whose detection probability is greater than a preset first threshold; selecting multiple target audio frames from the multiple candidate audio frames whose detection probability is greater than a preset second threshold; wherein the second threshold is greater than the first threshold; and obtaining multiple valid audio segments based on the multiple target audio frames; wherein one valid audio segment contains multiple temporally consecutive target audio frames.

[0007] In an optional implementation, generating initial text with speaker tags based on each valid audio segment and its text includes: performing voiceprint recognition on each valid audio segment to obtain each speaking segment and its speaker in the valid audio segment; determining the paragraph corresponding to the speaking segment in the text of each speaking segment, and marking the speaking segment's speaker identifier in the paragraph to obtain the labeled text of the valid audio segment; and clustering the labeled text of multiple valid audio segments to obtain the initial text with speaker tags.

[0008] In an optional implementation, the step of extracting meeting elements from the initial text using a preset language model and generating structured meeting text based on the meeting elements includes: using the language model, generating multiple element information based on the initial text and preset multiple element types according to a preset first prompt word, to obtain meeting elements containing the multiple element information; using the language model, checking and updating the meeting elements based on the initial text according to a preset second prompt word, to obtain updated meeting elements; and using the language model, generating structured meeting text based on the updated meeting elements according to a preset third prompt word.

[0009] In an optional implementation, generating meeting minutes corresponding to the meeting audio based on the meeting text and a preset business knowledge base for the business type includes: selecting a target template that matches the meeting text from a plurality of preset meeting minutes templates; obtaining target business knowledge associated with the meeting text from a plurality of business knowledge in the business knowledge base; and generating meeting minutes corresponding to the meeting audio based on the meeting text and the target business knowledge according to the target template.

[0010] In an optional implementation, the target speech recognition model is trained as follows: acquiring a set of business terms for the business type and multiple historical meeting audios; dividing each historical meeting audio into multiple audio segments, and labeling the content of each audio segment according to the set of business terms to obtain a text label for each audio segment; and training a basic speech recognition model based on the multiple audio segments and their text labels to obtain the target speech recognition model.

[0011] Secondly, this application also provides a meeting minutes generation device, the device comprising: an extraction module for extracting multiple valid audio segments from meeting audio; a processing module for performing speech recognition on each of the valid audio segments using a preset target speech recognition model to obtain text for each of the valid audio segments; wherein the target speech recognition model is matched with the business type to which the meeting audio belongs; generating initial text with speaker tags based on each of the valid audio segments and their text; extracting meeting elements from the initial text using a preset language model, and generating structured meeting text based on the meeting elements; and a generation module for generating meeting minutes corresponding to the meeting audio based on the meeting text and a preset business knowledge base for the business type.

[0012] Thirdly, this application provides a computer device including a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the steps of the method described in any of the foregoing embodiments.

[0013] Fourthly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in any of the foregoing embodiments.

[0014] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the method described in any of the foregoing embodiments.

[0015] This application provides a meeting minutes generation method, apparatus, device, storage medium, and program product. The method includes: extracting multiple valid audio segments from meeting audio; performing speech recognition on each valid audio segment using a preset target speech recognition model to obtain text for each valid audio segment; wherein the target speech recognition model matches the business type to which the meeting audio belongs; generating initial text with speaker tags based on each valid audio segment and its text; extracting meeting elements from the initial text using a preset language model and generating structured meeting text based on the meeting elements; and generating meeting minutes corresponding to the meeting audio based on the meeting text and a preset business knowledge base for the business type. By using a target speech recognition model that matches the business type to which the meeting audio belongs for recognition, the accuracy of speech recognition is ensured. Furthermore, by using a language model to extract meeting elements and combining it with a business knowledge base to generate meeting minutes, the generation of meeting minutes is adapted to specific business scenarios, thereby improving the accuracy of the meeting minutes content. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 The illustration shows an application scenario provided by an embodiment of this application;

[0018] Figure 2 A flowchart illustrating a meeting minutes generation method provided in an embodiment of this application is shown.

[0019] Figure 3 A flowchart illustrating another meeting minutes generation method provided in an embodiment of this application is shown;

[0020] Figure 4 A flowchart illustrating another meeting minutes generation method provided in an embodiment of this application is shown;

[0021] Figure 5 This paper illustrates a functional block diagram of a meeting minutes generation device provided in an embodiment of this application;

[0022] Figure 6 A schematic diagram of the internal structure of a computer device provided in an embodiment of this application is shown. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0024] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0025] In related technologies, although there are some speech-to-text tools or meeting recording software that can automatically generate meeting minutes, they have the following defects: (1) Insufficient speech recognition accuracy. General speech recognition models have low accuracy in recognizing professional terms. For example, professional terms such as "switching machine", "track maintenance", and "train control system" in the railway field are often misrecognized, affecting the quality of meeting minutes. (2) Lack of speaker differentiation ability. Existing tools can usually only generate single text transcriptions and cannot distinguish the content of different speakers' speeches, resulting in the loss of key information in the meeting minutes and difficulty in reconstructing the meeting discussion process. (3) Content understanding remains superficial. Simple speech-to-text tools can only complete transcription and cannot understand the core points of the meeting content, which still requires manual sorting. (4) Lack of domain knowledge support. General tools cannot be combined with the enterprise's internal knowledge base. During the process of generating meeting minutes, they cannot be linked to relevant business documents, historical resolutions and standard processes, and the generated meeting minutes lack professional depth and decision-making basis. (5) Data security risks. Most speech recognition services are typically deployed on public clouds. Uploading meeting recordings containing sensitive information to the cloud poses a data leakage risk and fails to meet the security compliance requirements of critical infrastructure sectors such as energy and transportation. Therefore, this application provides a method for generating meeting minutes to address these issues.

[0026] The meeting minutes generation method provided in this application embodiment can be applied to, for example, Figure 1In the application environment shown, the terminal communicates with the server via a network. The terminal can be, but is not limited to, various personal computers, laptops, smartphones, and tablets. The server can be a standalone physical server, a server cluster, or a distributed system composed of multiple physical servers. It is understood that in some embodiments, the terminal can send conference audio to the server, and the server executes the meeting minutes generation method provided in this application embodiment based on the conference audio; in other embodiments, the terminal can also obtain the conference audio based on user interaction and execute the meeting minutes generation method provided in this application embodiment based on the conference audio.

[0027] The following section will use a server as the execution entity to describe the steps of each method provided in the embodiments of this application, as well as the corresponding technical effects. Please refer to... Figure 2 This is a flowchart illustrating a meeting minutes generation method provided in an embodiment of this application, including steps 210 to 250.

[0028] Step 210: Extract multiple valid audio segments from the conference audio.

[0029] In this embodiment, a user can log in to the interactive AI business support system via a terminal and perform interactive operations on the system's webpage to select the meeting minutes generation function and upload meeting audio files. Based on the user's interactive operations, the terminal sends the received meeting audio file to the server. The server receives the meeting audio file and processes it to extract multiple valid audio segments.

[0030] Step 220: Using a preset target speech recognition model, perform speech recognition on each valid audio segment to obtain the text of each valid audio segment.

[0031] Among them, the target speech recognition model is matched with the business type to which the conference audio belongs.

[0032] In this embodiment, to improve the accuracy of speech recognition, a speech recognition model matching the business type of the conference audio can be pre-trained to obtain the target speech recognition model. For example, for the business type of railway management, a basic speech recognition model such as the Sense Voice-Small model can be trained using the business terminology set of this business type to obtain a target speech recognition model that matches the business type of railway management.

[0033] Furthermore, each valid audio segment is sequentially input into the target speech recognition model. The target speech recognition model maps the acoustic features of each valid audio segment into a character sequence, thus obtaining the text of each valid audio segment. Understandably, because the target speech recognition model is matched to the business type, the error rate of specialized vocabulary in the recognition results is significantly reduced, thereby ensuring the accuracy of the text for each valid audio segment.

[0034] Step 230: Generate initial text with speaker tags based on each valid audio segment and its text.

[0035] Understandably, plain text content lacks differentiation of speakers, while meeting minutes need to clearly identify the speaker to whom each statement belongs. Therefore, in this embodiment, on the one hand, voiceprint features are extracted using the audio signal of each valid audio segment, and on the other hand, the text corresponding to that audio segment is aligned by timestamp. Furthermore, different sound sources within each valid audio segment are clustered to identify the speech period corresponding to each speaker. Then, based on the time alignment relationship, speaker identifiers are assigned to the paragraphs in the text corresponding to each speech period. Finally, the texts of all valid audio segments are concatenated to obtain an initial text with speaker tags.

[0036] Step 240: Using a preset language model, extract meeting elements from the initial text and generate structured meeting text based on the meeting elements.

[0037] Understandably, initial text is typically a conversational, loosely structured sequence of natural language containing repetitive or irrelevant chatter, while meeting minutes need to be condensed to extract key points. Therefore, in this embodiment, the initial text and pre-set prompts are input into a pre-defined language model to guide it in recognizing and extracting meeting elements. That is, the language model performs semantic understanding on the initial text to extract content related to meeting elements and generate a structured meeting text. Understandably, compared to the initial text, the meeting text removes verbose redundancy, resulting in a clearer information hierarchy.

[0038] Step 250: Based on the meeting text and a business knowledge base preset for the business type, generate meeting minutes corresponding to the meeting audio.

[0039] It is understandable that meeting minutes for different business types have different focuses and writing standards, and content generated solely by language models may lack domain depth. Therefore, in the application embodiment, a pre-built business knowledge base is invoked to obtain business knowledge such as background information, historical resolutions, definitions of professional terms, and key performance indicators related to the meeting text. The obtained business knowledge and meeting text are then combined to generate meeting minutes corresponding to the meeting audio.

[0040] As can be seen, this application embodiment ensures the accuracy of speech recognition by using a target speech recognition model that matches the business type to which the meeting audio belongs. Furthermore, it uses a language model to extract meeting elements and combines them with a business knowledge base to generate meeting minutes, thereby adapting the generation of meeting minutes to specific business scenarios and improving the accuracy of the meeting minutes content.

[0041] Optionally, for step 210 above, this application embodiment provides a possible implementation method, namely: preprocessing the conference audio to obtain initial audio; using a preset speech detection model, performing human voice detection on each audio frame in the initial audio to obtain the detection probability of each audio frame, the detection probability being used to represent the probability that the audio frame contains human voice; selecting multiple candidate audio frames from multiple audio frames whose detection probability is greater than a preset first threshold; selecting multiple target audio frames from multiple candidate audio frames whose detection probability is greater than a preset second threshold; wherein, the second threshold is greater than the first threshold; obtaining multiple valid audio segments based on multiple target audio frames; wherein, a valid audio segment contains multiple temporally consecutive target audio frames.

[0042] It is understandable that the original conference audio may contain issues such as low-frequency noise, high-frequency interference, echo, or inconsistent signal amplitude. Therefore, in this embodiment, the server preprocesses the conference audio file upon receiving it. For example, if the conference audio file is in formats such as WAV (Waveform Audio File Format), MP3 (Moving Picture Experts Group Audio Layer III), AAC (Advanced Audio Coding), or M4A (MPEG-4 Audio), then audio and video processing software like FFmpeg (Fast Forward Moving Picture Experts Group) can be used to uniformly convert the conference audio to a 16kHz sampling rate mono format, ensuring sound quality while reducing file size and processing time. Furthermore, based on the acoustic characteristics of the conference room, a model can be trained using a large amount of conference recording data to obtain a specialized denoising model. This denoising model is then used to eliminate background noise, air conditioning noise, keyboard typing noise, and other noises in the conference audio, resulting in an initial audio with a higher signal-to-noise ratio and more balanced performance.

[0043] Subsequently, using a pre-defined speech detection model, human voice detection is performed on each audio frame in the initial audio, yielding the detection probability for each audio frame. For example, the initial audio is divided into frames with fixed frame lengths and frame shifts, resulting in multiple audio frames arranged in chronological order. Each audio frame is then input into the pre-defined speech detection model, which analyzes the acoustic features of the audio frame, such as Mel-frequency cepstral coefficients, zero-crossing rate, and spectral energy distribution, to output the detection probability of that audio frame. This detection probability represents the probability that the audio frame contains human voice, thus obtaining the detection probability for each audio frame.

[0044] Next, from multiple audio frames, several candidate audio frames with a detection probability greater than a preset first threshold are selected. This can be understood as the first threshold being set to a relatively low value, its purpose being to initially filter out audio frames that may contain human voices but have low confidence, while avoiding missing frames with weak energy or those at the start / end of speech due to an excessively high threshold. Specifically, the detection probability of each audio frame is compared with the first threshold, and audio frames with a detection probability lower than or equal to the first threshold are discarded, while audio frames with a detection probability greater than the first threshold are marked as candidate audio frames, thus obtaining multiple candidate audio frames.

[0045] Next, from multiple candidate audio frames, several target audio frames with a detection probability greater than a preset second threshold are selected, and the second threshold is greater than the first threshold. This can be understood as follows: the first threshold is used to capture audio frames that may contain human voices, while the second threshold is used to filter out high-confidence audio frames containing human voices from these candidate audio frames, ensuring that the selected audio frames definitely contain real speaker speech. Specifically, the detection probability of each candidate audio frame is compared with the second threshold, and candidate audio frames with a detection probability lower than or equal to the second threshold are discarded, while candidate audio frames with a detection probability greater than the second threshold are marked as target audio frames. This results in multiple target audio frames, filtering out silent or purely background noise audio frames. It can be understood that through this dual-threshold filtering mechanism, blurred frames at the boundary between human and non-human voices, as well as falsely detected noise frames, can be effectively filtered out, thereby obtaining a highly reliable sequence of target audio frames.

[0046] Finally, based on multiple target audio frames, multiple valid audio segments are obtained. For example, all target audio frames can be traversed in chronological order, and those target audio frames that are temporally adjacent can be grouped into a continuous audio segment. Each such continuous segment is a valid audio segment, and all audio frames within a valid audio segment have a human voice detection probability higher than the second threshold, representing a high-quality, uninterrupted speaker's speech. That is, each valid audio segment corresponds to a complete human speech segment in the conference audio.

[0047] Optionally, for step 230 above, this application embodiment provides a possible implementation method, namely: for each valid audio segment, perform voiceprint recognition on the valid audio segment to obtain each speaking segment and its speaker in the valid audio segment; for each speaking segment, determine the paragraph corresponding to the speaking segment in the text of the valid audio segment, and mark the speaker identifier of the speaking segment in the paragraph to obtain the labeled text of the valid audio segment; cluster based on the labeled text of multiple valid audio segments to obtain the initial text with speaker labels.

[0048] In this embodiment, the voiceprint features of multiple participants can be pre-collected, that is, the voiceprint features of each participant can be registered in a voiceprint database to facilitate direct identification of the speaker's identity. It is understood that a valid audio segment may contain multiple alternating speeches by different speakers, for example, speaker A finishes speaking and speaker B immediately follows. Voiceprint recognition technology can segment and merge speech signals based on the unique biometric features of different voices.

[0049] It is understood that the processing method for each valid audio segment in this application embodiment is similar. For the sake of brevity, the following description uses a single valid audio segment as an example. First, the voiceprint feature vector of each continuous human voice segment is extracted from the valid audio segment, and the speaker's identity is determined by voiceprint clustering or comparison with a voiceprint database. That is, endpoint detection is performed on the speech within the valid audio segment, dividing the continuous speech region into multiple independent speech segments, each speech segment corresponding to an uninterrupted speech delivered by the same speaker.

[0050] Then, for each speech segment, the system extracts its voiceprint features and calculates the similarity with each voiceprint feature in the voiceprint database. When the similarity exceeds a preset threshold, the speaker's identity for the speech segment can be determined. If no voiceprint feature in the voiceprint database can be matched, a new temporary speaker identifier is assigned to the speech segment, thus obtaining the start and end time information of each speech segment in the valid audio segment and its corresponding speaker identifier.

[0051] Next, since the text of a valid audio segment is aligned with its audio signal in time, the speakers in the text can be labeled based on the timestamp mapping to obtain labeled text. That is, each character or word in the text of a valid audio segment has its start and end times on the audio timeline, and each speech segment itself also has a clearly defined start and end time interval. Therefore, based on this time interval, character sequences within the same time interval can be retrieved from the text of the valid audio segment, and these character sequences can be identified as the paragraphs corresponding to the speech segments. These paragraphs are text blocks that are strictly aligned with the speech segments in time. The speaker identifiers of the speech segments are then labeled before or after these paragraphs, for example, using identifiers such as Speaker1, Speaker2, etc., to distinguish the speakers, thus obtaining the labeled text of the valid audio segment.

[0052] Finally, clustering is performed on the labeled text of multiple valid audio segments to group different identifiers with voiceprint feature similarity higher than a preset threshold into the same actual speaker, that is, to correct and unify the speaker identifiers in each labeled text. Then, the labeled text of each valid audio segment is spliced ​​together according to the original time sequence of the actual meeting to form a complete and continuous text record, which is the initial text with speaker tags.

[0053] Optionally, for step 240 above, this application embodiment provides a possible implementation method, please refer to [link to relevant documentation]. Figure 3 Step 240 includes steps 241 to 243.

[0054] Step 241: Using a language model, based on the preset first prompt word, generate multiple element information according to the initial text and multiple preset element types, to obtain meeting elements containing multiple element information.

[0055] Step 242: Using a language model, the meeting elements are checked and updated based on the initial text according to the preset second prompt word, resulting in updated meeting elements.

[0056] Step 243: Using a language model, generate structured meeting text based on the updated meeting elements and a preset third prompt word.

[0057] In this embodiment, a language model is first used to generate various element information based on the initial text and preset element types, according to a preset first prompt word, resulting in meeting elements containing multiple element information. It is understood that a language model is a large language model trained on a large corpus, capable of understanding natural language instructions and performing tasks such as text generation and information extraction. For example, preset element types can include meeting metadata such as meeting time, location, participants, and meeting topic; decision-making matters such as discussion topics, key conclusions, to-do items, and resolution results; and action matters such as task description, responsible person, and task deadline. Furthermore, the first prompt word can explicitly tell the language model in the form of a natural language instruction which types of element information to extract from the initial text, as well as the output format requirements. Then, the first prompt word can be concatenated with the initial text and input into the language model. The language model performs semantic understanding and information filtering on the initial text, identifying text fragments or summary statements corresponding to each element type, and generating structured meeting elements.

[0058] Then, using a language model, based on the pre-set second cue words, the meeting elements are checked and updated according to the initial text to obtain the updated meeting elements. It is understandable that during the first extraction process, the language model may have insufficient understanding of the context or inadequate consideration of the relationships between elements, leading to omissions, errors, or contradictions with other parts of the initial text. Therefore, the second cue words can guide the language model to verify and supplement it. That is, the second cue words, the initial text, and the meeting elements are input into the language model, and the language model re-examines the complete initial text, comparing each sentence with existing meeting elements to check for any uncaptured important information and verify the accuracy of the extracted elements, thus obtaining the updated meeting elements.

[0059] Finally, using a language model, structured meeting text is generated based on the updated meeting elements according to preset third cue words. This can be understood as the updated meeting elements not yet forming a coherent and readable meeting transcript. Therefore, the third cue words can be used to clarify the final presentation format and organization of the meeting text. That is, the third cue words and the updated meeting elements are input into the language model, and the language model, based on the format constraints in the third cue words, expands the information of each element into paragraphs or bullet points in natural language. Simultaneously, it arranges the elements logically according to their relationships, and adds necessary connecting words and transitional phrases to make the final text read smoothly and systematically, thus obtaining the structured meeting text.

[0060] As can be seen, the embodiments of this application improve the completeness and accuracy of meeting elements and ensure that the meeting text meets the preset structured requirements by calling the language model in three stages: the initial extraction of the first prompt word-driven model, the cross-checking and updating of the second prompt word-driven model, and the generation of structured meeting text by the third prompt word-driven model.

[0061] Optionally, for step 250 above, this application embodiment provides a possible implementation method, namely: selecting a target template that matches the meeting text from a plurality of preset meeting minutes templates; obtaining target business knowledge associated with the meeting text from a plurality of business knowledge from a business knowledge base; and generating meeting minutes corresponding to the meeting audio based on the meeting text and the target business knowledge according to the target template.

[0062] In this embodiment, a target template matching the meeting text is first selected from a set of preset meeting minutes templates. It is understood that different business types, or different meeting natures within the same business type, may have significantly different applicable meeting minutes formats and writing styles; therefore, multiple meeting minutes templates can be pre-stored.

[0063] For example, meeting minutes templates can cover different types of meetings such as business analysis meetings, safety production meetings, technical review meetings, and departmental meetings. Each meeting minutes template defines the heading hierarchy, paragraph structure, required fields, and writing guidelines for each section. System administrators can also customize the template structure through a visual interface, such as adding, deleting, and adjusting chapters, and setting required and optional fields. Furthermore, the templates support dynamic fields, such as automatically inserting meeting dates, automatic numbering, and automatically generating a list of attendees.

[0064] Then, content analysis can be performed on the meeting text to extract key features, such as keywords in the meeting theme, combinations of element types, and the distribution of speaker roles. The extracted key features are then matched with the scenario descriptions or tags applicable to each template, and the template with the highest matching degree is selected as the target template.

[0065] Subsequently, target business knowledge related to the meeting text is retrieved from multiple business knowledge sources in the business knowledge base. It can be understood that the business knowledge base is a pre-built collection of knowledge specific to the business type to which the meeting audio belongs. It stores background information, historical resolutions, terminology definitions, key performance indicators (KPIs), related project information, and internal company rules, regulations, and management procedures related to that business area. The meeting text is then used as the basis for querying the business knowledge base. For example, key entities can be extracted from the meeting text, and these entities can be used as search terms to match and search within the business knowledge base. Vector similarity retrieval or keyword exact matching methods are then used to filter out one or more business knowledge items with the highest relevance to the meeting text content, thus obtaining the target business knowledge. For instance, when the meeting text includes discussions on equipment maintenance, the target business knowledge retrieved includes technical manuals and maintenance procedures; when the meeting text includes the development of safety measures, the target business knowledge retrieved includes safety standards and historical accident cases.

[0066] Finally, based on the target template, meeting minutes corresponding to the meeting audio are generated using the meeting text and target business knowledge. This can be understood as follows: the meeting text records the conclusions and tasks discussed in the meeting, while the target business knowledge provides background information or supplementary information to support these conclusions. Integrating the two results in a more complete and professional minutes. For example, the structure of the target template can be analyzed to clarify the required content type for each field, and then the corresponding information can be extracted from the meeting text for filling. For sections requiring background knowledge or related information, relevant content can be extracted from the target business knowledge for filling. During the filling process, a rule-based text concatenation method can be used, or the language model can be invoked again, inputting the structural description of the target template, the meeting text, and the target business knowledge together. The language model then generates a complete meeting minutes text according to the template format. Furthermore, the source of the target business knowledge can be indicated in appropriate locations within the meeting minutes as notes or appendices for readers to trace back, thus obtaining the meeting minutes corresponding to the meeting audio.

[0067] As can be seen, the embodiments of this application ensure the standardization of the meeting minutes format through template matching and introduce business backgrounds that are deeply related to the meeting content through knowledge retrieval. As a result, the generated meeting minutes not only record the discussion content of the meeting itself, but also provide the business knowledge required to support these discussion contents, thereby improving the reference value and practicality of the meeting minutes.

[0068] Optionally, for the target speech recognition model described above, this application embodiment provides an implementation method for training the target language recognition model; please refer to [link to relevant documentation]. Figure 4 This includes steps 260 to 280.

[0069] Step 260: Obtain the business terminology set for the business type and multiple historical meeting audio recordings.

[0070] Step 270: Divide each historical meeting audio into multiple audio segments, and label the content of each audio segment according to the business terminology set to obtain the text tag for each audio segment.

[0071] Step 280: Based on multiple audio segments and their text labels, train the basic speech recognition model to obtain the target speech recognition model.

[0072] In this embodiment, the first step is to acquire a set of business terminology for the specific business type and multiple historical meeting audio recordings. It should be noted that different business types of meetings have their own unique professional vocabulary systems. General-purpose speech recognition models often exhibit high error rates when processing these professional terms, thus requiring specialized training for specific business domains. Therefore, in this embodiment, a set of business terminology corresponding to the business type can be acquired. This terminology set can be pre-constructed by collecting standard terminology dictionaries for that domain, high-frequency professional terms from past meeting records, industry standard documents, etc., and contains a large number of words that are rare in general corpora but frequently appear in this business type of meeting. Multiple historical meeting audio recordings for that business type can also be collected, and these audio recordings should be real meeting recordings that have been collected and archived and belong to the same business type as the meeting audio to be processed.

[0073] Subsequently, each historical meeting audio is divided into multiple audio segments, and the content of each audio segment is labeled according to a set of business terms, resulting in a text tag for each audio segment. For example, speech activity detection is performed on each historical meeting audio, dividing it into multiple audio segments of appropriate length. Each segment typically corresponds to one or several consecutive sentences, facilitating subsequent labeling and training. For each segmented audio segment, the speech in the segment can be transcribed word-for-word into text by labelers or auxiliary labeling systems, combined with professional vocabulary from the business terminology set. Due to the existence of the business terminology set, professional terms encountered during the labeling process can be transcribed according to standard spelling, avoiding inconsistencies in labeling due to ambiguity in terminology spelling. This yields a text tag corresponding to each audio segment, which is the accurate textual expression of the speech content of that segment.

[0074] Finally, based on multiple audio segments and their text labels, a basic speech recognition model is trained to obtain the target speech recognition model. For example, the basic speech recognition model can use the Sense Voice-Small model, taking each audio segment as input and its corresponding text label as the expected output, and using supervised learning to optimize the parameters of the basic speech recognition model. During training, the model extracts acoustic features from each audio segment, generates a feature sequence, and then calculates the predicted text sequence through internal model computation. It also calculates the loss function value between the predicted text sequence and the text label, and updates the model's weight parameters based on the loss value using a backpropagation algorithm. Through multiple iterations, until the model's recognition accuracy reaches the preset requirements, the trained model is then used as the target language recognition model.

[0075] Understandably, since the audio segments in the training data come from historical meeting audio of the same business type, and the text labels strictly follow the vocabulary norms of the business terminology set, the model can learn the acoustic-text mapping relationship specific to this business domain during the training process, thereby improving the recognition accuracy of the speech recognition model.

[0076] For ease of understanding, an example is provided in this application to illustrate the complete process.

[0077] For example, Xiao Zhang, an employee in the general office of a heavy-haul railway company, needs to record a 20-minute project review meeting and generate a standardized meeting minutes. Xiao Zhang first logs into the interactive AI business support system via a terminal, selects the "Intelligent Meeting Minutes Generation" function module on the webpage, and uploads the meeting recording file. The terminal then sends the meeting recording file to the server.

[0078] After receiving the meeting recording file, the server preprocesses it. For example, the server converts the M4A format meeting recording file into a 16kHz sampling rate mono WAV format using audio and video processing software like FFmpeg, and uses a noise reduction model to remove background air conditioning noise and keyboard typing sounds from the recording. Furthermore, the server performs speech activity detection on the preprocessed audio, using a speech detection model to identify human voice segments in the recording, filtering out silent sections at the beginning and blank segments in the middle, thus obtaining multiple valid audio segments. This reduces the processing time for invalid audio by approximately 25%.

[0079] Subsequently, the server utilizes a target speech recognition model to perform speech recognition on each valid audio segment, obtaining the text of each valid audio segment. This target speech recognition model is trained using a railway-specific terminology database, significantly improving the recognition accuracy for railway-related terms such as "switch machine," "track maintenance window," and "train control system." In actual testing, a 20-minute recording was processed in approximately 15 seconds during the speech recognition stage, achieving an accuracy rate of 96.3%. Next, the server performs speaker separation based on each valid audio segment and its text. For example, through voiceprint feature extraction and clustering algorithms, it successfully distinguished the speeches of five participants, including General Manager Wang, Engineer Li, and Manager Zhang, and labeled the transcribed text with speaker tags, forming initial texts with speaker tags such as, "General Manager Wang: Regarding the rail replacement plan for section K205, I suggest implementation during the third track maintenance window this month. Engineer Li: I agree with General Manager Wang's opinion, but we need to coordinate with the materials department in advance to prepare spare parts."

[0080] Next, the server utilizes a locally deployed large language model to extract key information from the meeting through a multi-round inference strategy. The first round, based on the initial text, identifies the meeting topic as "Review of the Rail Replacement Plan for Section K205," with 5 participants, a meeting date of August 15, 2025, and includes three main topics: discussion of the replacement plan, coordination of material preparation, and construction schedule, as well as two decisions and four pending tasks, thus obtaining the meeting elements. The second round verifies and supplements the meeting elements, obtaining updated meeting elements. The third round generates a structured meeting text based on the updated meeting elements.

[0081] Finally, based on the meeting text, the server selects a target template that matches it from multiple preset meeting minutes models, such as the "Technical Review Meeting" template. Based on the meeting text, it retrieves target business knowledge from the business knowledge base, such as the relevant procedures and clauses of the "Rail Replacement Operation Standard". Then, according to the target template, based on the meeting text and the target business knowledge, it generates meeting minutes. The meeting minutes include basic meeting information, discussion records, decision-making matters, and a list of to-do items.

[0082] The entire process of the server generating meeting minutes was completed within 32 seconds. The generated meeting minutes received a logical score of 4.9 and an accuracy rate of 97.8%. The server then sends the meeting minutes to the terminal, where users can preview, adjust, and download the minutes on the system page.

[0083] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0084] To perform the corresponding steps in the above embodiments and various possible methods, an implementation of a meeting minutes generation device is given below. Please refer to... Figure 5 This is a functional block diagram of the meeting minutes generation device provided in this application embodiment. It should be noted that the basic principle and technical effects of the meeting minutes generation device 300 provided in this embodiment are the same as those in the above embodiments. For the sake of brevity, any parts not mentioned in this embodiment can be referred to the corresponding content in the above embodiments. The meeting minutes generation device 300 includes:

[0085] Extraction module 310 is used to extract multiple valid audio segments from conference audio.

[0086] The processing module 320 is used to perform speech recognition on each valid audio segment using a preset target speech recognition model to obtain the text of each valid audio segment; wherein, the target speech recognition model is matched with the business type to which the conference audio belongs; based on each valid audio segment and its text, an initial text with speaker tags is generated; using a preset language model, conference elements are extracted from the initial text, and a structured conference text is generated based on the conference elements.

[0087] The generation module 330 is used to generate meeting minutes corresponding to the meeting audio based on the meeting text and a business knowledge base preset for the business type.

[0088] Optionally, the extraction module 310 is specifically used for: preprocessing the conference audio to obtain initial audio; using a preset speech detection model to perform human voice detection on each audio frame in the initial audio, obtaining the detection probability of each audio frame, the detection probability being used to represent the probability that the audio frame contains human voice; selecting multiple candidate audio frames from multiple audio frames whose detection probability is greater than a preset first threshold; selecting multiple target audio frames from multiple candidate audio frames whose detection probability is greater than a preset second threshold; wherein the second threshold is greater than the first threshold; and obtaining multiple valid audio segments based on the multiple target audio frames; wherein a valid audio segment contains multiple temporally consecutive target audio frames.

[0089] Optionally, the processing module 320 is specifically used for: performing voiceprint recognition on each valid audio segment to obtain each speaking segment and its speaker in the valid audio segment; determining the paragraph corresponding to the speaking segment in the text of each speaking segment, and marking the speaker identifier of the speaking segment in the paragraph to obtain the labeled text of the valid audio segment; and clustering based on the labeled text of multiple valid audio segments to obtain the initial text with speaker labels.

[0090] Optionally, the processing module 320 is specifically used to: use a language model to generate multiple element information based on the initial text and multiple preset element types according to a preset first prompt word, to obtain meeting elements containing multiple element information; use a language model to check and update the meeting elements based on the initial text according to a preset second prompt word, to obtain updated meeting elements; and use a language model to generate structured meeting text based on the updated meeting elements according to a preset third prompt word.

[0091] Optionally, the processing module 320 is specifically used to: select a target template that matches the meeting text from a set of preset meeting minutes templates; obtain target business knowledge associated with the meeting text from a set of business knowledge from a business knowledge base; and generate meeting minutes corresponding to the meeting audio based on the meeting text and the target business knowledge, according to the target template.

[0092] Optionally, the meeting minutes generation device 300 further includes a training module 340, which is used to acquire a set of business terms for the business type and multiple historical meeting audios; divide each historical meeting audio into multiple audio segments, and annotate the content of each audio segment according to the set of business terms to obtain a text label for each audio segment; and train a basic speech recognition model based on multiple audio segments and their text labels to obtain a target speech recognition model.

[0093] The modules in the aforementioned meeting minutes generation device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of the computer device in software form, so that the processor can call and execute the operations corresponding to each module.

[0094] This application also provides a computer device, which can be a terminal or a server, and its internal structure diagram can be as follows. Figure 6 As shown, the computer device includes a processor, memory, input / output interfaces, a communication interface, a display unit, and an input device. The processor, memory, and input / output interfaces are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interfaces. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interfaces are used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a method for generating meeting minutes.

[0095] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0096] This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the meeting minutes generation method disclosed in this application.

[0097] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the meeting minutes generation method disclosed in this application.

[0098] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0099] In summary, the meeting minutes generation method, apparatus, equipment, storage medium, and program products provided in this application have the following technical effects: (1) High speech recognition accuracy. In actual testing, a 20-minute audio file completes format conversion, speech transcription, and minutes generation within 32 seconds, with a speech recognition accuracy of 96.3%, and an accuracy of over 93.8% in noisy environments. Enhanced by a professional terminology dictionary, the accuracy of railway industry professional terminology recognition is significantly improved. (2) Precise speaker differentiation. In small to medium-sized meetings with 4 to 8 people, the speaker differentiation accuracy reaches over 95%, accurately distinguishing the content of different speakers and achieving a complete record of "who said what at what time". (3) Good noise filtering effect. Enhanced by a speech detection model, the background noise filtering rate reaches 98.5%, significantly improving the quality of speech transcription and the accuracy of key content extraction. (4) Complete and accurate content extraction. The generated meeting minutes cover complete information such as meeting theme, important topics, key decisions and to-do items. The content logic score averages 4.9 points and the accuracy rate is 97.8%. (5) High processing efficiency. The time for compiling minutes that used to take several hours is reduced to minutes. The processing time for a 20-minute meeting recording is only 32 seconds. The system supports efficient multi-threaded concurrent processing and can process 20 audio tasks at the same time. (6) Knowledge integration enhances professionalism. Through deep integration with the enterprise knowledge base, the minutes content can be automatically associated with relevant business documents, technical procedures and historical cases, which enhances the professional value of the minutes and the compliance of the decision-making basis. (7) Strong security. The entire process is deployed locally. All data processing is completed within the enterprise, which meets the stringent requirements for data security and compliance in key infrastructure fields such as energy and transportation.

[0100] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0101] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for generating meeting minutes, characterized in that, The method includes: Extract multiple valid audio segments from the meeting audio; Using a preset target speech recognition model, speech recognition is performed on each of the valid audio segments to obtain the text of each valid audio segment; wherein, the target speech recognition model is matched with the business type to which the conference audio belongs; Based on each valid audio segment and its text, generate initial text with speaker tags; Using a preset language model, meeting elements are extracted from the initial text, and structured meeting text is generated based on the meeting elements; Based on the meeting text and a business knowledge base preset for the business type, a meeting summary corresponding to the meeting audio is generated.

2. The method according to claim 1, characterized in that, The extraction of multiple valid audio segments from the conference audio includes: The conference audio is preprocessed to obtain initial audio; Using a preset speech detection model, human voice detection is performed on each audio frame in the initial audio to obtain the detection probability of each audio frame. The detection probability is used to represent the probability that the audio frame contains human voice. From the plurality of audio frames, select a plurality of candidate audio frames whose detection probability is greater than a preset first threshold; From the candidate audio frames, select a plurality of target audio frames whose detection probability is greater than a preset second threshold; wherein the second threshold is greater than the first threshold; Based on multiple target audio frames, multiple valid audio segments are obtained; wherein, one valid audio segment contains multiple time-continuous target audio frames.

3. The method according to claim 1, characterized in that, The step of generating initial text with speaker tags based on each valid audio segment and its text includes: For each of the valid audio segments, voiceprint recognition is performed on the valid audio segment to obtain each speech segment and its speaker in the valid audio segment; For each of the aforementioned speech segments, the paragraph corresponding to the speech segment is determined in the text of the valid audio segment, and the speaker's identifier for the speech segment is marked in the paragraph to obtain the marked text of the valid audio segment; Clustering is performed on the labeled text of multiple valid audio segments to obtain initial text with speaker labels.

4. The method according to claim 1, characterized in that, The step of extracting meeting elements from the initial text using a preset language model and generating structured meeting text based on the meeting elements includes: Using the language model, based on the preset first prompt word, multiple element information is generated from the initial text and multiple preset element types to obtain meeting elements containing the multiple element information; Using the language model, the meeting elements are checked and updated based on the initial text according to the preset second prompt word, resulting in updated meeting elements; Using the language model, a structured meeting text is generated based on the updated meeting elements and a preset third prompt word.

5. The method according to claim 1, characterized in that, The process of generating meeting minutes corresponding to the meeting audio based on the meeting text and a pre-set business knowledge base for the business type includes: Select the target template that matches the meeting text from a set of preset meeting minutes templates; Obtain the target business knowledge associated with the meeting text from multiple business knowledge sources in the business knowledge base; Based on the target template, the meeting minutes corresponding to the meeting audio are generated according to the meeting text and the target business knowledge.

6. The method according to claim 1, characterized in that, The target speech recognition model was trained in the following manner: Obtain the business terminology set for the aforementioned business type and multiple historical meeting audio recordings; Each of the historical meeting audio segments is divided into multiple audio segments, and the content of each audio segment is labeled according to the business terminology set to obtain a text tag for each audio segment; The basic speech recognition model is trained based on multiple audio segments and their text labels to obtain the target speech recognition model.

7. A meeting minutes generation device, characterized in that, The device includes: The extraction module is used to extract multiple valid audio segments from the conference audio. The processing module is used to perform speech recognition on each of the effective audio segments using a preset target speech recognition model to obtain the text of each effective audio segment; wherein the target speech recognition model is matched with the business type to which the conference audio belongs; generate initial text with speaker tags based on each effective audio segment and its text; extract conference elements from the initial text using a preset language model, and generate structured conference text based on the conference elements; The generation module is used to generate meeting minutes corresponding to the meeting audio based on the meeting text and a business knowledge base preset for the business type.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images