Conference management method and system based on natural language processing and retrieval enhancement

Abstract

The invention provides a conference management method and system based on natural language processing and retrieval enhancement, and belongs to the technical field of conference management, and the method comprises the steps: outputting a conference reservation result based on key parameters; if the reservation is successful, acquiring audio stream data in the conference process, and performing voice segmentation, initial voiceprint extraction and initial clustering on the audio stream data to output an original voice segment and speaker reference voice; target speaker extraction, accurate voiceprint extraction and final clustering are carried out based on the original voice segment and the speaker reference voice to obtain a pure voice text stream with a voiceprint label; performing dynamic abstract generation, enhanced retrieval and verification on the pure voice text stream; according to the conference management system and method, the intelligent management of the whole process of the conference can be covered, and meanwhile, the whole process management of the conference from voice-driven reservation, real-time content analysis to automatic task distribution is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of conference management, and in particular relates to a conference management method and system based on natural language processing and retrieval enhancement. Background Art

[0002] In modern collaborative scenarios, efficient meeting management is crucial to improving organizational effectiveness. Existing systems have significant deficiencies in intelligence and in-depth information processing: 1. The reservation process is cumbersome: It relies on manual queries and form filling, lacks natural language understanding capabilities, and cannot achieve accurate and fast conference room matching through voice commands; 2. Inefficient information processing during meetings: Key content relies on manual recording, with a lack of real-time automated summarization capabilities, and voiceprint recognition cannot accurately link speech content with speakers; 3. Insufficient post-meeting knowledge utilization: Meeting minutes become "information islands," making it difficult to achieve in-depth semantic analysis through traditional keyword searches. This makes it difficult to reuse information such as historical decisions and task commitments. 4. Delayed implementation of meeting conclusions: The system lacks the ability to extract tasks, requiring manual analysis and generation of to-do items after the meeting. This results in delayed task assignment, unclear execution nodes, and slow resolution implementation.

[0003] Therefore, there is an urgent need for an innovative solution that integrates natural language processing, voiceprint recognition, and enhanced retrieval to achieve a conference management method from voice-driven booking, real-time content analysis to automatic task distribution. Summary of the Invention

[0004] In order to solve the above technical problems, the present invention provides a conference management method and system based on natural language processing and retrieval enhancement to solve the problems in the above background technology.

[0005] In a first aspect, the invention provides the following technical solution: a conference management method based on natural language processing and retrieval enhancement, comprising: Obtaining a conference reservation instruction issued by a user, extracting key parameters from the conference reservation instruction, and outputting a conference reservation result based on the key parameters; If the conference reservation result is successful, obtain audio stream data during the conference, perform voice segmentation, initial voiceprint extraction and initial clustering on the audio stream data to output original voice segments and speaker reference voice; Performing target speaker extraction, precise voiceprint extraction, and final clustering based on the original voice segment and the speaker reference voice to obtain a clean voice text stream with voiceprint labels; Performing dynamic summary generation, enhanced retrieval, and conflict error checking on the clean voice text stream to obtain conference information; The meeting information is subjected to responsibility identification and voiceprint binding to output final meeting management data.

[0006] Compared with the existing technology, the present application has the following beneficial effects: This application proposes a voice-driven intelligent conference booking and interaction innovation. By parsing the voice / text command booking mode, users can complete conference room reservations through natural language. It can automatically extract key parameters and proactively ask questions when information is incomplete, completely breaking away from the reliance on traditional forms. At the same time, it can query the conference room system status in real time and intelligently recommend the optimal conference room solution, ensuring the immediacy and accuracy of the reservation results. To address the problems of voiceprint mixing and clustering errors when multiple people speak at the same time in traditional methods, an innovative extraction model is introduced. It can efficiently extract the pure voice of the target speaker from the mixed speech based on the reference voice of the target speaker. Iterative voiceprint enhancement and clustering: Through the iterative process of initial clustering, reference voice selection, extraction, and precise clustering, the discrimination and purity of voiceprint features are significantly improved, and the accuracy of speaker clustering and recognition in scenarios with multiple people speaking at the same time is greatly improved. Finally, a meeting summary is dynamically generated, retaining resolutions and core data, and can be automatically filled into a Markdown structured template. The present invention can accurately identify the responsible verbs and related elements in the speech and directly bind the speaker's voiceprint ID to the task triplet to achieve speaking and responsibility attribution, avoiding task assignment ambiguity.

[0007] Preferably, the step of extracting key parameters from the conference reservation instruction and outputting the conference reservation result based on the key parameters includes: Using a preset intent parsing algorithm to extract key parameters from the meeting reservation instruction; Determining whether the key parameters are complete; if the key parameters are complete, taking the key parameters as complete parameters; if the key parameters are incomplete, questioning the user based on the context of the conference reservation instruction and the matching slot information to obtain complete parameters; Based on the complete parameters, an optimal solution is output through target recommendation technology, and the optimal solution is published in the form of MCP to output a meeting reservation result.

[0008] Preferably, the key parameters include time, number of people and equipment requirements.

[0009] Preferably, the steps of performing speech segmentation, initial voiceprint extraction and initial clustering on the audio stream data to output original speech segments and speaker reference speech include: detecting speech segments and silence segments in the audio stream data, and segmenting the audio stream data based on the speech segments and the silence segments by fixed duration or silence intervals to obtain a plurality of original speech segments; Performing a first voiceprint feature extraction on the original speech segment to obtain an initial voiceprint segment; Calculating a first cosine similarity between the initial voiceprint segments, and performing a first clustering process on the initial voiceprint segments based on the first cosine similarity and using a Kmeans clustering method to obtain a plurality of initial clusters; Speech screening is performed in each of the initial clusters according to a preset threshold to obtain a speaker reference speech.

[0010] Preferably, the step of performing target speaker extraction, accurate voiceprint extraction, and final clustering based on the original voice segment and the speaker reference voice to obtain a clean voice text stream with voiceprint labels includes: Obtaining a pre-trained extraction model, inputting the original speech segment and the speaker reference speech into the pre-trained extraction model, and using the original speech segment as a mixed input and the speaker reference speech as a conditional input to obtain a pure target person speech segment; Performing a second voiceprint feature extraction on the clean target person's voice segment to obtain an accurate voiceprint segment; Calculating a second cosine similarity between the precise voiceprint segments, and performing a second clustering process on the precise voiceprint segments based on the second cosine similarity using a Kmeans clustering method to obtain a plurality of precise clusters; The start time and end time of each speaker are located according to the precise clustering clusters and a unique speaker role is assigned to each precise clustering cluster to obtain a clean speech text stream with voiceprint labels.

[0011] Preferably, the step of performing dynamic summary generation, enhanced retrieval, and conflict error checking on the clean voice text stream to obtain conference information includes: Generating a real-time summary of the clean speech text using a dynamic summary method to obtain a dynamic summary; Using a Siamese network, the dynamic summary is input as a whole into a pre-trained BERT model for enhanced retrieval to output a high-dimensional sentence vector; The sentence vector is compared with a preset authoritative data source in real time and potential conflicts and errors are warned to output verification result data, which is automatically filled into a Markdown template to obtain meeting information.

[0012] Preferably, the step of performing responsibility identification and voiceprint binding on the conference information to output final conference management data includes: Input the meeting information into the pre-trained BiLSTM-CRF model to output the target task and responsible person labels, and perform logical judgment on the responsible person labels; If the responsible person tag identifies the speaker himself, the identity corresponding to the voiceprint ID corresponding to the current speaker will be used as the person responsible for the task; If the responsible person tag is identified as someone other than the speaker, the identification result of the responsible person tag is used as the person responsible for the task, and the voiceprint ID of the current speaker is recorded as the assignor or authorizer of the task to output the meeting task data; The conference information and the conference task data are integrated into a structured task triplet and attached with the voiceprint ID and conference source to output the final conference management data.

[0013] In a second aspect, the invention provides the following technical solution: a conference management system based on natural language processing and retrieval enhancement, the system comprising: A reservation module is used to obtain a conference reservation instruction issued by a user, extract key parameters from the conference reservation instruction, and output a conference reservation result based on the key parameters; A reference module is configured to, if the conference reservation result is successful, obtain audio stream data during the conference, perform voice segmentation, initial voiceprint extraction and initial clustering on the audio stream data, and output original voice segments and speaker reference voices; A clean module, configured to perform target speaker extraction, precise voiceprint extraction, and final clustering based on the original speech segment and the speaker reference speech, so as to obtain a clean speech text stream with voiceprint labels; A retrieval module, configured to perform dynamic summary generation, enhanced retrieval, and conflict error checking on the clean voice text stream to obtain conference information; The output module is used to perform responsibility identification and voiceprint binding on the conference information to output final conference management data.

[0014] In a third aspect, the present invention provides the following technical solution: a computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor; when the processor executes the computer program, the conference management method based on natural language processing and retrieval enhancement as described above is implemented.

[0015] In a fourth aspect, the present invention provides the following technical solution: a storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the conference management method based on natural language processing and retrieval enhancement as described above. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A flowchart of a conference management method based on natural language processing and retrieval enhancement provided in Example 1 of the present invention; Figure 2 This is a structural block diagram of a conference management system based on natural language processing and retrieval enhancement provided in Example 2 of the present invention; Figure 3 A schematic diagram of the hardware structure of a computer provided in another embodiment of the present invention.

[0018] The present invention will be further described below with reference to the accompanying drawings and their descriptions. DETAILED DESCRIPTION

[0019] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0020] In the description of the embodiments of the present invention, it should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc., indicating the orientation or position relationship, are based on the orientation or position relationship shown in the accompanying drawings, and are only for the convenience of describing the embodiments of the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as limiting the present invention.

[0021] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of the technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present invention, "plurality" means two or more, unless otherwise specifically defined.

[0022] In the embodiments of the present invention, unless otherwise expressly specified or limited, the terms "installed," "connected," "connected," "fixed," etc. should be understood in a broad sense. For example, they may refer to fixed connection, detachable connection, or integration; mechanical connection or electrical connection; direct connection or indirect connection through an intermediate medium; internal communication between two components or interaction between two components. Those skilled in the art will understand the specific meanings of the above terms in the embodiments of the present invention based on specific circumstances.

[0023] Example 1 In one embodiment of the present invention, Figure 1 As shown, a conference management method based on natural language processing and retrieval enhancement includes: S1. Obtaining a conference reservation instruction issued by a user, extracting key parameters from the conference reservation instruction, and outputting a conference reservation result based on the key parameters; Specifically, for the meeting reservation instruction, the instruction here can be a voice instruction or a text instruction, such as "a meeting room for 10 people at 10 o'clock tomorrow". Here, "tomorrow", "10 o'clock" and "10 people" are all key parameters. The corresponding meeting reservation result can be output through the key parameters. There are two situations for the meeting reservation result, one is a successful reservation, such as "a meeting of 10 has been reserved for you at 10 o'clock tomorrow", etc., and the other is a failed reservation, such as "the meeting room you reserved has been reserved at 10 o'clock tomorrow", etc.

[0024] Wherein, the step S1 includes: S11, extracting key parameters from the meeting reservation instruction using a preset intent parsing algorithm; Among them, the key parameters include time, number of people and equipment requirements. The preset intention parsing algorithm here is specifically a pre-trained LLM model in the existing technology. The pre-trained LLM model can be obtained by inputting data from the conference field into the model for training.

[0025] S12. Determine whether the key parameters are complete. If the key parameters are complete, use the key parameters as complete parameters. If the key parameters are incomplete, question the user based on the context of the conference reservation instruction and the matching slot information to obtain complete parameters. Specifically, if there are incomplete key parameters, follow-up questions will be asked based on the context and matching slot information, such as "Please confirm the meeting topic", "Please confirm the meeting time", "Please confirm the number of meeting participants", etc. That is, for key parameters, they should include meeting time, number of meeting participants, equipment requirements, etc., and may also include the meeting topic. If the extracted key parameters are missing one or more of them, it is judged that the key parameters are incomplete and need to be asked.

[0026] S13. Outputting an optimal solution based on the complete parameters through target recommendation technology, and publishing the optimal solution in the form of MCP to output a meeting reservation result; Specifically, in this step, the conference assistant agent obtains the optimal solution recommended in real time by the conference room system through function calling technologies such as MCP, publishes the optimal solution in the form of MCP, intelligently avoids conflicting meeting locations and meeting times, and then outputs the meeting reservation results. The target recommendation technology here is the pre-trained MCP Tongyi model, which can be obtained by inputting data from the conference field into the model for training. At the same time, the optimal solution output here is the recommended optimal conference room. At the same time, the optimal solution is published in the form of MCP, which can provide content to end users accurately and efficiently by integrating, coordinating and allocating upstream and downstream resources.

[0027] S2. If the conference reservation result is successful, obtain audio stream data during the conference, perform voice segmentation, initial voiceprint extraction and initial clustering on the audio stream data to output original voice segments and speaker reference voice; Wherein, the step S2 includes: S21, detecting speech segments and silence segments in the audio stream data, and segmenting the audio stream data based on the speech segments and the silence segments by fixed duration or silence intervals to obtain a plurality of original speech segments; Specifically, the VAD method can be used here to complete the detection of speech segments and silence segments, and then the speech segments detected by VAD are divided into shorter speech segments, and then divided by fixed duration or based on silence intervals to obtain original speech segments. These original speech segments may contain single-person speech or multiple people speaking at the same time.

[0028] S22. Performing a first voiceprint feature extraction on the original voice segment to obtain an initial voiceprint segment; Specifically, a pre-trained wav2Emb2 model may be used here to extract the first voiceprint feature. The pre-trained wav2Emb2 model may be trained by inputting voice segment data.

[0029] S23, calculating a first cosine similarity between the initial voiceprint segments, and performing a first clustering process on the initial voiceprint segments based on the first cosine similarity and using a Kmeans clustering method to obtain a plurality of initial clusters; Specifically, by classifying the initial voiceprint segments whose first cosine similarity is greater than the similarity threshold into the same cluster and clustering them using the Kmeans clustering method, an initial cluster can be obtained. In this step, the purpose of clustering by the first cosine similarity is a pre-clustering process. In order to pre-classify the initial voiceprint segments with a relatively close distance into the same cluster, and then cluster them using the Kmeans clustering method, a more accurate clustering result can be obtained, and the time required for clustering can also be shortened.

[0030] S24, performing speech screening in each of the initial clusters according to a preset threshold to obtain a speaker reference speech; Specifically, the purpose of screening is to select one or more relatively pure voice segments with high confidence as the speaker's reference voice. During the voice screening process, each voiceprint segment in each initial cluster has a corresponding confidence level. The confidence level is compared with a preset threshold, and the voiceprint segments with a confidence level higher than the preset threshold are used as the speaker's reference voice. The voice segments screened out according to the confidence level are relatively pure voice segments with high confidence levels. Among them, in the actual clustering process, when a segment is assigned to a cluster, its distance / similarity score with the cluster center (or the representative point within the cluster) can itself be used as the confidence judgment. If there are multiple judgments above the preset threshold, multiple segments can be selected as the speaker reference speech. Because it is mentioned above that one or more speech segments with high confidence and relatively pureness are screened out as the speaker reference speech, it can be multiple segments.

[0031] S3. Performing target speaker extraction, precise voiceprint extraction, and final clustering based on the original voice segment and the speaker reference voice to obtain a clean voice text stream with voiceprint labels; Specifically, the target speaker extraction here is to extract the speech segments corresponding to each speaker, that is, the speech data of the target speaker during the meeting.

[0032] Wherein, the step S3 includes: S31, obtaining a pre-trained extraction model, inputting the original speech segment and the speaker reference speech into the pre-trained extraction model, and using the original speech segment as a mixed input and the speaker reference speech as a conditional input to obtain a pure target person speech segment; Specifically, traditional methods can create a mixed voiceprint when multiple people speak simultaneously, leading to clustering errors. To optimize this process, the present invention introduces a pre-trained SpEx+ model algorithm. SpEx+ extracts the target speaker's pure voice from a mixed speech using a reference speech segment. After extracting a purer single-speaker speech segment through SpEx+, voiceprint feature extraction can be performed to obtain a more discriminative voiceprint embedding, thereby improving clustering and recognition accuracy. After the initial clustering, the more reliable clustering result can be selected as the SpEx+ reference speech for a second round of extraction and clustering, further improving the results.

[0033] Specifically, the extraction model here is the SpEx+ model, which can be trained by inputting speech data. After using the SpEx+ model, the pure speech of the target speaker corresponding to the reference speech can be extracted from the mixed input. Therefore, for an original segment that may contain the voices of multiple people, an attempt is made to extract the voice of each known speaker in the segment.

[0034] S32, performing a second voiceprint feature extraction on the clean target person's voice segment to obtain an accurate voiceprint segment; Specifically, for the pure target person voice segments of each target that are now purer and extracted by the SpEx+ model, voiceprint feature extraction is performed again. The precise voiceprint segments extracted this time will be more accurate than those extracted in step S22 because the interfering sounds (other speakers) have been greatly weakened.

[0035] S33, calculating a second cosine similarity between the precise voiceprint segments, and performing a second clustering process on the precise voiceprint segments based on the second cosine similarity using a Kmeans clustering method to obtain a plurality of precise clusters; Specifically, the similarity calculation process and clustering process here are the same as step S23, but this step can perform more accurate speaker clustering. Due to the improvement of voiceprint quality and the effective separation of overlapping speech, the clustering result will be significantly better than the preliminary clustering.

[0036] S34, locating the start time and end time of each speaker according to the precise clustering cluster and assigning a unique speaker role to each precise clustering cluster to obtain a clean speech text stream with voiceprint labels; Specifically, the start and end time points (transition points) of each speaker are accurately located based on the precise clustering clusters, and a unique speaker role is assigned to each precise cluster, such as role 1, role 2, ..., role N. The assigned speaker role at this time is the voiceprint label. Each precise cluster has a corresponding voiceprint label, i.e., the speaker role.

[0037] S4, performing dynamic summary generation, enhanced retrieval, and conflict error checking on the clean voice text stream to obtain conference information; Specifically, in this step, the purpose of enhanced retrieval is to overcome the limitations of traditional keyword retrieval and to understand the user's intentions at a deeper level, while verification is to avoid conflicts in meeting content.

[0038] Wherein, the step S4 includes: S41, using a dynamic summarization method to generate a real-time summary of the clean speech text to obtain a dynamic summary; Specifically, here, summaries are generated in real time based on the pre-trained T5 model, retaining the resolution / data core. The T5 model can be trained by inputting text data. The dynamic summarization method here is to generate summaries in real time through the pre-trained T5 model.

[0039] S42. Using a twin network, the dynamic summary is input as a whole into a pre-trained BERT model for enhanced retrieval to output a high-dimensional sentence vector; Specifically, in this step, sentence vectors are output through the semantic layer. The semantic layer is the core of the "enhanced closed-loop search" function. Its fundamental goal is to break through the limitations of traditional keyword search and achieve a deep understanding and intelligent matching of historical meeting decisions. This ensures that not only can the text be "understood", but also the true intention behind it can be "understood", thereby providing the most valuable historical information for current issues. Traditional word embedding models rely primarily on vocabulary overlap. This significantly reduces retrieval performance when different expressions have similar meanings (e.g., "budget approval" versus "financial appropriation resolution"). This paper addresses this pain point through its unique "Siamese Network" architecture. Rather than encoding individual words, it treats entire sentences or phrases as a whole and feeds them into a specially fine-tuned BERT network. Fine-tuning involves training the BERT network with conference-related data, ultimately generating a high-dimensional, fixed-length vector (i.e., a "sentence embedding"). The Siamese network here is a pre-trained network trained by feeding it data from the conference domain. The Siamese network encodes sentences as a whole, with the input and output of the Siamese network being the dynamic summary and the encoded vector after encoding the dynamic summary as a whole, respectively. First, the model is labeled with the similarity / dissimilarity relationship (or specific similarity score) between sentence pairs, rather than a pre-calculated "correct" sentence vector. Second, the two branches of the Siamese network generate sentence vectors E1 and E2, which serve as intermediate representations. The goal of training is to ensure that a metric (distance, similarity) between E1 and E2 accurately reflects the semantic relationship labels between the input sentence pairs. Backpropagation adjusts the parameters of the shared BERT encoder by minimizing a loss function based on the relationship labels. This process gradually shapes the entire vector space: semantically similar sentences (in the context of conferences) are mapped to nearby points in the vector space, while semantically dissimilar sentences are mapped to distant points.

[0040] S43: Compare the sentence vector with a preset authoritative data source in real time and issue warnings for potential conflicts and errors to output verification result data, and automatically fill the Markdown template with the verification result data to obtain meeting information; Specifically, for the verification process, by comparing the key data mentioned in the meeting with the authoritative data source within the company in real time, ensuring that decisions are based on accurate information, and issuing early warnings for potential conflicts and errors, thus forming a closed loop of information verification, and then automatically filling in the Markdown template (topic / resolution / to-do), and outputting the meeting information. The potential conflicts and errors here can be determined by comparing the sentence vector with the data in the preset authoritative database. If there is a large error in the logic, timeline, and context understanding between the two, it is considered that there are potential conflicts and errors. The Markdown template here can contain various Markdown elements, such as titles, lists, links, pictures, code blocks, etc., and can be customized as needed to help users quickly create documents that conform to a specific format or structure. At the same time, the authoritative data source here mainly includes The authoritative data of the target enterprise can be extracted from the enterprise's internal database. During the actual comparison process, the sentence vectors are converted into structured entities and the entity relationships therein are identified. Then, actual business rules and temporal logic are defined to detect conflicts. For example, if the project status in the entity is mentioned as completed, but the project status in the authoritative data is in progress, the sales target is greater than the market capacity limit in the authoritative data, and the task is assigned to person A, but person A in the authoritative data has resigned, these are all logical conflicts. For example, if the new deadline of project A in the entity is earlier than the planned completion date of the predecessor project of project A, the activity start date is earlier than the resource approval date, and the event occurrence time is not within the valid time period of the event in the authoritative data, these are all timeline conflicts. The events and dependencies stored in the authoritative data are used to check timeline consistency and causal logic (for example, task C must start after task B is completed). When the entity relationship in the entity does not match the real time in the authoritative data, it is a context understanding conflict. In actual situations, the verification result data will retain the successfully verified data, that is, the data without errors and conflicts. These data contain meeting information, and these data can be filled into the Markdown template. The Markdown template is used to help users quickly create documents that conform to a specific format or structure.

[0041] S5. Perform responsibility identification and voiceprint binding on the conference information to output final conference management data; Wherein, the step S5 includes: S51: Input the meeting information into a pre-trained BiLSTM-CRF model to output target tasks and responsible person labels, and perform logical judgment on the responsible person labels; The meeting information here specifically refers to the information in the aforementioned Markdown template. The pre-trained BiLSTM-CRF model is trained by inputting data from the meeting domain. The speech clips obtained in the above steps are processed by the pre-trained BERT model to generate text data. After subsequent processing, the text-based meeting information is output. The target task here specifically refers to the specific content of an event mentioned in the meeting, which is part of the meeting task data. The target task is not output by the model but rather exists within the meeting information itself.

[0042] Specifically, the context of task assignment often depends on the context. For example, "I suggest that Role A be responsible for this matter" and "The project Role A was responsible for yesterday progressed smoothly" are text messages from meeting information, and the two "responsible" have completely different meanings. BiLSTM can read the entire sentence from both the front-end and back-end directions, more accurately understanding the true meaning of responsibility verbs such as "responsible" and "follow up" in the current context and determining whether they constitute a new task. The CRF layer can learn the dependencies between annotations (for example, the "task description-start" label is more likely to be followed by the "task description-internal" label rather than the "deadline" label). This ensures the consistency and accuracy of entity extraction, avoids illogical label combinations, and significantly improves the overall robustness of the model. In this example, the responsible person label is the identity information or specific role identified in the meeting information. The annotation here is determined by the BiLSTM reading the entire sentence from both the front-end and back-end directions to more accurately understand the true meaning of responsibility verbs such as "responsible" and "follow up" in the current context. After understanding the true meaning of the current context, the corresponding annotation is generated.

[0043] Among them, the BiLSTM-CRF model is specifically used to take the meeting information as the input of the bidirectional LSTM, perform sequence modeling through the bidirectional LSTM, obtain the hidden state representation of each position, and finally use the CRF layer to perform global sequence labeling to obtain the final labeling result. The BiLSTM-CRF model is a model in the prior art, and its structure is the same as that of the model in the prior art, so it will not be repeated here. The input and output of the BiLSTM-CRF model are the text data and annotations in the meeting data, respectively. The annotation here is the responsible person label.

[0044] S52: If the responsible person tag identifies the speaker as the speaker, the identity corresponding to the voiceprint ID corresponding to the current speaker is used as the responsible person for the task; Specifically, this step is a process of self-acknowledgement. If the T-PER tag (responsible person tag) identifies "I" or "we", or if no other responsible person is clearly identified in the sentence (the speaker himself is assumed to be the speaker in the context), the employee identity corresponding to the current speaker's voiceprint ID will be confirmed as the person responsible for the task. The voiceprint ID here is equivalent to the voiceprint tag mentioned above, that is, the speaker role.

[0045] S53. If the responsible person tag identifies someone other than the speaker, the identification result of the responsible person tag is used as the person responsible for the task, and the voiceprint ID of the current speaker is recorded as the assignor or authorizer of the task to output the meeting task data. Among them, if the T-PER tag recognizes a specific name (such as "Role A") or department (such as "Department B"), the system will confirm "Role A" or "Department B" as the person responsible for the task, and record the voiceprint ID of the current speaker as the "assigner" or "authorizer" of the task.

[0046] S54: Integrate the meeting information and the meeting task data into a structured task triplet and attach the voiceprint ID and the meeting source to output final meeting management data; Specifically, for the task triplet, it includes [task description, responsible person, deadline], and is accompanied by metadata such as the authorized person's voiceprint ID and the source of the meeting. For the final conference management data output, it includes structured meeting minutes and signed task cards. The source of the meeting here can be manually determined during the process of booking a conference room, which facilitates the subsequent traceability process. The task description is the target task. At the same time, the conference task data also includes the time information of the task, such as the start time, duration, deadline, etc. Therefore, the task description and deadline can be obtained from the conference task data. It should be noted that in the above steps, the time information can be obtained from the speaker's voice data after being processed through the above series of steps, and it is generated together with the specific task information.

[0047] In summary, the conference management method based on natural language processing and retrieval enhancement in the above embodiments of the present invention proposes a voice-driven intelligent conference reservation and interaction innovation. By parsing the voice / text command reservation mode, users can complete conference room reservations through natural language, automatically extract key parameters, and actively ask questions when the information is incomplete, completely getting rid of the traditional form reliance. At the same time, the conference room system status is queried in real time, and the optimal conference room plan is intelligently recommended to ensure the immediacy and accuracy of the reservation results. In response to the problems of voiceprint mixing and clustering errors when multiple people speak at the same time in traditional methods, an extraction model is innovatively introduced. It can efficiently extract the pure voice from the mixed voice based on the reference voice of the target speaker, iterative voiceprint enhancement and clustering: Through the iterative process of initial clustering, reference voice selection, extraction, and precise clustering, the discrimination and purity of voiceprint features are significantly improved, and the accuracy of speaker clustering and recognition in scenarios where multiple people speak at the same time is greatly improved. Finally, the meeting summary is dynamically generated, retaining resolutions and core data, and can be automatically filled into the Markdown structured template. The present invention can accurately identify the responsible verbs and related elements in the speech, and directly bind the speaker's voiceprint ID to the task triplet to realize the attribution of responsibility when speaking, avoiding task assignment ambiguity.

[0048] Example 2 like Figure 2 As shown, in a second embodiment of the present invention, a conference management system based on natural language processing and retrieval enhancement is provided, the system comprising: Reservation module 1, used to obtain a conference reservation instruction issued by a user, extract key parameters from the conference reservation instruction, and output a conference reservation result based on the key parameters; Reference module 2 is configured to obtain audio stream data during the conference if the conference reservation result is successful, perform voice segmentation, initial voiceprint extraction and initial clustering on the audio stream data, and output original voice segments and speaker reference voice; Clean module 3, used to perform target speaker extraction, accurate voiceprint extraction and final clustering based on the original voice segment and the speaker reference voice to obtain a clean voice text stream with voiceprint labels; Retrieval module 4, used for performing dynamic summary generation, enhanced retrieval and conflict error checking on the clean voice text stream to obtain conference information; The output module 5 is used to perform responsibility identification and voiceprint binding on the conference information to output final conference management data.

[0049] Wherein, the predetermined module 1 includes: An extraction submodule, configured to extract key parameters from the conference reservation instruction using a preset intent parsing algorithm; a judgment submodule, configured to judge whether the key parameters are complete; if the key parameters are complete, taking the key parameters as complete parameters; if the key parameters are incomplete, questioning the user based on the context of the conference reservation instruction and the matching slot information to obtain complete parameters; The publishing submodule is used to output the optimal solution based on the complete parameters and through the target recommendation technology, and publish the optimal solution in the form of MCP to output the meeting reservation result.

[0050] The reference module 2 includes: a detection submodule, configured to detect speech segments and silence segments in the audio stream data, and segment the audio stream data based on the speech segments and the silence segments by fixed duration or silence intervals to obtain a plurality of original speech segments; a first voiceprint extraction submodule, configured to extract a first voiceprint feature from the original speech segment to obtain an initial voiceprint segment; a first clustering submodule, configured to calculate a first cosine similarity between the initial voiceprint segments, and perform a first clustering process on the initial voiceprint segments based on the first cosine similarity using a Kmeans clustering method to obtain a plurality of initial clusters; The screening submodule is used to screen the speech in each of the initial clusters according to a preset threshold to obtain the speaker's reference speech.

[0051] The clean module 3 includes: An input submodule is used to obtain a pre-trained extraction model, input the original speech segment and the speaker reference speech into the pre-trained extraction model, and use the original speech segment as a mixed input and the speaker reference speech as a conditional input to obtain a pure target person speech segment; A second voiceprint extraction submodule is used to extract a second voiceprint feature from the clean target person's voice segment to obtain an accurate voiceprint segment; a second clustering submodule, configured to calculate a second cosine similarity between the precise voiceprint segments, and perform a second clustering process on the precise voiceprint segments based on the second cosine similarity using a Kmeans clustering method to obtain a plurality of precise clusters; The allocation submodule is used to locate the start time and end time of each speaker according to the precise clustering cluster and assign a unique speaker role to each precise clustering cluster to obtain a clean speech text stream with voiceprint labels.

[0052] The retrieval module 4 includes: A generating submodule, configured to generate a real-time summary of the clean speech text using a dynamic summarization method to obtain a dynamic summary; A vector generation submodule is used to use a Siamese network to input the dynamic summary as a whole into a pre-trained BERT model for enhanced retrieval to output a high-dimensional sentence vector; The early warning submodule is used to compare the sentence vector with the preset authoritative data source in real time and issue early warnings for potential conflicts and errors, output verification result data, and automatically fill the Markdown template with the verification result data to obtain meeting information.

[0053] The output module 5 includes: A logic submodule is used to input the meeting information into a pre-trained BiLSTM-CRF model to output target tasks and responsible person labels, and perform logical judgment on the responsible person labels; The first binding submodule is used to set the identity corresponding to the voiceprint ID corresponding to the current speaker as the person responsible for the task if the person responsible label is identified as the speaker himself; The second binding submodule is used to, if the responsible person tag is identified as someone other than the speaker, use the identification result of the responsible person tag as the person responsible for the task, and record the voiceprint ID of the current speaker as the assignor or authorizer of the task to output the meeting task data; The final output submodule is used to integrate the conference information and the conference task data into a structured task triple and attach the voiceprint ID and conference source to output the final conference management data.

[0054] In other embodiments of the present invention, the embodiments of the present invention provide the following technical solutions: a computer comprising a memory 102, a processor 101, and a computer program stored on the memory 102 and executable on the processor 101; the processor 101 implements the conference management method based on natural language processing and retrieval enhancement as described above when executing the computer program.

[0055] Specifically, the processor 101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured as one or more integrated circuits for implementing the embodiments of the present invention.

[0056] Memory 102 may include a large-capacity memory for data or instructions. By way of example, and not limitation, memory 102 may include a hard disk drive (HDD), a floppy disk drive, a solid-state drive (SSD), flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. Where appropriate, memory 102 may include removable or non-removable (or fixed) media. Where appropriate, memory 102 may be internal or external to the data processing device. In certain embodiments, memory 102 is non-volatile memory. In certain embodiments, memory 102 includes read-only memory (ROM) and random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable PROM (EEPROM), an electrically alterable ROM (EAROM) or a flash memory (FLASH), or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), where the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0057] The memory 102 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 101 .

[0058] The processor 101 implements the above-mentioned conference management method based on natural language processing and retrieval enhancement by reading and executing computer program instructions stored in the memory 102.

[0059] In some embodiments, the computer may further include a communication interface 103 and a bus 100. Figure 3 As shown, the processor 101 , the memory 102 , and the communication interface 103 are connected via a bus 100 and communicate with each other.

[0060] The communication interface 103 is used to implement communication between the various modules, devices, units, and / or equipment in the embodiments of the present invention. The communication interface 103 can also implement data communication with other components such as external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0061] Bus 100 includes hardware, software, or both, and couples components of a computer device to each other. Bus 100 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 100 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses, or a combination of two or more of these. Bus 100 may include one or more buses, where appropriate. Although embodiments of the present invention describe and illustrate a particular bus, the present invention contemplates any suitable bus or interconnect.

[0062] The computer can execute the conference management method based on natural language processing and retrieval enhancement of the present invention based on the acquired conference management system based on natural language processing and retrieval enhancement, thereby realizing conference management based on natural language processing and retrieval enhancement.

[0063] In some further embodiments of the present invention, in combination with the above-mentioned conference management method based on natural language processing and retrieval enhancement, the embodiments of the present invention provide the following technical solutions: a storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the above-mentioned conference management method based on natural language processing and retrieval enhancement.

[0064] Those skilled in the art will appreciate that the logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device), or in conjunction with such instruction execution system, apparatus, or device. For purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by an instruction execution system, apparatus, or device, or in conjunction with such instruction execution system, apparatus, or device.

[0065] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0066] It should be understood that various components of the present invention may be implemented using hardware, software, firmware, or a combination thereof. In the aforementioned embodiments, multiple steps or methods may be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one or a combination of the following technologies known in the art may be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field-programmable gate array (FPGA), etc.

[0067] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0068] The above-described embodiments merely represent several implementations of the present invention, and while their descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the patent. It should be noted that a person of ordinary skill in the art would be able to make numerous variations and improvements without departing from the spirit of the present invention, and these variations and improvements fall within the scope of protection of the present invention. Therefore, the scope of protection of the patent for this invention shall be determined by the appended claims.

Claims

1. A conference management method based on natural language processing and retrieval enhancement, characterized in that: include: Obtaining a conference reservation instruction issued by a user, extracting key parameters from the conference reservation instruction, and outputting a conference reservation result based on the key parameters; If the conference reservation result is successful, obtain audio stream data during the conference, perform voice segmentation, initial voiceprint extraction and initial clustering on the audio stream data to output original voice segments and speaker reference voice; Performing target speaker extraction, precise voiceprint extraction, and final clustering based on the original voice segment and the speaker reference voice to obtain a clean voice text stream with voiceprint labels; Performing dynamic summary generation, enhanced retrieval, and conflict error checking on the clean voice text stream to obtain conference information; The meeting information is subjected to responsibility identification and voiceprint binding to output final meeting management data.

2. The conference management method based on natural language processing and retrieval enhancement according to claim 1 is characterized in that: The step of extracting key parameters from the conference reservation instruction and outputting the conference reservation result based on the key parameters includes: Using a preset intent parsing algorithm to extract key parameters from the meeting reservation instruction; Determining whether the key parameters are complete; if the key parameters are complete, taking the key parameters as complete parameters; if the key parameters are incomplete, questioning the user based on the context of the conference reservation instruction and the matching slot information to obtain complete parameters; Based on the complete parameters, an optimal solution is output through target recommendation technology, and the optimal solution is published in the form of MCP to output a meeting reservation result.

3. The conference management method based on natural language processing and retrieval enhancement according to claim 2 is characterized in that: The key parameters include time, number of people and equipment requirements.

4. The conference management method based on natural language processing and retrieval enhancement according to claim 1 is characterized in that: The steps of performing speech segmentation, initial voiceprint extraction and initial clustering on the audio stream data to output original speech segments and speaker reference speech include: detecting speech segments and silence segments in the audio stream data, and segmenting the audio stream data based on the speech segments and the silence segments by fixed duration or silence intervals to obtain a plurality of original speech segments; Performing a first voiceprint feature extraction on the original speech segment to obtain an initial voiceprint segment; Calculating a first cosine similarity between the initial voiceprint segments, and performing a first clustering process on the initial voiceprint segments based on the first cosine similarity and using a Kmeans clustering method to obtain a plurality of initial clusters; Speech screening is performed in each of the initial clusters according to a preset threshold to obtain a speaker reference speech.

5. The conference management method based on natural language processing and retrieval enhancement according to claim 1 is characterized in that: The steps of performing target speaker extraction, precise voiceprint extraction, and final clustering based on the original voice segment and the speaker reference voice to obtain a clean voice text stream with voiceprint labels include: Obtaining a pre-trained extraction model, inputting the original speech segment and the speaker reference speech into the pre-trained extraction model, and using the original speech segment as a mixed input and the speaker reference speech as a conditional input to obtain a pure target person speech segment; Performing a second voiceprint feature extraction on the clean target person's voice segment to obtain an accurate voiceprint segment; Calculating a second cosine similarity between the precise voiceprint segments, and performing a second clustering process on the precise voiceprint segments based on the second cosine similarity using a Kmeans clustering method to obtain a plurality of precise clusters; The start time and end time of each speaker are located according to the precise clustering clusters and a unique speaker role is assigned to each precise clustering cluster to obtain a clean speech text stream with voiceprint labels.

6. The conference management method based on natural language processing and retrieval enhancement according to claim 1 is characterized in that: The step of performing dynamic summary generation, enhanced retrieval, and conflict error checking on the clean voice text stream to obtain conference information includes: Generating a real-time summary of the clean speech text using a dynamic summary method to obtain a dynamic summary; Using a Siamese network, the dynamic summary is input as a whole into a pre-trained BERT model for enhanced retrieval to output a high-dimensional sentence vector; The sentence vector is compared with a preset authoritative data source in real time and potential conflicts and errors are warned to output verification result data, which is automatically filled into a Markdown template to obtain meeting information.

7. The conference management method based on natural language processing and retrieval enhancement according to claim 1 is characterized in that: The step of performing responsibility identification and voiceprint binding on the conference information to output final conference management data includes: Input the meeting information into the pre-trained BiLSTM-CRF model to output the target task and responsible person labels, and perform logical judgment on the responsible person labels; If the responsible person tag identifies the speaker himself, the identity corresponding to the voiceprint ID corresponding to the current speaker will be used as the person responsible for the task; If the responsible person tag is identified as someone other than the speaker, the identification result of the responsible person tag is used as the person responsible for the task, and the voiceprint ID of the current speaker is recorded as the assignor or authorizer of the task to output the meeting task data; The conference information and the conference task data are integrated into a structured task triplet and attached with the voiceprint ID and conference source to output the final conference management data.

8. A conference management system based on natural language processing and retrieval enhancement, characterized in that: The system comprises: A reservation module is used to obtain a conference reservation instruction issued by a user, extract key parameters from the conference reservation instruction, and output a conference reservation result based on the key parameters; A reference module is configured to, if the conference reservation result is successful, obtain audio stream data during the conference, perform voice segmentation, initial voiceprint extraction and initial clustering on the audio stream data, and output original voice segments and speaker reference voices; A clean module, configured to perform target speaker extraction, precise voiceprint extraction, and final clustering based on the original speech segment and the speaker reference speech, so as to obtain a clean speech text stream with voiceprint labels; A retrieval module, configured to perform dynamic summary generation, enhanced retrieval, and conflict error checking on the clean voice text stream to obtain conference information; The output module is used to perform responsibility identification and voiceprint binding on the conference information to output final conference management data.

9. A computer comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the conference management method based on natural language processing and retrieval enhancement is implemented as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, implements the conference management method based on natural language processing and retrieval enhancement according to any one of claims 1 to 7.

Images