A conference management method and system based on natural language processing and retrieval enhancement

Abstract

The application provides a conference management method and system based on natural language processing and retrieval enhancement, and belongs to the technical field of conference management. The method comprises the following steps: outputting a conference reservation result based on key parameters; if the reservation is successful, acquiring audio stream data in the conference process, performing speech segmentation, initial voiceprint extraction and initial clustering on the audio stream data to output original speech segments and speaker reference speech; performing target speaker extraction, accurate voiceprint extraction and final clustering based on the original speech segments and the speaker reference speech to obtain pure speech text stream with a voiceprint label; performing dynamic abstract generation, enhanced retrieval and verification on the pure speech text stream; performing responsibility recognition and voiceprint binding on the conference information to output final conference management data. The application can cover intelligent management of the whole conference process, and realize conference whole-process management from voice-driven reservation, real-time content analysis to automatic task distribution.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of conference management, and particularly relates to a conference management method and system based on natural language processing and retrieval enhancement. BACKGROUND

[0002] In modern collaboration scenarios, efficient conference management is crucial to improving organizational efficiency. Existing systems have significant deficiencies in terms of intelligence and information depth processing:

[0003] 1. Complicated reservation process: relying on manual queries and form filling, lacking natural language understanding ability, unable to achieve accurate and fast conference room matching through voice commands;

[0004] 2. Inefficient information processing during the meeting: relying on manual recording of key content, lacking real-time automated summary capability, and unable to accurately associate speech content and speakers through voiceprint recognition;

[0005] 3. Insufficient post-meeting knowledge utilization: conference records become "information islands", traditional keyword search cannot achieve semantic-level depth analysis, making it difficult to reuse historical decision-making, task commitment, and other information;

[0006] 4. Slow implementation of meeting conclusions: the system lacks task extraction capability, and post-meeting manual analysis is required to generate to-do lists, resulting in delayed task assignment, ambiguous execution nodes, and slow resolution progress.

[0007] Therefore, there is an urgent need for an innovative solution that integrates natural language processing, voiceprint recognition, and enhanced retrieval to achieve a conference management method from voice-driven reservation, real-time content analysis, to automatic task distribution. SUMMARY

[0008] To solve the above technical problems, the application provides a conference management method and system based on natural language processing and retrieval enhancement to solve the problems in the background art.

[0009] In a first aspect, the application provides the following technical solution: a conference management method based on natural language processing and retrieval enhancement, comprising:

[0010] Obtaining a conference reservation instruction issued by a user, extracting key parameters from the conference reservation instruction, and outputting a conference reservation result based on the key parameters;

[0011] If the conference reservation result is successful, obtaining audio stream data during the conference, performing voice segmentation, initial voiceprint extraction, and initial clustering on the audio stream data to output original voice segments and speaker reference voice;

[0012] perform target speaker extraction, accurate voiceprint extraction and final clustering on the original speech segment and the speaker reference voice to obtain a pure speech text stream with a voiceprint label;

[0013] perform dynamic summary generation, enhanced retrieval and conflict error checking on the pure speech text stream to obtain conference information;

[0014] perform responsibility identification and voiceprint binding on the conference information to output final conference management data.

[0015] Compared with the prior art, the application has the beneficial effects that: the application proposes an intelligent conference reservation and interaction innovation driven by voice, through analyzing a voice / text instruction reservation mode, a user can complete conference room reservation through natural language, key parameters can be automatically extracted, and when information is incomplete, the user can be actively questioned to completely get rid of the dependence on traditional forms, at the same time, the system state of the conference room is queried in real time, an optimal conference room scheme is intelligently recommended, and the immediacy and accuracy of the reservation result are ensured, in view of the problem of voiceprint mixing and clustering error in the traditional method when multiple people speak at the same time, an extraction model is innovatively introduced, the pure voice of a target speaker can be efficiently extracted from mixed voice according to the reference voice of the target speaker, and iterative voiceprint enhancement and clustering are performed: through an iterative process of initial clustering, reference voice selection, extraction and accurate clustering, the distinguishability and purity of the voiceprint feature are significantly improved, and the speaker clustering and recognition accuracy in the multiple-person overlapping speaking scene are greatly improved, finally, a conference summary is dynamically generated, resolutions and core data are retained, and the conference summary can be automatically filled into a Markdown structured template, the application can accurately identify responsibility verbs and related elements in a speech, and the voiceprint ID of a speaker can be directly bound to a task triple to realize that speaking means responsibility attribution, and task assignment ambiguity is avoided.

[0016] Preferably, the step of extracting key parameters from the conference reservation instruction and outputting a conference reservation result based on the key parameters comprises:

[0017] extracting key parameters from the conference reservation instruction by using a preset intent analysis algorithm;

[0018] determining whether the key parameters are complete, if the key parameters are complete, regarding the key parameters as complete parameters, and if the key parameters are not complete, asking the user questions according to the context of the conference reservation instruction and matched slot information to obtain complete parameters;

[0019] outputting an optimal solution based on the complete parameters and through a target recommendation technology, and publishing the optimal solution in the form of an MCP to output a conference reservation result.

[0020] Preferably, the key parameters include time, number of people and equipment demand.

[0021] Preferably, the step of performing voice segmentation, initial voiceprint extraction and initial clustering on the audio stream data to output original voice segments and speaker reference voices comprises:

[0022] detecting voice segments and silence segments in the audio stream data, and dividing the audio stream data based on the voice segments and the silence segments by a fixed time length or silence interval to obtain a plurality of original voice segments;

[0023] performing first voiceprint feature extraction on the original voice segments to obtain initial voiceprint segments;

[0024] calculating first cosine similarity between the initial voiceprint segments, and performing first clustering processing on the initial voiceprint segments based on the first cosine similarity and using a Kmeans clustering method to obtain a plurality of initial clustering clusters;

[0025] performing voice screening in each of the initial clustering clusters according to a preset threshold to obtain speaker reference voices.

[0026] Preferably, the step of performing target speaker extraction, accurate voiceprint extraction and final clustering based on the original voice segments and the speaker reference voices to obtain a pure voice text stream with a voiceprint label comprises:

[0027] obtaining a pre-trained extraction model, inputting the original voice segments and the speaker reference voices into the pre-trained extraction model, and inputting the original voice segments as mixed input and inputting the speaker reference voices as conditional input to obtain pure target person voice segments;

[0028] performing second voiceprint feature extraction on the pure target person voice segments to obtain accurate voiceprint segments;

[0029] calculating second cosine similarity between the accurate voiceprint segments, and performing second clustering processing on the accurate voiceprint segments based on the second cosine similarity and using a Kmeans clustering method to obtain a plurality of accurate clustering clusters;

[0030] locating a starting time and an ending time of each speaker according to the accurate clustering clusters and assigning a unique speaker role to each of the accurate clustering clusters to obtain a pure voice text stream with a voiceprint label.

[0031] Preferably, the step of performing dynamic summary generation, enhanced retrieval and conflict error checking on the pure voice text stream to obtain conference information comprises:

[0032] performing real-time summary generation on the pure voice text using a dynamic summary method to obtain a dynamic summary;

[0033] The dynamic summary is input into a pre-trained BERT model as a whole to enhance retrieval, so as to output a high-dimensional sentence vector;

[0034] The sentence vector is compared with a preset authoritative data source in real time, and potential conflicts and errors are warned, so as to output verification result data, and the verification result data is automatically filled into a Markdown template, so as to obtain conference information.

[0035] Preferably, the step of identifying responsibility and binding voiceprint of the conference information to output final conference management data comprises:

[0036] The conference information is input into a pre-trained BiLSTM-CRF model to output a target task and a responsibility label, and the responsibility label is logically judged;

[0037] If the responsibility label is identified as the speaker himself, the identity corresponding to the voiceprint ID of the current speaker is taken as the responsibility person of the task;

[0038] If the responsibility label is identified as a non-speaker, the identification result of the responsibility label is taken as the responsibility person of the task, and the voiceprint ID of the current speaker is recorded as the assignor or the authorized person of the task, so as to output conference task data;

[0039] The conference information and the conference task data are integrated into a structured task triple and are attached with a voiceprint ID and a conference source, so as to output final conference management data.

[0040] In a second aspect, the present application provides the following technical scheme, a conference management system based on natural language processing and retrieval enhancement, the system comprises:

[0041] A predetermined module is configured to obtain a conference reservation instruction issued by a user, extract key parameters from the conference reservation instruction, and output a conference reservation result based on the key parameters;

[0042] A reference module is configured to, if the conference reservation result is reservation success, obtain audio stream data in a conference process, perform speech segmentation, initial voiceprint extraction and initial clustering on the audio stream data, and output original speech segments and speaker reference speech;

[0043] A pure module is configured to perform target speaker extraction, accurate voiceprint extraction and final clustering based on the original speech segments and the speaker reference speech, so as to obtain a pure speech text stream with a voiceprint label;

[0044] A retrieval module is configured to perform dynamic summary generation, enhanced retrieval and conflict error verification on the pure speech text stream, so as to obtain conference information;

[0045] an output module configured to perform responsibility recognition and voiceprint binding on the conference information to output final conference management data.

[0046] In a third aspect, the present application provides the following technical solution: a computer comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor implements the conference management method based on natural language processing and retrieval enhancement as described above when executing the computer program.

[0047] In a fourth aspect, the present application provides the following technical solution: a storage medium having a computer program stored thereon, wherein the computer program implements the conference management method based on natural language processing and retrieval enhancement as described above when executed by a processor. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0049] Figure 1 Flow chart of the conference management method based on natural language processing and retrieval enhancement provided for the first embodiment of the present application;

[0050] Figure 2 Structural block diagram of the conference management system based on natural language processing and retrieval enhancement provided for the second embodiment of the present application;

[0051] Figure 3 Hardware structure schematic diagram of the computer provided for another embodiment of the present application.

[0052] The present application will be further described below in combination with the drawings and the description of the drawings. DETAILED DESCRIPTION

[0053] The embodiments of the present application will be described in detail below, and the examples of the embodiments are shown in the drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are intended to explain the embodiments of the present application, and cannot be understood as a limitation of the present application.

[0054] In the description of the embodiments of the application, it should be understood that the terms "length", "width", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings based on the orientation or positional relationship shown in the drawings, and are only for the purpose of facilitating the description of the embodiments of the application and simplifying the description, and do not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the application.

[0055] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features. In the description of the embodiments of the application, the meaning of "a plurality of" is two or more, unless otherwise explicitly specified and limited.

[0056] In the embodiments of the application, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connection", "fixing" and the like should be understood in a broad sense, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements. For those skilled in the art, the specific meaning of the above terms in the embodiments of the application can be understood according to the specific circumstances.

[0057] Embodiment one

[0058] In one embodiment of the application, as shown in Figure 1 A conference management method based on natural language processing and retrieval enhancement, comprising:

[0059] S1, obtaining a conference reservation instruction issued by a user, extracting key parameters from the conference reservation instruction, and outputting a conference reservation result based on the key parameters;

[0060] Specifically, for the conference reservation instruction, the instruction here can be a voice instruction or a text instruction, for example "tomorrow 10 o'clock 10 people conference room", here "tomorrow", "10 o'clock", "10 people" are all key parameters, through which the responding conference reservation result can be output. The conference reservation result has two cases, one is reservation success, for example "10 conference has been reserved for you tomorrow at 10 o'clock", etc., the other is reservation failure, for example "the conference room you reserved tomorrow at 10 o'clock has been reserved" etc.

[0061] The step S1 comprises:

[0062] S11, extracting key parameters from the conference reservation instruction by using a preset intention analysis algorithm;

[0063] The key parameters include time, number of people, and equipment requirements. The preset intention analysis algorithm is specifically a pre-trained LLM model in the prior art, which can be obtained by inputting data in the conference field into the model for training.

[0064] S12, determining whether the key parameters are complete. If the key parameters are complete, the key parameters are taken as complete parameters. If the key parameters are not complete, the user is asked according to the context of the conference reservation instruction and the matched slot information to obtain complete parameters.

[0065] Specifically, if the key parameters are not complete, the context and the matched slot information are asked, such as "please confirm the conference theme", "please confirm the conference time", "please confirm the number of people", etc. For key parameters, they should include conference time, number of people, equipment requirements, and possibly conference theme. If one or several of the extracted key parameters are missing, it is determined that the key parameters are not complete and need to be asked.

[0066] S13, outputting an optimal solution based on the complete parameters and by using a target recommendation technology, and publishing the optimal solution in the form of MCP to output a conference reservation result.

[0067] Specifically, in this step, the conference assistant agent obtains the real-time recommended optimal solution of the conference room system by using MCP and other function calling technologies, publishes the optimal solution in the form of MCP, intelligently avoids conflicting conference locations and conference times, and then outputs the conference reservation result. The target recommendation technology is a pre-trained MCP general model, which can be obtained by inputting data in the conference field into the model for training. At the same time, the output optimal solution is the recommended optimal conference room, and the MCP form publishes the optimal solution, which can integrate, coordinate, and allocate upstream and downstream resources to provide content accurately and efficiently to end users.

[0068] S2, if the conference reservation result is successful, obtaining audio stream data in the conference process, performing voice segmentation, initial voiceprint extraction, and initial clustering on the audio stream data to output original voice segments and speaker reference voice.

[0069] The step S2 includes:

[0070] S21, detecting a speech segment and a silence segment in the audio stream data, and splitting the audio stream data based on the speech segment and the silence segment in a fixed time length or silence interval manner to obtain a plurality of original speech segments;

[0071] Specifically, the speech segment and the silence segment can be detected by the VAD method, and then the speech segment detected by the VAD is split into shorter speech segments, and then the original speech segments are obtained by splitting in a fixed time length or based on a silence interval.

[0072] S22, performing first voiceprint feature extraction on the original speech segments to obtain initial voiceprint segments;

[0073] Specifically, the pre-trained wav2Emb2 model can be used for first voiceprint feature extraction, and the pre-trained wav2Emb2 model can be obtained by inputting speech segment data for training.

[0074] S23, calculating first cosine similarity between the initial voiceprint segments, and performing first clustering processing on the initial voiceprint segments based on the first cosine similarity and using a Kmeans clustering method to obtain a plurality of initial clustering clusters;

[0075] Specifically, the initial voiceprint segments with a first cosine similarity greater than a similarity threshold are classified into the same clustering cluster, and the Kmeans clustering method is used for clustering, so that the initial clustering cluster is obtained. In this step, the purpose of clustering by the first cosine similarity is a pre-clustering process. In order to pre-classify the initial voiceprint segments with a relatively close distance into the same clustering cluster, the Kmeans clustering method is used for clustering, so that a more accurate clustering result can be obtained, and the clustering time can be shortened.

[0076] S24, performing speech screening in each initial clustering cluster according to a preset threshold to obtain a speaker reference speech;

[0077] Specifically, the purpose of screening is to screen one or more relatively pure speech segments with a higher confidence as a speaker reference speech. In the speech screening process, each voiceprint segment in each initial clustering cluster has a corresponding confidence. The confidence is compared with the preset threshold, and the voiceprint segment with a confidence higher than the preset threshold is taken as the speaker reference speech. The speech segment screened according to the confidence is a relatively pure speech segment with a higher confidence.

[0078] In the actual clustering process, when a segment is assigned to a cluster, the distance / similarity score with the cluster center (or cluster representative point) can be used as the confidence for judgment. If there are multiple segments with a confidence score higher than a preset threshold, the multiple segments can be selected as the speaker reference voice.

[0079] S3, performing target speaker extraction, accurate voiceprint extraction, and final clustering based on the original voice segment and the speaker reference voice to obtain a pure voice text stream with a voiceprint label;

[0080] Specifically, the target speaker extraction is performed to extract the voice segment corresponding to each speaker, i.e., the voice data of the target speaker during the conference.

[0081] The step S3 includes:

[0082] S31, obtaining a pre-trained extraction model, inputting the original voice segment and the speaker reference voice into the pre-trained extraction model, and inputting the original voice segment as a mixed input and the speaker reference voice as a conditional input to obtain a pure target person voice segment;

[0083] Specifically, for the traditional method, when multiple people speak at the same time, the extracted voiceprints are mixed, resulting in clustering errors. To optimize this process, the present application introduces a pre-trained SpEx+ model algorithm. SpEx+ can extract the pure voice of a target speaker from mixed voice based on a reference voice of the target speaker. After extracting a more pure single voice segment by SpEx+, voiceprint feature extraction is performed, which can obtain a voiceprint embedding with higher discrimination, thereby improving the accuracy of clustering and recognition. After the first clustering, a more reliable clustering result can be selected as the reference voice of SpEx+ for the second extraction and clustering, further improving the effect.

[0084] Specifically, the extraction model is an SpEx+ model, which can be trained by inputting voice data. After using the SpEx+ model, the pure voice of the target speaker corresponding to the reference voice can be extracted from the mixed input. Therefore, for an original segment that may contain the voices of multiple people, the voice of each known speaker in the segment is extracted.

[0085] S32, performing second voiceprint feature extraction on the pure target person voice segment to obtain an accurate voiceprint segment;

[0086] Specifically, for the now purer target speaker voice segments of each target extracted by the SpEx+ model, re-extracting the voiceprint features, the precise voiceprint segments obtained this time will be more accurate than those extracted in step S22, because the interference sound (other speakers) has been greatly weakened.

[0087] S33, calculate the second cosine similarity between the precise voiceprint segments, and perform a second clustering process on the precise voiceprint segments based on the second cosine similarity and using a Kmeans clustering method, to obtain a plurality of precise clustering clusters;

[0088] Specifically, the similarity calculation process and the clustering process here are the same as step S23, but this step can perform more accurate speaker clustering, and due to the improvement of voiceprint quality and the effective separation of overlapping speech, the clustering result will be significantly better than the preliminary clustering.

[0089] S34, according to the precise clustering cluster, positioning the starting time and ending time of each speaker and assigning a unique speaker role to each of the precise clustering clusters, to obtain a pure voice text stream with a voiceprint label;

[0090] Specifically, then according to the precise clustering cluster, the starting and ending time points (conversion points) of each speaker are accurately positioned, and a unique speaker role is assigned to each of the precise clustering clusters, such as role 1, role 2, …, role N, at this time the assigned speaker role is the voiceprint label, and each precise clustering cluster has a corresponding voiceprint label, i.e. speaker role.

[0091] S4, performing dynamic summary generation, enhanced retrieval and conflict error checking on the pure voice text stream to obtain conference information;

[0092] Specifically, in this step, the purpose of enhanced retrieval is to overcome the limitations of traditional keyword retrieval and to better understand the user's intent, and the checking is to avoid conflicts in the conference content.

[0093] Among them, the step S4 includes:

[0094] S41, using a dynamic summary method to generate a real-time summary of the pure voice text to obtain a dynamic summary;

[0095] Specifically, here the summary is generated in real time based on a pre-trained T5 model, which retains the resolution / data core, and the T5 model can be trained by inputting text data, and the dynamic summary method here is to generate a summary in real time through a pre-trained T5 model.

[0096] S42, inputting the dynamic summary as a whole into a pre-trained BERT model using a Siamese network for enhanced retrieval to output a high-dimensional sentence vector;

[0097] Specifically, in this step, the semantic layer is used to complete the output of the sentence vector, and the semantic layer is the core of the "enhanced retrieval closed loop" function. The ultimate goal is to break through the limitations of traditional keyword retrieval, achieve deep understanding and intelligent matching of historical decisions of the meeting, and ensure that not only the text can be understood, but also the true intention behind it can be understood, so as to provide the most valuable historical information for the current issue;

[0098] The traditional word vector model mainly depends on the degree of lexical overlap. When the expression is different but the meaning is similar (such as "budget approval" and "financial allocation decision"), the retrieval effect will be greatly reduced. The present application solves this pain point through its unique "Siamese Network" structure. Instead of encoding individual words, the entire sentence or phrase is input into a specially fine-tuned BERT network. The fine-tuning here refers to training the BERT network with data from the meeting field to ultimately generate a high-dimensional fixed-length vector (i.e. "sentence vector" or Sentence Embedding);

[0099] The Siamese network is a pre-trained network obtained by inputting data from the meeting field into the network for training. The Siamese network is used to encode the sentence as a whole. The input and output of the Siamese network are the dynamic summary and the encoded vector output after the dynamic summary is encoded as a whole, respectively. First, the label of the model is the similarity / unsimilarity relationship (or specific similarity score) between the sentence pair, not a pre-computed "correct" sentence vector. Secondly, the two branches of the Siamese network generate sentence vectors E1 and E2, which are intermediate representations. The purpose of training is to make some measure (distance, similarity) between E1 and E2 accurately reflect the semantic relationship label between the input sentence pair. By minimizing the loss function based on the relationship label, backpropagation will adjust the parameters of the shared BERT encoder. This process gradually shapes the entire vector space: semantically similar sentences (in the context of the meeting field) will be mapped to adjacent points in the vector space; semantically dissimilar sentences will be mapped to points far apart.

[0100] S43, comparing the sentence vector with a preset authoritative data source in real time and warning potential conflicts and errors to output verification result data, and automatically filling the Markdown template with the verification result data to obtain meeting information;

[0101] Specifically, for the verification process, by comparing the key data mentioned in the meeting with the authoritative data source within the company in real time, ensuring that decisions are based on accurate information, and issuing early warnings for potential conflicts and errors, thus forming a closed loop of information verification, and then automatically filling in the Markdown template (topic / resolution / to-do), and outputting the meeting information. The potential conflicts and errors here can be determined by comparing the sentence vector with the data in the preset authoritative database. If there is a large error in the logic, timeline, and context understanding between the two, it is considered that there are potential conflicts and errors. The Markdown template here can contain various Markdown elements, such as titles, lists, links, pictures, code blocks, etc., and can be customized as needed to help users quickly create documents that conform to a specific format or structure. At the same time, the authoritative data source here mainly includes The authoritative data of the target enterprise can be extracted from the enterprise's internal database. During the actual comparison process, the sentence vectors are converted into structured entities and the entity relationships therein are identified. Then, actual business rules and temporal logic are defined to detect conflicts. For example, if the project status in the entity is mentioned as completed, but the project status in the authoritative data is in progress, the sales target is greater than the market capacity limit in the authoritative data, and the task is assigned to person A, but person A in the authoritative data has resigned, these are all logical conflicts. For example, if the new deadline of project A in the entity is earlier than the planned completion date of the predecessor project of project A, the activity start date is earlier than the resource approval date, and the event occurrence time is not within the valid time period of the event in the authoritative data, these are all timeline conflicts. The events and dependencies stored in the authoritative data are used to check timeline consistency and causal logic (for example, task C must start after task B is completed). When the entity relationship in the entity does not match the real time in the authoritative data, it is a context understanding conflict.

[0102] In actual situations, the verification result data will retain the successfully verified data, that is, the data without errors and conflicts. These data contain meeting information, and these data can be filled into the Markdown template. The Markdown template is used to help users quickly create documents that conform to a specific format or structure.

[0103] S5. Perform responsibility identification and voiceprint binding on the conference information to output final conference management data;

[0104] Wherein, the step S5 includes:

[0105] S51: Input the meeting information into a pre-trained BiLSTM-CRF model to output target tasks and responsible person labels, and perform logical judgment on the responsible person labels;

[0106] Wherein, the conference information here refers to the information in the above-mentioned Markdown template, the pre-trained BiLSTM-CRF model here is trained by inputting the data in the conference field into the model, and since the voice segment obtained in the above-mentioned step will generate text data after being processed by the pre-trained BERT model, and after subsequent processing, the text format conference information can be output, the target task here refers to the specific content of a certain event mentioned in the conference, which belongs to a part of the conference task data, and the target task is not output by the model, but exists in the conference information itself.

[0107] Specifically, the context of task allocation often depends on the context. For example, "I suggest that role A is responsible for this matter" and "Role A's project yesterday is progressing smoothly", which are text information in the conference information, the two "responsible" here have completely different meanings. BiLSTM can read the entire sentence from both the front and back directions, thereby more accurately understanding the true meaning of responsibility verbs such as "responsible" and "follow up" in the current context, determining whether it constitutes a new task, and the CRF layer can learn the dependency between labels (for example, a "task description-internal" label is more likely to follow a "task description-start" label than a "deadline" label), ensuring the coherence and accuracy of entity extraction, avoiding illogical label combinations, and significantly improving the overall robustness of the model. For the above example, the responsibility label is the identity information or specific role identified in the conference information, the labeling here can be determined in the process of BiLSTM reading the entire sentence from both the front and back directions, more accurately understanding the true meaning of responsibility verbs such as "responsible" and "follow up" in the current context, and after understanding the true meaning of the current context, the corresponding label can be generated.

[0108] Wherein, the BiLSTM-CRF model is specifically used to input the conference information into the bidirectional LSTM, perform sequence modeling through the bidirectional LSTM to obtain the hidden state representation of each position, and finally use the CRF layer for global sequence labeling to obtain the final labeling result. The BiLSTM-CRF model is a model in the prior art, and its structure is the same as that of the model in the prior art, so it will not be described here. The input and output of the BiLSTM-CRF model are text data in the conference data and labels, respectively. The labels here are the responsibility labels.

[0109] S52, if the responsibility label is identified as the speaker himself, the identity corresponding to the voiceprint ID corresponding to the current speaker is taken as the responsibility of the task;

[0110] Specifically, this step is a self-claiming process. If the T-PER tag (responsible person tag) identifies "I", "we", or fails to identify other responsible persons in the sentence (the speaker himself is the default in the context), the voiceprint ID corresponding to the current speaker's employee identity is confirmed as the task responsible person. Here, the voiceprint ID is equivalent to the voiceprint tag mentioned above, that is, the speaker role.

[0111] S53, if the responsible person tag identifies a non-speaker, the identification result of the responsible person tag is recorded as the task responsible person, and the voiceprint ID of the current speaker is recorded as the task assignor or authorized person to output the conference task data;

[0112] If the T-PER tag identifies a specific name (such as "Role A") or department (such as "Department B"), the system confirms "Role A" or "Department B" as the task responsible person, and records the voiceprint ID of the current speaker as the task "assignor" or "authorized person".

[0113] S54, integrating the conference information and the conference task data into structured task triples and attaching voiceprint ID and conference source to output the final conference management data;

[0114] Specifically, for task triples, it includes [task description, responsible person, deadline], and is attached with authorized person voiceprint ID, conference source and other metadata. For the output final conference management data, it includes structured conference minutes and signed task cards. The conference source can be determined manually during the scheduled meeting to facilitate the subsequent traceability process. The task description is the target task, and the time information of the task, such as start time, duration, deadline, etc., is also included in the conference task data. Therefore, the task description and deadline can be obtained from the conference task data. It should be noted that in the above steps, the time information can be obtained from the speaker's voice data after the above series of steps, and it is generated together with the specific task information.

[0115] In summary, the conference management method based on natural language processing and retrieval enhancement in the above embodiments of the present application proposes a voice-driven intelligent conference reservation and interaction innovation. By analyzing the voice / text instruction reservation mode, the user can complete the conference room reservation through natural language, automatically extract key parameters, and actively ask questions when the information is incomplete to completely eliminate the dependence on traditional forms. At the same time, the system state of the conference room is queried in real time, the optimal conference room solution is intelligently recommended, and the immediacy and accuracy of the reservation result are ensured. In view of the problem of voiceprint mixing and clustering errors when multiple people speak at the same time in the traditional method, an extraction model is innovatively introduced. The pure voice of the target speaker can be efficiently extracted from the mixed voice according to the reference voice of the target speaker. Iterative voiceprint enhancement and clustering: through the iterative process of initial clustering, reference voice selection, extraction, and accurate clustering, the discrimination and purity of the voiceprint feature are significantly improved, and the speaker clustering and recognition accuracy in the multi-person overlapping speaking scene are greatly improved. Finally, a conference summary is dynamically generated, the resolutions and core data are retained, and the Markdown structured template can be automatically filled. The present application can accurately identify the responsibility verbs and related elements in the speech, and directly bind the voiceprint ID of the speaker to the task triple to realize the responsibility attribution as soon as the speaker speaks, thereby avoiding task assignment ambiguity.

[0116] Embodiment two

[0117] As Figure 2 shown, embodiment two of the present application provides a conference management system based on natural language processing and retrieval enhancement, which comprises:

[0118] A reservation module 1 is configured to obtain a conference reservation instruction issued by a user, extract key parameters from the conference reservation instruction, and output a conference reservation result based on the key parameters.

[0119] A reference module 2 is configured to, if the conference reservation result is reservation success, obtain audio stream data in a conference process, perform voice segmentation, initial voiceprint extraction, and initial clustering on the audio stream data to output original voice segments and speaker reference voice.

[0120] A pure module 3 is configured to perform target speaker extraction, accurate voiceprint extraction, and final clustering based on the original voice segments and the speaker reference voice to obtain a pure voice text stream with a voiceprint label.

[0121] A retrieval module 4 is configured to perform dynamic summary generation, enhanced retrieval, and conflict error checking on the pure voice text stream to obtain conference information.

[0122] An output module 5 is configured to perform responsibility recognition and voiceprint binding on the conference information to output final conference management data.

[0123] The reservation module 1 comprises:

[0124] an extraction submodule configured to extract a key parameter from the conference reservation instruction by using a preset intention analysis algorithm;

[0125] a judgment submodule configured to judge whether the key parameter is complete, if the key parameter is complete, the key parameter is taken as a complete parameter, if the key parameter is not complete, the user is asked according to the context of the conference reservation instruction and matched slot information to obtain a complete parameter;

[0126] a publishing submodule configured to output an optimal solution by a target recommendation technology based on the complete parameter, and publish the optimal solution in the form of an MCP to output a conference reservation result.

[0127] The reference module 2 comprises:

[0128] a detection submodule configured to detect a speech segment and a silence segment in the audio stream data, and cut the audio stream data by a fixed time length or a silence interval based on the speech segment and the silence segment to obtain a plurality of original speech segments;

[0129] a first voiceprint extraction submodule configured to extract a first voiceprint feature from the original speech segments to obtain an initial voiceprint segment;

[0130] a first clustering submodule configured to calculate a first cosine similarity between the initial voiceprint segments, and perform a first clustering processing on the initial voiceprint segments based on the first cosine similarity and using a Kmeans clustering method to obtain a plurality of initial clustering clusters;

[0131] a screening submodule configured to perform a speech screening in each of the initial clustering clusters according to a preset threshold to obtain a speaker reference speech.

[0132] The pure module 3 comprises:

[0133] an input submodule configured to obtain a pre-trained extraction model, input the original speech segments and the speaker reference speech into the pre-trained extraction model, and take the original speech segments as a mixed input and take the speaker reference speech as a conditional input to obtain a pure target person speech segment;

[0134] a second voiceprint extraction submodule configured to extract a second voiceprint feature from the pure target person speech segment to obtain an accurate voiceprint segment;

[0135] a second clustering submodule configured to calculate a second cosine similarity between the accurate voiceprint segments, and perform a second clustering processing on the accurate voiceprint segments based on the second cosine similarity and using a Kmeans clustering method to obtain a plurality of accurate clustering clusters.

[0136] an assigning sub-module, configured to locate a start time and an end time of each speaker according to the accurate clustering and assign a unique speaker role to each of the accurate clustering to obtain a pure speech text stream with a voiceprint label.

[0137] The retrieval module 4 comprises:

[0138] a generating sub-module, configured to generate a real-time summary of the pure speech text using a dynamic summary method to obtain a dynamic summary;

[0139] a vector generating sub-module, configured to input the dynamic summary as a whole into a pre-trained BERT model for enhanced retrieval to output a high-dimensional sentence vector;

[0140] a warning sub-module, configured to compare the sentence vector with a preset authoritative data source in real time and warn of potential conflicts and errors to output verification result data, and fill the verification result data into a Markdown template to obtain conference information.

[0141] The output module 5 comprises:

[0142] a logic sub-module, configured to input the conference information into a pre-trained BiLSTM-CRF model to output a target task and a responsibility label, and perform logical judgment on the responsibility label;

[0143] a first binding sub-module, configured to, if the responsibility label is identified as the speaker himself / herself, take the identity corresponding to the voiceprint ID corresponding to the current speaker as the responsibility of the task;

[0144] a second binding sub-module, configured to, if the responsibility label is identified as a non-speaker, take the identification result of the responsibility label as the responsibility of the task, and record the voiceprint ID of the current speaker as the assigner or the authorized person of the task to output conference task data;

[0145] a final output sub-module, configured to integrate the conference information and the conference task data into a structured task triple and attach a voiceprint ID and a conference source to output final conference management data.

[0146] In some embodiments of the present application, the present application provides the following technical solutions: a computer comprising a memory 102, a processor 101, and a computer program stored on the memory 102 and executable on the processor 101, wherein the processor 101 implements the conference management method based on natural language processing and retrieval enhancement as described above when executing the computer program.

[0147] In particular, the processor 101 can include a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the present application.

[0148] The memory 102 can include mass storage for data or instructions. By way of example, and not limitation, the memory 102 can include a Hard Disk Drive (HDD), a floppy disk drive, a solid-state drive (SSD), a flash drive, a Compact Disk Read Only Memory (CD-ROM), a Digital Versatile Disk (DVD), a Blu-ray disk, a magnetic tape, or a Universal Serial Bus (USB) drive, or a combination of two or more of these. The memory 102 can be removable and / or non-removable (or fixed) as appropriate. The memory 102 can be internal or external as appropriate. In certain embodiments, the memory 102 is a non-volatile memory. In certain embodiments, the memory 102 includes a Read-Only Memory (ROM) and a Random Access Memory (RAM). The ROM can be a mask-programmed ROM, a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically EPROM (EEPROM), an Electrically Alterable ROM (EAROM), or a FLASH memory, or a combination of two or more of these, as appropriate. The RAM can be a Static Random-Access Memory (SRAM) or a Dynamic Random Access Memory (DRAM), which can be a Fast Page Mode Dynamic Random Access Memory (FPMDRAM), an Extended Data Output Dynamic Random Access Memory (EDODRAM), a Synchronous Dynamic Random-Access Memory (SDRAM), or the like, as appropriate.

[0149] The memory 102 can be used to store or buffer various data files needed for processing and / or communication, and possible computer program instructions executed by the processor 101.

[0150] The processor 101 reads and executes the computer program instructions stored in the memory 102 to implement the conference management method based on natural language processing and retrieval enhancement as described above.

[0151] In some embodiments, the computer can further include a communication interface 103 and a bus 100. As shown, the processor 101, the memory 102, and the communication interface 103 are connected by the bus 100 and complete communication with each other. Figure 3

[0152] The communication interface 103 is used to realize the communication between the modules, devices, units and / or equipment in the embodiments of the present application. The communication interface 103 can also realize data communication with other components, such as external devices, image / data acquisition devices, databases, external storage, image / data processing workstations, etc.

[0153] ​Bus 100 includes hardware, software, or both, to couple components of computer device to each other and to couple components of computer device to other devices. Bus 100 includes, but is not limited to, at least one of the following: a data bus, an address bus, a control bus, an expansion bus, a local bus, and the like. By way of example and not limitation, bus 100 can include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or another suitable bus or a combination of two or more of these. Where appropriate, bus 100 can include one or more buses. Although the present embodiments describe and show a particular bus, the present embodiments contemplate any suitable bus or interconnect.

[0154] The computer can execute the natural language processing and retrieval enhanced conference management method based on the natural language processing and retrieval enhanced conference management system, thereby realizing the natural language processing and retrieval enhanced conference management.

[0155] In some embodiments of the present application, in combination with the natural language processing and retrieval enhanced conference management method described above, the present embodiments provide the following technical solutions: a storage medium, the storage medium has a computer program stored thereon, and the computer program is executed by a processor to implement the natural language processing and retrieval enhanced conference management method described above.

[0156] Those skilled in the art will appreciate that the logic and / or steps represented in the flow diagrams, or otherwise described herein, for example, can be thought of as a list of executable instructions for implementing logic functions, and can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, processor-containing system, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions, or a combination thereof. For the purposes of this specification, a "computer-readable medium" can be any apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0157] More specific examples (a non-exhaustive list) of the computer-readable medium include the following: an electrical connection (electronic) having one or more wires, a portable computer diskette (magnetic), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Additionally, the computer-readable medium can also be paper or another suitable medium upon which the program is printed, as the program can be electronically captured, for example, via an optical scanner, then compiled, interpreted, or otherwise processed, and stored in a computer memory in a form that is then employable by a computer. In this context, the above-mentioned media are considered machine-readable media.

[0158] It should be understood that aspects of the application can be implemented in hardware, software, firmware or any combination thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following technologies, known in the art, or a combination thereof, can be used: discrete logic circuitry having logic gates for implementing logic functions upon data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), and so forth.

[0159] The technical features of the above-described embodiments can be combined in any manner, and for the sake of brevity, not all possible combinations are described, however, any combination of the technical features is considered to be within the scope of the present specification.

[0160] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent. It should be noted that for ordinary skilled persons in the art, without departing from the concept of the present application, several modifications and improvements can be made, which are within the protection scope of the present application. Therefore, the protection scope of the present application patent should be subject to the appended claims.

Claims

1. A meeting management method based on natural language processing and retrieval enhancement, characterized in that, include: Obtaining a conference reservation instruction issued by a user, extracting key parameters from the conference reservation instruction, and outputting a conference reservation result based on the key parameters; If the conference reservation result is successful, obtain audio stream data during the conference, perform voice segmentation, initial voiceprint extraction and initial clustering on the audio stream data to output original voice segments and speaker reference voice; Based on the original speech segment and the speaker reference speech, target speaker extraction, precise voiceprint extraction and final clustering are performed to obtain a clean speech text stream with voiceprint labels; wherein, a pre-trained extraction model is obtained, the original speech segment and the speaker reference speech are input into the pre-trained extraction model, and the original speech segment is used as a mixed input and the speaker reference speech is used as a conditional input to obtain a clean target person speech segment; a second voiceprint feature extraction is performed on the clean target person speech segment to obtain a precise voiceprint segment; a second cosine similarity between the precise voiceprint segments is calculated, and based on the second cosine similarity, a second clustering process is performed on the precise voiceprint segments using the Kmeans clustering method to obtain a number of precise clustering clusters; the start time and end time of each speaker are located according to the precise clustering clusters and a unique speaker role is assigned to each precise clustering cluster to obtain a clean speech text stream with voiceprint labels; Dynamic summary generation, enhanced retrieval, and conflict error verification are performed on the clean voice text stream to obtain meeting information; wherein, a dynamic summary method is used to generate a real-time summary of the clean voice text to obtain a dynamic summary; the dynamic summary is used as a whole input into a pre-trained BERT model using a twin network to perform enhanced retrieval to output a high-dimensional sentence vector; the sentence vector is compared with a preset authoritative data source in real time and potential conflicts and errors are warned to output verification result data, and the verification result data is automatically filled into a Markdown template to obtain meeting information; The meeting information is subjected to responsibility identification and voiceprint binding to output the final meeting management data; wherein, the meeting information is input into the pre-trained BiLSTM-CRF model to output the target task and the responsible person label, and the responsible person label is logically judged; if the responsible person label is identified as the speaker himself, the identity corresponding to the voiceprint ID corresponding to the current speaker is used as the person responsible for the task; if the responsible person label is identified as someone other than the speaker himself, the identification result of the responsible person label is used as the person responsible for the task, and the voiceprint ID of the current speaker is recorded as the assignor or authorizer of the task to output the meeting task data; the meeting information and the meeting task data are integrated into a structured task triplet with the voiceprint ID and the meeting source attached to output the final meeting management data.

2. The meeting management method based on natural language processing and retrieval enhancement according to claim 1, characterized in that, The step of extracting key parameters from the conference reservation instruction and outputting the conference reservation result based on the key parameters includes: Using a preset intent parsing algorithm to extract key parameters from the meeting reservation instruction; determining whether the key parameters are complete, if the key parameters are complete, taking the key parameters as complete parameters, if the key parameters are not complete, asking the user for complete parameters according to the context of the conference reservation instruction and matched slot information; outputting an optimal solution based on the complete parameters and through a target recommendation technology, and publishing the optimal solution in the form of MCP to output a conference reservation result.

3. The meeting management method based on natural language processing and retrieval enhancement according to claim 2, characterized in that, The key parameters include time, number of people, and equipment requirements.

4. The meeting management method based on natural language processing and retrieval enhancement according to claim 1, characterized in that, The step of performing speech segmentation, initial voiceprint extraction, and initial clustering on the audio stream data to output original speech segments and speaker reference speech includes: detecting speech segments and silence segments in the audio stream data, and cutting the audio stream data based on the speech segments and the silence segments through a fixed time length or a silence interval to obtain a plurality of original speech segments; performing first voiceprint feature extraction on the original speech segments to obtain initial voiceprint segments; calculating first cosine similarity between the initial voiceprint segments, and performing first clustering processing on the initial voiceprint segments based on the first cosine similarity and using a Kmeans clustering method to obtain a plurality of initial clustering clusters; performing speech screening according to a preset threshold in each initial clustering cluster to obtain speaker reference speech.

5. A meeting management system based on natural language processing and retrieval enhancement, characterized in that, The system includes: a reservation module configured to obtain a conference reservation instruction issued by a user, extract key parameters from the conference reservation instruction, and output a conference reservation result based on the key parameters; a reference module configured to, if the conference reservation result is reservation success, obtain audio stream data in a conference process, perform speech segmentation, initial voiceprint extraction, and initial clustering on the audio stream data to output original speech segments and speaker reference speech; a pure module configured to perform target speaker extraction, accurate voiceprint extraction, and final clustering based on the original speech segments and the speaker reference speech to obtain a pure speech text stream with a voiceprint label; wherein the pure module is configured to obtain a pre-trained extraction model, input the original speech segments and the speaker reference speech into the pre-trained extraction model, take the original speech segments as mixed input and take the speaker reference speech as conditional input to obtain pure target person speech segments, perform second voiceprint feature extraction on the pure target person speech segments to obtain accurate voiceprint segments, calculate second cosine similarity between the accurate voiceprint segments, perform second clustering processing on the accurate voiceprint segments based on the second cosine similarity and using a Kmeans clustering method to obtain a plurality of accurate clustering clusters, and locate a starting time and an ending time of each speaker according to the accurate clustering clusters and assign a unique speaker role to each accurate clustering cluster to obtain a pure speech text stream with a voiceprint label. The retrieval module is used for dynamic summary generation, enhanced retrieval and conflict error checking of the pure voice text stream to obtain conference information; wherein, the dynamic summary method is used for real-time summary generation of the pure voice text to obtain a dynamic summary; the twin network is used for inputting the dynamic summary into a pre-trained BERT model as a whole for enhanced retrieval to output a high-dimensional sentence vector; the sentence vector is compared with a preset authoritative data source in real time, and potential conflicts and errors are prewarned to output checking result data, the checking result data is automatically filled into a Markdown template to obtain conference information; The output module is used for responsibility identification and voiceprint binding of the conference information to output final conference management data; wherein, the conference information is input into a pre-trained BiLSTM-CRF model to output a target task and a responsibility label, and the responsibility label is logically judged; if the responsibility label is identified as the speaker himself, the identity corresponding to the voiceprint ID corresponding to the current speaker is taken as the responsibility person of the task; if the responsibility label is identified as a non-speaker himself, the identification result of the responsibility label is taken as the responsibility person of the task, and the voiceprint ID of the current speaker is recorded as the assignor or the authorized person of the task to output conference task data; the conference information and the conference task data are integrated into a structured task triple and are attached with a voiceprint ID and a conference source to output final conference management data.

6. A computer comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to realize the conference management method based on natural language processing and enhanced retrieval according to any one of claims 1 to 4.

7. A storage medium, characterized by The storage medium has a computer program stored thereon, and the computer program is executed by the processor to realize the conference management method based on natural language processing and enhanced retrieval according to any one of claims 1 to 4.

Images