Conference summary generation method and device, electronic equipment and storage medium

Abstract

The invention discloses a conference summary generation method and device and electronic equipment, and belongs to the technical field of artificial intelligence. The method comprises the following steps: performing voice recognition and voiceprint recognition on audio data of a target conference, and obtaining a speech draft arrangement result for distinguishing participants; carrying out element extraction on the basis of the draft arrangement result and handwritten conference records of current participants of the target conference to obtain conference element information; content understanding based on conference element information weighting is carried out on the draft arrangement result, and a conference overview and / or a conference to-do list of the target conference are / is obtained; and generating a conference summary based on the conference overview and / or the conference backlog and the draft arrangement result. According to the method, the conference elements in the speaking of the participants are extracted by combining the handwritten conference record, and the content understanding is weighted based on the conference elements, so that the integrity and accuracy of the key information in the generated conference summary are further improved, and the key content of the conference can be highlighted.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a method, device, electronic device, and computer-readable storage medium for generating meeting minutes. Background Art

[0002] Traditional meeting minutes are recorded and organized manually, which is not only time-consuming and laborious, but also easy to miss content. With the rapid development of artificial intelligence technology and natural language processing technology, some innovative solutions to improve office efficiency and accuracy have emerged in the fields of smart office and smart meeting minutes. For example, some solutions extract the audio signal of the meeting, transcribe the audio signal, and convert it into text to form meeting minutes. Some solutions can also perform voiceprint recognition on audio signals, distinguish speakers, and generate meeting minutes. In order to improve the readability of meeting minutes, some solutions use keyword extraction, key sentence extraction and other solutions to generate meeting minutes.

[0003] However, pure speech transcription is greatly affected by the speaker's accent and the colloquialism of the speech. The information conveyed by the transcribed text may be incomplete or biased, and it still requires a lot of extra effort to organize. On the other hand, when the meeting time is long or there are many meetings, only taking the full text of the meeting speech transcription as the meeting minutes or only extracting some of the key information of the meeting without doing more information extraction, integration and processing will result in unclear and scattered meeting minutes, and it is easy to miss key information, which is not conducive to the induction of information and the improvement of work efficiency. The extraction of meeting information is limited by the limitations of key information extraction technology. How to extract accurate and concise meeting minutes is a problem that needs to be explored continuously.

[0004] It can be seen that the method for generating meeting minutes in the prior art still needs to be improved. Summary of the invention

[0005] The embodiments of the present application provide a method and device for generating meeting minutes, which are helpful to improve the information integrity and accuracy of the generated meeting minutes, thereby improving office efficiency.

[0006] In a first aspect, an embodiment of the present application provides a method for generating meeting minutes, including:

[0007] Perform speech recognition and voiceprint recognition on the audio data of the target meeting to obtain the transcript results that distinguish the participants;

[0008] Based on the speech editing results and the handwritten meeting minutes of the current participants of the target meeting, element extraction is performed to obtain meeting element information;

[0009] Performing content understanding on the speech editing results based on the weighted meeting element information to obtain a meeting overview and / or meeting to-do items of the target meeting;

[0010] Based on the meeting overview and / or meeting to-do items, and the transcript editing results, a meeting minutes is generated.

[0011] In a second aspect, an embodiment of the present application provides a device for generating meeting minutes, including:

[0012] The speech transcript editing module is used to perform speech recognition and voiceprint recognition on the audio data of the target meeting to obtain speech transcript editing results that distinguish the participants;

[0013] A conference element information acquisition module, used to extract elements and acquire conference element information based on the speech manuscript arrangement result and the handwritten meeting minutes of the current participants of the target meeting;

[0014] A meeting overview and to-do generating module, configured to perform content understanding on the speech editing result based on the weighted meeting element information to obtain a meeting overview and / or meeting to-do items of the target meeting;

[0015] The meeting minutes generation module is used to generate meeting minutes based on the meeting overview and / or meeting to-do items, and the speech manuscript arrangement results.

[0016] In a third aspect, an embodiment of the present application further discloses an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the meeting minutes generation method described in the embodiment of the present application when executing the computer program.

[0017] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the meeting minutes generation method disclosed in the embodiment of the present application.

[0018] The meeting minutes generation method disclosed in the embodiment of the present application performs speech recognition and voiceprint recognition on the audio data of the target meeting to obtain speech manuscript arrangement results that distinguish the participants; based on the speech manuscript arrangement results and the handwritten meeting records of the current participants of the target meeting, element extraction is performed to obtain meeting element information; the speech manuscript arrangement results are subjected to content understanding based on the weighted meeting element information to obtain the meeting overview and / or meeting to-do items of the target meeting; based on the meeting overview and / or meeting to-do items, and the speech manuscript arrangement results, meeting minutes are generated. This method further improves the completeness and accuracy of key information in the generated meeting minutes by extracting meeting elements from the speeches of participants in combination with handwritten meeting records, and weighting the content understanding based on the meeting elements, and can highlight the key content of the meeting.

[0019] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0021] Figure 1 It is a flow chart of the method for generating meeting minutes disclosed in the embodiment of the present application;

[0022] Figure 2 It is a schematic diagram of training data of the large model for speech editing disclosed in the embodiment of the present application;

[0023] Figure 3 It is a schematic diagram of training data of the conference understanding large model disclosed in the embodiment of the present application;

[0024] Figure 4 It is a schematic diagram of the structure of the meeting minutes generating device disclosed in the embodiment of the present application;

[0025] Figure 5 A block diagram schematically shows an electronic device for executing the method according to the present application; and

[0026] Figure 6 A storage unit for holding or carrying a program code for implementing the method according to the present application is schematically shown. DETAILED DESCRIPTION

[0027] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0028] Reference Figure 1 , a method for generating meeting minutes disclosed in an embodiment of the present application includes: steps 110 to 140.

[0029] Step 110, perform speech recognition and voiceprint recognition on the audio data of the target conference to obtain speech transcripts that distinguish the participants.

[0030] Among them, the method for obtaining the audio data of the target conference refers to the prior art and will not be repeated in the embodiments of the present application.

[0031] Optionally, the performing speech recognition and voiceprint recognition on the audio data of the target conference to obtain speech editing results that distinguish the participants includes: performing speech recognition and voiceprint recognition on the audio data of the target conference to obtain speech text corresponding to each participant; dividing the speech text corresponding to each participant into text paragraphs that are independent of context semantics, to obtain speech paragraphs that distinguish the participants; inputting each of the speech paragraphs into a pre-fine-tuned speech editing large model for parallel reasoning, to obtain speech editing results that distinguish the participants.

[0032] Optionally, voiceprint recognition can be first performed on the audio data of the target meeting, and the audio data can be segmented according to the voiceprint recognition results to obtain audio data segments of different participants who are speaking. Thereafter, speech recognition can be further performed on each audio data segment to obtain the speech texts of different participants, which are referred to as "script text" in the embodiments of the present application.

[0033] In other embodiments of the present application, other methods may be used to perform speech recognition and voiceprint recognition on the audio data of the target conference to obtain the speech manuscript text corresponding to each participant, which will not be listed one by one in the embodiments of the present application. The embodiments of the present application do not limit the specific implementation method of performing speech recognition and voiceprint recognition on the audio data of the target conference to obtain the speech manuscript text corresponding to each participant.

[0034] In practical applications, if some participants make a large speech with a long speech content and a long speech time, a long text paragraph will appear in the speech text of the participant obtained after voice recognition. Similar text paragraphs reduce the generation efficiency of the large speech editing model. In some embodiments of the present application, each speech text is segmented into independent text paragraphs with no contextual semantics between paragraphs based on the semantic similarity between adjacent sentences in the speech text of the same participant, so as to obtain speech paragraphs that distinguish the participant.

[0035] Optionally, the speech text corresponding to each participant is divided into text paragraphs that are semantically unrelated to the context to obtain speech paragraphs that distinguish the participants, including: for the current text paragraph corresponding to the designated participant, according to the relationship between the semantic similarity between adjacent sentences and a preset repeated speech threshold and a preset semantic similarity threshold, the semantically similar sentences in the current text paragraph are divided into the same paragraph to obtain the speech paragraph corresponding to the designated participant.

[0036] According to the method for obtaining the speech text described above, the speech text corresponding to the same participant may include one text paragraph or multiple text paragraphs, and each text paragraph may include one or more sentences. When the speech text corresponding to the same participant includes multiple text paragraphs, the multiple text paragraphs may be adjacent text paragraphs or non-adjacent text paragraphs. When performing text paragraph segmentation, each text paragraph in the speech text can be traversed, and each text paragraph can be segmented separately.

[0037] In some optional embodiments, based on the relationship between the semantic similarity between adjacent sentences and a preset repeated utterance threshold and a preset semantic similarity threshold, the semantically similar sentences in the current text paragraph are segmented into the same paragraph, including: obtaining the encoding vector of each sentence in the current text paragraph; calculating the normalized autocorrelation matrix between adjacent sentences in the segment based on the encoding vector of each sentence, wherein the values ​​of the diagonal matrix elements in the normalized autocorrelation matrix of adjacent sentences in the segment represent the semantic similarity between the corresponding adjacent sentences; based on the relationship between the semantic similarity and the preset repeated utterance threshold and the preset semantic similarity threshold, the semantically similar sentences in the current text paragraph are segmented into the same paragraph.

[0038] In the embodiment of the present application, the vectorization module of the speech manuscript arrangement model can be used to vectorize the current text paragraph, and then the vectorization result is segmented according to the sentence to obtain the encoding vector of each sentence. Then, according to the encoding vector of each sentence, the normalized autocorrelation matrix of adjacent sentences is established.

[0039] Optionally, a normalized autocorrelation matrix between adjacent sentences in a segment is calculated based on the coding vectors of each sentence, including: establishing a high-dimensional matrix of the coding vectors of the sentences in the segment based on the coding vectors of each sentence; calculating the autocorrelation matrix and the norm matrix of the high-dimensional matrix; dividing the autocorrelation matrix and the norm matrix at corresponding positions to obtain a text normalized autocorrelation matrix; clearing the matrix elements corresponding to non-adjacent sentences in the text normalized autocorrelation matrix to zero to obtain a normalized autocorrelation matrix of adjacent sentences in the segment.

[0040] For example, the internal parameters of the speech editing model can be used to vectorize the current text paragraph, and the vector can be segmented according to short sentences (for example, end-of-sentence markers such as periods, question marks, and exclamation marks), to obtain a high-dimensional matrix W = [w 1 ,w 2 ,w 3 ,…,w n ] T , where w i Represents the encoding vector for each sentence.

[0041] Optionally, the autocorrelation matrix Z of the high-dimensional matrix W can be calculated using the following formula:

[0042] in i and j represent the row and column labels of the matrix respectively, and n represents the number of sentences in the current text paragraph.

[0043] Optionally, the norm matrix ‖Z‖ of the high-dimensional matrix W can be calculated using the following formula:

[0044] where ||z|| ij =||w i ||*||w j ||, m represents the encoding vector dimension of the sentence.

[0045] Optionally, the autocorrelation matrix Z and the norm matrix ‖Z‖ are divided at corresponding positions, and the obtained text normalized autocorrelation matrix can be expressed as: Among them, each matrix element represents the semantic similarity of the corresponding two sentences. Then, the text is normalized to the autocorrelation matrix X, and x 12 、x 23 、x 34 ,...x n-1,n Except for the reserved position, the other positions are replaced with 0 to obtain the normalized autocorrelation matrix of adjacent texts.

[0046] In some optional embodiments, based on the magnitude relationship between the semantic similarity and the preset repeated speech threshold and the preset semantic similarity threshold, the semantically similar sentences in the current text paragraph are segmented into the same paragraph, including: in response to the semantic similarity being greater than the preset repeated speech threshold, deleting one of the adjacent sentences corresponding to the semantic similarity; in response to the semantic similarity being less than the preset repeated speech threshold and greater than the preset semantic similarity threshold, segmenting the adjacent sentences corresponding to the semantic similarity into the same paragraph; in response to the semantic similarity being less than the preset semantic similarity threshold, segmenting the adjacent sentences corresponding to the semantic similarity into different paragraphs. Wherein, the preset repeated speech threshold is greater than the preset semantic similarity threshold, and the larger the value of the semantic similarity is, the more similar the corresponding adjacent sentences are.

[0047] Taking the adjacent sentences i and i+1 in the current text paragraph as an example, the semantic similarity of these two adjacent sentences is x in the text normalized autocorrelation matrix X i,i+1 , if the semantic similarity x i,i+1 If the semantic similarity x is greater than the preset repeated speech threshold, the semantics of sentences i and i+1 are considered to be repeated, and one of the sentences needs to be deleted, such as deleting the shorter one. i,i+1 If the value is less than the preset repeated speech threshold and greater than the preset semantic similarity threshold, sentences i and i+1 are segmented into the same paragraph; otherwise, sentence i+1 is segmented into the next paragraph. Similarly, each sentence in the current text paragraph is segmented in turn to obtain one or more shorter paragraphs whose contextual semantics are unrelated.

[0048] Further, the conference participants corresponding to the current text paragraph are used as conference participants corresponding to the paragraphs obtained by segmenting the current text paragraph.

[0049] It takes a long time to calculate the similarity between adjacent sentences for segmentation. The embodiment of the present application can shorten the time of segmentation by directly calculating the normalized correlation matrix of all sentence text vectors within the speech paragraphs of the participants, thereby improving the speed of generating meeting minutes.

[0050] Next, the speech of each speech paragraph is sorted out respectively, and the speech sorting results of each speech paragraph are combined to obtain the speech sorting results of the target meeting.

[0051] In some optional embodiments, the step of inputting each of the speech paragraphs into a pre-fine-tuned speech organization large model for parallel reasoning to obtain speech organization results that distinguish between conference participants includes: using on-demand real-time distributed storage space to input each of the speech paragraphs into a pre-fine-tuned speech organization large model for parallel reasoning to obtain speech organization results that distinguish between conference participants.

[0052] The speech editing model is obtained by using the training data to perform supervised fine-tuning training on some parameters of the pre-trained generative model. The training scheme of the speech editing model is described below and will not be repeated here.

[0053] In the application stage, each speech paragraph obtained through paragraph segmentation is input into the speech editing large model obtained by pre-fine-tuning, and the input speech paragraph is inferred by the speech editing large model to obtain the speech editing results corresponding to each speech paragraph. For example, the encoding vector of each speech paragraph obtained after semantic segmentation is input into the speech editing large model for reasoning and natural language generation to obtain a vector of the reasoning result, and the vector is further decoded to obtain the corresponding speech text. Furthermore, the speech text output after decoding the speech editing large model is spliced ​​according to the paragraph order of the input speech paragraph, so as to quickly obtain the segmented written editing results of the super-long conference text that distinguishes the participants.

[0054] Optionally, in the process of inputting each speech paragraph obtained through paragraph segmentation into the speech editing large model obtained by pre-fine-tuning, a parallel reasoning method is adopted to improve the reasoning speed. Since the speech paragraphs input into the speech editing large model for reasoning are context-independent, the information of other speech paragraphs is not required to ensure the semantic integrity during reasoning. Therefore, the speech editing large model can adopt a multi-process method to reason on multiple speech paragraphs at the same time, and can still accurately generate the reasoning text.

[0055] On the other hand, in the segmented reasoning process, the embodiment of the present application allocates storage space while reasoning, and dynamically creates reasoning tasks by distributing storage space on demand in real time. After a paragraph of speech is reasoned, the video memory resources occupied by it are immediately released, and the video memory resources are quickly applied for the next paragraph of speech, and the paragraph text is reasoned and generated, thereby accelerating the reasoning process. The accelerated concurrent reasoning can complete the written arrangement and generation of the speech obtained by voice transcription of 10,000 words within 7 seconds, solving the problem of too long time to organize and generate the full text of the meeting in sequence while ensuring semantic integrity.

[0056] In the prior art, it is usually necessary to pre-set the maximum length of the current inference of the large model, and allocate parameter storage space for the current inference in advance according to the maximum length. If the actual inference does not reach the preset maximum length, it will cause a waste of video memory resources. In the embodiment of the present application, storage space is allocated as needed while inference is being performed. After the current inference task uses up the allocated smaller storage space, another storage space of the same size is allocated. By analogy, the unallocated storage space can be used to infer more parallel tasks, avoiding the waste of resources caused by pre-allocating a large amount of storage resources.

[0057] In addition, in an embodiment of the present application, a vector migration method of a large model for speech organization is used to calculate the semantic similarity of adjacent sentences, and the semantic similarity calculation is placed between the two steps of using the large model for speech organization to vectorize the text and model reasoning, thereby saving the extra time and space occupied by using other pre-quantization models to vectorize the text. The model vector trained with the conference text is used to calculate the similarity between texts, which has a higher segmentation accuracy than that obtained by calculating the similarity using other vectorization models.

[0058] Step 120 , extracting elements based on the speech editing results and the handwritten meeting minutes of the current participants of the target meeting to obtain meeting element information.

[0059] In the prior art, meeting minutes are generated only based on the audio data of the meeting. In the embodiments of the present application, full use is made of conference resources, and the speech transcript results obtained by transcribing the audio data of the meeting and the handwritten meeting records of the meeting are combined to jointly generate meeting minutes and other meeting-related outputs, thereby improving the accuracy of the generated meeting minutes and other meeting-related outputs.

[0060] Optionally, the extraction of elements based on the speech manuscript arrangement result and the handwritten meeting minutes of the current participants of the target meeting to obtain meeting element information includes: character recognition of the handwritten meeting minutes of the current participants of the target meeting to obtain the handwritten meeting minutes text; segmentation of the handwritten meeting minutes text based on a pre-constructed professional vocabulary to obtain the first segmentation; obtaining sentences including a specified number of the first segmentations in the speech manuscript arrangement result as the first meeting element information; selecting sentences whose similarity to the handwritten meeting minutes text meets a preset similarity condition from the sentences included in the speech manuscript arrangement result other than the first meeting element information as the second meeting element information; using the first meeting element information and the second meeting element information as the obtained meeting element information. The specified number is set according to specific needs. For example, the value of the specified number can be greater than or equal to 2.

[0061] Optionally, optical character scanning and recognition may be performed on the handwritten meeting minutes of the current participants of the target meeting to obtain the text of the handwritten meeting minutes of the current participants.

[0062] Next, the handwritten meeting minutes text is segmented by comparing it with a pre-built domain-specific word library, and stop words are removed to obtain the segmentation in the handwritten meeting minutes text as the first segmentation. The vocabulary in the professional word library is determined according to the application scenario of the meeting minutes generation method disclosed in the embodiment of the present application, including but not limited to: professional vocabulary in the medical industry, communication industry, e-commerce industry, etc.

[0063] By building a professional vocabulary for word segmentation, longer professional vocabulary can be completely segmented into one word, and words that are not in the professional vocabulary can be segmented according to daily habits.

[0064] Afterwards, sentences including two or more of the first participles in the speech editing result are selected as the first meeting element information.

[0065] On the other hand, the first word segmentation and other sentences in the speech editing result except the sentence serving as the first meeting element information can be further encoded to obtain the encoding vector of the first word segmentation and the encoding vector of other sentences in the speech editing result except the sentence serving as the first meeting element information. Then, the normalized text similarity matrix between the handwritten meeting minutes and each remaining speech editing text is calculated based on the encoding vector, and sentences in the speech editing text whose similarity with the handwritten meeting minutes is higher than a set threshold are selected, sorted according to the similarity and returned as the second meeting element information.

[0066] Optionally, the normalized text similarity matrix between the sentences in the handwritten meeting minutes and the edited text can be calculated by the following method: sentence-wise vectorization of the sentences in the edited text except for the sentences that serve as the first meeting element information to obtain a first high-dimensional matrix; row-wise vectorization of the text of the handwritten meeting minutes to obtain a second high-dimensional matrix; calculation of the correlation matrix and norm matrix of the first high-dimensional matrix and the second high-dimensional matrix; division of the correlation matrix and the norm matrix at corresponding positions to obtain a normalized text similarity matrix. For the specific implementation of calculating the first high-dimensional matrix and the second high-dimensional matrix, refer to the relevant description of calculating the high-dimensional matrix in the previous text; for the specific implementation of calculating the correlation matrix and the norm matrix of the first high-dimensional matrix and the second high-dimensional matrix, refer to the relevant description of calculating the correlation matrix and the norm matrix in the previous text. The matrix elements in the normalized text similarity matrix calculated by this method represent the semantic similarity between the corresponding row of handwritten meeting notes and the corresponding sentence in the edited text.

[0067] In the embodiment of the present application, the first conference element information and the second conference element information are taken together as the conference element information.

[0068] In some optional embodiments, after generating the meeting minutes based on the meeting overview and the result of the speech arrangement based on the element weighting, the method further includes: outputting the meeting minutes and the meeting keywords. The method for obtaining the meeting keywords is described below.

[0069] In some optional embodiments, after the handwritten meeting minutes text is segmented based on a pre-constructed professional vocabulary to obtain a first segmented word, the method further includes: segmenting the manuscript arrangement result based on a pre-constructed professional vocabulary to obtain a plurality of second segmented words; obtaining an importance index of each of the second segmented words based on the degree of association between the second segmented words; adjusting the importance index of the second segmented word based on the first segmented word; and obtaining meeting keywords based on the adjusted importance index of the second segmented word.

[0070] That is, the full text of the manuscript is segmented against a pre-built professional vocabulary, and stop words are removed. Combined with the first segmentation obtained in the previous step, highly relevant keywords are extracted.

[0071] For example, first, the speech editing result is segmented based on a pre-constructed professional vocabulary to obtain a number of second segmentations. Then, the correlation scores between the second segmentations are calculated based on the word vectors, and the correlation scores are iteratively summed to obtain the importance index of each second segmentation. Then, a higher weight is assigned to the second segmentations appearing in the handwritten meeting minutes. Based on the weight assigned to the second segmentations, the importance index of the second segmentations is adjusted. Finally, according to the adjusted importance index, a preset number of second segmentations with the highest importance indicated by the importance index are selected as meeting keywords.

[0072] The importance index vector of the second word is calculated by iteratively summing up the association scores: 1 ,s 2 ,s 3 , ..., s k ] as an example, where k represents the number of second participles. If a second participle appears in the handwritten meeting minutes, the weight of the second participle is 1+α, α>0, otherwise the weight of the second participle is 1, and the weight vector of each word is obtained [γ 1 , γ 2 , γ 3 , ..., γ k ], multiply the importance index vector by the corresponding position of the weight vector to obtain the final importance index vector of each second word segment.

[0073] Step 130 , performing content understanding on the speech editing result based on the weighted meeting element information to obtain a meeting overview and / or meeting to-do items of the target meeting.

[0074] Next, the speech editing results are weighted based on the meeting element information, and a meeting overview and / or meeting to-do items of the target meeting are generated through a pre-fine-tuned meeting understanding model.

[0075] Optionally, the content understanding of the speech arrangement results based on the weighted conference element information is performed on the speech arrangement results to obtain the meeting overview and / or meeting to-do items of the target meeting, including: dividing the speech arrangement results into at least one speech paragraph or multiple consecutive speech paragraphs based on topic relevance, wherein the paragraphs in each of the speech manuscripts correspond to the same topic; performing concurrent reasoning based on the weighted conference element information on each of the speech manuscripts through a large conference understanding model obtained by pre-fine-tuning to generate a meeting overview and / or meeting to-do items of the corresponding topic; splicing the meeting overviews of the corresponding topics according to the recording time information corresponding to the speech manuscript for generating the meeting overview to obtain the meeting overview of the target meeting, and / or splicing the meeting to-do items of different topics to obtain the meeting to-do items of the target meeting.

[0076] In order to improve the generation speed of the meeting overview, in an embodiment of the present application, the speech compilation results of each participant are first divided into multiple speech compilation results corresponding to different topics, and each speech compilation result includes one or more continuous speech paragraphs. The subsequent steps can generate the meeting overview and / or meeting to-do items of the corresponding topic based on the speech paragraphs corresponding to the same topic in parallel reasoning, and then splice the meeting overview generated by parallel reasoning to obtain the meeting overview of the target meeting, and splice the meeting to-do items to obtain the meeting to-do items of the target meeting, thereby improving the generation speed of the meeting overview and meeting to-do items.

[0077] Optionally, the manuscript arrangement result is divided into at least one manuscript consisting of one manuscript paragraph or multiple consecutive manuscript paragraphs based on topic relevance, including: obtaining the encoding vector of each manuscript paragraph in the manuscript arrangement result; calculating the normalized autocorrelation matrix between adjacent manuscript paragraphs based on the encoding vector, wherein the values ​​of the diagonal matrix elements in the normalized autocorrelation matrix represent the semantic similarity between the corresponding adjacent manuscript paragraphs; and dividing the consecutive paragraphs of the same topic in the manuscript arrangement result into the same manuscript according to the relationship between the semantic similarity and the preset topic relevance threshold.

[0078] Optionally, the pre-fine-tuned large conference comprehension model can be used to vectorize each speech paragraph in the speech arrangement result to obtain the encoding vector of each speech paragraph.

[0079] Optionally, a normalized autocorrelation matrix between adjacent manuscript paragraphs is calculated based on the coding vector, including: establishing a high-dimensional matrix of manuscript paragraphs according to the coding vectors of each manuscript paragraph; calculating the autocorrelation matrix and norm matrix of the high-dimensional matrix; dividing the autocorrelation matrix and the norm matrix at corresponding positions to obtain a text normalized autocorrelation matrix; clearing the matrix elements corresponding to non-adjacent manuscript paragraphs in the text normalized autocorrelation matrix to zero to obtain a normalized autocorrelation matrix of adjacent manuscript paragraphs.

[0080] For a specific implementation of calculating the normalized autocorrelation matrix between adjacent manuscript paragraphs based on the encoding vector, refer to the implementation of calculating the normalized autocorrelation matrix between adjacent sentences in a paragraph in the previous text, which will not be repeated here.

[0081] Next, according to the magnitude relationship between the semantic similarity and the preset topic-related threshold, the continuous paragraphs of the same topic in the manuscript arrangement result are segmented into the same manuscript. For example, when the semantic similarity represented by the matrix element is greater than the preset topic-related threshold, the corresponding adjacent manuscript paragraphs can be segmented into the same manuscript; if the semantic similarity represented by the matrix element is less than or equal to the preset topic-related threshold, the corresponding adjacent manuscript paragraphs can be segmented into different manuscripts.

[0082] Afterwards, each speech can be input into the conference comprehension model to perform conference overview reasoning.

[0083] Since each speech manuscript corresponds to a different topic and the content is independent of each other, each speech manuscript can be inferred independently. In the embodiment of the present application, a parallel reasoning method can be used to perform conference-related output reasoning by topic, which is conducive to shortening the cumulative time consumed by long text reasoning generation and avoiding the omission of key information caused by the distraction of the model due to the length of the text.

[0084] Optionally, the conference understanding large model obtained through pre-fine-tuning performs concurrent reasoning based on the weighted conference element information on each of the manuscripts to generate a conference overview and / or a conference to-do list for the corresponding topic, including: for the current manuscript, based on the conference element information in the current manuscript and the manuscript, generating a prompt text with weighted conference element information; based on the prompt text, triggering the pre-fine-tuned conference understanding large model to generate a conference overview and / or a conference to-do list for the corresponding topic.

[0085] In some optional embodiments, the prompt text includes but is not limited to: the text content of the current speech draft as input and output requirements. The input content includes the current speech draft, and the output requirements include but are not limited to: content requirements and data format requirements of the output meeting overview.

[0086] In some other optional embodiments, the output requirements also include: meeting to-do items.

[0087] The output of the meeting understanding model is determined according to the specific requirements of generating the meeting minutes. For example, if the meeting minutes do not need to include the meeting to-do items, the output requirements in the prompt text may not include the meeting to-do items.

[0088] For example, when generating prompt text for calling the conference comprehension big model based on each speech manuscript, text prompting the big model to focus on the meeting element information can be added to the prompt text, so that when the conference comprehension big model generates outputs such as a meeting overview based on the prompt text reasoning, it not only refers to the current speech manuscript, but also focuses on important sentences in the current speech manuscript, such as the first meeting element information and the second meeting element information in the meeting element information.

[0089] For example, a prompt message such as "This paragraph contains the main points of the meeting, please pay special attention" can be added before the paragraph of the speech containing the first meeting element information or the second meeting element information to prompt the meeting comprehension model to focus on the paragraph of the speech, thereby achieving the purpose of the weighted reasoning process based on the meeting element information.

[0090] Afterwards, the meeting overviews generated for different topics are sequentially spliced ​​according to the recording time sequence corresponding to the speech manuscripts used to generate the meeting overviews, so as to obtain the meeting overview of the target meeting.

[0091] On the other hand, when the output content includes meeting to-dos, the meeting to-do texts generated from the speech manuscripts for different topics in the target meeting can be spliced ​​together to obtain all the to-do items of the long meeting text without losing the content.

[0092] Step 140: Generate meeting minutes based on the meeting overview and / or meeting to-do items, and the transcript editing results.

[0093] Optionally, the generation of meeting minutes based on the meeting overview and / or meeting to-do items, and the speech manuscript arrangement result includes: in response to the length of the speech manuscript arrangement result being greater than a first length threshold, the meeting overview is used as a candidate input text, otherwise, the speech manuscript arrangement result is used as a candidate input text; in response to the length of the candidate input text being greater than a second length threshold, key information is extracted from the candidate input text to obtain an input text, otherwise, the candidate input text is used as the input text; the input text is input into a pre-fine-tuned meeting summary large model for reasoning to generate the meeting minutes body; and meeting minutes are generated based on the meeting minutes body and / or the meeting to-do items. The first length threshold and the second length threshold are set according to the input conditions and test results of the meeting summary large model, and the first length threshold is greater than the second length threshold.

[0094] In an embodiment of the present application, when generating meeting minutes according to the data output in the aforementioned steps, if the aforementioned steps generate a meeting to-do list, the meeting to-do list can be used as part of the meeting minutes, or the meeting minutes can be generated only based on the meeting overview and the speech manuscript editing results.

[0095] In an embodiment of the present application, it is necessary to judge the input of the meeting summary based on the character length of the speech editing result and the character length of the meeting overview. For example, when the length of the speech editing text exceeds 7500 characters, the meeting overview text can be used as the candidate input text for this part, otherwise the speech editing result is used as the candidate input text for this part. Then, it is determined whether the length of the candidate input text is greater than 2500 characters. If it is greater than 2500 characters, a text summary is extracted from the candidate input text using a high-correlation information extraction method as the actual input text, thereby shortening the actual input text length to less than 2500 characters according to the importance of the sentence, ensuring that all important meeting information is passed to the large model.

[0096] Optionally, extracting key information from the candidate input text to obtain the input text includes: obtaining an importance index of each sentence based on the degree of association between sentences in the candidate input text; weighting the importance index of each sentence based on preset weighting factors to obtain a weighted importance index of each sentence; extracting and concatenating the sentences according to the weighted importance index to obtain the input text.

[0097] In some optional embodiments, the sentences in the candidate input text may be vectorized to obtain the encoding vector of each sentence, and the encoding vector may be used to calculate the degree of association between each sentence, and then the degree of association may be iteratively summed to obtain the sum value as the importance index. The value of the importance index is positively correlated with the importance of the sentence.

[0098] Optionally, the preset weighted factors include, but are not limited to: one or more of the handwritten meeting minutes factors and the target meeting participant factors. For example, the sentence containing the first participle in the handwritten meeting minutes can be multiplied by a higher weight coefficient, and the speech of the target meeting participant can be multiplied by a higher weight coefficient, and the importance index of each sentence can be adjusted to obtain the weighted importance index of each sentence. Afterwards, sentences with high weighted importance indexes (i.e., important sentences) are selected as input texts in descending order of weighted importance indexes, the length of the input text is controlled within a specified range, and the selected important sentences are spliced ​​in the order of sentences in the original text to obtain the actual input text.

[0099] By using the above method to obtain the input text, the integrity of the meeting summary content is ensured, while avoiding the excessive consumption of computing resources and the distraction of model attention that may lead to the omission of key information due to the input of long meeting text. In addition, using different inputs to obtain meeting minutes for meetings of different lengths can ensure that concise summary results are obtained while the information is complete.

[0100] Finally, the actual input text obtained by screening is passed into the meeting summary model to obtain the meeting summary result as the main body of the meeting minutes. Optionally, the meeting summary model can use parallel reasoning operators to reason about the meeting summaries of different users.

[0101] After obtaining the subject of the meeting minutes, you can output the meeting minutes that only include the main body of the meeting minutes, or you can output the meeting minutes that include the main body of the meeting minutes and the meeting to-do items according to specific needs.

[0102] In some other optional embodiments, the conference element information further includes: conference element information determined according to target participants.

[0103] Optionally, after extracting elements based on the speech editing result and the handwritten meeting minutes of the current participants of the target meeting and obtaining the meeting element information, the method further includes: determining the target participants of the target meeting; obtaining the speech editing result corresponding to the target participants, and expanding the meeting element information. The target participants include but are not limited to: meeting leaders.

[0104] The target meeting participants can be determined by voice and voiceprint recognition, or can be manually set by a collection device that collects audio data of the target meeting. For example, after collecting the audio data of the target meeting and completing voice recognition and voiceprint recognition, the identified meeting participant number of the target meeting can be output for the user to select the target meeting participant. The speech of the target meeting participant has higher importance in the subsequent speech processing process.

[0105] In the subsequent process of content understanding based on the results of the speech compilation, when generating prompt text, for the speech paragraphs including the speeches of the target participants, prompt information such as "This paragraph is the speech of the leader, please pay special attention" can be spliced ​​in front of the corresponding paragraph to prompt the meeting comprehension model to focus on the speech of this paragraph, thereby achieving the purpose of the weighted reasoning process based on the said meeting element information.

[0106] In order to make this solution clearer, the training methods of the speech editing big model, the meeting understanding big model, and the meeting summary big model used in the embodiments of this application are illustrated below.

[0107] (I) Large Model for Manuscript Arrangement

[0108] The large speech editing model is used to infer and generate concise speech editing result text based on one or more input speech texts of the same participant.

[0109] Optionally, the large model for manuscript arrangement can be based on Figure 2 The training samples described in are used to fine-tune the pre-trained generative large model. Among them, the training samples for fine-tuning the speech editing large model can be a data pair including two fields, input and output, the input is: the speech transcribed text obtained by speech recognition of the audio data of the meeting, which is divided into short paragraphs that are unrelated to the context according to the speaker + semantic similarity; the output is: the corresponding written text after manually removing redundancy / correcting errors in the text). Use some of the above data to perform supervised fine-tuning training on some parameters of the existing pre-trained generative large model base to obtain a large model specifically for speech editing, that is, a speech editing large model. The specific implementation method of fine-tuning the pre-trained generative large model based on the training samples to obtain the speech editing large model refers to the prior art, which will not be repeated in the embodiments of the present application.

[0110] 2. Conference Understanding Model

[0111] The conference comprehension model is used to generate a conference overview and other content based on an input prompt text, wherein the prompt text includes: a speech manuscript corresponding to a specified topic for a specified participant, a weighted prompt text of conference element information, an output content description, and the like.

[0112] Optionally, by collecting audio data and handwritten meeting record data of various types of meetings in various departments and industries, using voice, voiceprint, optical character recognition, and the speech editing technology described in the previous embodiment, the written text of the meeting voice and the handwritten meeting record text of the entire meeting text are obtained according to the participants. After removing sensitive information from each written text of the meeting voice, the annotation personnel will split it into multiple short stories according to the subject content. The length of each short story is between 1,000 and 2,000 characters, and mark which leader is the speaker in the meeting, and at the same time give the meeting theme, main content and meeting to-do items contained in each short story.

[0113] Furthermore, the conference element information contained in each short story may be extracted by referring to the conference element information extraction method described above, such as sentences as the first conference element information, sentences as the second conference element information, and sentences of leaders' speeches.

[0114] Afterwards, training data is constructed based on the above information, and the training data includes several data pairs. Figure 3 As shown, each data pair includes two fields, input and output, wherein the input is: prompt text, which includes the result of the speech arrangement of the specified topic, prompt information of the weighted meeting elements, and output description information of the meeting overview and / or meeting to-do items, etc.; the output is: manually annotated meeting overview and / or meeting to-do items. Using several of the above data pairs, supervised fine-tuning training of some parameters of the existing pre-trained generative large model base is performed to obtain a large model specifically used to generate meeting-related content, namely, a meeting understanding large model. For the specific implementation method of fine-tuning the pre-trained generative large model based on training samples to obtain the meeting understanding large model, please refer to the prior art and will not be repeated in the embodiments of this application.

[0115] (III) Meeting summary model

[0116] The meeting summary large model is used to generate meeting minutes based on key meeting information within a given length. In the training stage of the meeting summary large model, the method for generating speech manuscript arrangement results and meeting overviews described above can be referred to, and speech manuscript arrangement results and meeting overviews can be generated for the audio data and handwritten meeting records of each meeting respectively, and the method described in the full text can be used to extract the input text actually input into the meeting summary large model from the speech manuscript arrangement results and meeting overviews of each meeting, and manually annotate the meeting minutes that need to be generated. Afterwards, based on the data pairs consisting of the input text and the annotated meeting minutes of each meeting, the existing pre-trained generative large model base is fine-tuned with partial parameters to obtain a large model specifically used to generate meeting minutes, namely, the meeting summary large model. For the specific implementation method of fine-tuning the pre-trained generative large model based on the training samples to obtain the meeting summary large model, please refer to the prior art, which will not be repeated in the embodiments of this application.

[0117] In summary, the meeting minutes generation method disclosed in the embodiment of the present application performs speech recognition and voiceprint recognition on the audio data of the target meeting to obtain speech manuscript arrangement results that distinguish the participants; based on the speech manuscript arrangement results and the handwritten meeting records of the current participants of the target meeting, element extraction is performed to obtain meeting element information; the speech manuscript arrangement results are subjected to weighted content understanding based on the meeting element information to obtain a meeting overview and / or meeting to-do items of the target meeting; based on the meeting overview and / or meeting to-do items, and the speech manuscript arrangement results, meeting minutes are generated. This method further improves the completeness and accuracy of key information in the generated meeting minutes by extracting meeting elements from the speeches of participants in combination with handwritten meeting records, and weighting the content understanding based on the meeting elements, and can highlight the key content of the meeting.

[0118] Furthermore, by segmenting the manuscript compilation results based on the participants and semantic similarity, dividing the manuscript paragraphs into chapters based on themes, and performing parallel reasoning based on paragraphs and topics, the speed of manuscript compilation and content comprehension is effectively improved, thereby further improving the efficiency of meeting minutes generation.

[0119] Reference Figure 4 The present application also discloses a device for generating meeting minutes, the device comprising:

[0120] The speech manuscript arrangement module 400 is used to perform speech recognition and voiceprint recognition on the audio data of the target conference to obtain speech manuscript arrangement results that distinguish the participants;

[0121] The conference element information acquisition module 410 is used to extract elements and acquire conference element information based on the speech manuscript arrangement result and the handwritten conference minutes of the current participants of the target conference;

[0122] A meeting overview and to-do generating module 420 is used to perform content understanding on the speech editing result based on the weighted meeting element information to obtain a meeting overview and / or meeting to-do items of the target meeting;

[0123] The meeting minutes generation module 430 is used to generate meeting minutes based on the meeting overview and / or meeting to-do items, and the speech editing results.

[0124] Optionally, extracting elements based on the speech manuscript arrangement result and the handwritten meeting minutes of the current participants of the target meeting to obtain meeting element information includes:

[0125] Performing character recognition on the handwritten meeting minutes of the current participants of the target meeting to obtain the handwritten meeting minutes text;

[0126] Performing word segmentation processing on the handwritten conference record text based on a pre-built professional word library to obtain a first word segmentation;

[0127] Acquire sentences including a specified number of the first participles in the speech manuscript arrangement result as first conference element information;

[0128] Selecting, from the sentences other than the first conference element information included in the speech manuscript arrangement result, sentences whose similarity with the handwritten conference record text meets a preset similarity condition as second conference element information;

[0129] The first conference element information and the second conference element information are used as the acquired conference element information.

[0130] Optionally, the step of performing content understanding on the speech editing result based on the weighted meeting element information to obtain a meeting overview and / or meeting to-do items of the target meeting includes:

[0131] Based on the topic relevance, the manuscript arrangement result is divided into at least one manuscript consisting of one manuscript paragraph or a plurality of consecutive manuscript paragraphs, wherein each paragraph in the manuscript corresponds to the same topic;

[0132] Through the pre-fine-tuned large model of conference understanding, weighted concurrent reasoning based on the conference element information is performed on each of the speech manuscripts to generate a conference overview and / or conference to-do items for the corresponding topic;

[0133] According to the recording time information corresponding to the speech manuscript used to generate the meeting overview, the meeting overviews of the corresponding topics are spliced ​​to obtain the meeting overview of the target meeting, and / or the meeting to-do items of different topics are spliced ​​to obtain the meeting to-do items of the target meeting.

[0134] Optionally, after extracting elements based on the speech editing result and the handwritten meeting minutes of the current participants of the target meeting to obtain the meeting element information, the device further includes:

[0135] A conference element information expansion module is used to determine the target participants of the target conference, and

[0136] The speech manuscript arrangement result corresponding to the target participant is obtained, and the conference element information is expanded.

[0137] Optionally, the conference understanding macro model obtained by pre-fine-tuning performs concurrent reasoning based on the weighted conference element information on each speech manuscript to generate a conference overview and / or conference to-do items for the corresponding topic, including:

[0138] For the current speech draft, based on the conference element information in the current speech draft and the speech draft, generating a prompt text weighted with respect to the conference element information;

[0139] The pre-fine-tuned conference comprehension model is triggered based on the prompt text to generate a conference overview and / or conference to-do items for the corresponding topic.

[0140] Optionally, generating meeting minutes based on the meeting overview and / or meeting to-do items and the speech transcript arrangement result includes:

[0141] In response to the length of the speech manuscript arrangement result being greater than a first length threshold, taking the meeting overview as a candidate input text; otherwise, taking the speech manuscript arrangement result as a candidate input text;

[0142] In response to the length of the candidate input text being greater than a second length threshold, extracting key information from the candidate input text to obtain an input text; otherwise, taking the candidate input text as the input text;

[0143] Inputting the input text into the pre-fine-tuned meeting summary model for reasoning to generate the main body of the meeting minutes;

[0144] Based on the meeting minutes body and / or the meeting to-do items, generate meeting minutes.

[0145] Optionally, extracting key information from the candidate input text to obtain the input text includes:

[0146] Obtaining an importance index for each sentence according to the degree of association between the sentences in the candidate input text;

[0147] Performing weighted processing on the importance index of each of the sentences based on preset weighting factors to obtain a weighted importance index of each of the sentences;

[0148] The sentences are extracted and concatenated according to the weighted importance index to obtain the input text.

[0149] Optionally, performing speech recognition and voiceprint recognition on the audio data of the target conference to obtain speech transcripts for distinguishing the participants includes:

[0150] Perform speech recognition and voiceprint recognition on the audio data of the target meeting to obtain the speech text corresponding to each participant;

[0151] The speech text corresponding to each participant is divided into text paragraphs that are semantically irrelevant to the context, so as to obtain speech paragraphs that distinguish the participants;

[0152] Each of the speech paragraphs is input into a pre-fine-tuned speech editing large model for parallel reasoning to obtain speech editing results that distinguish the participants.

[0153] The meeting minutes generating device disclosed in the embodiment of the present application is used to implement the meeting minutes generating method described in the embodiment of the present application. The specific implementation methods of each module of the device will not be repeated here, and reference can be made to the specific implementation methods of the corresponding steps in the method embodiment.

[0154] The meeting minutes generation device disclosed in the embodiment of the present application performs speech recognition and voiceprint recognition on the audio data of the target meeting to obtain speech manuscript arrangement results that distinguish the participants; based on the speech manuscript arrangement results and the handwritten meeting records of the current participants of the target meeting, element extraction is performed to obtain meeting element information; the speech manuscript arrangement results are subjected to weighted content understanding based on the meeting element information to obtain a meeting overview and / or meeting to-do items of the target meeting; based on the meeting overview and / or meeting to-do items, and the speech manuscript arrangement results, meeting minutes are generated. This device extracts meeting elements from the speeches of participants in combination with handwritten meeting records, and weights the content understanding based on the meeting elements, thereby further improving the completeness and accuracy of key information in the generated meeting minutes, and can highlight the key content of the meeting.

[0155] Furthermore, by segmenting the manuscript compilation results based on the participants and semantic similarity, dividing the manuscript paragraphs into chapters based on themes, and performing parallel reasoning based on paragraphs and topics, the speed of manuscript compilation and content comprehension is effectively improved, thereby further improving the efficiency of meeting minutes generation.

[0156] Each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0157] The above is a detailed introduction to a meeting minutes generation method and device provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

[0158] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0159] The various component embodiments of the present application can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all functions of some or all components in the electronic device according to the embodiment of the present application. The present application can also be implemented as a device or apparatus program (e.g., computer program and computer program product) for executing part or all of the methods described herein. Such a program implementing the present application can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0160] For example, Figure 5 An electronic device that can implement the method according to the present application is shown. The electronic device can be a PC, a mobile terminal, a personal digital assistant, a tablet computer, etc. The electronic device traditionally includes a processor 510 and a memory 520 and a program code 530 stored on the memory 520 and can be run on the processor 510, and the processor 510 implements the method described in the above embodiment when executing the program code 530. The memory 520 can be a computer program product or a computer-readable medium. The memory 520 can be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk or a ROM. The memory 520 has a storage space 5201 for the program code 530 of the computer program for executing any method step in the above method. For example, the storage space 5201 for the program code 530 can include various computer programs for implementing the various steps in the above method respectively. The program code 530 is a computer-readable code. These computer programs can be read from one or more computer program products or written into the one or more computer program products. These computer program products include program code carriers such as hard disks, compact disks (CDs), memory cards or floppy disks. The computer program includes a computer readable code, and when the computer readable code is executed on an electronic device, the electronic device is caused to execute the method according to the above embodiment.

[0161] The embodiment of the present application also discloses a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method for generating meeting minutes as described in the embodiment of the present application are implemented.

[0162] Such a computer program product may be a computer-readable storage medium, which may have a computer-readable storage medium having a Figure 5 The memory 520 in the electronic device shown in the figure is similar to the storage segment, storage space, etc. The program code can be compressed and stored in the computer-readable storage medium in an appropriate form. The computer-readable storage medium is generally as shown in the reference Figure 6 The portable or fixed storage unit. Generally, the storage unit includes computer readable code 530', which is a code read by a processor. When the code is executed by the processor, each step in the method described above is implemented.

[0163] The term "one embodiment", "embodiment" or "one or more embodiments" herein means that a particular feature, structure or characteristic described in conjunction with the embodiment is included in at least one embodiment of the present application. In addition, please note that the examples of the term "in one embodiment" here do not necessarily all refer to the same embodiment.

[0164] In the description provided herein, a large number of specific details are described. However, it is understood that the embodiments of the present application can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0165] In the claims, any reference signs placed between brackets shall not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present application may be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In a unit claim enumerating several means, several of these means may be embodied by the same item of hardware. The use of the words first, second, and third etc. does not indicate any order. These words may be interpreted as names.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating meeting minutes, characterized in that: The method comprises: Perform speech recognition and voiceprint recognition on the audio data of the target meeting to obtain the transcript results that distinguish the participants; Based on the speech editing results and the handwritten meeting minutes of the current participants of the target meeting, element extraction is performed to obtain meeting element information; Performing content understanding on the speech editing results based on the weighted meeting element information to obtain a meeting overview and / or meeting to-do items of the target meeting; Based on the meeting overview and / or meeting to-do items, and the transcript editing results, a meeting minutes is generated.

2. The method according to claim 1, characterized in that The extracting elements based on the speech editing result and the handwritten meeting minutes of the current participants of the target meeting to obtain the meeting element information includes: Performing character recognition on the handwritten meeting minutes of the current participants of the target meeting to obtain the handwritten meeting minutes text; Performing word segmentation processing on the handwritten conference record text based on a pre-built professional word library to obtain a first word segmentation; Acquire sentences including a specified number of the first participles in the speech manuscript arrangement result as first conference element information; Selecting, from the sentences other than the first conference element information included in the speech manuscript arrangement result, sentences whose similarity with the handwritten conference record text meets a preset similarity condition as second conference element information; The first conference element information and the second conference element information are used as the acquired conference element information.

3. The method according to claim 1, characterized in that The step of performing content understanding on the speech editing result based on the weighted meeting element information to obtain a meeting overview and / or meeting to-do items of the target meeting includes: Based on the topic relevance, the manuscript arrangement result is divided into at least one manuscript consisting of one manuscript paragraph or a plurality of consecutive manuscript paragraphs, wherein each paragraph in the manuscript corresponds to the same topic; Through the pre-fine-tuned large model of conference understanding, weighted concurrent reasoning based on the conference element information is performed on each of the speech manuscripts to generate a conference overview and / or conference to-do items for the corresponding topic; According to the recording time information corresponding to the speech manuscript used to generate the meeting overview, the meeting overviews of the corresponding topics are spliced ​​to obtain the meeting overview of the target meeting, and / or the meeting to-do items of different topics are spliced ​​to obtain the meeting to-do items of the target meeting.

4. The method according to claim 1, characterized in that: After extracting elements based on the speech editing result and the handwritten meeting minutes of the current participants of the target meeting to obtain the meeting element information, the method further includes: Determining target participants of the target meeting; The speech manuscript arrangement result corresponding to the target participant is obtained, and the conference element information is expanded.

5. The method according to claim 3 or 4, characterized in that: The conference comprehension macro model obtained by pre-fine-tuning performs concurrent reasoning based on the weighted conference element information on each speech manuscript to generate a conference overview and / or conference to-do items for the corresponding topic, including: For the current speech draft, based on the conference element information in the current speech draft and the speech draft, generating a prompt text weighted with respect to the conference element information; The pre-fine-tuned conference comprehension model is triggered based on the prompt text to generate a conference overview and / or conference to-do items for the corresponding topic.

6. The method according to claim 1, characterized in that Generating meeting minutes based on the meeting overview and / or meeting to-do items and the speech editing results includes: In response to the length of the speech manuscript arrangement result being greater than a first length threshold, taking the meeting overview as a candidate input text; otherwise, taking the speech manuscript arrangement result as a candidate input text; In response to the length of the candidate input text being greater than a second length threshold, extracting key information from the candidate input text to obtain an input text; otherwise, taking the candidate input text as the input text; Inputting the input text into the pre-fine-tuned meeting summary model for reasoning to generate the main body of the meeting minutes; Based on the meeting minutes body and / or the meeting to-do items, generate meeting minutes.

7. The method according to claim 6, characterized in that The step of extracting key information from the candidate input text to obtain the input text includes: Obtaining an importance index for each sentence according to the degree of association between the sentences in the candidate input text; Performing weighted processing on the importance index of each of the sentences based on preset weighting factors to obtain a weighted importance index of each of the sentences; The sentences are extracted and concatenated according to the weighted importance index to obtain the input text.

8. The method according to claim 1, characterized in that: The speech recognition and voiceprint recognition are performed on the audio data of the target conference to obtain the speech transcripts for distinguishing the participants, including: Perform speech recognition and voiceprint recognition on the audio data of the target meeting to obtain the speech text corresponding to each participant; The speech text corresponding to each participant is divided into text paragraphs that are semantically irrelevant to the context, so as to obtain speech paragraphs that distinguish the participants; Each of the speech paragraphs is input into a pre-fine-tuned speech editing large model for parallel reasoning to obtain speech editing results that distinguish the participants.

9. A device for generating meeting minutes, characterized in that: The device comprises: The speech transcript editing module is used to perform speech recognition and voiceprint recognition on the audio data of the target meeting to obtain speech transcript editing results that distinguish the participants; A conference element information acquisition module, used to extract elements and acquire conference element information based on the speech manuscript arrangement result and the handwritten meeting minutes of the current participants of the target meeting; A meeting overview and to-do generating module, configured to perform content understanding on the speech editing result based on the weighted meeting element information to obtain a meeting overview and / or meeting to-do items of the target meeting; The meeting minutes generation module is used to generate meeting minutes based on the meeting overview and / or meeting to-do items, and the speech manuscript arrangement results.

10. An electronic device comprising a memory, a processor, and a program code stored in the memory and executable on the processor, characterized in that: When the processor executes the program code, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium having a program code stored thereon, characterized in that: When the program code is executed by a processor, the steps of the method described in any one of claims 1 to 8 are implemented.

Images