Meeting record generation method and device, equipment and storage medium

Abstract

The present invention relates to the field of artificial intelligence, and discloses a method, apparatus, device, and storage medium for generating conference records, which are used to solve the technical problem of large errors in information content in automatically generated conference records in the prior art. The method comprises: obtaining a conference audio file to be recorded and identified; calling a preset speech recognition model to perform speech recognition on the conference audio file to obtain text content in the conference audio file; calling a preset voiceprint extraction model to perform voiceprint feature extraction on the conference audio file to obtain voiceprint features in the conference audio file, wherein there is at least one voiceprint feature; based on the voiceprint features, searching for participant information corresponding to the voiceprint features in a preset voiceprint information library; annotating the text content with speakers based on the participant information to obtain speaker information; determining the work project to which the meeting belongs based on the participant information, and generating a conference record based on the work project, the text content, and the corresponding speaker information.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, and in particular to a conference record generation method and device, equipment and a storage medium. BACKGROUND

[0002] With the development of artificial intelligence, speech recognition technology is gradually mature, and voice can be converted into text through speech recognition, which can assist or replace repeated human labor to record and summarize meetings. In addition, with the development of voiceprint recognition technology, the source of the voice can be determined through voiceprint recognition, and the combination of speech recognition and voiceprint recognition can distinguish and label the text of different speakers, which can make the conference record more accurate and efficient.

[0003] In the existing technology, the recognized text content can only be labeled according to the recognized voiceprint to distinguish the speech content of different speakers, and the specific speaker cannot be determined according to the work project of the meeting, resulting in that the conference record obtained by recognition is not accurate enough and has a large error with the audio. SUMMARY

[0004] The main purpose of the present application is to solve the technical problem of large information content error in the automatically generated conference record in the prior art.

[0005] The first aspect of the present application provides a conference record generation method, comprising: obtaining a conference audio file to be recorded and recognized; calling a preset speech recognition model to perform speech recognition on the conference audio file to obtain text content in the conference audio file; calling a preset voiceprint extraction model to perform voiceprint feature extraction on the conference audio file to obtain voiceprint features in the conference audio file, wherein the voiceprint features are at least one; according to the voiceprint features, searching for participant information corresponding to the voiceprint features in a preset voiceprint information library; according to the participant information, labeling the text content with a speaker to obtain speaker information; determining a work project to which the conference belongs according to the participant information, and generating a conference record according to the work project, the text content and the corresponding speaker information.

[0006] Optionally, in the first implementation manner of the first aspect of the present application, after the conference record is generated according to the work project, the text content and the corresponding speaker information, the method further comprises: performing text semantic recognition on the conference record content to extract core viewpoints and task content of the conference record; generating a conference summary based on the core viewpoints and generating a to-do list based on the task content; and updating the content of the conference record according to the conference summary and the to-do list.

[0007] Optionally, in the second implementation form of the first aspect of the present application, the speech recognition model comprises a feature extraction layer, a tone recognition layer and a text sequence matching layer, and the calling the preset speech recognition model to perform speech recognition on the conference audio file to obtain the text content in the conference audio file comprises: calling the feature extraction layer to perform spectral feature extraction on the conference audio file to obtain an audio feature spectrum; calling the tone recognition layer to split the audio feature spectrum to obtain a plurality of feature spectrum pieces, and performing pronunciation tone recognition on the plurality of feature spectrum pieces to obtain a pinyin sequence; and calling the text sequence matching layer to perform text sequence matching based on the pinyin sequence to obtain the text content of the conference audio file.

[0008] Optionally, in the third implementation form of the first aspect of the present application, the calling the preset voiceprint extraction model to perform voiceprint feature extraction on the conference audio file to obtain the voiceprint feature in the conference audio file comprises: inputting the conference audio file into the voiceprint extraction model to perform voiceprint feature extraction, to obtain acoustic spectral feature parameters, lexical feature parameters, prosodic feature parameters and accent feature parameters in the conference audio file, wherein the voiceprint extraction model is constructed in advance based on a radial basis function neural network; and generating voiceprint parameters based on the acoustic spectral feature parameters, the lexical feature parameters, the prosodic feature parameters and the accent feature parameters to obtain the voiceprint feature in the conference audio file.

[0009] Optionally, in the fourth implementation form of the first aspect of the present application, the generating the conference record according to the work item, the text content and the corresponding speaker information comprises: determining a current conference related field according to the work item information; searching a preset related field dictionary according to the conference related field; adjusting the text content according to the related field dictionary to correct inaccurate words to obtain corrected text content; and generating the conference record according to the corrected text content and the corresponding speaker information.

[0010] Optionally, in the fifth implementation form of the first aspect of the present application, before the conference audio file to be recorded and recognized is acquired, the method further comprises: collecting a speech sample of an employee, performing denoising processing on the speech sample to obtain a denoised speech sample; performing quality detection on the denoised speech sample to determine whether the speech quality meets a preset sample threshold; if yes, performing feature extraction on the denoised speech sample to obtain a registered voiceprint feature; acquiring preset employee information, associating the registered voiceprint feature with the employee information, and obtaining a voiceprint information library based on the employee information and the registered voiceprint feature.

[0011] Optionally, in a sixth implementation form of the first aspect of the present application, after the speaker annotation on the text content based on the participant information is performed, the method further includes: performing speech emotion recognition on the conference audio file based on the preset audio emotion recognition model to obtain emotion feature parameters; performing emotion state annotation on the text content based on the emotion feature parameters to obtain emotion state labels; outputting a probability of achieving a conference target based on the emotion state labels, and annotating the text content based on the conference target and the probability.

[0012] The second aspect of the present application provides a conference record generation device, which includes: an acquisition module configured to acquire a conference audio file to be recognized; a recognition module configured to perform speech recognition on the conference audio file based on a preset speech recognition model to obtain text content in the conference audio file; an extraction module configured to perform voiceprint feature extraction on the conference audio file based on a preset voiceprint extraction model to obtain voiceprint features in the conference audio file, wherein the voiceprint features are at least one; a search module configured to search for participant information corresponding to the voiceprint features in a preset voiceprint information library based on the voiceprint features; an annotation module configured to perform speaker annotation on the text content based on the participant information to obtain speaker information; and a generation module configured to determine a work project to which the conference belongs based on the participant information, and generate a conference record based on the work project, the text content, and the corresponding speaker information. Optionally, in a first implementation form of the second aspect of the present application, the conference record generation device further includes a content update module, which includes: an extraction unit configured to perform text semantic recognition on the conference record content to extract core viewpoints and task content of the conference record; a content generation unit configured to generate a conference summary based on the core viewpoints and to-do items based on the task content; and an update unit configured to update the content of the conference record based on the conference summary and the to-do items.

[0013] Optionally, in a second implementation form of the second aspect of the present application, the speech recognition model includes a feature extraction layer, a tone recognition layer, and a text sequence matching layer, and the recognition module includes: a spectrum extraction unit configured to perform spectrum feature extraction on the conference audio file based on the feature extraction layer to obtain an audio feature spectrum; a tone recognition unit configured to split the audio feature spectrum based on the tone recognition layer to obtain a plurality of feature spectrum pieces, and to perform pronunciation tone recognition on the plurality of feature spectrum pieces to obtain a pinyin sequence; and a text matching unit configured to perform text sequence matching based on the pinyin sequence based on the text sequence matching layer to obtain the text content of the conference audio file.

[0014] Optionally, in a third implementation form of the second aspect of the present application, the extraction module comprises: a parameter extraction unit, configured to input the conference audio file into a voiceprint extraction model to extract voiceprint features, to obtain acoustic spectral feature parameters, lexical feature parameters, prosodic feature parameters and accent feature parameters in the conference audio file, wherein the voiceprint extraction model is constructed in advance based on a radial basis function neural network; and a parameter calculation unit, configured to generate voiceprint parameters based on the acoustic spectral feature parameters, the lexical feature parameters, the prosodic feature parameters and the accent feature parameters, to obtain voiceprint features in the conference audio file.

[0015] Optionally, in a fourth implementation form of the second aspect of the present application, the generation module comprises: a domain determination unit, configured to determine a current conference related domain according to the work item information; a dictionary searching unit, configured to search a pre-stored related domain dictionary according to the conference related domain; a text correction unit, configured to adjust the text content according to the related domain dictionary, to correct inaccurate words, to obtain corrected text content; and a record generation unit, configured to generate a conference record according to the corrected text content and corresponding speaker information.

[0016] Optionally, in a fifth implementation form of the second aspect of the present application, the conference record generation apparatus further comprises a voiceprint library construction module, which comprises: a unit configured to collect a voice sample of an employee, to perform noise reduction processing on the voice sample to obtain a denoised voice sample; a unit configured to perform quality detection on the denoised voice sample to determine whether the voice quality meets a pre-set sample threshold; if yes, a unit configured to perform feature extraction on the denoised voice sample to obtain registered voiceprint features; a unit configured to obtain pre-stored employee information, to associate the registered voiceprint features with the employee information, and to obtain a voiceprint information library based on the employee information and the registered voiceprint features.

[0017] Optionally, in a sixth implementation form of the second aspect of the present application, the conference record generation apparatus further comprises a target labeling module, which comprises: an emotion recognition unit, configured to call a pre-stored audio emotion recognition model to perform voice emotion recognition on the conference audio file, to obtain emotion feature parameters; an emotion labeling unit, configured to perform emotion state labeling on the text content based on the emotion feature parameters, to obtain emotion state identifiers; and a target labeling unit, configured to output a probability of achieving a predicted conference target based on the emotion state identifiers, and to label the text content according to the predicted conference target and the probability.

[0018] The third aspect of the present application provides a conference record generation device, comprising a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to enable the conference record generation device to perform the steps of the conference record generation method described above.

[0019] The fourth aspect of the present application provides a computer readable storage medium, the computer readable storage medium storing instructions, when running on a computer, enabling the computer to perform the steps of the conference record generation method described above.

[0020] In the technical solution provided by the present application, a conference audio file to be recorded and recognized is obtained; a preset voice recognition model is called to perform voice recognition on the conference audio file, so as to obtain text content in the conference audio file; a preset voiceprint extraction model is called to perform voiceprint feature extraction on the conference audio file, so as to obtain voiceprint features in the conference audio file, wherein the voiceprint features are at least one; according to the voiceprint features, participant information corresponding to the voiceprint features is searched in a preset voiceprint information library; according to the participant information, speaker annotation is performed on the text content, so as to obtain speaker information; according to the participant information, a work project to which the conference belongs is determined, and a conference record is generated according to the work project, the text content and the corresponding speaker information. The technical solution of the embodiment of the present application improves the accuracy and detail of the information content of the automatically generated conference record. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 The figure is a schematic diagram of a first embodiment of the conference record generation method in the embodiment of the present application;

[0022] Figure 2 The figure is a schematic diagram of a second embodiment of the conference record generation method in the embodiment of the present application;

[0023] Figure 3 The figure is a schematic diagram of a third embodiment of the conference record generation method in the embodiment of the present application;

[0024] Figure 4 The figure is a schematic diagram of a fourth embodiment of the conference record generation method in the embodiment of the present application;

[0025] Figure 5 The figure is a schematic diagram of a CBHG model in the conference record generation method in the embodiment of the present application;

[0026] Figure 6 The figure is a schematic diagram of one embodiment of the conference record generation device in the embodiment of the present application;

[0027] Figure 7 The figure is a schematic diagram of another embodiment of the conference record generation device in the embodiment of the present application;

[0028] Figure 8 An embodiment of a conference record generation device in the embodiments of the present application is shown. DETAILED DESCRIPTION

[0029] The embodiments of the present application provide a conference record generation method, device, equipment and storage medium, obtain a conference audio file to be recorded and recognized; call a speech recognition model to perform speech recognition on the conference audio file, and obtain text content in the conference audio file; call a voiceprint extraction model to perform voiceprint feature extraction on the conference audio file, and obtain voiceprint features in the conference audio file, wherein the voiceprint features are at least one; according to the voiceprint features, find out corresponding participant information in a preset voiceprint information library; according to the participant information, perform speaker annotation on the text content, and obtain speaker information; according to the participant information, determine a work project to which the conference belongs, and generate a conference record according to the work project, the text content and the corresponding speaker information. The technical solution in the embodiments of the present application improves the accuracy and detail of information content of the automatically generated conference record.

[0030] The terms "first", "second", "third", "fourth" and the like in the description, claims, as well as throughout the tables and figures of the attached drawings, if and as exist, are used for distinguishing between similar objects and not necessarily for describing a specific sequential or chronological order. It is to be understood that the use of the terms so termed is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of efficient implementation irrespective of the specific order of steps presented in the claims or the description. Furthermore, the term "comprising" or "containing" and variations thereof as used in the description and the claims as well as the accompanying tables and figures, are to be interpreted in a non-exclusive manner, i.e. they are intended to cover the process, method, system, product, or apparatus comprising a list of steps or elements as well as any other step or element not expressly listed or inherent to such process, method, product, or apparatus.

[0031] For the sake of understanding, the specific flow of the embodiments of the present application is described below, please refer to Figure 1 An embodiment of a conference record generation method in the embodiments of the present application includes:

[0032] 101. Obtain a conference audio file to be recorded and recognized;

[0033] It can be understood that the execution subject of the present application can be a conference record generation device, and can also be a terminal or a server, and the specific implementation is not limited herein. The embodiments of the present application take the server as the execution subject for example.

[0034] The server in the embodiment can receive a conference record generation request, extract a corresponding conference audio file contained in the conference record generation request, and also can obtain speech in a conference scene requiring conference record generation in real time in response to the received conference record generation request to obtain the conference audio file.

[0035] Specifically, in the embodiment, when recording conference audio, the conference audio can be recorded by using a microphone array to improve the audio quality of the conference audio file, so that subsequent recognition is performed, the obtained conference audio is processed for the spatial characteristics of the sound field, and reverberation, echo and noise in the conference audio file are removed to obtain a conference audio file with background noise removed.

[0036] 102, call a preset voice recognition model to perform voice recognition on the conference audio file to obtain text content in the conference audio file;

[0037] The VAD (Voice Activity Detection) model is used to identify the mute time period in the conference audio file, wherein the mute time period refers to the interval of rest or pause in the conference, and the pause interval in the speech; the bottom noise and the speech endpoint of the current conference audio are obtained through the audio waveform of the mute time period. The conference audio file is denoised according to the bottom noise of the current conference audio, and the denoised conference audio file is segmented according to the speech endpoint to obtain a segmented conference audio file.

[0038] Subsequently, the segmented conference audio file is input into the preset voice recognition model, the voice recognition model first extracts the sound features of the segmented conference audio file to obtain the acoustic features of each segmented conference audio file, and then performs feature matching on the acoustic features according to the voice endpoint and the specific acoustic features obtained in this step to obtain the pronunciation corresponding to each character in the segmented conference audio file. Finally, a language model constructed by a deep neural network is used to generate the text content corresponding to the pronunciation content based on the pronunciation corresponding to each character.

[0039] 103, call a preset voiceprint extraction model to perform voiceprint feature extraction on the conference audio file to obtain voiceprint features in the conference audio file;

[0040] The segmented conference audio file obtained after the denoising and segmentation processing in the preceding step is input into a preset voiceprint extraction model to perform speech effect enhancement processing, and an enhanced segmented conference voice is obtained.

[0041] 104. According to the voiceprint feature, the voiceprint feature corresponding participant information is searched in the preset voiceprint information library.

[0042] According to the voiceprint feature data consistent with the voiceprint feature, the voiceprint feature data is searched in the preset voiceprint information library, and the participant code corresponding to the voiceprint feature is obtained according to the label of the voiceprint feature data. Then, the participant code is matched in the preset employee information library to obtain the specific participant information speaking in the current conference audio file. The participant information is pre-stored in the employee information library, including participant name, position, organization structure, project group and the like.

[0043] 105. According to the participant information, the speaker annotation of the text content is performed to obtain the speaker information.

[0044] 106. According to the participant information, the work project to which the conference belongs is determined, and the conference record is generated according to the work project, the text content and the corresponding speaker information.

[0045] According to the voiceprint feature of each conference audio recognized in the preceding step, the participant information of each conference audio is annotated, and the text content recognized in the conference audio is annotated according to the participant information, to obtain the specific speaker information of each text content.

[0046] After obtaining all the participant information in the current conference audio file, the work project to which the current conference belongs is determined based on all the speaker information in the current conference, and the specific content of the work project is obtained.

[0047] According to the specific speaker information of each text content and the specific content of the work project, the text content recognized is adjusted and annotated to generate the conference record.

[0048] The technical scheme of the embodiment of the application improves the accuracy and detail of the information content of the automatically generated conference record.

[0049] Please refer to Figure 2The second embodiment of the conference recording generation method in the embodiment of the present application comprises:

[0050] 201, obtaining a conference audio file to be recorded and recognized;

[0051] receiving a conference recording generation request, extracting the corresponding conference audio file contained in the conference recording generation request; or in response to the received conference recording generation request, obtaining the speech in the conference scene in real time, and obtaining the conference audio file.

[0052] 202, calling a feature extraction layer to perform spectral feature extraction on the conference audio file, and obtaining an audio feature spectrum;

[0053] 203, calling a tone recognition layer to divide the audio feature spectrum, obtaining a plurality of feature spectrum pieces, and performing pronunciation tone recognition on the plurality of feature spectrum pieces, and obtaining a pinyin sequence;

[0054] 204, calling a text sequence matching layer to perform text sequence matching based on the pinyin sequence, and obtaining the text content of the conference audio file;

[0055] Because the time of some conferences is long, the recording time in the audio file is long, and the file is large, in order to ensure the accuracy of recognition and improve the success rate of recognition, the conference audio file obtained in the above step is first segmented.

[0056] Specifically, when the audio file is converted, a preset VAD (Voice Activity Detection) model is called to detect the voice endpoint in the conference audio file, wherein the VAD model is also called a voice endpoint detection model or a voice boundary detection model; the VAD model is used to identify the silent time period in the conference audio file, wherein the source of the silent time period may be the interval of rest or pause in the conference, the specific position of the silent time period is labeled, and the voice endpoint is obtained.

[0057] Subsequently, a preconfigured segmentation rule is obtained, the preconfigured segmentation rule can predefine a segmentation file size threshold and / or a file time length threshold, and the conference audio file is segmented according to the position of the voice endpoint under the condition that the segmentation file size threshold and / or the file time length threshold are met, and the segmented conference audio file is obtained.

[0058] The obtained segmented conference audio file is input into a speech recognition model for text content recognition, wherein the speech recognition model includes a feature extraction layer, a tone recognition layer, and a text sequence matching layer. First, the feature extraction layer is called to extract spectral features of the conference audio file. Specifically, the segmented conference audio file is first subjected to a Fourier transform to obtain the spectrum of the conference audio file; the obtained spectrum is input into the feature extraction layer to extract feature information of the spectrum, and the feature information of the obtained spectrum is input into the tone recognition layer, wherein the tone recognition layer contains a pre-trained spectrum recognition tool, wherein the spectrum recognition tool is constructed according to a neural network algorithm; the tone recognition layer identifies the pronunciation corresponding to the spectrum based on the characteristics of the spectrum, and outputs the corresponding pinyin sequence based on the pronunciation.

[0059] Subsequently, the obtained pinyin sequence is input into the text matching tool, and the candidate Chinese character labels corresponding to each pinyin are screened out in the Chinese phonetic database, and the candidate Chinese character labels are screened according to the context information to obtain the corresponding Chinese character labels. The corresponding Chinese characters are extracted according to the Chinese character labels to obtain the text corresponding to the conference audio.

[0060] 205. Input the conference audio file into the voiceprint extraction model to extract voiceprint features, and obtain acoustic spectrum feature parameters, lexical feature parameters, prosodic feature parameters, and accent feature parameters in the conference audio file;

[0061] Obtain the conference audio file after denoising and segmentation processing in the above steps, and input the obtained segmented conference audio file into a preset voiceprint extraction model to extract voiceprint features, wherein the voiceprint extraction model is pre-constructed based on RBFNN (Radial Basis Function Neyral Network); the radial basis function neural network is a neural network constructed with radial basis function (RBF) as the "basis" of the hidden unit to form a hidden layer space. The radial basis function can be used as the main function of the fully connected layer and the ReLU (Rectified Linear Unit) layer in the neural network structure.

[0062] The conference audio file is input into the voiceprint extraction model built based on RBFNN to extract the feature parameters in the audio file. In this embodiment, the extracted voiceprint feature parameters include acoustic spectrum feature parameters, lexical feature parameters, prosodic feature parameters and accent feature parameters.

[0063] The acoustic spectrum feature parameter is obtained by extracting the power spectrum of the audio file, taking the inverse of the power spectrum, and performing inverse Fourier transform on the obtained inverse value; the lexical feature parameter and the prosody feature parameter are generated by evaluating the speech of each speech paragraph in the conference audio file based on the n-gram algorithm; and the accent feature parameter is generated by classifying according to the recognized language, dialect and accent information.

[0064] 206、based on the acoustic spectrum feature parameter, the lexical feature parameter, the prosody feature parameter and the accent feature parameter, a voiceprint parameter is generated, and a voiceprint feature in the conference audio file is obtained;

[0065] Based on the voiceprint feature parameter obtained above, a voiceprint parameter is generated, and a voiceprint feature in the conference audio file is obtained; specifically, the voiceprint feature can be a voiceprint sequence combined by the voiceprint parameters, and the voiceprint sequence is taken as the voiceprint feature of the current speech paragraph. The voiceprint features contained in the conference audio file are at least one.

[0066] 207、According to the voiceprint feature, the participant information corresponding to the voiceprint feature is searched in the preset voiceprint information library;

[0067] 208、According to the participant information, the speaker information is marked on the text content, and the speaker information is obtained;

[0068] 209、According to the participant information, the work project to which the conference belongs is determined, and the conference record is generated according to the work project, the text content and the corresponding speaker information.

[0069] The contents of steps 207-209 in the embodiment are basically the same as those of steps 104-106 in the foregoing embodiment, and therefore will not be described here.

[0070] The technical scheme of the embodiment of the application improves the accuracy of the automatically generated conference record, and improves the detailed level of the information content contained in the conference record.

[0071] Please refer to Figure 3 The third embodiment of the conference record generation method in the embodiment of the application includes:

[0072] 301、obtain a conference audio file to be recorded and recognized;

[0073] Receive a conference record generation request, extract the conference audio file contained in the conference record generation request, and perform segmentation processing on the conference audio file to obtain a segmented conference audio file.

[0074] 302. Calling a preset voiceprint extraction model to extract voiceprint features from the conference audio file to obtain the voiceprint features in the conference audio file;

[0075] The obtained segmented conference audio files are input into a preset voiceprint extraction model, and the voiceprint features in the valid speech are extracted to obtain the voiceprint features in each segmented conference audio file.

[0076] 303. Based on the voiceprint feature, search for the participant information corresponding to the voiceprint feature in a preset voiceprint information database;

[0077] According to the aforementioned identified voiceprint features, voiceprint feature data consistent with the voiceprint features is searched in the preset voiceprint information database, the corresponding participant code is queried, and then the specific participant information of the speaker contained in the current conference audio file is obtained according to the participant code.

[0078] 304. Determine the work project to which the meeting belongs based on the participant information, and determine the relevant areas of the current meeting based on the work project information;

[0079] 305. Searching for a relevant field dictionary in a preset field dictionary set according to the relevant field of the conference;

[0080] After obtaining the participant information obtained in the above steps, obtain the relevant field tags of the current meeting based on the participant name, position, organizational structure, project team and other information contained in the participant information, and search the preset dictionary set for the relevant field dictionary corresponding to the relevant field of the meeting based on the relevant field tags of the current meeting.

[0081] 306. Calling a preset speech recognition model to perform speech recognition on the conference audio file to obtain text content in the conference audio file;

[0082] In this embodiment, the recognition parameters of the preset speech recognition model are adjusted according to the relevant field dictionary to improve the matching probability of the words contained in the relevant field dictionary; the conference audio file is input into the adjusted speech recognition model, the voice endpoints contained in the conference audio file are identified, and the acoustic features are feature matched according to the specific acoustic features of the audio file to obtain the pronunciation corresponding to each character in the conference audio file, and the text content corresponding to the pronunciation content is generated based on the pronunciation corresponding to each character.

[0083] 307. Mark the text content with the speaker according to the participant information to obtain the speaker information;

[0084] Based on the voiceprint features of each conference audio segment identified above, each text content is marked accordingly to obtain the specific speaker information of each text content.

[0085] 308、invoke the preset audio emotion recognition model to perform voice emotion recognition on the conference audio file to obtain emotion feature parameters;

[0086] 309、perform emotion state labeling on the text content based on the emotion feature parameters to obtain emotion state labels;

[0087] 310、output a probability that the conference achieves the expected conference goal based on the emotion state labels, and label the text content according to the expected conference goal and the probability;

[0088] In the embodiment, the preset audio emotion recognition model is invoked to perform voice emotion recognition on the conference audio file, wherein the audio emotion recognition model is constructed in advance based on a deep neural network algorithm, and specifically, the emotion features of each piece of text in the conference audio file can be determined through the prosody or fundamental frequency features of the voice, and emotion feature parameters are obtained.

[0089] Each piece of text in the obtained text content is labeled with an emotion state based on the specific value of the emotion feature parameters to obtain emotion state labels, emotion state labels in all conference text content are counted, a probability that the conference achieves the expected conference goal is calculated, and the text content is labeled according to the expected conference goal and the probability.

[0090] 311、generate a conference record according to the text content and corresponding speaker information.

[0091] The identified text content is adjusted and labeled according to the specific speaker information of each piece of text content and the specific content of the work project to generate a conference record.

[0092] The technical scheme of the embodiment of the present application improves the accuracy and the detailed degree of information content of the automatically generated conference record, and can also automatically generate to-do lists and conference summaries according to the conference record, so as to facilitate subsequent specific follow-up operations on the conference content.

[0093] Please refer to Figure 4 and Figure 5 The fourth embodiment of the conference record generation method in the embodiment of the present application comprises:

[0094] In the present application, a company employee information database needs to be acquired in advance, wherein the employee information database contains employee information including employee number, name, position, and organization structure information. Voice sample audio of each employee is collected, and after obtaining the voice sample audio, the voice sample audio is denoised to obtain denoised voice samples. Then, quality detection is performed on the denoised voice samples to determine whether the voice quality meets a preset sample threshold. If not, the voice sample audio collection is performed again. If yes, feature extraction is performed on the denoised voice samples to obtain the registered voiceprint feature of each employee, and the registered voiceprint feature is associated with the employee information in the employee information database to obtain a voiceprint information library, so that when a voiceprint feature consistent with the registered voiceprint feature is matched, the corresponding employee information can be found through the voiceprint database.

[0095] 401. Obtain a conference audio file to be recorded and recognized;

[0096] In this step, a conference record generation request is received, and the corresponding conference audio file contained in the conference record generation request is extracted; or in response to the received conference record generation request, the speech in the conference scene that needs to generate conference records is obtained in real time to obtain the conference audio file.

[0097] 402. Call the feature extraction layer to perform spectral feature extraction on the conference audio file to obtain an audio feature spectrum;

[0098] 403. Call the tone recognition layer to split the audio feature spectrum to obtain a plurality of feature spectrum pieces, and perform pronunciation tone recognition on the plurality of feature spectrum pieces to obtain a pinyin sequence;

[0099] 404. Call the text sequence matching layer to perform text sequence matching based on the pinyin sequence to obtain the text content of the conference audio file;

[0100] The voice endpoint in the conference audio file is detected by calling a preset VAD (Voice Activity Detection) model to obtain the voice endpoint. The conference audio file is segmented according to the position of the voice endpoint to obtain a segmented conference audio file.

[0101] The segmented conference audio file is input into a speech recognition model for text content recognition, wherein the speech recognition model comprises a feature extraction layer, a tone recognition layer, and a text sequence matching layer. First, the feature extraction layer is called to perform spectral feature extraction on the conference audio file. Specifically, the Fourier transform is first performed on the segmented conference audio file to obtain the spectrum of the conference audio file. The obtained spectrum is input into the feature extraction layer to extract the feature information of the spectrum. The feature information of the spectrum is input into the tone recognition layer, wherein the tone recognition layer comprises a pre-trained spectrum recognition tool, and the spectrum recognition tool is constructed according to a neural network algorithm. The tone recognition layer identifies the pronunciation corresponding to the spectrum according to the feature of the spectrum, and outputs the corresponding pinyin sequence according to the pronunciation.

[0102] Subsequently, the obtained pinyin sequence is input into a text matching tool, candidate Chinese character labels corresponding to the pinyin are screened out from a Chinese character phonetic annotation database according to each pinyin, and the candidate Chinese character labels are screened according to context information to obtain corresponding Chinese character labels. The corresponding Chinese characters are extracted according to the Chinese character labels, and the text corresponding to the conference audio is obtained.

[0103] In the present embodiment, the text matching tool is constructed based on a CBHG model. Please refer to Figure 5 , wherein CBHG refers to a 1-D convolution bank+highway network+bidirectional GRU, which is a model architecture constructed by a 1-D convolution (1-D convolution bank / 1-D convolution bank), a highway layer, and a bidirectional GRU network (Gate Recurrent Unit). The GRU network is a bidirectional recurrent neural network (Bidirectional RNN, Bidirectional Recurrent Neural Network).

[0104] Specifically, the obtained pinyin sequence first passes through k 1-D convolutions (Conv1D banks) to model the information of the pinyin sequence, where these convolution kernels can also effectively model the contextual information. Then the output of the convolutional network is obtained. Among them, the convolution outputs are stacked together and max-pooled along the time axis to increase the invariance of the current information. The stride is set to 1 to maintain the temporal resolution; then it is input into several fixed-width 1-D convolution layers (Conv1D layers), and the output is added to the starting input sequence through the residual connection; then the result is input into a multi-layer highway network (Highway layers) to extract higher-level features; finally, a bidirectional GRU is added at the top to extract the contextual features of the sequence; finally, classification is performed based on the extracted sequence features, and the Chinese character most likely corresponding to the pinyin sequence is output.

[0105] 405. Input the conference audio file into the voiceprint extraction model to extract voiceprint features, and obtain acoustic spectrum feature parameters, lexical feature parameters, prosodic feature parameters, and accent feature parameters in the conference audio file;

[0106] 406. Generate voiceprint parameters based on the acoustic spectrum feature parameters, the lexical feature parameters, the prosodic feature parameters, and the accent feature parameters to obtain voiceprint features in the conference audio file;

[0107] 407. Based on the voiceprint feature, search for the participant information corresponding to the voiceprint feature in a preset voiceprint information database;

[0108] The contents of steps 405 to 407 in this embodiment are substantially the same as those of steps 205 to 207 in the aforementioned embodiment, and therefore are not described again here.

[0109] 408. Mark the text content with the speaker according to the participant information to obtain the speaker information;

[0110] Based on the voiceprint features of each conference audio segment identified above, each text content is marked accordingly to obtain the specific speaker information of each text content.

[0111] 409. Calling a preset audio emotion recognition model to perform speech emotion recognition on the conference audio file to obtain emotion feature parameters;

[0112] 410. Annotate the text content with an emotional state based on the emotional feature parameter to obtain an emotional state identifier;

[0113] 411, identify the probability of achieving the expected meeting goal based on the emotional state, and label the text content according to the expected meeting goal and the probability;

[0114] In this embodiment, a preset audio emotion recognition model is called to perform voice emotion recognition on the conference audio file, wherein the audio emotion recognition model is constructed in advance based on a deep neural network algorithm, and can specifically determine the emotional characteristics of each text in the conference audio file through information such as prosody or fundamental frequency characteristics of the voice, and obtain emotional characteristic parameters.

[0115] Based on the specific numerical value of the emotional characteristic parameters, each text in the obtained text content is marked with an emotional state to obtain an emotional state identifier; the emotional state identifiers in all conference text content are counted to calculate the probability of achieving the expected meeting goal for this meeting, and the text content is labeled according to the expected meeting goal and the probability.

[0116] 412, determine the work project to which the meeting belongs according to the participant information, and determine the relevant field of the current meeting according to the work project information;

[0117] 413, find the preset related field dictionary according to the meeting related field;

[0118] 414, adjust the text content according to the related field dictionary, and correct the inaccurate words to obtain corrected text content;

[0119] 415, generate a meeting record according to the corrected text content and the corresponding speaker information.

[0120] After obtaining the participant information obtained in the foregoing steps, the related field label of the current meeting is obtained according to the participant name, position, organization structure, project group and other information contained in the participant information, and the related field dictionary corresponding to the meeting related field is found out from the preset dictionary set according to the related field label of the current meeting. The generated text content is adjusted according to the content in the related field dictionary, and the inaccurate words are corrected to obtain corrected text content; and a meeting record is generated according to the corrected text content and the corresponding speaker information.

[0121] After obtaining the labeled meeting record content, text semantic recognition is performed on the meeting record content, the core viewpoints and task content of the meeting record are extracted; a conference abstract is generated based on the core viewpoints, so as to extract important content in the meeting; a to-do list is automatically generated based on the task content; and the content of the meeting record is updated according to the conference abstract and the to-do list.

[0122] The technical scheme of the embodiment of the present application improves the accuracy and the detail of the automatically generated conference record, and can automatically generate to-do lists and conference summaries according to the conference record, so as to facilitate subsequent specific follow-up operations on the conference content.

[0123] The conference record generation method in the embodiment of the present application is described above, and the conference record generation device in the embodiment of the present application is described below, please refer to Figure 6 An embodiment of the conference record generation device in the embodiment of the present application includes:

[0124] The acquisition module 601 is configured to acquire a conference audio file to be recognized and recorded;

[0125] The recognition module 602 is configured to call a preset speech recognition model to perform speech recognition on the conference audio file, and obtain text content in the conference audio file;

[0126] The extraction module 603 is configured to call a preset voiceprint extraction model to perform voiceprint feature extraction on the conference audio file, and obtain voiceprint features in the conference audio file, wherein the voiceprint features are at least one;

[0127] The search module 604 is configured to search for participant information corresponding to the voiceprint features in a preset voiceprint information library according to the voiceprint features;

[0128] The marking module 605 is configured to mark speakers according to the participant information, and obtain speaker information;

[0129] The generation module 606 is configured to determine a work item to which the conference belongs according to the participant information, and generate a conference record according to the work item, the text content, and the corresponding speaker information.

[0130] The technical scheme of the embodiment of the present application improves the accuracy and the detail of the automatically generated conference record, and can automatically generate to-do lists and conference summaries according to the conference record, so as to facilitate subsequent specific follow-up operations on the conference content.

[0131] Please refer to Figure 7 Another embodiment of the conference record generation device in the embodiment of the present application includes:

[0132] The acquisition module 601 is configured to acquire a conference audio file to be recognized and recorded;

[0133] The recognition module 602 is configured to call a preset speech recognition model to perform speech recognition on the conference audio file, and obtain text content in the conference audio file;

[0134] The extraction module 603 is configured to call a preset voiceprint extraction model to perform voiceprint feature extraction on the conference audio file, to obtain voiceprint features in the conference audio file, wherein the voiceprint features are at least one.

[0135] The searching module 604 is configured to search, according to the voiceprint features, participant information corresponding to the voiceprint features in a preset voiceprint information library.

[0136] The labeling module 605 is configured to perform speaker labeling on the text content according to the participant information, to obtain speaker information.

[0137] The generation module 606 is configured to determine a work item to which the conference belongs according to the participant information, and generate a conference record according to the work item, the text content and the corresponding speaker information.

[0138] In another embodiment of the present application, the conference record generation apparatus further includes a content updating module 607, and the content updating module 607 includes:

[0139] The extraction unit 6071 is configured to perform text semantic recognition on the conference record content, to extract core viewpoints and task content of the conference record.

[0140] The content generation unit 6072 is configured to generate a conference summary based on the core viewpoints and to-do items based on the task content.

[0141] The updating unit 6073 is configured to update the content of the conference record according to the conference summary and the to-do items.

[0142] In another embodiment of the present application, the speech recognition model includes a feature extraction layer, a tone recognition layer and a text sequence matching layer, and the recognition module 602 includes:

[0143] The spectrum extraction unit 6021 is configured to call the feature extraction layer to perform spectrum feature extraction on the conference audio file, to obtain an audio feature spectrum.

[0144] The tone recognition unit 6022 is configured to call the tone recognition layer to split the audio feature spectrum, to obtain a plurality of feature spectrum pieces, and to perform pronunciation tone recognition on the plurality of feature spectrum pieces, to obtain a pinyin sequence.

[0145] The text matching unit 6023 is configured to call the text sequence matching layer to perform text sequence matching based on the pinyin sequence, to obtain text content of the conference audio file.

[0146] In another embodiment of the present application, the extraction module 603 includes:

[0147] The parameter extraction unit 6031 is configured to input the conference audio file into a voiceprint extraction model to extract voiceprint features, and obtain acoustic spectrum feature parameters, lexical feature parameters, prosody feature parameters, and accent feature parameters in the conference audio file. The voiceprint extraction model is constructed in advance based on a radial basis function neural network.

[0148] The parameter calculation unit 6032 is configured to generate voiceprint parameters based on the acoustic spectrum feature parameters, the lexical feature parameters, the prosody feature parameters, and the accent feature parameters, and obtain voiceprint features in the conference audio file.

[0149] In another embodiment of the present application, the generation module 606 includes:

[0150] The domain determination unit 6061 is configured to determine a current conference related domain according to the work item information.

[0151] The dictionary searching unit 6062 is configured to search a pre-stored related domain dictionary according to the conference related domain.

[0152] The text correction unit 6063 is configured to adjust the text content according to the related domain dictionary, correct inaccurate words, and obtain corrected text content.

[0153] The record generation unit 6064 is configured to generate a conference record according to the corrected text content and corresponding speaker information.

[0154] In another embodiment of the present application, the conference record generation apparatus further includes a voiceprint library construction module, and the voiceprint library construction module includes:

[0155] A voice sample of an employee is collected, and the voice sample is subjected to denoising processing to obtain a denoised voice sample.

[0156] The denoised voice sample is subjected to quality detection to determine whether the voice quality meets a preset sample threshold.

[0157] If yes, feature extraction is performed on the denoised voice sample to obtain registered voiceprint features.

[0158] Pre-stored employee information is obtained, the registered voiceprint features are associated with the employee information, and a voiceprint information library is obtained based on the employee information and the registered voiceprint features.

[0159] In another embodiment of the present application, the conference record generation apparatus further includes a target labeling module, and the target labeling module includes:

[0160] An emotion recognition unit is configured to invoke a preset audio emotion recognition model to perform voice emotion recognition on the conference audio file to obtain emotion feature parameters.

[0161] An emotion labeling unit is configured to label an emotion state of the text content based on the emotion feature parameters to obtain an emotion state identifier.

[0162] A target labeling unit is configured to output a probability of achieving a conference target in the conference based on the emotion state identifier, and label the text content according to the conference target and the probability.

[0163] The technical solution of the embodiment of the present application improves the accuracy and the detailed level of information content of the automatically generated conference record, and can automatically generate a to-do list and a conference summary according to the conference record, so as to facilitate subsequent specific follow-up operations on the conference content.

[0164] The above Figure 6 And Figure 7 The conference record generation apparatus in the embodiment of the present application is described in detail from the perspective of a modular functional entity, and the conference record generation device in the embodiment of the present application is described in detail from the perspective of hardware processing.

[0165] Figure 8 Fig. 8 is a structural schematic diagram of a conference record generation device provided by the embodiment of the present application. The conference record generation device 800 can have great differences due to different configurations or performances, and can include one or more than one processor (central processing unit, CPU) 810 (for example, one or more than one processor) and a memory 820, and one or more than one storage medium 830 (for example, one or more than one mass storage device) storing an application program 833 or data 832. The memory 820 and the storage medium 830 can be temporary storage or persistent storage. The program stored in the storage medium 830 can include one or more than one module (not shown in the figure), and each module can include a series of instruction operations in the conference record generation device 800. Further, the processor 810 can be configured to communicate with the storage medium 830 and execute a series of instruction operations in the storage medium 830 on the conference record generation device 800.

[0166] The conference record generation device 800 can further include one or more than one power supply 840, one or more than one wired or wireless network interface 850, one or more than one input and output interface 860, and / or one or more than one operating system 831, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, etc. Those skilled in the art can understand that Figure 8The illustrated conference record generation device structure does not constitute a limitation on the conference record generation device, and can include more or fewer components than illustrated, or combine certain components, or arrange the components differently.

[0167] The present application also provides a computer device, which can be any device capable of executing the conference record generation method described in the above embodiments, and the computer device comprises a memory and a processor, the memory stores computer readable instructions, and the computer readable instructions are executed by the processor to make the processor execute the steps of the conference record generation method described in the above embodiments.

[0168] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium or a volatile computer readable storage medium, and the computer readable storage medium stores instructions, and the instructions make the computer execute the steps of the conference record generation method when the instructions are run on the computer.

[0169] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0170] The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, and includes a number of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in the various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0171] The above-described and above-embodied examples are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating meeting records, characterized in that: The conference record generation method includes: Obtaining the conference audio file to be recorded and recognized; Identify silent periods in conference audio files using a pre-set voice activity detection model, where silent periods refer to breaks or pauses in meetings, as well as pauses in speeches. The current conference audio noise floor and voice endpoints are obtained through the audio waveform during the silent period. De-noise the conference audio file based on the current conference audio background noise, and segment the de-noised conference audio file based on the voice endpoint to obtain segmented conference audio files. Calling a preset speech recognition model to perform speech recognition on the segmented conference audio files to obtain the text content of each conference audio segment; Calling a preset voiceprint extraction model to extract voiceprint features from the conference audio file to obtain voiceprint features in the conference audio file, wherein the voiceprint feature is at least one; According to the voiceprint feature, searching for the participant information corresponding to the voiceprint feature in a preset voiceprint information database; Marking the text content of each conference audio segment with the speaker according to the participant information to obtain the speaker information of each text content; Determine the work project to which the meeting belongs based on the participant information, and adjust and annotate the text content based on the work project and the speaker information of each text content to generate a meeting record; After labeling the text content of each conference audio segment with a speaker according to the participant information to obtain the speaker information of each text content, the method further includes: Calling a preset audio emotion recognition model to perform speech emotion recognition on the conference audio file to obtain emotion feature parameters, wherein the emotion feature parameters are obtained by judging the emotion features of each text in the conference audio file by the prosody or fundamental frequency features of the speech; Marking the text content with an emotional state based on the emotional characteristic parameters to obtain an emotional state identifier; The probability of the meeting achieving the expected meeting goal is output based on the emotional state identifier, and the text content is annotated according to the expected meeting goal and the probability.

2. The method for generating conference records according to claim 1, wherein: After generating the meeting record according to the work items, text content and corresponding speaker information, the method further includes: Performing text semantic recognition on the content of the meeting minutes to extract the core ideas and task content of the meeting minutes; Generate a meeting summary based on the core ideas and generate to-do items based on the task content; The content of the meeting minutes is updated according to the meeting summary and to-do items.

3. The method for generating conference records according to claim 2, wherein: The speech recognition model includes a feature extraction layer, a tone recognition layer, and a text sequence matching layer. The calling of the preset speech recognition model to perform speech recognition on the conference audio file to obtain the text content in the conference audio file includes: Calling the feature extraction layer to extract spectrum features of the conference audio file to obtain an audio feature spectrum; Calling the tone recognition layer to segment the audio feature spectrum to obtain a plurality of feature spectrum slices, and performing pronunciation tone recognition on the plurality of feature spectrum slices to obtain a pinyin sequence; The text sequence matching layer is called to match the text sequence based on the pinyin sequence to obtain the text content of the conference audio file.

4. The method for generating conference records according to claim 3, wherein: The calling of a preset voiceprint extraction model to extract voiceprint features from the conference audio file to obtain the voiceprint features in the conference audio file includes: Inputting the conference audio file into a voiceprint extraction model to extract voiceprint features, thereby obtaining acoustic spectrum feature parameters, lexical feature parameters, prosodic feature parameters, and accent feature parameters in the conference audio file, wherein the voiceprint extraction model is pre-constructed based on a radial basis function neural network; A voiceprint parameter is generated based on the acoustic spectrum feature parameter, the lexical feature parameter, the prosodic feature parameter, and the accent feature parameter to obtain the voiceprint feature in the conference audio file.

5. The method for generating conference records according to claim 4, wherein: Generating the meeting record according to the work items, text content and corresponding speaker information includes: Determine the relevant areas of the current meeting based on the work item information; Searching a preset dictionary of related fields according to the conference-related field; Adjusting the text content according to the related field dictionary, correcting inaccurately recognized words, and obtaining corrected text content; A meeting record is generated according to the revised text content and the corresponding speaker information.

6. The method for generating conference records according to any one of claims 1 to 5, characterized in that: Before obtaining the conference audio file to be recorded and identified, the method further includes: Collecting a voice sample of an employee and performing denoising processing on the voice sample to obtain a denoised voice sample; Performing a quality test on the denoised speech sample to determine whether the speech quality meets a preset sample threshold; If yes, then extract features from the denoised speech sample to obtain registered voiceprint features; Pre-set employee information is obtained, the registered voiceprint feature is associated with the employee information, and a voiceprint information database is obtained based on the employee information and the registered voiceprint feature.

7. A conference record generating device, characterized in that: The conference record generating device includes: An acquisition module is used to obtain a conference audio file to be recorded and recognized; identify silent periods in the conference audio file using a preset voice activity detection model, where the silent periods refer to breaks or pauses in the meeting, as well as pauses in speeches; obtain the current conference audio background noise and voice endpoints through the audio waveforms of the silent periods; denoise the conference audio file based on the current conference audio background noise, and segment the denoised conference audio file based on the voice endpoints to obtain segmented conference audio files; A recognition module is used to call a preset speech recognition model to perform speech recognition on the segmented conference audio files to obtain the text content of each conference audio segment; an extraction module, configured to call a preset voiceprint extraction model to extract voiceprint features from the conference audio file, and obtain a voiceprint feature in the conference audio file, wherein the voiceprint feature is at least one; A search module, configured to search for participant information corresponding to the voiceprint feature in a preset voiceprint information database according to the voiceprint feature; A tagging module is used to tag the text content of each conference audio segment with the speaker according to the participant information to obtain the speaker information of each text content; A generating module, configured to determine the work project to which the meeting belongs based on the participant information, and to adjust and annotate the text content based on the work project and the speaker information of each text content to generate a meeting record; The conference record generation device also includes a target labeling module, which includes: an emotion recognition unit, which is used to call a preset audio emotion recognition model to perform speech emotion recognition on the conference audio file to obtain emotion feature parameters, wherein the emotion feature parameters are obtained by judging the emotion features of each text in the conference audio file through the rhythm or fundamental frequency features of the speech; an emotion labeling unit, which is used to label the text content with an emotion state based on the emotion feature parameters to obtain an emotion state identifier; a target labeling unit, which is used to output the probability of the meeting achieving the expected meeting goals based on the emotion state identifier, and label the text content according to the expected meeting goals and the probability.

8. A conference record generating device, characterized in that: The conference record generating device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor calls the instructions in the memory to enable the meeting record generating device to execute the steps of the meeting record generating method according to any one of claims 1 to 6.

9. A computer-readable storage medium having instructions stored thereon, characterized in that: When the instructions are executed by the processor, the steps of the meeting record generation method as described in any one of claims 1 to 6 are implemented.

Images