Online conference transcription method and system capable of realizing intelligent conference summary

Abstract

The invention relates to the technical field of voice communication, and provides an online conference transcription method capable of realizing intelligent conference summary, which comprises the following steps: collecting audio data of each speaker in a conference process in real time through a microphone array, and carrying out primary processing on the collected audio signals; performing sound source positioning on the audio signal and determining a source direction of the audio signal according to a sound source positioning result; performing voiceprint recognition on the audio signal to extract voice spectrum features of the speaker, and marking the voiceprint of the speaker according to a voiceprint recognition result; and the weighted audio signal is used for character transcription, and the speaker is positioned to generate a complete conference summary. The problems that in a traditional audio transfer system, multiple speakers cannot be accurately recognized, speech content cannot be distinguished, and manual proofreading work cannot be reduced are effectively solved, and the accuracy and efficiency of conference recording are improved.

Description

Technical Field

[0001] The present invention relates to the field of voice communication technology, and in particular to an online conference transcription method and system capable of realizing intelligent conference minutes. Background Art

[0002] With the rapid development of globalization and information technology, remote work and hybrid office have driven the rapid growth of the enterprise communication and collaboration market. The application of advanced technologies such as cloud computing and artificial intelligence has not only led to a surge in demand for cloud conferences, cloud software platforms and video conferencing solutions, but also promoted the development and popularization of related hardware equipment. In this context, the growth of collaborative spaces such as small and medium-sized conference rooms and open meeting spaces has driven the increasing demand for audio conferencing terminals. Conference speakers and low-cost and easy-to-use plug-and-play solutions have achieved rapid growth.

[0003] In terms of processing conference content, efficient and high-quality organization of video, audio and text information has become increasingly important. Conference transcription, that is, converting the voice in the meeting into text records in real time, has become a basic requirement of the industry. The application of this technology covers a variety of scenarios such as meetings, training, interviews, etc., requiring the generated text materials to be not only complete, but also equipped with corresponding subtitles. The surge in market demand for video conferencing has also brought about intensified competition. Various companies including video conferencing hardware manufacturers, audio brands, solution providers, signal or voice transmission system manufacturers, DSP manufacturers, display and security manufacturers, and AI voice recognition manufacturers have entered this market. The participation of these companies has prompted the market, which was originally dominated by audio and video conferencing VC providers, to shift to a diversified manufacturer to jointly promote the industrial development model.

[0004] However, despite the continuous advancement of technology, existing conference audio systems still face several challenges: (1) Limitations of environmental noise suppression: For array microphone conference speakers, due to application scenarios and cost limitations, algorithms can achieve a certain degree of environmental noise suppression, but it still cannot meet the requirements of text transcription.

[0005] (2) The inaccuracy of long speech recognition and the fact that the text exported after speech recognition has no timestamp and is not segmented.

[0006] (3) It is basically impossible to distinguish the voices of different speakers in the meeting, which results in more manpower required for subsequent text proofreading. Summary of the invention

[0007] The purpose of the invention is to solve the above technical problems and propose an online conference transcription method and system that can realize intelligent conference minutes, which solves the problems of difficulty in distinguishing multiple speakers, low voice recognition accuracy, and large workload of manual proofreading. The purpose of the invention can be achieved through the following technical solutions: The present invention provides an online conference transcription method capable of realizing intelligent conference minutes, comprising the following steps: Step S100: collecting audio data of each speaker during the conference in real time through a microphone array, and performing preliminary processing on the collected audio signals; Step S200: Based on the preliminary processing of the audio signal, the time difference of each microphone receiving the audio signal is calculated, the sound source of the audio signal is localized, and the source direction of the audio signal is determined according to the sound source localization result; Step S300: performing voiceprint recognition on the audio signal to extract the speaker's speech spectrum features, and marking the speaker's voiceprint according to the voiceprint recognition result; Step S400: Based on the sound source recognition result and the voiceprint recognition result, a weighting coefficient of each frame of the audio signal is calculated, and each frame of the audio signal is weighted to select and merge the sound source localization label and the voiceprint label information; Step S500: using the weighted audio signal for text transcription and locating the speaker to generate a complete meeting minutes.

[0008] Furthermore, step S100 includes: Step S110: continuously and synchronously collecting audio signals through microphones in the microphone array to obtain audio data of all speakers during the conference; Step S120: performing preliminary processing on the collected audio signal, including echo cancellation, acoustic environment noise reduction and automatic gain.

[0009] Further, step S200 includes: Step S210: Calculate the source direction of the audio signal by using a differential time algorithm based on the time difference of the audio signal received by different microphones; Step S220: According to the sound source localization result, the phase array of the microphone array is adjusted to optimize the signal quality in the target direction, and at the same time, the audio signal is enhanced by slope shaping and power spectrum estimation to improve the positioning accuracy of the audio signal; Step S230: Match the sound source localization tag with the real-time audio frame, mark the sound source localization tag of each frame of audio data, and classify the audio information according to the sound source localization tag during subsequent audio processing, where the sound source localization tag is V doa =f(Vt).

[0010] Further, step S300 includes: Step S310: extracting speech spectrum features from the audio signal, capturing characteristic information of the sound, including pitch, timbre and speaking speed, and generating a unique voiceprint model for each speaker; Step S320: Based on the extracted speech spectrum features, compare them with the voiceprint model, and assign a unique voiceprint label to each speaker based on the comparison results, marking the speaker's voice features. The voiceprint label is P vp =f(Pt); where When the amount of data is insufficient or the labeling fails, the voiceprint label Pt is set to 0; When the voiceprint marking is successful, Pt={P1,P2,P3,......,Pδ,}, where δ is a natural number greater than 0.

[0011] Further, in step S400, for each frame of audio signal , the weighted sub-item formula in weighted processing is, ; ; in, Indicates the timestamp of each frame of audio signal. Indicates the value of the voiceprint tag at the current timestamp. Indicates the value of the sound source localization tag at the current timestamp.

[0012] Based on the same inventive concept, the present invention provides an online conference transcription system capable of realizing intelligent conference minutes, adopting the online conference transcription method as described above, comprising: The audio acquisition module is used to collect the audio data of each speaker during the meeting in real time through the microphone array and perform preliminary processing on the collected audio signals; The sound source localization module is used to calculate the time difference of each microphone receiving the audio signal during the preliminary processing of the audio signal, localize the sound source of the audio signal and determine the source direction of the audio signal according to the sound source localization result; The voiceprint recognition module is used to perform voiceprint recognition on the audio signal to extract the speaker's voice spectrum features, and mark the speaker's voiceprint according to the voiceprint recognition result; The data processing module is used to calculate the weighting coefficient of each frame of audio signal based on the sound source recognition results and the voiceprint recognition results, and perform weighted processing on each frame of audio signal to achieve the selection and fusion of sound source positioning labels and voiceprint label information; the weighted audio signal is used for text transcription to locate the speaker to generate complete meeting minutes.

[0013] Furthermore, the audio acquisition module includes: A collection unit, used to continuously and synchronously collect audio signals through microphones in the microphone array, so as to obtain audio data of all speakers during the conference; The pre-processing unit is used to perform preliminary processing on the collected audio signals, including echo cancellation, acoustic environment noise reduction and automatic gain.

[0014] Furthermore, the sound source localization module includes: A calculation unit, used to calculate the source direction of the audio signal by a differential time algorithm based on the time difference of the audio signal received by different microphones; The positioning unit is used to adjust the phase array of the microphone array according to the sound source positioning result, optimize the signal quality in the target direction, and enhance the audio signal through slope shaping and power spectrum estimation to improve the positioning accuracy of the audio signal; Labeling unit: used to correspond the sound source localization label with the real-time audio frame, mark the sound source localization label of each frame of audio data, and classify the audio information according to the sound source localization label in subsequent audio processing, where the sound source localization label is V doa =f(Vt).

[0015] Furthermore, the voiceprint recognition module includes: The feature extraction unit is used to extract speech spectrum features from the audio signal, capture the characteristic information of the sound, including pitch, timbre and speaking speed, and generate a unique voiceprint model for each speaker; The matching labeling unit is used to compare the extracted speech spectrum features with the voiceprint model, and assign a unique voiceprint label to each speaker based on the comparison results to mark the speaker's voice features. The voiceprint label is P vp =f(Pt); when the amount of data is insufficient or the marking fails, the voiceprint label Pt is set to 0; when the voiceprint marking is successful, Pt={P1,P2,P3,......,Pδ,}, δ is a natural number greater than 0.

[0016] Furthermore, for each frame of audio signal , the weighted sub-item formula in weighted processing is, ; ; in, Indicates the timestamp of each frame of audio signal. Indicates the value of the voiceprint tag at the current timestamp. Indicates the value of the sound source localization tag at the current timestamp.

[0017] Compared with the prior art, the present invention has at least one of the following technical effects: The present invention combines sound source localization, voiceprint recognition and audio weighted processing to achieve accurate speech transcription and speaker differentiation during a meeting, effectively solving the problems in traditional audio transcription systems that are unable to accurately identify multiple speakers, distinguish speech content and reduce manual proofreading work. It performs particularly well in noisy environments and multi-speaker scenarios, reduces later manual intervention, and improves the accuracy and efficiency of meeting records. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solution of the embodiment of the present invention, the following is a brief introduction to the drawings required for describing the embodiment: Figure 1 A flowchart of the steps of an online meeting transcription method for realizing intelligent meeting minutes. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and specific implementation methods. It should be understood that the specific implementation methods described herein are only used to explain the present invention and are not used to limit the present invention.

[0020] Example 1 In the existing technology, long speech recognition and transcription have high algorithm model requirements, especially in multi-person conference scenarios, facing many challenges. For example: (1) The amount of audio data involved in long speech transcription is huge, requiring the speech recognition system to be able to efficiently process and accurately recognize the speech content. However, due to the differences in the timbre, language expression habits and pronunciation of speakers, the accuracy of the speech recognition system in multi-speaker scenarios is often affected.

[0021] (2) In a multi-person conference, different speakers sit in different positions, and the propagation paths of sound are also different, resulting in direct wave attenuation and path masking effects. Due to the different attenuation of direct waves, the path masking effect will affect the accuracy of speech recognition transcription, further reducing the accuracy of speech recognition. Therefore, the existing technology has certain limitations when dealing with audio signal processing and transcription in complex conference environments, and an innovative method that can optimize audio signal processing and improve transcription accuracy is urgently needed.

[0022] Therefore, based on in-depth thinking about the limitations of the existing technology, the inventor proposed a new solution, which effectively solved the challenges in audio signal processing in multi-person conference scenarios by combining sound source localization DOA and voiceprint recognition. The invention uses a microphone array to collect the audio data of each speaker during the meeting in real time, and combines the sound source localization information and voiceprint features to achieve accurate sound source separation and speech recognition in the transcription process, thereby significantly improving the accuracy of transcription. At the same time, the voices of different speakers are reasonably distinguished through weighted processing methods, which solves the effects of timbre differences, language habits and sitting positions, and realizes efficient transcription in complex conference environments. This innovative method not only improves the accuracy of conference transcription, but also reduces the workload of subsequent text proofreading, and has important application value. The specific implementation methods are as follows: The present invention provides an online conference transcription method capable of realizing intelligent conference minutes, comprising the following steps: Step S100: collecting audio data of each speaker during the conference in real time through a microphone array, and performing preliminary processing on the collected audio signals; Step S200: Based on the preliminary processing of the audio signal, the time difference of each microphone receiving the audio signal is calculated, the sound source of the audio signal is localized, and the source direction of the audio signal is determined according to the sound source localization result; Step S300: performing voiceprint recognition on the audio signal to extract the speaker's speech spectrum features, and marking the speaker's voiceprint according to the voiceprint recognition result; Step S400: Based on the sound source recognition result and the voiceprint recognition result, a weighting coefficient of each frame of the audio signal is calculated, and each frame of the audio signal is weighted to select and merge the sound source localization label and the voiceprint label information; Step S500: transcribe the weighted audio signal into text through a speech recognition algorithm, locate the speaker, and generate a complete meeting minutes.

[0023] The intelligent conference transcription method proposed in this embodiment solves many pain points in the current intelligent conference industry by combining conference speakers and cloud conference transcription systems. The conference speaker products are based on different algorithm requirements and are equipped with different microphone arrays to ultimately achieve clear human voice output. Intelligent voice products. The microphone array is composed of multiple (≥2) microphones. These microphones are used as acoustic sensors to sample and process the spatial characteristics of the sound field. Through a microphone array with a specific arrangement, the spatial characteristics of the sound field can be fully sampled, and the collected audio signals can be processed in combination with the corresponding algorithms, solving common acoustic problems such as echo cancellation, noise suppression, and signal enhancement, thereby ensuring clear output of human voices during the meeting. At the same time, the conference transcription system uses the cloud processing function to perform real-time speech recognition on the audio signals received by the conference speakers and convert the voice content into text. The cloud system can not only process a single audio signal, but also perform in-depth analysis based on information such as audio characteristics, context, and speaker identity to achieve high-accuracy transcription.

[0024] Furthermore, step S100 includes: Step S110: continuously and synchronously collecting audio signals through microphones in the microphone array to obtain audio data of all speakers during the conference; Step S120: performing preliminary processing on the collected audio signal, including echo cancellation, acoustic environment noise reduction and automatic gain.

[0025] Further, step S200 includes: Step S210: Calculate the source direction of the audio signal by using a differential time algorithm based on the time difference of the audio signal received by different microphones; Step S220: According to the sound source localization result, the phase array of the microphone array is adjusted to optimize the signal quality in the target direction, and at the same time, the audio signal is enhanced by slope shaping and power spectrum estimation to improve the positioning accuracy of the audio signal; Step S230: Match the sound source localization tag with the real-time audio frame, mark the sound source localization tag of each frame of audio data, and classify the audio information according to the sound source localization tag during subsequent audio processing, where the sound source localization tag is V doa =f(Vt).

[0026] When the conference speaker is in use, the uplink audio signal collected by the pickup unit of the conference speaker is processed by the algorithm processing unit using the 3A algorithm, namely, echo cancellation AEC, acoustic environment noise reduction ANC, and automatic gain AGC. After processing, the audio signal is sent to the network end through the data transmission unit, and finally transmitted to the conference peer through the Internet. The downlink audio is transmitted from the Internet to the speaker SPK of the local machine through the data transmission unit for playback.

[0027] The traditional 3A algorithm performs echo cancellation AEC, acoustic environment noise reduction ANC and automatic gain control AGC in sequence. However, the breakthrough of this patent is that DOA direction arrival information is added during the audio transmission process, and then the "sound source positioning" DOA Tab is realized based on the 3A algorithm. Sound source positioning can accurately determine the location of the sound source, which means that the direction of the sound source can be accurately identified.

[0028] Specifically, in actual meetings, since speakers often keep moving around, there may even be multiple speakers speaking at the same time, which will affect the stability and accuracy of sound source localization. Therefore, this patent proposes a key technology, which is to adjust the processing method of the audio signal in real time according to the sound source localization results, so that the audio signal can always accurately match the corresponding sound source position, thereby realizing the synchronous processing of real-time audio frames and their sound source localization information. Ultimately, through this technology, it can be ensured that in a complex conference environment, even if the speaker position changes or multiple speakers exist at the same time, the audio can be accurately processed and transmitted in real time efficiently.

[0029] Therefore, for the transmission of audio frames, a DOA Tab is added. In the process of processing the link based on the call noise reduction 3A algorithm, DOA information, that is, sound source positioning information, is added. "Sound source positioning" means that the location of the "sound source" can be accurately located; the sound field information is obtained based on the conference speaker microphone array, and the basic unit parameters of the phase array are adjusted so that signals at certain angles obtain constructive interference, while signals at other angles obtain destructive interference. The output signals of each array element are weighted, summed, and filtered, and finally the voice signal in the desired direction is output, which is equivalent to forming a "beam". Using the principles of beamforming and power spectrum estimation, the signal is processed to determine the direction of the signal wave, so that the source can be accurately directed. The final technical details will have certain differences based on the differences in the phase array, which will not be described in detail in this patent. The key point is to correspond the precise positioning direction of the audio with the real-time voice frame. Thereby realizing the DOA Tab of the transmitted audio frame.

[0030] Further, step S300 includes: Step S310: Extract the speech spectrum features from the audio signal, capture the characteristic information of the sound, including pitch, timbre and speaking speed, and generate a unique voiceprint model for each speaker; the voiceprint is a sound wave spectrum that carries speech information and is captured and displayed by an electroacoustic instrument. Voiceprints have high specificity and stability characteristics and can provide unique identification between individuals. Especially after adulthood, people's voiceprint features usually remain stable for a long time and are almost unaffected by time. .

[0031] Step S320: Based on the extracted speech spectrum features, compare them with the voiceprint model, and assign a unique voiceprint label to each speaker based on the comparison results, marking the speaker's voice features. The voiceprint label is P vp =f(Pt); where When the amount of data is insufficient or the labeling fails, the voiceprint label Pt is set to 0; When the voiceprint marking is successful, Pt={P1,P2,P3,......,Pδ,}, where δ is a natural number greater than 0.

[0032] In actual meetings, some complex situations are often encountered, such as the speaker moving around continuously or multiple speakers speaking alternately at the same time. In addition, since the speaker may turn his back to the conference equipment during the meeting, the sound propagation path is affected, resulting in a more serious sound masking effect. In order to solve these complex meeting environment problems, this embodiment proposes a new solution.

[0033] Specifically, the breakthrough point 2 of the solution in this embodiment is that in the process of audio transcription and recognition, voiceprint information is added, that is, different speakers are distinguished by the voiceprint of the person. In the process of transcribing the content, the Vocal print Tab voiceprint tag is introduced. By combining the speech model and the sound model, different sound sources are distinguished according to the "distance" of the sound propagation in the sound field, and each sound source is individually voiceprinted. In this way, in the transcription process, the identity of each sound source can be confirmed by comparing the voiceprint, so as to more accurately mark and distinguish different speakers.

[0034] It should be noted that the accuracy of voiceprint marking is limited by several factors: a. The duration of a single person's continuous speech; b. The effective speech duration; c. The number of speech corpora. Therefore, the generation and update of Vocal print Tab is a cumulative and iterative process. As time goes by, when effective voiceprint markings are gradually formed, these markings will correspond to real-time audio frames, thereby realizing Vocal print Tab marking of each frame of audio signal. Through this marking (Pvp=f(Pt)), the system can accurately distinguish different speakers and improve the accuracy and effectiveness of conference transcription.

[0035] Further, in step S400, for each frame of audio signal , the weighted sub-item formula in weighted processing is, ; ; in, Indicates the timestamp of each frame of audio signal. Indicates the value of the voiceprint tag at the current timestamp. Indicates the value of the sound source localization tag at the current timestamp.

[0036] Specifically, for each transcribed content, this embodiment combines the sound source localization information and voiceprint information of the person and uses weighted processing to achieve accurate transcription differentiation. The specific steps of the weighted algorithm can be expressed as follows: When the data volume of the voiceprint tag is insufficient, the tagging fails. t =0; When the voiceprint is successfully marked, set P t ={P1, P2, P3, ..., Pδ}, where δ represents a natural number greater than 0, representing multiple possible voiceprint markers; The DOA (Direction of Arrival) of the transmitted audio frame is labeled V doa =f(Vt), i.e., sound source localization information; The weighting coefficient α ranges from (0, 1) and is divided into the following two cases: When Pt=0, α takes the value of 0.9, which means that it relies more on the sound source localization information; When Pt is successfully marked continuously, α takes the value of 0.7, which means the influence of voiceprint information is enhanced; The weighted sub-item calculation formula is: ,in, Indicates the timestamp of each frame of audio signal. Indicates the value of the voiceprint tag at the current timestamp. Indicates the value of the sound source localization tag at the current timestamp.

[0037] Through this weighted algorithm, the system can effectively select and integrate the sound source location tag information and voiceprint information tag of the person, so as to achieve accurate distinction of each frame of audio signal. This process ensures that the speaker in the long-term speech transcription can be accurately distinguished in the conference transcription, thereby greatly reducing the manpower required for subsequent text proofreading and improving the efficiency and accuracy of transcription.

[0038] Example 2 Based on the same inventive concept, the present invention provides an online conference transcription system capable of realizing intelligent conference minutes, adopting the online conference transcription method as described above, comprising: The audio acquisition module is used to collect the audio data of each speaker during the meeting in real time through the microphone array and perform preliminary processing on the collected audio signals; The sound source localization module is used to perform voiceprint recognition on the audio signal to extract the speaker's voice spectrum characteristics, and mark the speaker's voiceprint according to the voiceprint recognition result; The voiceprint recognition module is used to calculate the time difference of each microphone receiving the audio signal during the preliminary processing of the audio signal, locate the sound source of the audio signal and determine the source direction of the audio signal based on the sound source location result; The data processing module is used to calculate the weighting coefficient of each frame of audio signal based on the sound source recognition results and the voiceprint recognition results, and perform weighted processing on each frame of audio signal to achieve the selection and fusion of the sound source positioning label and the voiceprint label information; the weighted audio signal is used for text transcription to locate the speaker to generate a complete meeting minutes.

[0039] Furthermore, the audio acquisition module includes: A collection unit, used to continuously and synchronously collect audio signals through microphones in the microphone array, so as to obtain audio data of all speakers during the conference; The pre-processing unit is used to perform preliminary processing on the collected audio signals, including echo cancellation, acoustic environment noise reduction and automatic gain.

[0040] Furthermore, the sound source localization module includes: A calculation unit, used to calculate the source direction of the audio signal by a differential time algorithm based on the time difference of the audio signal received by different microphones; The positioning unit is used to adjust the phase array of the microphone array according to the sound source positioning result, optimize the signal quality in the target direction, and enhance the audio signal through slope shaping and power spectrum estimation to improve the positioning accuracy of the audio signal; Labeling unit: used to correspond the sound source localization label with the real-time audio frame, mark the sound source localization label of each frame of audio data, and classify the audio information according to the sound source localization label in subsequent audio processing, where the sound source localization label is V doa =f(Vt).

[0041] Furthermore, the voiceprint recognition module includes: The feature extraction unit is used to extract speech spectrum features from the audio signal, capture the characteristic information of the sound, including pitch, timbre and speaking speed, and generate a unique voiceprint model for each speaker; The matching labeling unit is used to compare the extracted speech spectrum features with the voiceprint model, and assign a unique voiceprint label to each speaker based on the comparison results to mark the speaker's voice features. The voiceprint label is P vp =f(Pt); when the amount of data is insufficient or the marking fails, the voiceprint label Pt is set to 0; when the voiceprint marking is successful, Pt={P1,P2,P3,......,Pδ,}, δ is a natural number greater than 0.

[0042] Furthermore, for each frame of audio signal , the weighted sub-item formula in weighted processing is, ; ; in, Indicates the timestamp of each frame of audio signal. Indicates the value of the voiceprint tag at the current timestamp. Indicates the value of the sound source localization tag at the current timestamp.

[0043] Although the present invention has been disclosed as above in the form of a preferred embodiment, it is not intended to limit the present invention. Any person skilled in the art may make possible changes and modifications to the technical solution of the present invention by using the methods and technical contents disclosed above without departing from the spirit and scope of the present invention. Therefore, any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention shall fall within the protection scope of the technical solution of the present invention.

Claims

1. A method for online conference transcription capable of realizing intelligent conference minutes, characterized in that: The following steps are involved: Step S100: collecting audio data of each speaker during the conference in real time through a microphone array, and performing preliminary processing on the collected audio signals; Step S200: Based on the preliminary processing of the audio signal, the time difference of each microphone receiving the audio signal is calculated, the sound source of the audio signal is localized, and the source direction of the audio signal is determined according to the sound source localization result; Step S300: performing voiceprint recognition on the audio signal to extract the speech spectrum features of the speaker, and marking the voiceprint of the speaker according to the voiceprint recognition result; Step S400: Based on the sound source recognition result and the voiceprint recognition result, a weighting coefficient of each frame of the audio signal is calculated, and weighted processing is performed on each frame of the audio signal to select and merge the sound source localization label and the voiceprint label information; Step S500: using the weighted audio signal for text transcription and locating the speaker to generate a complete meeting minutes.

2. The online conference transcription method according to claim 1, characterized in that: The step S100 includes: Step S110: continuously and synchronously collecting audio signals through microphones in the microphone array to obtain the audio data of all speakers during the conference; Step S120: performing preliminary processing on the collected audio signal, including echo cancellation, acoustic environment noise reduction and automatic gain.

3. The online conference transcription method according to claim 1, characterized in that: The step S200 includes: Step S210: Calculating the source direction of the audio signal by using a differential time algorithm based on the time difference between the audio signals received by different microphones; Step S220: adjusting the phase array of the microphone array according to the sound source localization result, optimizing the signal quality in the target direction, and enhancing the audio signal through slope shaping and power spectrum estimation to improve the positioning accuracy of the audio signal; Step S230: Match the sound source localization tag with the real-time audio frame, mark the sound source localization tag of each frame of the audio data, and classify the audio information according to the sound source localization tag during subsequent audio processing, wherein the sound source localization tag is V doa =f(Vt).

4. The online conference transcription method according to claim 1, characterized in that: The step S300 includes: Step S310: extracting the speech spectrum features from the audio signal, capturing characteristic information of the sound, including pitch, timbre and speaking speed, and generating a unique voiceprint model for each speaker; Step S320: Based on the extracted speech spectrum features, the voiceprint model is compared, and a unique voiceprint label is assigned to each speaker according to the comparison result, marking the voice features of the speaker. The voiceprint label is P vp =f(Pt); where When the amount of data is insufficient or the marking fails, the voiceprint label Pt is set to 0; When the voiceprint marking is successful, Pt={P1, P2, P3, ..., Pδ,}, where δ is a natural number greater than 0.

5. The online conference transcription method according to claim 3 or 4, characterized in that: In step S400, for each frame of the audio signal , the weighted sub-item formula in weighted processing is, ； ； in, represents the timestamp of each frame of the audio signal, represents the value of the voiceprint tag at the current timestamp, The value of the sound source localization tag representing the current timestamp.

6. An online conference transcription system capable of realizing intelligent conference minutes, using the online conference transcription method as described in claims 1 to 5, characterized in that: include: The audio acquisition module is used to collect the audio data of each speaker during the meeting in real time through the microphone array and perform preliminary processing on the collected audio signals; A sound source localization module, used to calculate the time difference of each microphone receiving the audio signal during the preliminary processing of the audio signal, localize the sound source of the audio signal and determine the source direction of the audio signal according to the sound source localization result; A voiceprint recognition module, configured to perform voiceprint recognition on the audio signal to extract the speech spectrum characteristics of the speaker, and mark the voiceprint of the speaker according to the voiceprint recognition result; A data processing module is used to calculate the weighting coefficient of each frame of audio signal based on the sound source recognition result and the voiceprint recognition result, and perform weighted processing on each frame of audio signal to achieve the selection and fusion of sound source positioning label and voiceprint label information; the weighted audio signal is used for text transcription to locate the speaker to generate complete meeting minutes.

7. The intelligent replenishment analysis method according to claim 6, characterized in that: The audio acquisition module comprises: A collection unit, used to continuously and synchronously collect audio signals through microphones in the microphone array, so as to obtain the audio data of all speakers during the conference; The pre-processing unit is used to perform preliminary processing on the collected audio signal, including echo cancellation, acoustic environment noise reduction and automatic gain.

8. The intelligent replenishment analysis method according to claim 6, characterized in that: The sound source localization module comprises: A calculation unit, configured to calculate the source direction of the audio signal by a differential time algorithm based on the time difference between the audio signals received by different microphones; A positioning unit, configured to adjust the phase array of the microphone array according to the sound source positioning result, optimize the signal quality in the target direction, and enhance the audio signal through slope shaping and power spectrum estimation to improve the positioning accuracy of the audio signal; A labeling unit is used to correspond the sound source localization tag with the real-time audio frame, mark the sound source localization tag of each frame of the audio data, and classify the audio information according to the sound source localization tag during subsequent audio processing, wherein the sound source localization tag is V doa =f(Vt).

9. The intelligent replenishment analysis method according to claim 6, characterized in that: The voiceprint recognition module includes: A feature extraction unit, configured to extract the speech spectrum features from the audio signal, capture characteristic information of the sound, including pitch, timbre and speech speed, and generate a unique voiceprint model for each speaker; A matching and marking unit is used to compare the extracted speech spectrum features with the voiceprint model, and assign a unique voiceprint label to each speaker based on the comparison result to mark the voice features of the speaker. The voiceprint label is P vp =f(Pt); wherein, when the amount of data is insufficient or the marking fails, the voiceprint label Pt is set to 0; when the voiceprint marking is successful, Pt={P1,P2,P3,......,Pδ,}, where δ is a natural number greater than 0.

10. The intelligent replenishment analysis method according to claim 8 or 9, characterized in that: For each frame of the audio signal , the weighted sub-item formula in weighted processing is, ； ； in, represents the timestamp of each frame of the audio signal, represents the value of the voiceprint tag at the current timestamp, The value of the sound source localization tag representing the current timestamp.

Images