An intelligent conference system based on directional sound pickup device

Abstract

The application relates to the technical field of conference data recording, and discloses an intelligent conference system based on a directional sound pickup device, which comprises a directional sound pickup device module, which is used for collecting audio and video data of a conference area, uploading speaker audio after noise reduction to an audio processing and transcription module; an audio and video information processing module, which is used for confirming the number, position and identity information of speakers according to the audio and video data; the audio processing and transcription module is used for processing the received speaker audio, and then calling a transcription engine service to transcribe the speaker audio into text; a text processing and generation module is used for collating the text, realizing role correspondence and semantic recognition, and generating conference text and an abstract summary; a client is used for displaying or retrieving conference text and recording, and supports real-time editing, post-conference summary collation and operation control of the directional sound pickup device module. The application ensures reliable role separation and reduces noise interference, thereby improving transcription accuracy, user experience and conference efficiency.

Description

Technical Field

[0001] This application relates to the technical field of conference data recording, and in particular to an intelligent conference system based on a directional sound pickup device. Background Technology

[0002] In existing technologies, conference systems typically rely on omnidirectional microphones for audio acquisition. These devices indiscriminately capture sound from all directions, including the speaker's voice, background noise, and other interference. This omnidirectional acquisition method results in low-quality raw audio signals containing significant noise and reverberation, severely impacting the accuracy of subsequent speech transcription. When the transcription error rate is high, meeting records are often incomplete or distorted, making it difficult to accurately understand the meeting content.

[0003] Furthermore, traditional conferencing systems lack effective role separation capabilities, failing to distinguish the voices of different speakers, leading to messy transcriptions and requiring extensive manual editing of meeting minutes afterward—a process that is inefficient and prone to errors. Although some systems have introduced basic speech enhancement techniques, the inherent limitations of omnidirectional microphones limit noise cancellation effectiveness, especially in multi-person meeting environments where the problem is more pronounced.

[0004] Furthermore, existing systems typically do not support real-time editing and visual audio-text comparison, preventing users from promptly correcting transcription errors and impacting the reliability of meeting minutes. Regarding personalization, existing systems struggle to integrate user-specific vocabulary lists or dynamically update trending online terms, resulting in low recognition rates for technical terms or emerging vocabulary. High transcription latency also affects the real-time interactive experience. Simultaneously, existing systems rely on single technologies for speaker location and identity verification, such as voiceprint or facial recognition alone, which lacks accuracy in complex environments. Therefore, there is an urgent need in this field for a meeting system that can improve audio quality, achieve accurate role separation, support intelligent semantic understanding, and enhance user experience. Summary of the Invention

[0005] To provide a conferencing system that improves audio quality, enables precise role separation, supports intelligent semantic understanding, and enhances user experience, this application provides an intelligent conferencing system based on a directional sound pickup device, employing the following technical solution:

[0006] A smart conferencing system based on a directional sound pickup device includes a directional sound pickup device module, an audio and video information processing module, an audio processing and transcription module, a text processing and generation module, and a client.

[0007] The directional audio pickup module is used to collect audio and video data of the conference area, and upload the speaker's audio to the audio processing and transcription module after noise reduction.

[0008] The audio and video information processing module is used to confirm the number, location, and identity information of speakers based on the audio and video data;

[0009] The audio processing and transcription module is used to process the received speaker audio and then call the transcription engine service to transcribe the speaker audio into text.

[0010] The text processing and generation module is used to organize the text, realize role correspondence and semantic recognition, and generate meeting text and summary overview;

[0011] The client is used to display or retrieve the meeting text and recordings, supports real-time editing and post-meeting minutes organization, and controls the operation of the directional sound pickup device module.

[0012] By adopting the above technical solutions, the audio and video information processing module enables accurate speaker location and identity verification. In conjunction with the directional sound pickup device module, it can improve audio quality, ensure reliable role separation, and reduce noise interference, thereby improving transcription accuracy. The text processing and generation module automatically generates meeting summaries, reducing manual processing time. The client provides intuitive editing and search functions, improving user experience and meeting efficiency.

[0013] Optionally, the directional sound pickup device module includes:

[0014] Noise suppression unit, used to suppress ambient noise;

[0015] Reverberation suppression unit, used to suppress ambient reverberation;

[0016] The voice enhancement unit is used to enhance audio data within a specified beam range;

[0017] The audio compression unit is used to compress the enhanced audio data.

[0018] By adopting the above technical solutions, the noise suppression and reverberation suppression units effectively purify the audio signal and improve the quality of the original data; the speech enhancement unit accurately focuses on the target speech and avoids background interference; and the audio compression unit optimizes data transmission, adapts to the network environment, and ensures the real-time performance of the system.

[0019] Optionally, the directional pickup device module includes at least one directional pickup device unit;

[0020] The directional sound pickup device unit includes a directional sound pickup component for acquiring audio data in a specified direction;

[0021] The specified direction is configured in real time based on the speaker's estimated location area.

[0022] By adopting the above technical solutions, the system dynamically tracks the speaker's movement through real-time configuration of the specified direction, reducing background noise and maintaining the accuracy of audio acquisition; the multi-unit design covers the entire conference area, improving the system's adaptability and reliability.

[0023] Optionally, the directional sound pickup device module further includes:

[0024] The video acquisition unit is used to acquire video data centered on the specified direction, including the boundary of the conference area;

[0025] The audio and video information processing module includes:

[0026] The processing unit is used to configure the specified direction in real time and perform facial recognition to confirm the speaker's identity based on the video data;

[0027] The attention allocation in the face recognition focuses on the specified direction.

[0028] Optionally, the audio and video information processing module further includes multiple sound pickup feedback groups, each of which includes two microphones that are set at the same height on the straight boundary of the conference area;

[0029] The processing unit is further configured to select a real-time positioning group from the sound pickup feedback group according to a preset rule, and determine the estimated location area of ​​the speaker based on at least two real-time positioning groups located on non-parallel straight line boundaries.

[0030] The preset rules include:

[0031] The processing unit performs similarity verification on the audio data received by the pickup feedback group that is greater than a specified volume threshold, and selects at least one group of audio segments with a similarity greater than a specified ratio. If the time difference of the same audio in the audio segment group is less than a specified time difference value, the corresponding pickup feedback group is allowed to be selected as the real-time positioning group; otherwise, it is not allowed.

[0032] By adopting the above technical solutions, the reliability of positioning data is ensured through similarity verification and time difference checks; multi-group collaboration improves positioning accuracy, reduces misjudgments, and enhances the robustness of the system in complex environments.

[0033] Optionally, the process of determining the estimated location region includes:

[0034] When the number of real-time positioning groups is 2, the intersection point is determined according to the midpoint normal of all real-time positioning groups perpendicular to the boundary of the straight line. With the intersection point as the center, a circle with a diameter of 2 / 3 of the minimum microphone spacing in the real-time positioning group is used as the estimated position area.

[0035] When the number of real-time positioning groups is greater than 2, the enclosed area is determined based on the midpoint normal of all real-time positioning groups perpendicular to the straight line boundary. The estimated position area is a circle with a diameter of 2 / 3 of the minimum microphone spacing within the real-time positioning group, centered on the centroid of the enclosed area.

[0036] Optionally, if the spacing between microphones within the real-time positioning group is greater than a specified spacing threshold, then before confirming the intersection or the enclosed area, for each real-time positioning group, the midpoint normal is offset around the midpoint as the rotation center according to the same audio time difference. The offset direction is the microphone corresponding to the earlier received audio segment in the audio segment group, and the offset angle is positively correlated with the magnitude of the same audio time difference.

[0037] Optionally, the directional pickup assembly is configured with a pickup cover whose pickup surface is controlled, and the control logic of the pickup surface is as follows:

[0038] When the specified direction changes, the pickup surface increases;

[0039] When the static time in the specified direction exceeds a specified duration, the pickup surface is reduced to a specified size;

[0040] The specified size is positively correlated with the intra-group spacing of the real-time positioning group.

[0041] By adopting the above technical solutions, the adaptive pickup surface optimizes the audio acquisition range, maintaining a high signal-to-noise ratio in dynamic environments; reducing the pickup surface reduces background noise and improves audio quality.

[0042] Optionally, the client supports visual phonetic-text comparison, highlights the corresponding text when playing audio, and allows users to edit the text in real time. The text processing and generation module supports uploading personalized word lists and updating trending online words to improve transcription accuracy.

[0043] By adopting the above technical solutions, visual comparison makes it easier for users to check and correct the transcribed content, improving text accuracy; personalized word lists and hot word updates enhance the transcription adaptability, ensuring the system's accuracy and user experience in diverse scenarios.

[0044] In summary, this application includes at least one of the following beneficial technical effects:

[0045] This application achieves accurate speaker location and identity verification through an audio and video information processing module. In conjunction with a directional sound pickup device module, it can improve audio quality, ensure reliable role separation, and reduce noise interference, thereby improving transcription accuracy. The text processing and generation module automatically generates meeting summaries, reducing manual processing time. The client provides intuitive editing and search functions, improving user experience and meeting efficiency. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the intelligent meeting system architecture in this application. Detailed Implementation

[0047] The embodiments of this application are described in detail below, and examples of the embodiments are shown in the accompanying drawings.

[0048] In the description of this specification, the references to "certain embodiments," "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples" refer to specific features, structures, materials, or characteristics described in connection with the described embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0049] This application discloses an intelligent conference system based on a directional microphone device, referring to... Figure 1 It includes a directional sound pickup device module, an audio and video information processing module, an audio processing and transcription module, a text processing and generation module, and a client.

[0050] The directional audio pickup module is used to collect audio and video data of the conference area, and upload the speaker's audio to the audio processing and transcription module after noise reduction.

[0051] The audio and video information processing module is used to confirm the number, location, and identity information of speakers based on the audio and video data;

[0052] The audio processing and transcription module is used to process the received speaker audio and then call the transcription engine service to transcribe the speaker audio into text.

[0053] The text processing and generation module is used to organize the text, realize role correspondence and semantic recognition, and generate meeting text and summary overview;

[0054] The client is used to display or retrieve the meeting text and recordings, supports real-time editing and post-meeting minutes organization, and controls the operation of the directional sound pickup device module.

[0055] In this embodiment of the application, the GooseCard uses an audio and video information processing module to achieve accurate speaker location and identity verification. In conjunction with the directional sound pickup device module, it can improve audio quality, ensure reliable role separation, and reduce noise interference, thereby improving transcription accuracy. The text processing and generation module automatically generates meeting summaries, reducing manual processing time. The client provides intuitive editing and retrieval functions, improving user experience and meeting efficiency.

[0056] Optionally, the directional sound pickup device module includes:

[0057] A noise suppression unit is used to suppress ambient noise in order to obtain clean target speech;

[0058] The reverberation suppression unit is used to suppress ambient reverberation in order to obtain clean target speech;

[0059] The voice enhancement unit is used to enhance audio data within a specified beam range;

[0060] The audio compression unit compresses the enhanced audio data using the G.711 compression algorithm or OGG compression, which reduces bandwidth and ensures audio transmission stability; the noise suppression and reverberation suppression unit effectively purifies the audio signal and improves the quality of the original data; the speech enhancement unit accurately focuses on the target speech and avoids background interference; the audio compression unit optimizes data transmission, adapts to the network environment, and ensures system real-time performance.

[0061] Optionally, the directional sound pickup device module includes at least one directional sound pickup device unit and a video acquisition unit;

[0062] The directional sound pickup device unit includes a directional sound pickup component for acquiring audio data in a specified direction;

[0063] The video acquisition unit is used to acquire video data centered on the specified direction, including the boundary of the conference area;

[0064] The audio and video information processing module includes:

[0065] The processing unit is used to configure the specified direction in real time and perform facial recognition to confirm the speaker's identity based on the video data;

[0066] The specified direction is configured in real time based on the speaker's estimated location area, and the attention allocation of the face recognition focuses on the specified direction.

[0067] In this embodiment, the video acquisition unit and the directional audio pickup device unit are bound together, and the acquisition direction of both is consistent. By configuring the specified direction in real time, the speaker's movement can be dynamically tracked, reducing noise input and maintaining the accuracy of audio acquisition. The multi-unit design covers the entire conference area, improving the system's adaptability and reliability.

[0068] In addition, the directional pickup component is equipped with a pickup cover with a controlled pickup surface, and the control logic of the pickup surface is as follows:

[0069] When the specified direction changes, the pickup surface increases;

[0070] When the static time in the specified direction exceeds 2 seconds, the pickup surface is reduced to the specified size;

[0071] The specified size is positively correlated with the intra-group spacing of the real-time positioning group.

[0072] The specified size configuration needs to be calibrated according to the actual situation of the meeting area. The smaller the spacing within the real-time positioning group, the higher the positioning accuracy. The pickup surface can also be minimized to a certain extent, thereby obtaining a purer speaker audio.

[0073] In this embodiment, the audio and video information processing module further includes multiple sound pickup feedback groups; specifically, for a designated meeting area, a virtual wall with at least three straight boundaries that are as consistent as possible with the boundary of the meeting area can be pre-established. The sound pickup feedback groups on each straight boundary are set at the same height, and each sound pickup feedback group includes two microphones set at the same height on the straight boundary of the meeting area. The spacing between the sound pickup feedback groups should be smaller than the internal spacing of the sound pickup feedback groups.

[0074] The processing unit is further configured to select a real-time positioning group from the sound pickup feedback group according to a preset rule, and determine the estimated location area of ​​the speaker based on at least two real-time positioning groups located on non-parallel straight line boundaries.

[0075] The preset rules include:

[0076] The processing unit performs similarity verification on the audio data received by the pickup feedback group that is greater than a specified volume threshold, and selects at least one group of audio segments with a similarity greater than a specified ratio. If the time difference of the same audio in the audio segment group is less than a specified time difference value, the corresponding pickup feedback group is allowed to be selected as the real-time positioning group; otherwise, it is not allowed.

[0077] Taking a sound feedback group A and a sound feedback group B on a straight boundary as an example, with the specified time difference configured as 1ms, sound feedback group A includes microphone A1 and microphone A2, and sound feedback group B includes microphone B1 and microphone B2. Since the main speakers are usually 1-3 people, the specified ratio can be configured to 85% to reduce computing power consumption. Assuming that the time difference between the same audio segment detected by microphones A1 and A2 is 0.5ms, and the time difference between the same audio segment detected by microphones B1 and B2 is 1.4ms, then sound feedback group A can be selected as the real-time positioning group. However, the time difference between the same audio segment detected by sound feedback group B is too large, and it is impossible to accurately determine the speaker's position based on the midpoint normal of microphones B1 and B2. Therefore, it is not allowed to be selected as the real-time positioning group. With this design, the speaker's location in the venue can be captured by the corresponding real-time positioning group. The upper limit of the acquisition accuracy depends on the similarity verification accuracy, the same-frequency time difference calculation accuracy, the number and density of the sound pickup feedback groups, and the total number of straight boundary lines. Since the similarity verification and same-frequency time difference calculation methods are relatively mature technical solutions, the upper limit of subsequent positioning accuracy improvement is relatively easy to improve. Different packages and different pricing levels can be designed to meet the needs of large, medium, small and various high-end and low-end conference scenarios in the market, which has broad commercial prospects.

[0078] Optionally, the process of determining the estimated location region includes:

[0079] When the number of real-time positioning groups is 2 (this case is rare, generally suitable for small meeting scenarios, with slightly lower accuracy, suitable for customers with lower budgets), the intersection point is determined according to the midpoint normal of all real-time positioning groups perpendicular to the boundary of the straight line. With the intersection point as the center, a circle with a diameter of 2 / 3 of the minimum microphone spacing in the real-time positioning group is used as the estimated position area.

[0080] When the number of real-time positioning groups is greater than 2, the enclosed area is determined based on the midpoint normal of all real-time positioning groups perpendicular to the straight line boundary. The estimated position area is a circle with a diameter of 2 / 3 of the minimum microphone spacing within the real-time positioning group, centered on the centroid of the enclosed area.

[0081] Optionally, if the spacing between microphones within the real-time positioning group is greater than a specified spacing threshold (indicating that the density of the microphone feedback group may be low) or the customer seeks higher precision in speaker positioning, then before confirming the intersection or the enclosed area, for each real-time positioning group, the midpoint normal is offset around the midpoint as the rotation center according to the same audio time difference. The offset direction is the microphone corresponding to the earlier received audio segment in the audio segment group, and the offset angle is positively correlated with the magnitude of the same audio time difference, such as a linear correspondence of 0.1 radians / millisecond or a non-linear correspondence starting at 0.08 radians / millisecond.

[0082] Thus, this embodiment achieves a full-process conference service by capturing video and audio data and processing the data, from fuzzy audio data location to accurate video location, and following the audio pickup, transcription, and recording.

[0083] Optionally, the client supports visual audio-text comparison, highlighting the corresponding text while playing audio, allowing users to edit the text in real time, and supporting role-based viewing / full-text viewing: it supports role-based viewing of meeting information, allowing users to view the speaker's speaking information in the current meeting and the corresponding speaking timestamp by clicking on the corresponding speaker; the client is also configured with a meeting management module: it needs to provide management of all real-time recorded audio text data transcribed by this system, displayed in a list format; it supports viewing and editing at any time through the content management module, and can be quickly searched based on information such as name.

[0084] In addition, the intelligent meeting system also supports the following features and personalized, in-depth customization functions:

[0085] Upload contacts and personalized word lists, improve the accuracy of personal vocabulary, use the user's device as a unique ID, and link different devices through a language pass;

[0086] Customization features: Daily updates of trending online terms, quarterly updates of the self-trained model;

[0087] Service capabilities: Command word grammar recognition, web hot word search;

[0088] Features: Supports 8K / 16KPCM audio, supports AMR / Speex audio, transmission bandwidth is 16k / 23kbps, recognition result latency is no more than 2 seconds; also supports customized business processes after recognition.

[0089] Although embodiments of this application have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting this application. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of this application.

Claims

1. An intelligent conference system based on a directional sound pickup device, characterized in that, It includes a directional sound pickup device module, an audio and video information processing module, an audio processing and transcription module, a text processing and generation module, and a client. The directional audio pickup module is used to collect audio and video data of the conference area, and upload the speaker's audio to the audio processing and transcription module after noise reduction. The audio and video information processing module is used to confirm the number, location, and identity information of speakers based on the audio and video data; The audio processing and transcription module is used to process the received speaker audio and then call the transcription engine service to transcribe the speaker audio into text. The text processing and generation module is used to organize the text, realize role correspondence and semantic recognition, and generate meeting text and summary overview; The client is used to display or retrieve the meeting text and recordings, and supports real-time editing, post-meeting minutes compilation, and operation control of the directional sound pickup device module. The directional pickup device module includes at least one directional pickup device unit; The directional sound pickup device unit includes a directional sound pickup component for acquiring audio data in a specified direction; The specified direction is configured in real time based on the speaker's estimated location area; The directional sound pickup device module also includes: The video acquisition unit is used to acquire video data containing the boundary of the conference area centered on the specified direction; The audio and video information processing module includes: The processing unit is used to configure the specified direction in real time and perform facial recognition to confirm the speaker's identity based on the video data; The attention allocation in the face recognition focuses on the specified direction; The audio and video information processing module also includes multiple sound pickup feedback groups, each of which includes two microphones that are set at the same height on the straight boundary of the conference area. The processing unit is further configured to select a real-time positioning group from the sound pickup feedback group according to a preset rule, and determine the estimated location area of ​​the speaker based on at least two real-time positioning groups located on non-parallel straight line boundaries. The preset rules include: The processing unit performs similarity verification on the audio data received by the pickup feedback group that is greater than a specified volume threshold, and selects at least one group of audio segments with a similarity greater than a specified ratio. If the time difference of the same audio in the audio segment group is less than a specified time difference value, the corresponding pickup feedback group is allowed to be selected as the real-time positioning group; otherwise, it is not allowed.

2. The intelligent conference system based on a directional sound pickup device according to claim 1, characterized in that, The directional sound pickup device module includes: Noise suppression unit, used to suppress ambient noise; Reverberation suppression unit, used to suppress ambient reverberation; The voice enhancement unit is used to enhance audio data within a specified beam range; The audio compression unit is used to compress the enhanced audio data.

3. The intelligent conference system based on a directional sound pickup device according to claim 1, characterized in that, The process of determining the estimated location region includes: When the number of real-time positioning groups is 2, the intersection point is determined according to the midpoint normal of all real-time positioning groups perpendicular to the boundary of the straight line. With the intersection point as the center, a circle with a diameter of 2 / 3 of the minimum microphone spacing in the real-time positioning group is used as the estimated position area. When the number of real-time positioning groups is greater than 2, the enclosed area is determined based on the midpoint normal of all real-time positioning groups perpendicular to the straight line boundary. The estimated position area is a circle with a diameter of 2 / 3 of the minimum microphone spacing within the real-time positioning group, centered on the centroid of the enclosed area.

4. The intelligent conference system based on a directional sound pickup device according to claim 3, characterized in that, If the spacing between microphones within the real-time positioning group is greater than a specified spacing threshold, then before confirming the intersection or the enclosed area, for each real-time positioning group, the midpoint normal is offset around the midpoint as the rotation center according to the same audio time difference. The offset direction is the microphone corresponding to the earlier received audio segment in the audio segment group, and the offset angle is positively correlated with the magnitude of the same audio time difference.

5. The intelligent conference system based on a directional sound pickup device according to claim 1, characterized in that, The directional pickup component is equipped with a pickup cover with a controlled pickup surface. The control logic for the pickup surface is as follows: When the specified direction changes, the pickup surface increases; When the static time in the specified direction exceeds a specified duration, the pickup surface is reduced to a specified size; The specified size is positively correlated with the intra-group spacing of the real-time positioning group.

6. The intelligent conference system based on a directional sound pickup device according to claim 1, characterized in that, The client supports visual phonetic-text comparison, highlights the corresponding text when playing audio, and allows users to edit the text in real time. The text processing and generation module supports uploading personalized word lists and updating trending online words to improve transcription accuracy.

Images