An intelligent conference ethics protection method and system based on voiceprint recognition

Abstract

This invention belongs to the field of data security technology, specifically a method and system for intelligent meeting ethics protection based on voiceprint recognition. The method includes: during the meeting initialization phase, a lightweight voiceprint recognition program is run in a browser sandbox environment to collect real-time voice recordings of participants and construct a temporary voiceprint baseline whitelist stored only in a local cache; during the meeting, the system collects microphone audio streams in real time, extracts voiceprint features from speech segments using a sliding window mechanism, and compares the similarity with the local whitelist; when uninvited participants' voices are detected (i.e., the similarity is below a preset threshold), the system immediately triggers a tiered ethical response mechanism, including visual waveform warnings and audio stream transmission blocking. This invention achieves real-time ethical monitoring and proactive protection of the meeting environment without uploading user biometric data to a server, effectively solving the privacy and ethical infringement problem caused by unauthorized voices in an open environment.

Description

Technical Field

[0001] This invention belongs to the field of data security technology, specifically, it relates to a method and system for protecting ethical conduct in intelligent meetings based on voiceprint recognition. Background Technology

[0002] With the rapid development of artificial intelligence technology, AI-powered intelligent conferencing systems have become the infrastructure for remote collaboration, greatly improving meeting efficiency with their automatic recording, transcription, and intelligent analysis functions. However, along with this convenience comes increasingly prominent ethical and privacy protection issues. Existing mainstream conferencing system designs often follow a technology-centric approach. While they strengthen protection through passive means such as access passwords, waiting rooms, or data encryption, they neglect the ethical issues of use in complex real-world scenarios. In practical applications, participants are often in uncontrollable open environments such as cafes, homes, or shared office spaces. This fluidity of environment presents serious ethical challenges.

[0003] First, there is the risk of privacy intrusion due to low awareness. Existing systems lack dynamic perception and ethical filtering mechanisms for ambient sounds. When the voices of unrelated individuals (such as family members or passersby) accidentally interrupt the meeting, these uninvited participants are actually being collected without their knowledge. They are unaware that their voices are being recorded, transcribed, and even analyzed by AI models in real time by the meeting system. This seriously infringes upon the uninvited participants' right to know and their right to privacy, constituting an ethical blind spot in the meeting environment.

[0004] Secondly, there is a lack of ethical oversight. Current AI-powered intelligent conferencing systems lack proactive compliance design when faced with such environmental interference. The system fails to provide users or those being recorded with immediate information and choice when irrelevant sounds intervene. This technology-centric design results in users lacking autonomy and channels for dynamic consent when faced with the excessive collection of their privacy data (especially biometric data).

[0005] Therefore, there is an urgent need for an ethical adjustment scheme that transcends mere technical protection and embeds the principle of informed consent into the functional design. This scheme needs to achieve real-time identification and ethical blocking of the voices of uninvited participants without affecting the normal use of the conferencing system, shifting from passive defense to proactive, user-centric ethical protection, in order to address the challenges of lack of informed consent and ethical compliance faced by current conferencing systems in complex environments. Summary of the Invention

[0006] This invention aims to address the problems mentioned in the background section by providing a method and system for intelligent meeting ethics protection based on voiceprint recognition. This invention achieves real-time ethical monitoring and proactive protection of the meeting environment without uploading user biometric data to a server, effectively resolving the privacy and ethical infringement issues caused by unauthorized voices of non-participants in open environments.

[0007] The technical solution of the present invention is described in detail below.

[0008] A smart meeting ethics protection method based on voiceprint recognition runs a lightweight voiceprint recognition model within a sandbox on the client browser. This constructs a dynamic whitelist of participants without data leaving the domain and monitors microphone audio streams in real time. Upon detecting unauthorized third-party voices, the system immediately triggers an ethics warning mechanism, thereby ensuring the privacy and compliance of the meeting. The method includes the following steps: (1) Local dynamic benchmark construction: During the initialization phase of the meeting, the client loads the voiceprint recognition model in the browser sandbox environment; the real-time voice of the participants is collected locally on the client, and after the validity of the input samples is evaluated, the model extracts voiceprint feature data to construct a temporary whitelist; (2) Real-time verification of attendees’ identities: During the meeting, the original audio stream of the microphone is captured and voice activity is detected to filter out silent segments; the effective voice frame is captured by a sliding window mechanism, the real-time voiceprint features in the effective voice frame are extracted, and the similarity between the voiceprint features of the effective voice frame and the voiceprint features of the temporary whitelist is calculated. If the similarity of N consecutive frames is lower than the preset threshold, the current sound source is determined to be a non-attendee. (3) Tiered ethical blocking: When a person is determined to be an uninvited participant, the system will execute prompting and intervention responses according to the configuration.

[0009] In this invention, during step (1), when evaluating the validity of the input samples, samples with excessive reverberation and background noise are removed. A voiceprint recognition model is then used to extract high-dimensional voiceprint feature data from the participants' audio recordings. This is data used as a temporary whitelist.

[0010] In this invention, in step (1), the voiceprint recognition model is loaded in the browser sandbox environment using WebAssembly technology; the voiceprint feature data is only stored in the local cache of the client and a lifecycle is set, and the data is automatically destroyed when the meeting ends command is triggered, and the voiceprint feature data is not uploaded to the server.

[0011] In this invention, in step (2), the original audio stream of the microphone is captured using AudioWorklet technology, and the real-time voiceprint features are extracted locally by the WebAssembly program running in the browser sandbox.

[0012] In this invention, in step (3), the prompting response (i.e., warning feedback) specifically involves: color-changing the audio waveform in the user interface to provide feedback to the user; the intervention response (mandatory informed consent) specifically involves: when the user has enabled the strong intervention mode setting, blocking the transmission of the audio sampling stream through the application layer interface or the underlying flow control interface, and triggering informed consent interaction; that is, if the user has enabled the strong intervention mode in the system settings, the user interface will pause speaking, a pop-up window will prompt that there is a meeting security risk, and the user will be forced to choose to turn off the microphone or confirm to continue the operation; if it is not enabled, only the prompting response will be maintained.

[0013] This invention also provides an intelligent meeting ethics protection system based on voiceprint recognition, used to implement the above method, comprising: Real-time monitoring module: used to perform frame-by-frame processing and noise filtering on the audio stream during the meeting, and output valid audio stream samples; Voiceprint recognition module: Deployed on the user's client, it uses a lightweight voiceprint recognition model to receive the user's valid audio stream and extract voiceprint feature data, build a local temporary whitelist, and verify the identity of the participants through voiceprint comparison; Hierarchical Ethics Interaction Control Module: Used to receive identity determination results and execute prompting and intervention responses.

[0014] In this invention, both the voiceprint recognition module and the real-time monitoring module run in a browser sandbox environment based on WebAssembly technology, ensuring that the original voiceprint feature data does not leak out of the user's terminal device during the calculation process.

[0015] Compared with the prior art, the present invention has the following beneficial effects: (1) Data sovereignty protection: All voiceprint extraction and comparison are completed in the local browser sandbox, without the need to upload sensitive biometric data to the cloud server, completely eliminating users' concerns about privacy leakage.

[0016] (2) Ethical protection throughout the process: Unlike the traditional trust model of joining a meeting, this invention provides continuous verification of dynamic informed adjustment, which can cope with the ethical risks brought about by changes in the environment during the meeting. Attached Figure Description

[0017] Figure 1 is a schematic diagram of a real-time identity verification and ethical protection system and method for intelligent conference participants based on voiceprint recognition. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described in detail below with reference to specific embodiments.

[0019] like Figure 1As shown, this invention provides an intelligent meeting ethics protection method based on voiceprint recognition. This method is deployed in a network conferencing system that includes client terminals and a conferencing server. It establishes a temporary voiceprint baseline locally on the client side to monitor and warn of unauthorized voice intervention in real time. The specific steps are as follows: (1) Dynamic whitelist construction: During the meeting initialization phase, real-time voice recordings of participants are collected locally on the client side. After evaluating the validity of the recorded samples, voiceprint features are extracted to construct a temporary whitelist; specifically including: ① Personnel verification: The client displays interactive prompts to participants, guiding them to respond in real time with their voices, evaluating the validity of the input samples in real time, and removing samples with excessive reverberation or background noise to ensure that the generated whitelist benchmark is clean; ② Feature Extraction: Using a voiceprint recognition model loaded by a WebAssembly program running in a browser sandbox, high-dimensional voiceprint feature data is extracted from the audio of the participants reading aloud. ; ③ Privacy storage: Store the data in the client cache as whitelist data and set a lifecycle so that the data is automatically destroyed when the meeting ends. The temporary whitelist voiceprint reference is stored only in the client session storage space, and this storage space has a lifecycle attribute that is automatically destroyed when the meeting ends, and the voiceprint feature data is not uploaded to the server; (2) Real-time verification of attendee identity: During the meeting, the audio stream input from the microphone is collected in real time and valid voice frames are extracted. The voiceprint features of the current voice segment are extracted, and the similarity between the voiceprint features and the features in the temporary whitelist is calculated locally. The real-time verification of attendee identity adopts a sliding window and voice activity detection mechanism, specifically including: ① Perform speech activity detection on the real-time audio stream and filter out silent segments; ② A sliding window mechanism is used to extract valid speech frames and extract the speaker signature data of the current frame. ; ③Calculation Each feature data in the temporary whitelist Voiceprint similarity; ④ If the maximum similarity value of multiple consecutive frames is lower than the preset judgment threshold If the result is "uninvited participants intervened", the output will be "uninvited participants intervened".

[0020] (3) Ethical Warning: When the similarity calculated from multiple consecutive frames is lower than a preset safety threshold, the system determines that the current sound is from uninvited participants and immediately triggers a tiered ethical response mechanism, including a visual warning on the client-side front end and the execution of an audio stream blocking strategy. The tiered ethical response operation includes at least one of the following: ① Prompt response: Change the rendering state of the sound column or waveform on the front-end interface to provide feedback to the user; ② Interventional response: When the system detects that the user has enabled the strong intervention mode, it blocks the transmission of the audio sampling stream through the application layer interface or the underlying flow control interface and triggers informed consent interaction.

[0021] This invention also provides an intelligent meeting ethics protection system based on voiceprint recognition, used to implement the above-described method, comprising: Real-time monitoring module: used for voice activity detection and frame segmentation of the audio stream during the meeting; Voiceprint recognition module: Deployed on the user terminal, it is used to receive the user's valid audio stream and extract voiceprint feature data, build a local temporary whitelist, and perform voiceprint comparison; The hierarchical ethics interaction control module is used to receive identity determination results, control the rendering of visual warnings on the user interface, and manage the blocking or transmission of audio streams. Both the aforementioned voiceprint recognition module and real-time monitoring module run in a browser sandbox environment based on WebAssembly technology, ensuring that the original voiceprint feature data does not leak out of the user's terminal device during the calculation process.

[0022] Both the aforementioned voiceprint recognition module and real-time monitoring module run in a browser sandbox environment based on WebAssembly technology, ensuring that the original voiceprint feature data does not leak out of the user's terminal device during the calculation process.

[0023] The following are specific examples.

[0024] Example 1: Construction of a local dynamic whitelist based on WebAssembly This embodiment details the voiceprint identity registration process during the meeting initialization phase.

[0025] (1) Environment Loading: When a user enters the meeting interface, the browser's background thread asynchronously loads the compiled WebAssembly format voiceprint recognition program. This program can be implemented using existing mature edge-side voiceprint inference SDKs (such as third-party technologies that support browser-side operation, such as Picovoice Eagle). Subsequently, the voiceprint recognition program loads a lightweight voiceprint recognition model (i.e., the weights and configuration files of the voiceprint extraction model) in an independent browser sandbox memory space, achieving physical isolation from the external network environment.

[0026] (2) Personnel Verification: The system triggers dynamic interactive commands on the front-end interface to guide users to generate instant voice responses. The system dynamically verifies the recording process through real-time quality feedback returned by the voiceprint recognition program. If the audio sampling shows discontinuous acoustic features, missing effective human voices, or substandard acoustic quality assessment (such as severe reverberation or low signal-to-noise ratio), the system will trigger a feedback suppression mechanism to pause the recording progress bar and filter out the invalid audio segment, ensuring that the constructed temporary voiceprint whitelist benchmark has high uniqueness and anti-interference capabilities.

[0027] (3) Feature extraction: For the verified speech data, frame segmentation and windowing are performed. The WebAssembly voiceprint recognition program extracts clear voiceprint feature data from the audio locally, denoted as... .

[0028] (4) Privacy storage: This data is directly added to the temporary whitelist for this meeting and stored in the browser's cache. The system is configured to release this memory area immediately when the tab is closed or the meeting ends, ensuring that no data is left behind.

[0029] Example 2: Real-time monitoring based on sliding window This embodiment describes how to accurately identify uninvited attendees in a continuous audio stream.

[0030] (1) Audio Stream Acquisition: The microphone input stream is acquired with extremely low latency using the browser's AudioContext and AudioWorkletNode interfaces. In the specific implementation, the client uses the browser's AudioContext interface to create an audio processing context and loads the audio processing module in an independent thread through the AudioWorkletNode interface. The system starts microphone acquisition through a subscription mechanism, and the audio stream is resampled to 16000Hz in real time to match the hard requirements of the voiceprint model for the characteristic frequency distribution. Whenever the acquisition buffer reaches the preset frame length, the background processing thread asynchronously pushes the encapsulated PCM audio frame to the WebAssembly voiceprint recognition program running in an independent thread through a message passing mechanism, thereby realizing non-blocking voiceprint feature extraction.

[0031] (2) Voice Activity Detection: To reduce unnecessary computation and power consumption, the system performs voice activity detection on each audio frame before it enters the voiceprint recognition process. The root mean square energy (RMS) of each audio sample is calculated to characterize the overall energy level of that frame. When the energy of an audio frame is detected to be below a preset threshold, the frame is determined to be a silent or background noise segment, and the voiceprint comparison logic is skipped to reduce computational overhead. This avoids further processing and prevents false alarms caused by background noise fluctuations. By eliminating background noise and low-volume segments in this way, and processing only audio frames containing valid human voices, the system resource consumption is effectively reduced and recognition stability is improved.

[0032] (3) Sliding window comparison: For valid audio frames detected through speech activity, the system uses a sliding window mechanism to perform temporal slicing processing on continuous speech. The collected continuous audio frames are constructed into an analysis window according to a fixed time length, with each window covering approximately 1 to 2 seconds of continuous speech data; overlapping areas are set between adjacent windows, allowing the windows to slide forward at small time steps (e.g., hundreds of milliseconds). In this way, the system can process long-duration speech streams in batches while ensuring temporal continuity, enabling voiceprint feature extraction to be both real-time and responsive to short-term speech changes.

[0033] (4) Local inference: The constructed audio window is passed to the WebAssembly voiceprint recognition program to extract the voiceprint feature vector of the current audio data. .

[0034] (5) Similarity Calculation: After obtaining the voiceprint feature vector of the audio data to be detected through the WebAssembly voiceprint recognition program, the client will calculate the similarity. With each feature vector in the whitelist The similarity of voiceprints between them is used as the matching score. The similarity calculation is based on mature metric algorithms in existing technologies, such as cosine similarity algorithms, traditional machine learning algorithms, and end-to-end scoring mechanisms based on deep neural networks. (Preset identity verification threshold) If the speaker is identified as the participant (i.e., a whitelisted user), then audio data transmission or subsequent business logic execution is permitted; if... If a frame is detected, it is marked as a suspicious frame. To avoid false alarms caused by occasional noise, the system maintains a queue of length [length missing]. The decision result queue is only used when the queue exceeds [a certain number of results]. The system only officially triggers an ethics alert when all frames are determined to be from uninvited attendees.

[0035] Example 3: Ethical Blockade and Interactive Feedback This embodiment describes the system's tiered response strategy after detecting a risk.

[0036] (1) Visual warning: Once the system detects that there is an uninvited person's voice access, the front-end main thread captures the event and modifies the UI style: the rendering color of the user's voice wave visualization component changes from green to red flashing, and prompts "Unfamiliar voice detected in the environment, please pay attention to privacy".

[0037] (2) Informed Consent Pop-up: The system first reads the user's security configuration parameters. If the user has selected the "Strong Intervention Mode" option and the risk continues for more than the set time without user intervention, the system will automatically take over, adjust the microphone input gain to 0, cut off the audio upload link to the conference server, and a mandatory informed consent pop-up will appear in the center of the screen, prompting the system to detect that non-participants are speaking in the environment. For the safety of the conference, the user must confirm the operation and choose to mute the microphone or confirm to continue the conference. If the user does not enable this option, the system will continue to maintain a visual warning but will not cut off the audio link to ensure the continuity of the conference.

Claims

1. A method for protecting ethical conduct in intelligent meetings based on voiceprint recognition, characterized in that, It utilizes a lightweight voiceprint recognition model running within a sandbox in the client browser to build a dynamic whitelist of participants without data leaving the domain. It also monitors microphone audio streams in real time. Upon detecting unauthorized third-party voices, the system immediately triggers an ethical warning mechanism, thus ensuring the privacy and compliance of the meeting. This includes the following steps: (1) Local dynamic benchmark construction: During the initialization phase of the meeting, the client loads the voiceprint recognition model in the browser sandbox environment; the real-time voice of the participants is collected locally on the client, and after the validity of the input samples is evaluated, the model extracts voiceprint feature data to construct a temporary whitelist; (2) Real-time verification of attendees’ identities: During the meeting, the original audio stream of the microphone is captured and voice activity is detected to filter out silent segments; the effective voice frame is captured by a sliding window mechanism, the real-time voiceprint features in the effective voice frame are extracted, and the similarity between the voiceprint features of the effective voice frame and the voiceprint features of the temporary whitelist is calculated. If the similarity of N consecutive frames is lower than the preset threshold, the current sound source is determined to be a non-attendee. (3) Tiered ethical blocking: When a person is determined to be an uninvited participant, the system will execute prompting and intervention responses according to the configuration.

2. The intelligent meeting ethics protection method according to claim 1, characterized in that, In step (1), when evaluating the validity of the input samples, samples with excessive reverberation and background noise are removed. High-dimensional voiceprint feature data are extracted from the audio of the participants' reading using a voiceprint recognition model. This is data used as a temporary whitelist.

3. The intelligent meeting ethics protection method according to claim 1, characterized in that, In step (1), the voiceprint recognition model is loaded in the browser sandbox environment using WebAssembly technology; Voiceprint feature data is stored only in the client's local cache and has a lifecycle set. The data is automatically destroyed when the meeting ends and the command is triggered. The voiceprint feature data is not uploaded to the server.

4. The intelligent meeting ethics protection method according to claim 1, characterized in that, In step (2), the original audio stream from the microphone is captured using AudioWorklet technology, and real-time voiceprint features are extracted in the local WebAssembly program.

5. The intelligent meeting ethics protection method according to claim 1, characterized in that, In step (3), the prompting response is specifically: the audio waveform is rendered in different colors on the user interface to provide feedback to the user; the intervention response is specifically: when the user has enabled the strong intervention mode setting, the transmission of the audio sampling stream is blocked through the application layer interface or the underlying flow control interface, and the informed consent interaction is triggered.

6. A smart meeting ethics protection system based on voiceprint recognition, used to implement the method of any one of claims 1 to 5, characterized in that, include: Real-time monitoring module: used to perform frame-by-frame processing and noise filtering on the audio stream during the meeting, and output valid audio stream samples; Voiceprint recognition module: Deployed on the user's client, it uses a lightweight voiceprint recognition model to receive the user's valid audio stream and extract voiceprint feature data, build a local temporary whitelist, and verify the identity of the participants through voiceprint comparison; Hierarchical Ethics Interaction Control Module: Used to receive identity determination results and execute prompting and intervention responses.

7. The intelligent meeting ethics protection system according to claim 6, characterized in that: Both the voiceprint recognition module and the real-time monitoring module are based on WebAssembly technology and run in a browser sandbox environment to ensure that the original voiceprint feature data does not leak out of the user's terminal device during the calculation process.

Images