Voice transcription and summary generation method based on deep integrated security conference system

Abstract

The invention relates to a voice transcription and summary generation method based on a deep integrated security conference system, belongs to the technical field of voice transcription and summary generation, and solves the problems that a traditional voice transcription and summary generation technology is low in efficiency, information is omitted and potential safety hazards exist. Comprising the steps that a voice recognition client side collects audio and sends the collected audio to a voice recognition server side through an audio sending thread; the voice recognition server side adopts a quantized off-line model paraform-offset-zh to carry out segmentation recognition and secondary whole segment recognition on the audio, a transfer text is generated, and a conference summary is generated based on the transfer text; the voice recognition client receives the transcription text generated by the voice recognition server through a transcription text receiving link; wherein the audio sending thread and the transliteration text receiving link of the client side are mutually independent, and transmission is carried out through WebSocket over TLS. Therefore, the conference summary is generated by real-time transfer of the conference voice.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of speech transcription and summary generation, in particular to a speech transcription and summary generation method based on deep integration of a secure conference system. BACKGROUND

[0002] With the advancement of digital transformation, speech transcription technology has become a core infrastructure in medical, judicial, and government-enterprise conference scenarios, which can convert conference speech into text in real time, solving the problems of low efficiency, easy omission, and time-consuming in traditional manual recording, especially in confidential meeting scenarios. Because of the prohibition of carrying external storage devices, the traditional method of handwriting or audio recording pen after the meeting and manual transcription highlights the disadvantages of poor information integrity, complicated process, and high risk of leakage, and there is an urgent need for a safe and efficient speech transcription technical solution.

[0003] The current integration scheme of speech recognition and conference system still has many technical defects, which is difficult to meet the needs of secure conferences: the operation flexibility is insufficient, and only a single control terminal (web page or client) is supported to execute transcription start and stop, which cannot adapt to multi-role collaborative management, and the operation convenience is poor, which is easy to cause misoperation or delay. Data security risks are prominent, long-term storage of conference audio and transcription text lacks an automatic destruction mechanism after the meeting; some use wireless transmission, and data is easy to be intercepted; hardware modification destroys the original security architecture, increasing the risk of leakage. Strong hardware dependence and poor compatibility require additional deployment of high-performance hardware such as GPU, increasing procurement and maintenance costs, and there is compatibility risk; modifying the connection of existing microphones, mixing consoles, etc. with the original conference system has poor adaptability, affecting stability. Real-time and accuracy are difficult to balance, offline operation is required in confidential scenarios, and the existing offline model lacks real-time performance, which cannot realize synchronous output of speech and text; single recognition mode is prone to homonym errors and sentence segmentation deviations, affecting information accuracy. Long-time conference processing capacity is weak, and there is limited support for long-time conferences exceeding 4 hours, which is easy to cause storage pressure, transcription interruption, and inefficient query; conference summaries still rely on manual processing, which is time-consuming and labor-intensive, and it is difficult to quickly extract core information.

[0004] In summary, how to realize the lightweight integration of speech recognition and conference system without adding hardware and destroying the security architecture, while balancing transcription real-time performance, accuracy, and data security, and supporting long-time conference processing and intelligent summary generation, has become a technical problem to be solved in the field of secure conference speech transcription. SUMMARY

[0005] In view of the above analysis, the embodiments of the present application aim to provide a speech transcription and summary generation method based on deep integration of a secure conference system, to solve the problems of low efficiency, information omission, and security risks in traditional speech transcription and summary generation technology.

[0006] In one aspect, the embodiment of the present application provides a voice transcription and minutes generation method based on a deep integration security conference system, comprising the following steps:

[0007] The voice recognition client collects audio and sends the collected audio to the voice recognition server through an audio sending thread;

[0008] The voice recognition server uses the quantized offline model paraformer-offline-zh to perform segmented recognition and secondary whole segment recognition on the audio, generates a transcription text, and generates a conference minutes based on the transcription text;

[0009] The voice recognition client receives the transcription text generated by the voice recognition server through a transcription text receiving link;

[0010] The audio sending thread and the transcription text receiving link of the client are independent of each other and are transmitted through WebSocket over TLS.

[0011] Based on the further improvement of the above method, the segmented recognition divides the whole audio stream into short segments, generates segmented transcription texts through the paraformer-offline-zh offline model, and stores them in a database; the secondary whole segment recognition is triggered when a preset number of short segment recognitions is completed or a stop signal is received, retrieves the corresponding whole segment or remaining audio, generates a whole segment transcription text through the paraformer-offline-zh offline model, deletes the corresponding segmented transcription text in the database, and stores the whole segment transcription text for data updating.

[0012] Based on the further improvement of the above method, the quantized offline model paraformer-offline-zh is based on an official pre-quantized ONNX model, and maps FP32 floating-point numbers to the limited integer interval of INT8 through the ONNX Runtime tool with a scaling factor and a zero point.

[0013] Based on the further improvement of the above method, the method further comprises: monitoring the WebSocket over TLS connection state in real time, and when explicit disconnection or connection is detected, using an "exponential backoff + random jitter" strategy to implement automatic disconnection and reconnection.

[0014] Based on the further improvement of the above method, the segmented recognition comprises: the voice recognition server segments the whole audio stream, divides each t seconds, generates k short segments, assigns a unique identification information to each short segment, the identification information includes a conference ID, a segment serial number and a time interval; calls the quantized offline model paraformer-offline-zh model for voice recognition, generates a corresponding segmented transcription text, and stores it in the database table corresponding to the conference in the order of the segment serial number.

[0015] Further improvement based on the above method, when the voice recognition server completes the preset number of short fragment recognition, the server calls the whole audio stream, uses the paraformer-offline-zh offline model to perform whole speech recognition on the whole audio stream, and generates a whole transcription text;

[0016] When the stop signal is received, the server summarizes all audio data that is not completed for whole recognition, splices the complete remaining audio stream in the receiving order; calls the paraformer-offline-zh offline model to perform whole recognition on the remaining audio stream, and generates a corresponding whole transcription text.

[0017] Further improvement based on the above method, the automatic disconnection and reconnection has the following specific steps: explicit disconnection is captured by listening to the websockets.ConnectionclosedError event of the WebSocket over TLS connection; the client sends a ping frame to the server every x seconds, and if the server pong response is not received for y consecutive times, it is determined that the connection is dead; if explicit disconnection or connection is detected, reconnection is triggered.

[0018] Further improvement based on the above method, the method further comprises: when the server stores the transcription text, a new data record is inserted in the database table every h time interval, and the segmented transcription text and the whole transcription text for h time interval from the current time are stored.

[0019] On the other hand, the embodiment of the application provides a voice transcription and minutes generation system based on a deep integration security conference system, which is characterized in that the system comprises a voice recognition client, a voice recognition server and a communication module.

[0020] The voice recognition client collects audio and sends the collected audio to the voice recognition server through an audio sending thread in the communication module;

[0021] The voice recognition server uses the quantized offline model paraformer-offline-zh to perform segmented recognition and secondary whole recognition on the audio, generates a transcription text, generates a conference minutes based on the transcription text, and sends the transcription text to the voice recognition client through a transcription text receiving link in the communication module;

[0022] The communication module comprises an audio sending thread and a transcription text receiving link, and the audio sending thread and the transcription text receiving link are independent of each other and are transmitted through WebSocket over TLS.

[0023] Based on further improvements to the above system, the system also includes a conference system. The conference system server extracts the stored segmented transcribed text and whole transcribed text from the database table in real time, splices them together according to the segment number, and pushes them to the front-end page to display the transcribed results in real time.

[0024] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0025] 1. Achieve lightweight integration of speech recognition with the conference system without adding new hardware. Only the client is deployed on the chairman's machine, reusing the original wired audio equipment and architecture. The speech recognition service runs independently and interacts through encrypted API, with no compatibility risks or additional costs.

[0026] 2. The dual-mode control channel of "web page + chairman's machine tray icon" allows only the chairman's machine to start, stop and transcribe, and the operation status is synchronized in real time with an error rate of ≤0.1%, balancing convenience and zero interference in the meeting;

[0027] 3. Through the offline model of "segmentation + secondary whole segment recognition" and quantization optimization, the average transcription latency is ≤1s and the accuracy is over 95%, taking into account both offline requirements and real-time accuracy in classified scenarios.

[0028] 4. Supports long meetings of 8 hours or more, automatically segments and stores 2-hour meetings to reduce database pressure, and integrates a large language model to polish and correct meeting minutes, reducing manual processing costs;

[0029] 5. Optimize the client disconnection reconnection and recording file generation mechanism. When disconnected, the ReconnectManager module enables fast reconnection within ≤2 seconds, and the data loss rate during short-term network fluctuations is <0.01%. The recording file generation is no longer dependent on the server and is generated locally immediately when recording is terminated, improving system stability and data reliability and ensuring uninterrupted transcription of meetings.

[0030] In this invention, the above-described technical solutions can be combined with each other to achieve more preferred combinations. Other features and advantages of this invention will be set forth in the following description, and some advantages may become apparent from the description or be learned by practicing the invention. The objects and other advantages of this invention can be realized and obtained from what is particularly pointed out in the description and drawings. Attached Figure Description

[0031] The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Throughout the drawings, the same reference numerals denote the same parts.

[0032] Figure 1 This is a flowchart of the speech transcription and minutes generation method based on a deeply integrated secure conferencing system in Embodiment 1 of the present invention;

[0033] Figure 2 This is a schematic diagram of the speech transcription and minutes generation system based on a deeply integrated secure conferencing system in Embodiment 2 of the present invention. Detailed Implementation

[0034] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings, which form part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not intended to limit the scope of the present invention.

[0035] Example 1,

[0036] A specific embodiment of the present invention discloses a method for speech transcription and minutes generation based on a deeply integrated secure conferencing system, such as... Figure 1 As shown, it includes the following steps:

[0037] S1: The speech recognition client collects audio and sends the collected audio to the speech recognition server through the audio sending thread.

[0038] It should be noted that the client uses the stream_open series of APIs to call the associated microphone device to collect raw audio data. Then, the raw audio data is preprocessed and compressed into low-bandwidth OPUS format. A connection is established with the server via WebSocket over TLS, and the encoded OPUS format audio data is divided into small blocks and continuously and asynchronously sent to the server at a frequency of tens of data packets per second. The collected audio files are synchronously stored in a specific directory on the client's local machine, and the local audio files are irreversibly deleted after the meeting ends.

[0039] It should be noted that TLS encrypts the communication link end-to-end, ensuring that audio data is encrypted during transmission and effectively preventing man-in-the-middle attacks.

[0040] It should be noted that the server receives audio data packets sent by the client and puts the received data packets into a buffer. Each data packet corresponds to a short segment for segment recognition. When the data in the buffer reaches a certain length, or a stop signal is received, or the conference duration reaches the long-term storage time threshold h, it is assembled into a complete audio segment, corresponding to the entire recognized audio stream.

[0041] It should be noted that this invention reuses existing conference room equipment: microphones, mixing consoles, and the chairman's computer audio interface, without requiring any new hardware. The conference room microphones are connected to the mixing console via wired connections. The mixing console can control the volume of each microphone. The mixing console is connected to the chairman's computer audio interface. The chairman's computer identifies the audio interface through an audio driver and receives the audio stream. The voice recognition client is set up on the chairman's computer and acquires the audio stream via API.

[0042] It should be noted that the preprocessing of the original audio signal includes noise reduction and extraction of effective speech segments. Preferably, adaptive filtering and WebRTC VAD technology based on deep learning are used respectively. The specific implementation process is as follows:

[0043] Preferably, for time-varying environmental noises such as air conditioning noise, keyboard typing noise, and background conversations contained in the audio collected in the conference scene, an adaptive filter is constructed using the Normalized Least Mean Square (NLMS) algorithm to achieve accurate noise suppression. In the preprocessing stage, the original audio is first normalized to a mono PCM format with a sampling rate of 16kHz and a bit depth of 16bit. After eliminating the DC offset introduced by the hardware, it is framed according to a frame length of 20ms and a frame shift of 10ms, and Hanning window weighting is used to reduce inter-frame spectral leakage to ensure the stability of the filter. The filter is initialized with a 128th-order coefficient vector (initially all zeros) and a step size factor μ ranging from 0.01 to 0.05. An initial noise reference signal is collected during the 3-second silence period before the meeting, and the reference signal is updated in real time during transmission in conjunction with the silence segments detected by VAD to ensure matching with the current environmental noise characteristics. For each frame with noise, the noise reference signal is linearly filtered using the current filter coefficients to generate estimated noise. The error signal (i.e., preliminary denoised speech) is obtained by subtracting the estimated noise from the noisy frequency frame sample by sample. Then, based on the minimum mean square error criterion, the filter coefficients are dynamically updated in combination with the step size factor and reference signal energy normalization processing to achieve adaptive tracking and continuous suppression of time-varying noise, balancing noise reduction effect and real-time processing efficiency.

[0044] Preferably, the denoised speech output by the NLMS algorithm is filtered using WebRTC VAD technology based on deep learning. The detection threshold is dynamically adjusted by statistically analyzing the noise energy of the denoised silent frames in real time. When the noise energy is higher than the -40dBFS benchmark value, the threshold is increased to avoid false noise judgment. When it is lower than the benchmark value, the threshold is decreased to reduce the omission of short speech, thus adapting to the noise fluctuation characteristics of the conference scenario. Mel spectrogram features with 40 Mel filter channels are extracted from each frame of denoised speech. These features can effectively characterize the speech frequency distribution and suppress residual noise interference. They are then input into a pre-trained CNN+LSTM deep learning model, which outputs frame-level valid speech probability values ​​(0-1 interval). Frames with probabilities higher than a dynamic threshold are marked as "speech frames," and those without are marked as "non-speech frames." Subsequently, smoothing processing and a 200ms hang-over tail are used to avoid speech fragmentation caused by short silences. Isolated false detection segments with a duration of less than 300ms are also removed. Finally, continuous and high-purity valid speech segments are output as the pre-processed original audio signal, providing high-quality input for the subsequent speech recognition of the Paraformer-large model.

[0045] S2: The speech recognition server uses the quantized offline model paraformer-offline-zh to perform segmented recognition and secondary whole-segment recognition of the audio, generate transcribed text, and generate meeting minutes based on the transcribed text.

[0046] It should be noted that, considering the need to achieve real-time speech-to-text transcription in confidential meeting scenarios, online models cannot be used. Therefore, the offline model paraformer-offline-zh was chosen for speech recognition. This model is designed for offline scenarios, and its design philosophy is to first acquire the complete audio and then recognize it as a whole. It cannot achieve real-time output while recording and transcribing. Therefore, this invention adopts an architecture of segmented recognition and secondary whole-segment recognition, which achieves real-time speech-to-text transcription output with a response time of up to a second while also ensuring the accuracy of speech recognition.

[0047] Specifically, the segment recognition divides the entire audio stream into short segments, which are then recognized by the paraformer-offline-zh offline model to generate segmented transcribed text, which is then stored in the database. The secondary whole-segment recognition is triggered when a preset number of short segments have been recognized or when a stop signal is received. The corresponding whole segment or remaining audio is retrieved, recognized by the paraformer-offline-zh offline model to generate whole-segment transcribed text, the corresponding segmented transcribed text is deleted from the database, and the whole-segment transcribed text is then stored to update the data.

[0048] Specifically, the quantized offline model paraformer-offline-zh is based on the official pre-quantized ONNX model, and uses the ONNX Runtime tool to map FP32 floating-point numbers to the finite integer range of INT8 with scaling factors and zeros.

[0049] It should be noted that by running the original paraformer-offline-zh model and the quantized paraformer-offline-zh model on the same test set, and comparing the model size, memory usage, inference speed, and power consumption before and after quantization, the following results were obtained: the model size before quantization was 400MB, and after quantization it was 200MB, a reduction of 75%; the memory usage before quantization was high, and after quantization it was extremely low, a reduction of 70%-80%; the inference speed before quantization was 1x, and after quantization it was 1.5-2x, an improvement of 50%-100%; and the power consumption before quantization was the baseline, while after quantization it was significantly reduced.

[0050] Specifically, the segmentation recognition includes: the speech recognition server segments the entire audio stream into segments, dividing it into k short segments per t seconds, and assigning unique identification information to each short segment, which includes the meeting ID, segment number, and time interval; calling the quantized offline model paraformer-offline-zh model for speech recognition, generating corresponding segmented transcribed text, and storing it in the database table corresponding to the meeting in order of segment number.

[0051] Specifically, after the speech recognition server completes the recognition of a preset number of short segments, the server retrieves the entire audio stream and uses the paraformer-offline-zh offline model to perform full-segment speech recognition on the entire audio stream, generating the entire transcribed text.

[0052] When a stop signal is received, the server aggregates all audio data that has not yet been fully recognized and splices them together into a complete remaining audio stream in the order they were received. The server then calls the paraformer-offline-zh offline model to perform full-segment recognition on the remaining audio stream and generates the corresponding full-segment transcribed text.

[0053] Specifically, when the server stores the transcribed text, it inserts a new data record into the database table every h hours, storing the segmented transcribed text and the entire transcribed text for h hours from the current time.

[0054] It should be noted that a long-term storage time threshold h is preset on the speech recognition server. The server monitors the transcription duration of the current meeting in real time, starting from the moment the first audio stream is received. When the cumulative duration reaches h, the block storage mechanism is triggered for the first time. The server inserts a new data record into the database table to store the segmented transcribed text from h to 2h and the whole transcribed text. Thereafter, every h, a new data record is inserted into the database table to store the segmented transcribed text and the whole transcribed text for a duration of h from the current moment, forming a time-series transcribed text storage structure.

[0055] It should be noted that this invention employs a dual-mode control channel—both web-based and tray icon-based—to control the start and stop of voice recognition. Both the conference system's web interface and the chairman's client have voice recognition start / stop buttons on their tray icons, allowing for immediate start and stop at any time, with both methods having equal priority. Specifically, when the meeting begins, the conference system's web interface sends meeting information to the voice recognition client: meeting ID, meeting room organization ID, organizer, etc. After the voice recognition client starts recording, it notifies the conference system terminal to update the voice recognition status via a UDP message. At this point, the conference system terminal can send a message to the voice recognition client to stop recording. When the meeting ends, the conference system's web interface sends a clear message to the recognition client. The start and end of recording / recognition are synchronized, ensuring that the conference system's web interface and client remain synchronized.

[0056] S3: The speech recognition client receives the transcribed text generated by the speech recognition server through the transcribed text receiving link;

[0057] The audio sending thread and the transcribed text receiving link on the client side are independent of each other and are transmitted via WebSocket over TLS.

[0058] Specifically, the method also includes: real-time monitoring of the WebSocket over TLS connection status, and when an explicit disconnection or connection freeze is detected, an "exponential backoff + random jitter" strategy is adopted to automatically disconnect and reconnect.

[0059] Specifically, the automatic disconnection and reconnection process involves the following steps: capturing explicit disconnections by listening to the websockets.ConmectionclosedError event of the WebSocket over TLS connection; the client sends ping frames to the server every x seconds, and if no ping response is received from the server y times consecutively, the connection is considered to be dead;

[0060] If a visible disconnection or connection freeze is detected, a reconnection will be triggered.

[0061] It should be noted that before reconnecting, the local network availability is checked, the initial WSS TLS configuration is reused, and the validity of the server certificate is verified. If the certificate is invalid, the reconnection is terminated and the user is notified.

[0062] The strategy employs "exponential backoff + random jitter," with an initial reconnection interval of z seconds. After each failed retry, the interval doubles and increases by ±a% random value. The maximum number of retries is b in intranet scenarios and c in weak public network scenarios, with b... <c;

[0063] During the reconnection process, the audio sending thread synchronously writes the encoded OPUS audio frames to the local recording file and the memory cache queue. After the reconnection is successful, it sends a request to the server for the reception progress with the sequence number of the last successfully sent audio frame. The cache queue data is then resent in order according to the sequence number, and the server uses the sequence number to remove duplicates.

[0064] When reconnecting, the client carries the meeting ID and session token. After the server verifies them, it restores the transcription context. If the reconnection is successful, real-time audio transmission and result reception will resume. If it fails, the retry will stop and the user will be notified. Local recording will not be affected.

[0065] It should be noted that, to ensure meeting security, audio data and transcribed text generated during the meeting must be encrypted during transmission and storage.

[0066] Preferably, after the audio data is processed by the speech recognition client, an application-layer encryption operation is performed before writing it to the disk. A 32-byte independent key and a 16-byte initialization vector are generated by a cryptographically secure pseudo-random number generator. The former meets the key length specification of the AES-256 algorithm, and the latter adapts to the 128-bit block size requirement of CBC mode. The original audio data is first padded with PKCS#7 to solve the problem of the length not being a multiple of the block size. Then, AES-256-CBC encryption is performed with the generated key and initialization vector as parameters. The encrypted data is Base64 encoded to form structured data containing the initialization vector, encoded ciphertext, and key identifier. The independent key is not stored in the same database as the ciphertext, but is stored in a dedicated key management system equipped with a hardware security module or a cryptographic security container. A mapping relationship between ciphertext and key is established through a unique key identifier. The system supports automatic key rotation, off-site backup, and operation auditing to ensure key security.

[0067] Preferably, the structured data is transmitted to the server via the WebSocket Secure protocol. Before transmission, the client and server complete a handshake process based on the TLS 1.3 protocol, verify the legality of the server certificate, negotiate the session encryption key, and establish an encrypted connection. After that, the structured data is encapsulated in the WebSocket frame format. The frame payload is encrypted with the session key and a frame masking mechanism and SHA-256 integrity verification are enabled to effectively prevent eavesdropping, tampering, and replay attacks. During the storage phase, a role-based access control model is adopted to implement the principle of least privilege. Fine-grained roles such as administrators, authorized participants, and auditors are defined, and the boundaries of permissions are clearly defined. Access operations require dual verification of multi-factor authentication and short-term session tokens. All operations are recorded in a complete audit log containing the subject's identity, timestamp, and data identifier to support full traceability.

[0068] Preferably, the transcribed text is also encrypted, transmitted, and stored using the above-described method. When the transcribed text is displayed on the front-end page of the conference system, the same encryption method and parameters are used for decryption before the recognized content is displayed on the page. Furthermore, the front-end uses dynamic rendering and DOM protection technology to prohibit copying and screenshotting, ensuring the security of sensitive information during the display process.

[0069] Preferably, data lifecycle management is triggered after the meeting. A configurable buffer period of 1 to 7 days is initiated after the meeting for emergency traceability. After the buffer period, the system automatically executes the destruction process. First, the corresponding independent key is permanently deleted through the key management system. Then, the encrypted data stored on the server is overwritten and deleted according to the DoD 5220.22-M standard. At the same time, sensitive data identifiers in the audit log are de-identified, and only the operation traces required for compliance are retained. If early deletion or extension of the storage period is required, multi-level approval is required and the operation reason is recorded to ensure that the data is secure and controllable throughout its entire lifecycle.

[0070] Preferably, transcribed text, audio files, and meeting minutes are classified according to the meeting's security level by default, and the security level is displayed in the file name to ensure that the security level is controllable.

[0071] Example 2

[0072] Another specific embodiment of the present invention discloses a speech transcription and minutes generation system based on a deeply integrated secure conferencing system, such as... Figure 2 This achieves the speech transcription and minutes generation method based on a deeply integrated secure conferencing system as described in Example 1. The specific implementation of each module is described in the corresponding section of Example 1. The system includes a speech recognition client, a speech recognition server, and a communication module.

[0073] The speech recognition client collects audio and sends the collected audio to the speech recognition server through the audio sending thread in the communication module.

[0074] The speech recognition server uses the quantized offline model paraformer-offline-zh to perform segment recognition and secondary whole-segment recognition on the audio, generates transcribed text, generates meeting minutes based on the transcribed text, and sends the transcribed text to the speech recognition client through the transcribed text receiving link in the communication module.

[0075] The communication module includes an audio sending thread and a transcribed text receiving link, which are independent of each other and are transmitted via WebSocket over TLS.

[0076] Specifically, the system also includes a conferencing system. The conferencing system server extracts the stored segmented and whole-segment transcribed text from the database table in real time, splices them together according to the segment number, and pushes them to the front-end page to display the transcription results in real time.

[0077] For example, the system can also connect to large language models such as Tongyi Qianwen to perform contextual semantic verification, complete professional terms, and modify and add punctuation by integrating prosodic features to correct errors and polish the transcribed text, thereby optimizing the meeting minutes.

[0078] Those skilled in the art will understand that all or part of the processes implementing the methods of the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0079] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for speech transcription and minutes generation based on a deeply integrated secure conferencing system, characterized in that, Includes the following steps: The speech recognition client collects audio and sends the collected audio to the speech recognition server through the audio sending thread; The speech recognition server uses the quantized offline model paraformer-offline-zh to perform segmented recognition and secondary whole-segment recognition of the audio, generate transcribed text, and generate meeting minutes based on the transcribed text; The speech recognition client receives the transcribed text generated by the speech recognition server through the transcribed text receiving link. The audio sending thread and the transcribed text receiving link on the client side are independent of each other and are transmitted via WebSocket over TLS.

2. The speech transcription and minutes generation method based on a deeply integrated secure conferencing system according to claim 1, characterized in that, The segment recognition divides the entire audio stream into short segments, which are then recognized by the paraformer-offline-zh offline model to generate segmented transcribed text, which is stored in the database. The secondary whole segment recognition is triggered when a preset number of short segments are recognized or a stop signal is received. The corresponding whole segment or remaining audio is retrieved, recognized by the paraformer-offline-zh offline model to generate whole segment transcribed text, the corresponding segmented transcribed text is deleted from the database, and the whole segment transcribed text is stored to update the data.

3. The speech transcription and minutes generation method based on a deeply integrated secure conferencing system according to claim 1, characterized in that, The quantized offline model paraformer-offline-zh is based on the official pre-quantized ONNX model. It uses the ONNX Runtime tool to map FP32 floating-point numbers to the finite integer range of INT8 with scaling factors and zeros.

4. The speech transcription and minutes generation method based on a deeply integrated secure conferencing system according to claim 1, characterized in that, The method also includes: real-time monitoring of the WebSocket over TLS connection status, and when an explicit disconnection or connection deadlock is detected, an "exponential backoff + random jitter" strategy is adopted to automatically disconnect and reconnect.

5. The speech transcription and minutes generation method based on a deeply integrated secure conferencing system according to claim 2, characterized in that, The segmented recognition includes: the speech recognition server segments the entire audio stream into segments, dividing it into k short segments per t seconds. Each short segment is assigned a unique identifier, which includes the meeting ID, segment number, and time interval. The quantized offline model paraformer-offline-zh is then called to perform speech recognition, generating corresponding segmented transcribed text, which is stored in the database table corresponding to the meeting in order of segment number.

6. The speech transcription and minutes generation method based on a deeply integrated secure conferencing system according to claim 1, characterized in that, After the speech recognition server completes the recognition of a preset number of short segments, the server retrieves the entire audio stream and uses the paraformer-offline-zh offline model to perform full-segment speech recognition on the entire audio stream, generating the entire transcribed text. When a stop signal is received, the server aggregates all audio data that has not yet been fully recognized and splices them together into a complete remaining audio stream in the order they were received. The server then calls the paraformer-offline-zh offline model to perform full-segment recognition on the remaining audio stream and generates the corresponding full-segment transcribed text.

7. The speech transcription and minutes generation method based on a deeply integrated secure conferencing system according to claim 1, characterized in that, The automatic disconnection and reconnection process involves the following steps: capturing explicit disconnections by listening to the websockets.ConmectionclosedError event of the WebSocket over TLS connection; the client sends ping frames to the server every x seconds, and if no ping response is received from the server y times consecutively, the connection is considered to be dead; If a visible disconnection or connection freeze is detected, a reconnection will be triggered.

8. The speech transcription and minutes generation method based on a deeply integrated secure conferencing system according to claim 1, characterized in that, The method further includes: when the server stores the transcribed text, it inserts a new data record into the database table every h time intervals, storing the segmented transcribed text and the whole transcribed text for h time intervals from the current time.

9. A speech transcription and minutes generation system based on a deeply integrated secure conferencing system, characterized in that, The system includes a speech recognition client, a speech recognition server, and a communication module. The speech recognition client collects audio and sends the collected audio to the speech recognition server through the audio sending thread in the communication module. The speech recognition server uses the quantized offline model paraformer-offline-zh to perform segment recognition and secondary whole-segment recognition on the audio, generates transcribed text, generates meeting minutes based on the transcribed text, and sends the transcribed text to the speech recognition client through the transcribed text receiving link in the communication module. The communication module includes an audio sending thread and a transcribed text receiving link, which are independent of each other and are transmitted via WebSocket over TLS.

10. The speech transcription and minutes generation system based on a deeply integrated secure conferencing system according to claim 9, characterized in that, The system also includes a conferencing system. The conferencing system server extracts the stored segmented and whole-segment transcribed text from the database table in real time, splices them together according to the segment number, and pushes them to the front-end page to display the transcription results in real time.

Images