Multi-language intelligent conference system based on wireless AI architecture and information processing method thereof

Abstract

The invention relates to the technical field of intelligent conference systems, in particular to a multi-language intelligent conference system based on wireless AI architecture and an information processing method thereof.The multi-language intelligent conference system comprises a plurality of PAD clients and a miniature AI server, and the PAD clients and the AI server form private wireless local area network connection through server built-in Wi-Fi hotspots; the PAD client comprises a voice acquisition module, a display module and a communication module; the AI server comprises a voice recognition module, a machine translation module, a conference summary generation module and a communication module. According to the invention, rapid deployment of the conference system is realized through a wireless networking mode, multi-language real-time translation and automatic generation of the conference summary are realized by using the AI technology, and the conference efficiency is improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent conference system technology, specifically to a multilingual intelligent conference system based on a wireless AI architecture and its information processing method. Background Technology

[0002] With the acceleration of globalization, the demand for cross-language conferences is increasing. Traditional conference systems have the following problems:

[0003] 1. Requires a complex cabling system, making deployment inconvenient;

[0004] 2. The lack of real-time translation functionality affects meeting efficiency;

[0005] 3. Meeting minutes require manual recording, which can easily lead to the omission of important information;

[0006] The system has poor scalability and is difficult to adapt to the needs of meetings of different sizes. Summary of the Invention

[0007] The purpose of this invention is to provide a multilingual intelligent conference system based on a wireless AI architecture and its information processing method, enabling rapid deployment of the conference system, real-time multilingual translation, and automatic generation of meeting minutes.

[0008] The embodiments of this application adopt the following technical solutions:

[0009] The first aspect of this invention proposes a multilingual intelligent conferencing system based on a wireless AI architecture, comprising:

[0010] At least multiple PAD clients and one miniature AI server are connected via a private wireless local area network formed by the server's built-in Wi-Fi hotspot to achieve encrypted data transmission.

[0011] The PAD client is used for collecting voice, displaying information, and communication.

[0012] The AI ​​server is used to recognize, translate, and generate meeting minutes from the collected voice information.

[0013] As one technical solution of the present invention, the PAD client includes:

[0014] The voice acquisition module collects user voice information and adds a unique user ID tag;

[0015] The display module is used to display the source language text, the target language translation, and meeting minutes;

[0016] The first communication module is used to establish a connection with the AI ​​server.

[0017] As one technical solution of the present invention, the AI ​​server includes:

[0018] The speech recognition module is used to convert the user's voice audio stream into text information;

[0019] A machine translation module is used to translate the text information into the target language;

[0020] The meeting minutes generation module is used to extract and semantically aggregate information to generate structured meeting minutes.

[0021] The second communication module is used to create private Wi-Fi hotspots and manage client connections.

[0022] As one technical solution of the present invention, the system further includes the construction of a speech recognition model:

[0023] Capture local features of user speech, combine them with context to form a cascaded structure, and introduce user ID embedding vectors;

[0024] The base model is trained using a standard speech dataset, enhanced by injecting simulated conference noise, and its anti-interference ability is improved by using an adversarial loss function and a block attention mechanism is used for delay processing.

[0025] During the speech-to-text decoding process, a conference domain terminology library is preloaded to add extra weight to the candidate paths for term matching, and the last n sentences of historical text are cached to enhance the probability of repeated terms.

[0026] A second aspect of this invention proposes an information processing method for a multilingual intelligent conferencing system based on a wireless AI architecture, comprising the following steps:

[0027] Step S101: The PAD client collects voice data with user ID tags and sends it to the AI ​​server via a private wireless LAN.

[0028] Step S201: The speech recognition module of the AI ​​server converts the speech data into text information and associates it with the user ID timestamp;

[0029] Step S301: The machine translation module translates the text information into a preset target language and generates a translation of the comparison data.

[0030] Step S401: Distribute the translated text to the corresponding PAD client for display according to user ID;

[0031] Step S501: The meeting minutes generation module analyzes the speech text in real time, extracts resolutions / tasks, generates minutes, and distributes them.

[0032] As a technical solution of the present invention, in step S101, the user ID binding includes:

[0033] When a user logs into the PAD client for the first time, a preset text is read aloud, the voice signal feature vector is extracted to generate a voiceprint template and stored;

[0034] Collect and extract the feature vector of the user's current speech signal;

[0035] Calculate the cosine similarity between the current feature vector and the user's pre-registered voiceprint template;

[0036] If the similarity is greater than or equal to the preset threshold, the user ID will be automatically bound; otherwise, a second identity verification will be triggered to generate tagged voice data encapsulation.

[0037] As one technical solution of the present invention, the generation of tagged voice data encapsulation includes:

[0038] Construct a JSON data packet;

[0039] An additional checksum is performed on the complete data packet and appended to the protocol header to ensure data integrity and traceability; among which,

[0040] The voice data is encapsulated as a JSON format data structure containing an audio stream, user ID, and timestamp, and transmitted via UDP protocol with a CRC32 checksum appended to the UDP packet header.

[0041] As a technical solution of the present invention, in step S201, the speech recognition module of the AI ​​server converts the speech data into text information and associates it with the user ID timestamp, specifically including:

[0042] Receive encrypted data packets sent by the PAD client, verify them, extract the audio stream, and decode it;

[0043] Extract audio stream features and add timestamp location markers;

[0044] Load the pre-associated user embedding vector based on the user ID in the data packet;

[0045] Dynamically adjust the decoder weights and output a text sequence;

[0046] The recognized text is bound to the user ID and timestamp in the original data packet;

[0047] The output text, along with the user ID and timestamp, is encapsulated in a JSON structure, including the start and end times of the speech. As a technical solution of this invention, in step S301, the machine translation module translates the text information according to a preset target language, generating a comparative translation, specifically including:

[0048] It receives structured data from speech recognition output, retrieves the speaker's historical translations as a context cache, and concatenates them into the current text input;

[0049] Load the conference domain terminology database, add weights to the candidate paths matched by the terminology database in the neural machine translation model, and generate word-level correspondences between the source language and the target language through the model attention alignment matrix;

[0050] Users can set output modes, including bilingual comparison and / or direct output of the target text.

[0051] The terminology database contains professional vocabulary and abbreviation mappings for the conference field. The output modes include plain translation format and bilingual comparison format, and the bilingual comparison format includes word position alignment index.

[0052] As a technical solution of the present invention, in step S501, the meeting minutes generation module analyzes the spoken text in real time, extracts resolutions / task items to generate minutes and distributes them, specifically including:

[0053] Aggregate multiple users' speech texts by timestamp;

[0054] Entity recognition and intent classification are performed simultaneously using a joint model;

[0055] When the intent is a resolution or task, a predefined template is populated to generate structured minutes and distributed.

[0056] The above-described technical solutions adopted in the embodiments of this application can achieve the following beneficial effects:

[0057] 1. No wiring required, quick deployment; 2. Supports real-time translation of multiple languages; 3. Automatically generates meeting minutes;

[0058] 4. The system has good scalability; 5. It is easy to use and has low maintenance costs.

[0059] The specific beneficial technical effects are described in detail in the following specific embodiments. Attached Figure Description

[0060] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0061] Figure 1 Here is a schematic diagram of the system structure of the present invention;

[0062] Figure 2 Here is a schematic diagram of the method flow of the present invention;

[0063] Figure 3 Here is a schematic diagram of the user ID binding process in the method of this invention;

[0064] Figure 4 Here is a schematic diagram of the tagged voice data encapsulation process in the method of this invention;

[0065] Figure 5 Here is a schematic diagram of the specific implementation process of step S201 in one embodiment of the method of the present invention;

[0066] Figure 6 Here is a schematic diagram of the specific implementation process of step S301 in one embodiment of the method of the present invention;

[0067] Figure 7 Here is a flowchart of the user ID binding and data transmission process in the method of this invention. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. Conformer is a model that combines a convolutional neural network (CNN) and a transformer.

[0069] refer to Figure 1 A multilingual intelligent conferencing system based on a wireless AI architecture includes: at least multiple PAD clients and a micro AI server. The PAD clients and the AI ​​server are connected via a private wireless local area network formed by the server's built-in Wi-Fi hotspot to achieve encrypted data transmission. The PAD clients are used to collect voice, display information, and communicate. The AI ​​server is used to recognize, translate, and generate meeting minutes from the collected voice information.

[0070] The PAD client includes a voice acquisition module for collecting user voice information and adding a unique user ID tag; a display module for displaying source language text, target language translation, and meeting minutes; and a first communication module for establishing a connection with the AI ​​server.

[0071] The AI ​​server includes a speech recognition module for converting user speech audio streams into text information; a machine translation module for translating the text information into a target language; a meeting minutes generation module for extracting and semantically aggregating data to generate structured meeting minutes; and a second communication module for creating private Wi-Fi hotspots and managing client connections.

[0072] The system also includes the construction of a speech recognition model: capturing local features of user speech, combining them with context to form a cascaded structure, and introducing a user ID embedding vector; training the base model using a standard speech dataset, injecting simulated conference noise for enhanced training, utilizing an adversarial loss function to improve the model's anti-interference ability, and employing a block attention mechanism for delay processing; during the speech-to-text decoding process, preloading a conference domain terminology database, adding extra weight to candidate paths for term matching, and caching the last n sentences of historical text to achieve probability enhancement for repeated terms. Specifically:

[0073] First, a hybrid neural network encoder is constructed, using a Conformer layer to capture local speech features (such as connected speech and plosives), followed by a Transformer layer to model long-distance contextual relationships, forming a cascaded structure. The output layer incorporates a user ID embedding vector to dynamically adjust decoding weights, achieving accent adaptation. Next, incremental adversarial training is performed. The base model is first trained using a standard speech dataset, then simulated conference noise (e.g., a mixture of keyboard sounds, air conditioner noise, and coughing sounds) is injected for incremental training. An adversarial loss function enhances the model's robustness against interference, and a block-based attention mechanism is used for latency control, for example, keeping the latency within 300ms. Finally, dynamic terminology injection decoding is performed, preloading a conference domain terminology library (e.g., "KPI," "ROI"). During speech-to-text decoding, additional weights are added to candidate paths for terminology matching (based on the candidate paths), while caching the last 10 historical text sentences to enhance the probability of repeated terms.

[0074] As an example, when a user says "Q3 financial report ROI met target, needs to be reviewed", the model responds as follows:

[0075] The Conformer layer is used to capture the continuous reading characteristics of "financial report" and adapt it to the Hong Kong pronunciation by user ID; ignoring background keyboard sounds, block processing ensures a response within 200ms, and the term "ROI" is weighted to avoid mistranslation as "input-output ratio".

[0076] A second aspect of this invention proposes an information processing method for a multilingual intelligent conferencing system based on a wireless AI architecture, comprising the following steps:

[0077] refer to Figure 6 The S101 PAD client collects voice data tagged with user IDs and sends it to the AI ​​server via a private wireless LAN.

[0078] Specifically, the user ID binding process includes: when a user logs into the PAD client for the first time, reading a preset text aloud, extracting the speech signal feature vector to generate a voiceprint template and storing it; collecting and extracting the user's current speech signal feature vector; calculating the cosine similarity between the current feature vector and the user's pre-registered voiceprint template; if the similarity is greater than or equal to a preset threshold, the user ID is automatically bound; otherwise, a secondary identity verification is triggered, generating tagged speech data encapsulation. The specific steps are as follows:

[0079] a. User identity initialization, its implementation method:

[0080] Voiceprint pre-registration involves the user reading a preset text upon first login to the PAD, extracting the Mel-frequency cepstral coefficients (MFCC) feature vector to generate and store a voiceprint template; manual identification input allows users to select identity labels (e.g., "Speaker A" or "Translator B") via a graphical interface. This addresses the identity confusion issue in multi-user scenarios and provides a unique identifier for voice data.

[0081] b. Real-time binding of voice capture with identity, the implementation method of which is as follows:

[0082] When voice acquisition is initiated, the MFCC feature vector of the current audio is extracted; the cosine similarity score (Score) between this vector and the pre-stored voiceprint template is calculated; if Score ≥ threshold θ (θ = 0.85), the corresponding user ID is automatically bound; if Score < θ, a manual confirmation interface is triggered to force the user to select an identity label. This achieves real-time dynamic association between the voice stream and the user ID, improving accuracy.

[0083] c. Encapsulation of tagged voice data, and its implementation method:

[0084] Construct a JSON-formatted data packet; add a data verification layer, specifically by calculating a CRC32 checksum for the complete data packet and appending it to the UDP header. This ensures data integrity and traceability, and reduces the transmission error rate.

[0085] Therefore, the voice data is encapsulated into a JSON format data structure containing the audio stream, user ID, and timestamp, and transmitted via the UDP protocol with a CRC32 checksum appended to the UDP packet header.

[0086] S201, the AI ​​server's speech recognition module converts the speech data into text information and associates it with a user ID and timestamp. Specifically, it receives encrypted data packets sent by the PAD client, extracts and decodes the audio stream after verification; extracts audio stream features and adds timestamp position identifiers; loads pre-associated user embedding vectors based on the user ID in the data packet; dynamically adjusts the decoder weights and outputs a text sequence; and binds the recognized text with the user ID and timestamp in the original data packet. The output text, user ID, and timestamp are encapsulated in a JSON structure, including the start and end timestamps of the speech. In one embodiment, this step is implemented as follows:

[0087] Audio preprocessing involves receiving encrypted data packets sent by the PAD client, verifying them using CRC32, and extracting the Base64-encoded PCM audio stream. Audio decoding is performed using a 24-bit depth / 48kHz sampling rate, and high-frequency attenuation is compensated using a pre-emphasis filter.

[0088] Sound feature extraction: 80-dimensional Mel spectrum is calculated and time frame position markers are added: 20ms per frame, 10ms step size, to generate a feature matrix with time markers.

[0089] Hybrid neural network inference,

[0090] Input the feature matrix into the Conformer layer to capture acoustic local dependencies; input the Conformer output into the Transformer layer to model long-range context.

[0091] User-adaptive decoding loads a pre-associated user embedding vector based on the user ID in the data packet and dynamically adjusts the decoder weights to output a text sequence;

[0092] The data is associated and encapsulated into a structured output, binding the recognized text with the user ID and timestamp in the original data packet to generate a JSON structure.

[0093] In the above embodiments, the decoding weight is dynamically adjusted by embedding the user ID to reduce the recognition error rate caused by accent differences; the Conformer-Transformer hybrid architecture is used to solve the problem of inaccurate recognition of conference terms; time frame position identification is used to solve the problem of disordered speaking order among multiple users; and JSON structured binding is used to prevent the loss of identity and text association.

[0094] S301, The machine translation module translates the text information according to the preset target language to generate a translated version of the comparison data. It receives the structured data output by speech recognition, retrieves the historical translation of the speaker as context cache and splices it to the current text input; loads the conference domain term library, increases the weight of the candidate paths matched by the term library in the neural machine translation model, and generates the word-level comparison relationship between the source language and the target language through the model attention alignment matrix; the user sets the output mode, including bilingual comparison and / or directly outputting the target text. Among them, the term library contains professional vocabulary and abbreviation mappings in the conference domain, and the output modes include pure translation format and bilingual comparison format, and the bilingual comparison format contains word position alignment indexes.

[0095] Introduce and implement a caching mechanism for context-related translation: receive the structured data (including text, user ID, timestamp) output by speech recognition, retrieve the last N historical translations (N = 10) of this user as context cache to solve the problem of ambiguous pronoun reference in conference spoken language (such as "this solution" referring to the previous resolution);

[0096] Domain adaptive translation, load the conference domain term library, for example: preset mapping tables such as "KPI / Key Performance Indicator", "ROI / Return on Investment", etc. In the Beam Search of the neural machine translation model (Transformer architecture), increase the weight of the candidate paths matched by the terms. Ensure the consistency of professional term translation, for example, "tape-out" is always translated as "tape-out" instead of "tape output".

[0097] Bilingual comparison generation, alignment matrix parsing, through the attention alignment matrix of the translation model, generate the word-level alignment relationship between the source language and the target language, and output any of the following modes according to the user's setting. For example, Mode A, bilingual comparison. Mode B, pure translation, directly output the target text.

[0098] S401, Distribute the translated text to the corresponding PAD client for display according to the user ID.

[0099] S501, The meeting minutes generation module analyzes the spoken text in real time, extracts resolutions / task items to generate meeting minutes and distributes them. Aggregate the spoken text in real time, extract resolutions / task items through entity recognition and intention analysis, and generate structured meeting minutes according to the timeline, specifically including: aggregating the spoken text of multiple users according to the timestamp; synchronously performing entity recognition and intention classification through a joint model; when the intention is a resolution or a task, fill in a predefined template to generate a structured meeting minutes item and distribute it. The specific implementation is as follows:

[0100] The system outputs timestamps based on speech recognition, concatenates the text of multiple users' speeches in chronological order, and generates a time-series text stream. It then performs joint analysis combining entity and intent, extracting person / time / task entities and classifying intents, with sentences categorized as resolutions / tasks / discussions. Resolution items are generated in a structured manner (template filling), and when intent = resolution / task, a summary item is generated according to rules. For example, the input is: "Mr. Zhang proposes to postpone the test to next week" (intent = resolution, entity = [Mr. Zhang, test, next week]), resulting in the output: Resolution: The test is postponed to next week. Responsible person: Mr. Zhang. In the above embodiment, optionally, entity recognition uses the BERT-CRF model, and intent classification is based on BERT's CLS vectors. It should be further noted that joint analysis interpretation, for example, avoids misidentifying "within 7 days" as a time entity instead of a deadline.

[0101] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A multilingual intelligent conferencing system based on a wireless AI architecture, characterized in that: include: At least multiple PAD clients and one miniature AI server are connected via a private wireless local area network formed by the server's built-in Wi-Fi hotspot to achieve encrypted data transmission. The PAD client is used for collecting voice, displaying information, and communication. The AI ​​server is used to recognize, translate, and generate meeting minutes from the collected voice information.

2. The multilingual intelligent conferencing system based on a wireless AI architecture according to claim 1, characterized in that, The PAD client includes: The voice acquisition module collects user voice information and adds a unique user ID tag; The display module is used to display the source language text, the target language translation, and meeting minutes; The first communication module is used to establish a connection with the AI ​​server.

3. The multilingual intelligent conferencing system based on a wireless AI architecture according to claim 1, characterized in that, The AI ​​server includes: The speech recognition module is used to convert the user's voice audio stream into text information; A machine translation module is used to translate the text information into the target language; The meeting minutes generation module is used to extract and semantically aggregate information to generate structured meeting minutes. The second communication module is used to create private Wi-Fi hotspots and manage client connections.

4. A multilingual intelligent conferencing system based on a wireless AI architecture according to claim 3, characterized in that, It also includes the construction of speech recognition models: Capture local features of user speech, combine them with context to form a cascaded structure, and introduce user ID embedding vectors; The base model is trained using a standard speech dataset, enhanced by injecting simulated conference noise, and its anti-interference ability is improved by using an adversarial loss function and a block attention mechanism is used for delay processing. During the speech-to-text decoding process, a conference domain terminology library is preloaded to add extra weight to the candidate paths for term matching, and the last n sentences of historical text are cached to enhance the probability of repeated terms.

5. The information processing method of a multilingual intelligent conferencing system based on a wireless AI architecture according to any one of claims 1-4, characterized in that, Includes the following steps: Step S101: The PAD client collects voice data with user ID tags and sends it to the AI ​​server via a private wireless LAN. Step S201: The speech recognition module of the AI ​​server converts the speech data into text information and associates it with the user ID timestamp; Step S301: The machine translation module translates the text information into a preset target language and generates a translation of the comparison data. Step S401: Distribute the translated text to the corresponding PAD client for display according to user ID; Step S501: The meeting minutes generation module analyzes the speech text in real time, extracts resolutions / tasks, generates minutes, and distributes them.

6. The multilingual intelligent conference information processing method based on a wireless AI architecture according to claim 5, characterized in that, In step S101, the user ID binding includes: When a user logs into the PAD client for the first time, a preset text is read aloud, the voice signal feature vector is extracted to generate a voiceprint template and stored; Collect and extract the feature vector of the user's current speech signal; Calculate the cosine similarity between the current feature vector and the user's pre-registered voiceprint template; If the similarity is greater than or equal to the preset threshold, the user ID will be automatically bound; otherwise, a second identity verification will be triggered to generate tagged voice data encapsulation.

7. The multilingual intelligent conference information processing method based on a wireless AI architecture according to claim 6, characterized in that, The generation of labeled speech data encapsulation includes: Construct a JSON data packet; An additional checksum is performed on the complete data packet and appended to the protocol header to ensure data integrity and traceability; among which, The voice data is encapsulated as a JSON format data structure containing an audio stream, user ID, and timestamp, and transmitted via UDP protocol with a CRC32 checksum appended to the UDP packet header.

8. A multilingual intelligent conference information processing method based on a wireless AI architecture according to claim 7, characterized in that, In step S201, the AI ​​server's speech recognition module converts the speech data into text information and associates it with a user ID timestamp, specifically including: Receive encrypted data packets sent by the PAD client, verify them, extract the audio stream, and decode it; Extract audio stream features and add timestamp location markers; Load the pre-associated user embedding vector based on the user ID in the data packet; Dynamically adjust the decoder weights and output a text sequence; The recognized text is bound to the user ID and timestamp in the original data packet; The output text, along with the user ID and timestamp, is encapsulated in a JSON structure, including the start and end timestamps of the voice recording.

9. A multilingual intelligent conference information processing method based on a wireless AI architecture according to claim 8, characterized in that, In step S301, the machine translation module translates the text information according to a preset target language and generates a comparative translation, specifically including: It receives structured data from speech recognition output, retrieves the speaker's historical translations as a context cache, and concatenates them into the current text input; Load the conference domain terminology database, add weights to the candidate paths matched by the terminology database in the neural machine translation model, and generate word-level correspondences between the source language and the target language through the model attention alignment matrix; Users can set output modes, including bilingual comparison and / or direct output of the target text. The terminology database contains professional vocabulary and abbreviation mappings for the conference field. The output modes include plain translation format and bilingual comparison format, and the bilingual comparison format includes word position alignment index.

10. A multilingual intelligent conference information processing method based on a wireless AI architecture according to claim 5, characterized in that, In step S501, the meeting minutes generation module analyzes the spoken text in real time, extracts resolutions / tasks, generates minutes, and distributes them. Specifically, this includes: Aggregate multiple users' speech texts by timestamp; Entity recognition and intent classification are performed simultaneously using a joint model; When the intent is a resolution or task, a predefined template is populated to generate structured minutes and distributed.

Images