Speech recognition real-time interaction system and method based on artificial intelligence

Abstract

The invention discloses a voice recognition real-time interaction system and method based on artificial intelligence, and the method comprises the steps: deploying an edge calculation node on a distributed voice collection device, and carrying out the voice feature recognition and voice text conversion of collected conference voice data through the edge calculation node, voice characteristic values and text data of a plurality of nodes are obtained; connecting the plurality of edge computing nodes through a block chain, and transmitting the voice features and text data of the plurality of nodes to a central node; performing style prediction on the voice features and the text data of the plurality of nodes by using the central node to obtain a conference language style; and according to the instruction initiating node, performing interactive answering based on the conference language style. The invention relates to the technical field of voice interaction, and solves the technical problems that an existing voice interaction system is low in long text processing efficiency and insufficient in conference scene dynamic interaction experience feeling.

Description

Technical Field

[0001] The present invention belongs to the field of voice interaction and relates to artificial intelligence technology, specifically to a real-time interaction system and method for voice recognition based on artificial intelligence. Background Art

[0002] Existing voice interaction systems have achieved deep integration of voice dialogue, logical reasoning and contextual understanding, and have made important technological breakthroughs in areas such as voice signal processing, intent analysis algorithms and multi-round dialogue management.

[0003] However, in multi-person conference scenarios, traditional speech recognition-based conference recording systems generally lack multi-source separation and identity recognition mechanisms, making it impossible to accurately associate voice data with speaker roles. This results in meeting content recording remaining at the basic level of indiscriminate speech-to-text conversion. Furthermore, existing systems are limited to shallow analysis of conference texts, such as keyword retrieval, and lack the ability to conduct real-time, in-depth analysis of conference topic evolution, semantic logic, and scenario-based interaction needs. This makes it difficult to implement advanced interactive features such as intelligent summary generation based on conference content, real-time question-and-answer responses, and discussion guidance. These systems are unable to meet the requirements of modern intelligent conferences for efficient information processing and natural human-computer collaboration.

[0004] If existing AI voice interaction systems are directly applied to conference scenarios, when faced with long text processing tasks in conference speech recognition, when the data volume exceeds the sequence length limit of the pre-trained model, the system will be forced to interrupt analysis due to memory overflow or a surge in computational complexity. This can lead to problems such as information truncation and logical discontinuities in semantic parsing in long text scenarios, seriously affecting the integrity and accuracy of content processing. Furthermore, existing systems use predefined fixed language style templates and standardized intonation parameters for voice interaction, lacking real-time perception and adaptive mechanisms for the dynamic characteristics of conference scenarios. As a result, interactive responses lack emotional and contextual response design, leading to negative user experiences of "stiff machine responses" and "lack of in-depth understanding" in high-frequency interactive scenarios. This makes it difficult to meet the flexible interaction requirements for natural human-computer collaboration in smart meetings. Summary of the Invention

[0005] The present invention aims to solve at least one of the technical problems existing in the prior art; to this end, the present invention proposes a real-time interactive system and method for speech recognition based on artificial intelligence, which is used to solve the technical problems of low efficiency in processing long texts and insufficient dynamic interactive experience in conference scenarios in existing voice interaction systems.

[0006] To achieve the above objectives, the first aspect of the present invention provides a real-time interactive method for speech recognition based on artificial intelligence, comprising:

[0007] Deploy edge computing nodes on distributed voice collection devices and use them to perform voice feature recognition and voice-to-text conversion on the collected conference voice data to obtain voice feature values and text data for several nodes.

[0008] Connect several edge computing nodes through blockchain and transmit the voice features and text data of several nodes to the central node;

[0009] Use the central node to predict the style of the speech features and text data of several nodes to obtain the conference language style;

[0010] Initiate nodes according to instructions and provide interactive responses based on the conference language style.

[0011] Furthermore, the speech feature recognition includes:

[0012] A1 uses the energy entropy method or deep learning model to extract voice segments from conference voice data of different edge computing nodes to obtain a voice segment set, and then divides the voice segments into preset time intervals to obtain a frame sequence;

[0013] A2: Based on the voice segment set, count the number and cumulative duration of voice segments with the same timestamp on all edge computing nodes to obtain the number of voice overlaps L. N Overlap time with speech L T ;

[0014] A3, using edge computing nodes to perform frequency domain analysis on the frame sequence to obtain spectrum features;

[0015] A4, calculating the ratio of the harmonic component of the speech signal to the noise component based on the spectral characteristics to obtain the harmonic-to-noise ratio (HNR);

[0016] A5, using the autocorrelation method to calculate the fundamental frequency of each frame of speech in the frame sequence, to obtain the fundamental frequency sequence, and calculating the standard deviation of the fundamental frequency difference between adjacent frames based on the fundamental frequency sequence to obtain the fundamental frequency perturbation J;

[0017] A6, using the root mean square value to calculate the amplitude of each frame in the frame sequence to obtain an amplitude sequence, and calculating the standard deviation of the amplitude difference between adjacent frames based on the amplitude sequence to obtain the amplitude perturbation S;

[0018] A7, according to the formula Calculate the speech feature value F of the current edge computing node a Among them, LN max Indicates the preset maximum overlap times, T indicates the total duration of the meeting, H max represents the preset maximum harmonic-to-noise ratio, and k represents the overlap duration attenuation coefficient.

[0019] By analyzing speech overlap and voice quality, the system can accurately capture the intensity of interaction and clarity of speech within a meeting. The first term in the formula reflects the activity of the discussion, with high overlap indicating lively discussion and low overlap indicating orderly reporting. The second term distinguishes the clarity and stability of speech, used to determine the formality of the meeting. This multi-dimensional quantitative analysis enables the system to perceive the dynamic atmosphere of the meeting in real time, ensuring that the tone of responses matches the interactive pattern of the meeting.

[0020] Furthermore, the speech-to-text conversion includes:

[0021] Use edge computing nodes to preprocess voice data to obtain preprocessed voice data;

[0022] The preprocessed speech data is input into the speech recognition model in real time and then preprocessed to obtain text data; wherein the speech recognition model is deployed on several edge computing nodes.

[0023] Speech recognition models are deployed locally at edge nodes to pre-process and convert voice data into text in real time, achieving low-latency voice-to-text synchronization. This supports real-time interaction requirements such as listening and summarizing, and instant Q&A during meetings. Furthermore, multiple edge nodes process multiple speakers in parallel, avoiding the queuing delays associated with centralized processing and adapting to high-frequency interactions among multiple speakers.

[0024] Furthermore, connecting the plurality of edge computing nodes via a blockchain includes:

[0025] Assign a unique node ID to each edge computing node;

[0026] Record the speaker information corresponding to each edge computing node, and associate the speaker information, node ID, node voice feature value and text data with timestamp to obtain several data blocks;

[0027] Using blockchain technology to transmit several data blocks to the central node in real time;

[0028] After receiving several data blocks, the central node stores the data blocks in the corresponding storage pool according to the node ID.

[0029] By binding node ID, speaker information, and timestamp through blockchain, a traceable distributed data index is formed, enabling central nodes to quickly locate content of specific speakers, time periods, or topics, thereby improving question-and-answer efficiency. The tamper-proof nature of blockchain ensures data credibility and enhances the credibility of system responses.

[0030] Furthermore, the method of using the central node to perform style prediction on the speech feature values and text data of several nodes includes:

[0031] The speech feature values of several nodes are averaged to obtain the comprehensive speech feature value;

[0032] Calculate the comprehensive text feature value based on the text data of several nodes, and perform weighted summation of the comprehensive voice feature value and the comprehensive text feature value to obtain a comprehensive score;

[0033] Set a corresponding threshold based on the conference type, compare the comprehensive score with the threshold, and obtain the conference type of the current conference;

[0034] The language style corresponding to the conference type of the current conference is extracted from the language style library to obtain the conference language style.

[0035] The comprehensive voice feature values are used to reflect the real-time interactive status of the meeting, and the comprehensive text feature values are used to characterize the focus and structural attributes of the meeting. The two complement each other, enabling the system to "understand" the tone and atmosphere of the speech, and "understand" the professionalism and logical structure of the content, allowing the system to adapt to the dynamic changes in the meeting process, automatically match the corresponding language style library, and generate interactive answers that are highly consistent with the on-site atmosphere, thereby improving the naturalness and adaptability of the interaction to the scene.

[0036] Furthermore, the calculating of the comprehensive text feature value based on the text data of the plurality of nodes includes:

[0037] Count the proportion of the number of terms in the text data of several nodes to the total number of terms to obtain the term density TD;

[0038] Input the text data of several nodes into the topic modeling algorithm to obtain the topic distribution of the current meeting, and use the formula LE = -∑p i ×lnp i The topic entropy LE is calculated; wherein the topic distribution represents the probability distribution of text data on several topics, p i represents the probability of the i-th conference topic;

[0039] Calculate the average time interval TI between adjacent speeches based on the node ID and timestamp; wherein the adjacent speeches refer to speeches with consecutive timestamps but different node IDs;

[0040] Use the syntax analysis tool in the Python library to perform syntax analysis on the text data of several nodes and calculate the syntax complexity SC;

[0041] According to the formula Calculate the text feature value F t ; where α represents the term sensitivity coefficient, SC max Indicates the preset maximum syntactic complexity;

[0042] The text feature values of several nodes are averaged to obtain the comprehensive text feature value.

[0043] Term density reflects the domain attributes of the meeting, the dispersion or concentration of topic distribution reflects whether the discussion is divergent or focused, the length of intervals between speeches quantifies the frequency of interaction, and the complexity of sentence structure reflects the formality or colloquialism of expression. The comprehensive modeling of these features enables the system to accurately determine the type of meeting and adjust the structure and tone of responses accordingly—for example, using structured statements in professional settings and more open, guiding questions in free discussion settings.

[0044] Furthermore, the statistical process of the term vocabulary number includes:

[0045] Collect professional terms in several fields and classify them into several topics to obtain a term base containing topic names and term subsets;

[0046] Traverse the topic distribution of the current conference and check whether each conference topic in the topic distribution exists in the terminology library; if so, obtain the term subset corresponding to the conference topic from the terminology library; if not, mark the term vocabulary as 0;

[0047] The number of terms in the text data is counted based on the term subset to obtain the term vocabulary.

[0048] It should be noted that topic distribution represents the probability distribution of text data on several topics.

[0049] Furthermore, the interactive answering based on the conference language style includes:

[0050] Deploy pre-trained language models on edge computing nodes and central nodes, and deploy lightweight pre-trained sub-language models on central nodes. The pre-trained language model is used to answer instructions, and the pre-trained sub-language model is used to analyze the node ID required to answer the instructions.

[0051] Use the pre-trained language model of the edge computing node to summarize the text data of each node and obtain the summary data of each node;

[0052] The command is transmitted to the pre-trained sub-language model of the central node, and the node ID required to answer the command is analyzed based on the summary data of each node to obtain the target node ID;

[0053] The central node selects text data from the storage pool corresponding to the target node ID according to the target node ID to obtain the analysis text;

[0054] Determine whether the number of characters in the analyzed text is greater than the preset character processing number;

[0055] If yes, the pre-trained language model of the target node ID is used to analyze the core content of the text data of the target node ID to obtain several core texts, and the several core texts are transmitted to the instruction initiating node. The instruction initiating node is used to analyze the several core texts and provide interactive answers based on the conference language style.

[0056] If not, the analysis text is transmitted to the instruction initiating node, the analysis text is analyzed using the pre-trained language model of the instruction initiating node, and interactive answers are given based on the meeting language style.

[0057] When processing long texts, the pre-trained sub-language model is used to screen the required node IDs. The edge computing node extracts the core content and then transmits it to the central node or the corresponding edge computing node for analysis, enabling the system to efficiently handle complex node problems. This not only avoids the computing power consumption of directly processing ultra-long texts, but also ensures the accuracy of semantic analysis by focusing on key information, and improves the processing efficiency of long texts.

[0058] The second aspect of the present invention provides a real-time interactive system for speech recognition based on artificial intelligence, comprising:

[0059] Data collection module: used to deploy edge computing nodes on distributed voice collection devices, and use edge computing nodes to perform voice feature recognition and voice-to-text conversion on the collected conference voice data to obtain voice feature values and text data of several nodes;

[0060] Data transmission module: used to connect several edge computing nodes through the blockchain and transmit the voice features and text data of several nodes to the central node;

[0061] Style prediction module: used to use the central node to predict the style of the speech features and text data of several nodes to obtain the conference language style;

[0062] Interactive Q&A module: used to initiate nodes according to instructions and provide interactive answers based on the conference language style.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] (1) Multi-node design and edge computing significantly improve the accuracy and real-time performance of speech recognition. By deploying edge computing nodes on distributed speech acquisition devices, the system can perform local processing of conference voice data, including speech feature recognition and real-time speech-to-text conversion. The parallel processing capability of edge nodes effectively solves the data congestion problem in multi-speaker scenarios and avoids queuing delays in traditional centralized processing models.

[0065] (2) Blockchain-based data transmission and style prediction technology enhances the system's credibility and interactivity. By binding node IDs, speaker information, and timestamps through blockchain technology, a traceable distributed data index is constructed, ensuring that data cannot be tampered with and is easy to locate, greatly improving the efficiency and credibility of the question-and-answer session. At the same time, the central node dynamically predicts the meeting's language style by weightedly combining speech and text feature values. The system can adjust tone and answer strategies based on the type of meeting, improving the naturalness of interaction and adaptability to specific scenarios.

[0066] (3) The hierarchical processing mechanism and lightweight model optimize the efficiency and resource consumption of long text analysis. A hierarchical architecture is adopted in which pre-trained language models and sub-models work together. The central node quickly filters the target node ID through the lightweight sub-model, and the edge node is responsible for refining the core content of the text, avoiding the computing power burden caused by directly processing super-long texts. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0068] Figure 1 A flow chart of the real-time interactive method for speech recognition based on artificial intelligence provided by the present invention;

[0069] Figure 2 A schematic diagram of the framework of the artificial intelligence-based speech recognition real-time interactive system provided by the present invention;

[0070] Figure 3 This is a schematic diagram of the workflow of the interactive question-and-answer module provided by the present invention. DETAILED DESCRIPTION

[0071] The technical solutions of the present invention will be clearly and completely described below in conjunction with the embodiments. Obviously, the embodiments described are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0072] See also Figure 1-Figure 3 The first embodiment of the present invention provides a real-time interactive system for speech recognition based on artificial intelligence, comprising:

[0073] Data collection module: used to deploy edge computing nodes on distributed voice collection devices, and use edge computing nodes to perform voice feature recognition and voice-to-text conversion on the collected conference voice data to obtain voice feature values and text data of several nodes;

[0074] Data transmission module: used to connect several edge computing nodes through the blockchain and transmit the voice features and text data of several nodes to the central node;

[0075] Style prediction module: used to use the central node to predict the style of the speech features and text data of several nodes to obtain the conference language style;

[0076] Interactive Q&A module: used to initiate nodes according to instructions and provide interactive answers based on the conference language style.

[0077] It should be noted that the data acquisition module, data transmission module, style prediction module and interactive question-answering module of the present invention are communicatively connected.

[0078] The data acquisition module of the present invention includes a distributed voice acquisition device deployed with edge computing nodes. By associating with the edge computing node ID and speaker information, voice data is distributedly collected and processed to achieve real-time collection, feature extraction, and structured processing of language data. The specific process is as follows:

[0079] (1) Before executing the data acquisition module, the edge computing node needs to be initialized and deployed:

[0080] First, a unique node ID is assigned to each distributed voice collection device, which is solidified through system configuration and serves as the unique identifier for data traceability.

[0081] Then bind the speaker information corresponding to the node, including but not limited to name, role, etc., and establish a mapping relationship table The data is stored in the node's local secure storage area. NodeID represents the node ID, and SpeakerID represents the speaker ID. A lightweight speech recognition model and pre-trained language model are deployed on the edge computing node to perform real-time speech stream processing and local semantic analysis.

[0082] Then, a distributed microphone array is used to collect the conference voice signal, and the ambient noise is suppressed through beamforming technology to generate a time domain audio stream S(t). The energy entropy method or deep learning model, such as the LSTM-VAD model, is used to detect the valid voice segment to filter out silence and non-voice signals and output a set of voice segments. The voice segments are then divided into overlapping frames of 20-30ms or a preset time interval to generate a frame sequence {f1, f2, ..., f n}, get the preprocessed speech frame;

[0083] (2) Then, the pre-processed speech data is used to perform speech feature recognition and quantification calculation, specifically including:

[0084] According to the voice segment set, the number and cumulative duration of voice segments with the same timestamp in all edge computing nodes are counted to obtain the number of voice overlaps L N Overlap time with speech L T ;

[0085] Perform fast Fourier transform on the preprocessed speech frame to obtain spectral features, and calculate the ratio of the harmonic component to the noise component of the speech signal based on the spectral features to obtain the harmonic-to-noise ratio (HNR);

[0086] The fundamental frequency of each frame in the preprocessed speech frame is calculated using the autocorrelation method to obtain the fundamental frequency sequence. The standard deviation of the fundamental frequency difference between adjacent frames is calculated based on the fundamental frequency sequence to obtain the fundamental frequency perturbation J.

[0087] The single-frame amplitude of the preprocessed speech frame is quantized using the root mean square value, and then the standard deviation of the adjacent amplitude differences is calculated to obtain the amplitude perturbation S;

[0088] Finally, according to the formula Calculate the speech feature value F of the current edge computing node a Among them, LN max Indicates the preset maximum overlap times, T indicates the total duration of the meeting, H max represents the preset maximum harmonic-to-noise ratio, and k represents the overlap duration attenuation coefficient.

[0089] (3) Speech-to-text and text preprocessing:

[0090] The pre-processed speech frame sequence is input into the speech recognition model of the edge node, and the text data with timestamp is output. i , TS i}; Then, the recognized text is processed to remove stop words and punctuation marks, and the word segmentation tool is used to segment the words to generate word sequences, and the preprocessed text data is obtained to provide a basis for subsequent topic modeling and term statistics.

[0091] It should be noted that the deployment process of the edge computing node in this embodiment may include:

[0092] Edge computing nodes are deployed on distributed voice collection devices, equipped with low-power processors, memory and storage modules, and supporting wireless or 5G communication interfaces. Each edge computing node generates a globally unique identifier as a node ID during initialization, which is fixed through the system configuration file.

[0093] The speech recognition model and pre-trained language model are then deployed on the edge computing node to perform real-time localized speech-to-text and conversational voice question and answer. The pre-trained language model is a large language model built based on artificial intelligence algorithms. It can be an existing open source pre-trained model, such as BERT, GPT series, RoBERTa, DeepSeek, etc., or it can be independently built.

[0094] In the data transmission module, edge computing nodes use blockchain technology to build a trusted distributed network to achieve secure real-time transmission of voice features and text data, including:

[0095] First, the node ID, speaker information, and speech feature value F of the current node are a And text data is encapsulated as data block Block i , the structure is as follows: Block i ={NodeID, SpeakerInfo, Timestamp, AudioFeatures, TextData, PreviousHash}; the edge node broadcasts the data block to the consensus node through the P2P protocol, and after verification by the PBFT consensus mechanism or the PoS consensus mechanism, it is synchronized to the central node in real time; where SpeakerInfo represents the associated speaker information, Timestamp represents the timestamp of data collection, and AudioFeatures represents the speech feature vector sequence, including the speech feature value F a , the number of speech overlaps L N Overlap time with speech L T , harmonic-to-noise ratio HNR, fundamental frequency perturbation J, amplitude perturbation S, TextData represents the text data after speech-to-text conversion, PreviousHash represents the hash value of the top data block;

[0096] After receiving the data blocks, the central node classifies the data blocks according to the NodeID and stores them in the corresponding storage pool StoragePool: StoragePool = {NodeID→[Block1, Block2, ..]}.

[0097] The blockchain's chain structure and hash checksum ensure that data transmission from edge computing nodes to central nodes cannot be tampered with. Any malicious tampering of data will result in a hash value mismatch and be rejected by the consensus mechanism, thus providing a viable data foundation for subsequent conference style predictions. At the same time, the node ID is bound to the speaker information on the chain, and combined with the central node's storage pool classification mechanism, a three-level data index of device, person, and time is formed. This supports fast and complex queries, ensuring the efficiency of subsequent data processing and interactive analysis of text data.

[0098] After the central node receives the voice feature values and text data from all edge computing nodes, the style prediction module first analyzes the comprehensive text feature values based on the text data. It then performs a weighted fusion of the comprehensive text feature values and the comprehensive voice feature values to obtain a comprehensive score. This score is then compared with a preset threshold to determine the meeting type, and finally, a matching meeting language style is extracted from the language style library. By integrating multi-dimensional features such as the intensity of voice interaction, speech clarity, and the professionalism and topic focus of the text, the meeting scene is dynamically predicted, providing interactive responses with language strategies that adapt to the current meeting atmosphere, achieving a precise match between the tone and structure of the response and the meeting type.

[0099] Specifically, the process of obtaining the comprehensive text feature value in the style prediction module may include:

[0100] Count the proportion of the number of terms in the text data of several nodes to the total number of terms to obtain the term density TD;

[0101] Input the text data of several nodes into the topic modeling algorithm to obtain the topic distribution of the current meeting, and use the formula LE = -∑p i ×lnp i The topic entropy LE is calculated to measure the dispersion of conference topics; the topic distribution represents the probability distribution of text data on several topics, p i represents the probability of the i-th conference topic;

[0102] Then, we collect professional terms from several fields and classify them into several topics. We then build a term base containing topic names and term subsets to calculate the term density in the text data: we traverse the topic distribution of the current meeting. If the topic exists in the term base, we extract the corresponding term subset; if not, the term vocabulary is marked as 0. We count the number of words in the text data that belong to the term subset and calculate the term density TD: TD = term vocabulary number / total vocabulary number, to reflect the domain expertise of the meeting.

[0103] Based on the node ID and timestamp, the average time interval TI between adjacent speeches is calculated to evaluate the frequency of meeting interactions and indirectly reflect the meeting atmosphere. Adjacent speeches refer to speeches with consecutive timestamps but different node IDs.

[0104] Use the syntax analysis tool in the Python library to perform syntax analysis on the text data of several nodes and calculate the syntax complexity SC, which reflects the formality of the expression;

[0105] According to the formula Calculate the text feature value F t ; Among them, α represents the term sensitivity coefficient, and SC is obtained by experimental calibration. max Indicates the preset maximum syntactic complexity;

[0106] The text feature values of several nodes are averaged to obtain the comprehensive text feature value.

[0107] Then, similarly, the speech feature values transmitted by several edge computing nodes are averaged to obtain a comprehensive speech feature value, and then the comprehensive speech feature value and the comprehensive text feature value are weighted and summed to obtain a comprehensive score; the comprehensive score is compared with the set threshold of each conference type to obtain the conference type of the current conference; the language style corresponding to the conference type of the current conference is extracted from the language style library to obtain the conference language style.

[0108] The statistical process of terminology vocabulary includes:

[0109] In one embodiment, the conference thresholds for different types can be obtained from Table 1:

[0110] Table 1, comprehensive thresholds for different types of conferences:

[0111] Meeting Type Comprehensive score (S) threshold range Formal reporting meeting S≥0.8 Business discussion meeting 0.65≤S<0.8 Academic Seminar 0.7≤S<0.85 Divergent discussion 0.5≤S<0.65 Notification meeting S<0.5

[0112] By integrating speech and text features, the system can accurately determine the type of meeting and call corresponding tone templates and answer structures from a language style library, ensuring that responses naturally fit the meeting atmosphere. For example, in response to the high terminology density and complex syntax of formal meetings, the system can use data-based, structured representations to quickly extract core ideas. In response to the high frequency of interaction and scattered topics in divergent discussions, guided questions can be used to facilitate logical analysis and reduce information redundancy. This targeted approach to answering can enhance the naturalness of human-computer interaction and the efficiency of information transfer, reducing the cost of comprehension for participants.

[0113] Finally, in the interactive question-and-answer module, through the collaborative processing of edge computing nodes and central nodes, the hierarchical analysis of pre-trained language models, and the dynamic adaptation of the conference language style, the command parsing, data positioning, and intelligent response of the human-computer interaction process are realized. Finally, the answers that fit the scene are generated in combination with the conference style, ensuring the efficiency, naturalness and scene adaptability of human-computer interaction, and providing intelligent support for conference collaboration.

[0114] In one embodiment, interactive answering based on the conference language style may include the following steps:

[0115] Deploy pre-trained language models on edge computing nodes and central nodes, and deploy lightweight pre-trained sub-language models on central nodes. The pre-trained language model is used to answer instructions, and the pre-trained sub-language model is used to analyze the node ID required to answer the instructions.

[0116] Use the pre-trained language model of the edge computing node to summarize the text data of each node and obtain the summary data of each node;

[0117] The command is transmitted to the pre-trained sub-language model of the central node, and the node ID required to answer the command is analyzed based on the summary data of each node to obtain the target node ID;

[0118] The central node selects text data from the storage pool corresponding to the target node ID according to the target node ID to obtain the analysis text;

[0119] Determine whether the number of characters in the analyzed text is greater than the preset character processing number;

[0120] If yes, the pre-trained language model of the target node ID is used to analyze the core content of the text data of the corresponding node to obtain several core texts. The core texts are then transmitted to the instruction initiating node, which analyzes the core texts and provides interactive answers based on the conference language style.

[0121] If not, the analysis text is transmitted to the instruction initiating node, the analysis text is analyzed using the pre-trained language model of the instruction initiating node, and interactive answers are given based on the meeting language style.

[0122] Therefore, in the interactive question-and-answer module, regardless of whether the instruction-initiating node is an edge computing node or a central node, it is necessary to first determine the target node required to answer the instruction through a lightweight pre-trained sub-language model - that is, to perform semantic analysis based on the instruction and determine which nodes' text data need to be called. If the total length of the target node's text data exceeds the processing threshold of the pre-trained language model, the system will trigger the pre-trained language model of the corresponding edge computing node, perform core content extraction on its local text data based on the instruction content, and generate a structured set of key information, such as argument summaries, data indicators, logical relationship chains, etc. Subsequently, the core content will be transmitted to the instruction-initiating node, and the text semantic analysis and interactive answer generation will be completed in combination with the conference language style. Through the layered processing mechanism, the computing power bottleneck and semantic parsing efficiency problems of traditional large language models in long text scenarios are effectively solved, and efficient processing and precise interaction of complex conference texts are achieved.

[0123] For example, at an unusual academic seminar on "Climate Change and Coastal Adaptation," a participant sends a question through the central node: "Please summarize the solutions proposed at this seminar for addressing flood disasters in coastal cities, and what are the key research directions and implementation recommendations for the future?" The central node first uses the pre-trained sub-language model to parse the keywords in the instruction, obtaining: "response to flood disasters in coastal cities," "solutions," and "future research directions." Then, through semantic matching, it identifies the following speech nodes in the seminar that touch on this topic:

[0124] Researcher Li from NodeID-012: Focused on "Improvement of Flood Risk Assessment Model" and proposed technical solutions;

[0125] Professor Zhang from NodeID-035: Report on "Synergistic Effects of Ecological Engineering and Hard Flood Control Facilities," involving engineering solutions;

[0126] Dr. Chen from NodeID-078: Discussing "Urban Planning Adaptation under Climate Change Scenarios," including recommendations at the policy and planning levels;

[0127] The sub-model determines the text data of the three nodes mentioned above that needs to be retrieved based on the node ID-speaker-topic mapping relationship recorded in the blockchain. When the total number of characters exceeds the processing threshold of the pre-trained language model, the long text layer processing mechanism is triggered:

[0128] The central node distributes the instructions to the corresponding edge computing nodes, and each node extracts the core content based on its own speech content and instruction requirements:

[0129] NodeID-012 extracts the core content of technical solutions; NodeID-035 extracts the core content of engineering and ecological methods; NodeID-078 extracts the core content of planning and policy recommendations;

[0130] Each edge node transmits the refined core text to the central node, combines it with the predicted results of the conference language style, reorganizes the answer content according to the logic of "current problem → solution → future direction", and interactively answers the current question initiator.

[0131] The first embodiment of the present invention provides a real-time interactive method for speech recognition based on artificial intelligence, comprising:

[0132] Deploy edge computing nodes on distributed voice collection devices and use them to perform voice feature recognition and voice-to-text conversion on the collected conference voice data to obtain voice feature values and text data for several nodes.

[0133] Connect several edge computing nodes through blockchain and transmit the voice features and text data of several nodes to the central node;

[0134] Use the central node to predict the style of the speech features and text data of several nodes to obtain the conference language style;

[0135] Initiate nodes according to instructions and provide interactive responses based on the conference language style.

[0136] Some of the data in the above formula are calculated by removing the dimensions and taking their numerical values. The formula is a formula that is closest to the actual situation obtained by software simulation of a large amount of collected data; the preset parameters and preset thresholds in the formula are set by technical personnel in this field according to actual conditions or obtained through simulation of a large amount of data.

[0137] Working principle of the present invention:

[0138] By deploying edge computing nodes on distributed voice collection equipment, conference voice data is locally processed, including voice feature recognition and real-time voice-to-text conversion, and data congestion problems in multi-speaker scenarios are solved through parallel processing of edge nodes; blockchain technology is used to bind node IDs, speaker information and timestamps to build a traceable distributed data index, ensuring that data cannot be tampered with and is easy to locate, improving question-and-answer efficiency and credibility; the central node dynamically predicts the meeting language style through weighted comprehensive voice feature values and text feature values, enabling the system to adjust the tone and answer strategy according to the type of meeting; a layered architecture is adopted in which pre-trained language models and sub-models work together. The central node quickly filters the target node ID through lightweight sub-models, and the edge node is responsible for refining the core content of the text, solving the computing power burden problem of long text processing.

[0139] The above embodiments are only used to illustrate the technical method of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical method of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical method of the present invention.

Claims

1. A real-time interactive method for speech recognition based on artificial intelligence, characterized in that: include: Use the edge computing nodes on the distributed voice collection device to perform voice feature recognition and voice-to-text conversion on the collected conference voice data to obtain the voice feature values and text data of several nodes; The voice features and text data of several nodes are transmitted to the central node through blockchain technology; Use the central node to predict the style of the speech features and text data of several nodes to obtain the conference language style; After obtaining the question-and-answer instructions from the instruction-initiating node, interactive answers are given based on the conference language style; wherein the instruction-initiating node includes the edge computing node and the central node.

2. The method for real-time interaction of speech recognition based on artificial intelligence according to claim 1, characterized in that: The speech feature recognition includes: Use the energy entropy method or deep learning model to extract voice segments of conference voice data from different edge computing nodes to obtain a voice segment set, and then segment the voice segments according to preset time intervals to obtain a frame sequence; According to the voice segment set, the number and cumulative duration of voice segments with the same timestamp in all edge computing nodes are counted to obtain the number of voice overlaps L N Overlap time with speech L T ; Use edge computing nodes to perform frequency domain analysis on frame sequences to obtain spectrum characteristics; The ratio of the harmonic component to the noise component of the speech signal is calculated according to the spectral characteristics to obtain the harmonic-to-noise ratio HNR; The fundamental frequency of each frame of speech in the frame sequence is calculated using the autocorrelation method to obtain the fundamental frequency sequence. The standard deviation of the fundamental frequency difference between adjacent frames is calculated based on the fundamental frequency sequence to obtain the fundamental frequency perturbation J. The amplitude of each frame in the frame sequence is calculated using the root mean square value to obtain an amplitude sequence. The standard deviation of the amplitude difference between adjacent frames is calculated based on the amplitude sequence to obtain the amplitude perturbation S. The speech feature value of the current edge computing node is obtained according to the speech feature calculation formula.

3. The method for real-time interaction of speech recognition based on artificial intelligence according to claim 2, characterized in that: The voice feature calculation formula is: Among them, F a Represents the speech feature value, LN max Indicates the preset maximum overlap times, T indicates the total duration of the meeting, H max represents the preset maximum harmonic-to-noise ratio, and k represents the overlap duration attenuation coefficient.

4. The method for real-time interaction of speech recognition based on artificial intelligence according to claim 1, characterized in that: Connecting several edge computing nodes through blockchain includes: Assign a unique node ID to each edge computing node; Record the speaker information corresponding to each edge computing node, and associate the speaker information, node ID, node voice feature value and text data with timestamp to obtain several data blocks; Using blockchain technology to transmit several data blocks to the central node in real time; After receiving several data blocks, the central node stores the data blocks in the corresponding storage pool according to the node ID.

5. The method for real-time interaction of speech recognition based on artificial intelligence according to claim 1, characterized in that: The method of using a central node to perform style prediction on speech feature values and text data of a plurality of nodes includes: The speech feature values of several nodes are averaged to obtain the comprehensive speech feature value; Calculate the comprehensive text feature value based on the text data of several nodes, and perform weighted summation of the comprehensive voice feature value and the comprehensive text feature value to obtain a comprehensive score; Set a corresponding threshold based on the conference type, compare the comprehensive score with the threshold, and obtain the conference type of the current conference; The language style corresponding to the conference type of the current conference is extracted from the language style library to obtain the conference language style.

6. The method for real-time interaction of speech recognition based on artificial intelligence according to claim 5, characterized in that: The method of calculating the comprehensive text feature value based on the text data of a plurality of nodes includes: Count the proportion of the number of terms in the text data of several nodes to the total number of terms to obtain the term density TD; Input the text data of several nodes into the topic modeling algorithm to obtain the topic distribution of the current meeting, and use the formula LE = -∑p i ×lnp i The topic entropy LE is calculated; wherein the topic distribution represents the probability distribution of text data on several topics, p i represents the probability of the i-th conference topic; Calculate the average time interval TI between adjacent speeches based on the node ID and timestamp; wherein the adjacent speeches refer to speeches with consecutive timestamps but different node IDs; Use the syntax analysis tool in the Python library to perform syntax analysis on the text data of several nodes and calculate the syntax complexity SC; According to the text feature calculation formula, the text feature value F of a single node is obtained t ; The text feature values of several nodes are averaged to obtain the comprehensive text feature value.

7. The method for real-time interaction of speech recognition based on artificial intelligence according to claim 6, characterized in that: The text feature calculation formula is: Among them, α represents the term sensitivity coefficient, SC max Indicates the preset maximum syntactic complexity.

8. The method for real-time interaction of speech recognition based on artificial intelligence according to claim 6, characterized in that: The statistical process of the terminology vocabulary number includes: Collect professional terms in several fields and classify them into several topics to obtain a term base containing topic names and term subsets; Traverse the topic distribution of the current conference and check whether each conference topic in the topic distribution exists in the terminology library; if so, obtain the term subset corresponding to the conference topic from the terminology library; if not, mark the term vocabulary as 0; The number of terms in the text data is counted based on the term subset to obtain the term vocabulary.

9. The method for real-time interaction of speech recognition based on artificial intelligence according to claim 1, characterized in that: The interactive responses based on the conference language style include: Deploy pre-trained language models on edge computing nodes and central nodes, and deploy lightweight pre-trained sub-language models on central nodes. The pre-trained language model is used to answer commands, and the pre-trained sub-language model is used to analyze the node ID required to answer the command. Use the pre-trained language model of the edge computing node to summarize the text data of each node and obtain the summary data of each node; The command is transmitted to the pre-trained sub-language model of the central node, and the node ID required to answer the command is analyzed based on the summary data of each node to obtain the target node ID; The central node selects text data from the corresponding storage pool according to the target node ID and obtains the analysis text; Determine whether the number of characters in the analyzed text is greater than the preset character processing number; If yes, the pre-trained language model of the target node ID is used to analyze the core content of the text data of the corresponding node to obtain several core texts. The core texts are then transmitted to the instruction initiating node, which analyzes the core texts and provides interactive answers based on the conference language style. If not, the analysis text is transmitted to the instruction initiating node, the analysis text is analyzed using the pre-trained language model of the instruction initiating node, and interactive answers are given based on the meeting language style.

10. A real-time interactive speech recognition system based on artificial intelligence, applied to the real-time interactive speech recognition method based on artificial intelligence according to any one of claims 1 to 9, characterized in that: include: Data collection module: used to deploy edge computing nodes on distributed voice collection devices, and use edge computing nodes to perform voice feature recognition and voice-to-text conversion on the collected conference voice data to obtain voice feature values and text data of several nodes; Data transmission module: used to connect several edge computing nodes through the blockchain and transmit the voice features and text data of several nodes to the central node; Style prediction module: used to use the central node to predict the style of the speech features and text data of several nodes to obtain the conference language style; Interactive Q&A module: used to initiate nodes according to instructions and provide interactive answers based on the conference language style.

Images