Voice interactive chart dynamic generation method and system based on large language model

Abstract

The invention discloses a voice interactive chart dynamic generation method and system based on a large language model, and relates to the technical field of chart generation, and the method comprises the steps: collecting a user voice signal, carrying out the preliminary framing, calculating the short-time energy, and analyzing a mute segment set signal; frequency domain signals are extracted from the set signals, gain is calculated through a filter, noise reduction is carried out on the frequency domain signals, the zero-crossing rate and noise reduction short-time energy are calculated, effective voice frames are screened, peak detection is carried out, the average voice speed is calculated, voice speed normalization is carried out, and signal values of sampling points in the frames are extracted. According to the method, the noise power spectrum density is extracted through FFT on the mute section, a frequency domain model of background noise is effectively established, a follow-up filter can accurately act on an actually existing frequency band interference area, weakening of the voice main signal spectrum is avoided, the Mel filter bank is accessed after the power spectrum is calculated through the FFT after frame windowing, and the noise power spectrum density is calculated through the FFT after frame windowing. Energy can be redistributed on the logarithmic Mel scale according to human ear perception characteristics.

Description

Technical Field

[0001] The present invention relates to the technical field of chart generation, and in particular to a method and system for dynamically generating voice interactive charts based on a large language model. Background Art

[0002] With the rapid development of speech recognition and natural language processing technologies, the form of human-computer interaction is gradually evolving from traditional graphical interface input to a more natural and efficient voice interaction method. In this process, large language models rely on the vast corpus training capabilities of deep neural networks, demonstrating powerful capabilities in semantic understanding, intent recognition, and context modeling. At the same time, data visualization technology is also constantly improving, and users' requirements for the interactivity, dynamism, and intelligence of chart presentations are increasing.

[0003] However, most current voice-driven visualization generation is still in the primary stage of relying on template recognition or keyword matching. It is unable to achieve an in-depth understanding of users' natural language commands, and it is difficult to accurately map voice input into structured chart generation instructions, which seriously limits the interactive efficiency and practicality of voice and chart linkage. In existing chart generation methods, users often need to specify chart types, data dimensions and visualization styles through multiple rounds of interaction. The operation process is cumbersome and requires users to have a high level of chart construction knowledge. In terms of speech recognition, traditional ASR (automatic speech recognition) systems often rely on fixed dictionaries and lack flexibility. The accuracy rate drops significantly when faced with changes in speech speed, noise interference and pronunciation differences. In addition, existing technologies find it difficult to achieve integrated fusion from voice input to chart presentation. There is a lack of effective connection modeling mechanism between voice content and data semantics, and a dynamic linkage mechanism has not been established between structured semantic understanding and chart generation rules. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method and system for dynamically generating interactive speech charts based on a large language model to solve the problem in existing chart generation methods that users often need to specify chart types, data dimensions and visualization styles through multiple rounds of interaction. The operation process is cumbersome and requires users to have high knowledge of chart construction. In terms of speech recognition, traditional ASR (automatic speech recognition) systems often rely on fixed dictionaries and lack flexibility. The accuracy rate drops significantly when faced with changes in speech speed, noise interference and pronunciation differences. In addition, the existing technology is difficult to achieve integrated fusion from speech input to chart presentation. There is a lack of effective connection modeling mechanism between speech content and data semantics, and it has failed to establish a dynamic linkage mechanism between structured semantic understanding and chart generation rules.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for dynamically generating a speech interactive graph based on a large language model, comprising:

[0008] Collect user voice signals, perform preliminary frame division and calculate short-time energy, and analyze the aggregate signal of silent segments;

[0009] Extract the frequency domain signal from the collective signal, calculate the gain through the filter to reduce the noise of the frequency domain signal, calculate the zero-crossing rate and the short-time energy of the noise reduction to screen the valid speech frames, perform peak detection and calculate the average speaking rate for speech rate normalization, extract the signal value of the sampling point in the frame, calculate the power spectrum and perform filtering analysis on the feature sequence, calculate the conditional probability of the subwords in the BPE word segmentation table, and select the subword sequence with the highest probability;

[0010] Perform semantic recognition on subword sequences, output corresponding structured instructions, map them to DSL commands, perform verification and correction, obtain output instructions, structure user-uploaded data through graph neural networks, calculate the goodness of fit of trend functions, and determine the optimal trend;

[0011] Generate dynamic charts and provide text and voice explanations, encrypt chart data and back it up in the cloud.

[0012] As a preferred solution of the method for dynamically generating interactive speech charts based on a large language model according to the present invention, the steps of extracting the signal values of sampling points within a frame, calculating the power spectrum and performing filtering analysis on the feature sequence, and calculating the conditional probability of subwords in the BPE word segmentation table include:

[0013] Perform Fast Fourier Transform (FFT) on each frame of speech signal to obtain the frequency domain signal, and calculate the noise power spectrum density for the silent segment set and non-silent segment frames;

[0014] For each frequency point of each frame, the gain is calculated according to the noise power spectral density through the Wiener filter, and the gain of the frequency domain signal of each frame is adjusted to obtain the denoised frequency domain signal, which is further inverse Fourier transformed to obtain the time domain denoised signal;

[0015] Re-frame the noise reduction signal and calculate the noise reduction short-time energy of each frame, and calculate the zero-crossing rate of each frame;

[0016] A noise reduction threshold is set based on the sum of the mean and twice the standard deviation of the historical noise reduction short-time energy data, and a zero-crossing rate threshold is set based on the sum of the mean and twice the standard deviation of the historical zero-crossing rate data. A comparison is performed based on each frame of the noise reduction signal. If the noise reduction short-time energy of the noise reduction signal is greater than or equal to the noise reduction threshold, and the zero-crossing rate of the noise reduction signal is greater than or equal to the zero-crossing rate threshold, the corresponding frame is determined to be a valid speech frame, and the output is an index set of valid speech frames;

[0017] Perform peak detection of short-time energy of noise reduction on valid speech frames, count the number of syllables S, calculate the effective speech duration, and determine the average speaking speed based on the ratio of the number of syllables to the effective speech duration;

[0018] The standard speaking speed is determined based on historical experience. If the average speaking speed is higher than the standard speed, the noise reduction signal of the valid speech frame is time-stretched using the WSOLA algorithm to normalize the speaking speed.

[0019] For the effective speech signal after speech rate normalization, the frame length and frame shift are set to calculate the total number of frames. For each frame, the signal value of the sampling point within the frame is extracted, multiplied by the Hamming window, and fast Fourier transform is performed to obtain the frequency domain signal. The power spectrum is calculated and filtered through the Mel filter bank to obtain the Mel energy.

[0020] Take the logarithm of each Mel energy, obtain the logarithmic Mel energy, perform discrete cosine transform, and obtain the MFCC feature vector to form the feature sequence X;

[0021] A pre-trained end-to-end word-level deep neural network ASR model is used to output the conditional probability of the subwords in the BPE word segmentation table based on the input of the feature sequence, and obtain the probability of the entire output sequence;

[0022] The beam search algorithm is used to select the subword sequence with the highest probability from the probability distribution output by the model as the final ASR recognition result.

[0023] As a preferred solution of the method for dynamically generating voice interactive charts based on a large language model of the present invention, wherein: the structured instructions corresponding to the output are mapped to DSL commands, including:

[0024] Preprocessing of subword sequences involves removing non-linguistic symbols and using the pre-trained BERT large language model for semantic information recognition. Based on the pre-defined labels of the large language model, maximum likelihood estimation (MLE) is used to determine the probability that the user's intention is the pre-defined label. Based on the results of intent recognition, the model outputs the corresponding structured instructions.

[0025] Each instruction is represented as a set of standardized graph commands. The syntax rules of DSL commands are defined using BNF Backus-Naur Form. The structured instructions are mapped to DSL commands to generate a standardized graph command set.

[0026] A cyclic consistency check mechanism is introduced to perform consistency check between the current instruction and the previous instruction by calculating the similarity value between the current instruction set and the previous instruction set;

[0027] Sub-supervised learning method is used for feedback correction to obtain the final output instructions.

[0028] As a preferred solution of the method for dynamically generating speech interactive charts based on a large language model of the present invention, wherein: the calculation of the goodness of fit of the trend function and the determination of the optimal trend include:

[0029] Parse the chart data uploaded by users, use the graph neural network (GNN) to treat data types as nodes, set edges based on the relationship between data attributes, and structure the data table;

[0030] A trend function set is obtained based on a database, and the goodness of fit of the trend function is calculated based on structured data, and the trend function with the maximum goodness of fit is selected as the optimal trend.

[0031] As a preferred solution of the method for dynamically generating a voice interactive chart based on a large language model of the present invention, the method of generating a dynamic chart and performing text and voice interpretation includes:

[0032] Select the chart type based on the determination of the optimal trend function, draw the trend line in the chart according to the output instructions, and use the Matplotlib tool to generate dynamic charts;

[0033] Based on the changes in data points in the chart, including trends and extreme points, text-to-speech (TTS) method is used to provide text-to-speech explanations.

[0034] As a preferred solution of the method for dynamically generating interactive speech charts based on a large language model of the present invention, the method includes: collecting user speech signals, performing preliminary framing and calculating short-time energy, and analyzing the silent segment aggregate signal, including:

[0035] Use a high signal-to-noise ratio microphone array to collect speech signals and form an original speech signal sequence. Then divide the original speech signal sequence into frames and calculate the short-time energy of each frame.

[0036] By collecting the short-time energy mean plus twice the standard deviation of the silent segment as the energy threshold, the short-time energy frames less than or equal to the energy threshold are regarded as silent segments, and the short-time energy frames greater than the energy threshold are regarded as speech segments, and the silent segment set is formed according to the frame index of the silent segment.

[0037] As a preferred solution of the method for dynamically generating speech interactive charts based on a large language model of the present invention, the method includes:

[0038] The generated chart data is encrypted and stored using the symmetric encryption algorithm AES, a key generation function is used to generate the key, and the data is transmitted to the cloud for data backup through wireless transmission technology, and the storage is performed through key-value mapping.

[0039] In a second aspect, the present invention provides a system for dynamically generating speech interactive graphs based on a large language model, comprising:

[0040] The voice processing module uses a high signal-to-noise ratio microphone array to collect user voice, divides the original voice signal into frames, and calculates the short-time energy of each frame;

[0041] The voice activity detection module reduces noise in the frequency domain of the speech segment based on the noise power spectrum density, performs an inverse transform to restore the time domain signal, calculates the zero-crossing rate and noise reduction short-time energy of each frame, and determines the valid speech frame;

[0042] The speech rate normalization module performs peak detection on valid speech frames, calculates the average speech rate based on the number of syllables and the length of valid frames, and uses the WSOLA algorithm to perform time normalization if the speech rate is abnormal.

[0043] The speech recognition module frames the normalized speech signal, extracts the Mel-frequency cepstral coefficients, uses an end-to-end ASR model to obtain the BPE subword sequence and its conditional probability, and selects the output sequence with the maximum probability.

[0044] The instruction generation module cleans subword sequences, uses the BERT model for semantic recognition, maps the identified structured intent to DSL chart commands, and verifies and corrects the instructions.

[0045] The trend modeling module uses a graph neural network (GNN) to structure the chart data uploaded by users, obtain a set of trend functions, calculate the fitting error of each trend function, and select the optimal trend model;

[0046] The chart display module parses DSL commands to draw charts, draws trend lines based on trend functions, and uses them to realize dynamic playback of charts, synchronizing chart dynamics with data trend changes;

[0047] The intelligent voice explanation module analyzes the key points in the chart, automatically generates corresponding explanation text, and uses TTS technology to synthesize text into speech;

[0048] The cloud storage module encrypts dynamic chart data and completes chart data backup in the cloud through a key-value mapping structure.

[0049] In a third aspect, the present invention provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: when the computer program is executed by the processor, any step of the method for dynamically generating speech interactive charts based on a large language model as described in the first aspect of the present invention is implemented.

[0050] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein: when the computer program is executed by a processor, it implements any step of the method for dynamically generating speech interactive charts based on a large language model as described in the first aspect of the present invention.

[0051] The beneficial effects of the present invention are as follows: by performing FFT on the silent segment to extract the noise power spectral density, a frequency domain model of the background noise is effectively established. The subsequent filter can accurately act on the actual frequency band interference area to avoid weakening the spectrum of the main voice signal. The power spectrum is calculated by FFT after frame windowing and then connected to the Mel filter bank. The energy can be redistributed on the logarithmic Mel scale according to the human ear perception characteristics. The probability of user intention is calculated by maximum likelihood estimation. Based on the text content expressed by the user, this method can assign a corresponding probability to each possible intention, thereby providing a quantitative judgment of the user's intention. Using the mean square error as the consistency standard, it can not only accurately quantify the degree of fit of the function to the data trend, but also compare and optimize functions through a unified error value. By using structured description text for audio synthesis through the TTS system, not only accessibility is enhanced, so that visually impaired users can also receive chart information, but also the memorability and reception efficiency of information are enhanced through voice interpretation. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 This is a flow chart of the method for dynamically generating speech interactive charts based on a large language model in Example 1.

[0054] Figure 2 This is a structural diagram of the system for dynamically generating interactive speech charts based on a large language model in Example 1. DETAILED DESCRIPTION

[0055] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0056] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0057] Secondly, the term "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in various places throughout this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive of other embodiments.

[0058] Example 1, reference Figures 1 to 2 , which is the first embodiment of the present invention, provides a method for dynamically generating a speech interactive chart based on a large language model, comprising the following steps:

[0059] S1: Collect user voice signals, perform preliminary frame division and calculate short-time energy, and analyze the collective signal of the silent segment;

[0060] Preferably, collecting the user voice signal, performing preliminary framing and calculating the short-time energy, and analyzing the silent segment aggregate signal include:

[0061] Use a high signal-to-noise ratio microphone array to collect speech signals and form an original speech signal sequence. Frame the original speech signal sequence and calculate the short-time energy of each frame, which is expressed as:

[0062]

[0063] Among them E k represents the short-time energy of the k-th frame, N is the total number of samples, x represents the collected speech signal, and x[kN+i] represents the signal amplitude of the i-th sampling point in the k-th frame;

[0064] By collecting the short-time energy mean plus twice the standard deviation of the silent segment as the energy threshold, the short-time energy frames less than or equal to the energy threshold are regarded as silent segments, and the short-time energy frames greater than the energy threshold are regarded as speech segments, and the silent segment set is formed according to the frame index of the silent segment.

[0065] By using a microphone array with a high signal-to-noise ratio for voice collection, the interference of background noise can be significantly suppressed in the signal capture stage, and the proportion of meaningful components in the voice signal can be enhanced, so that the subsequent voice processing module no longer relies on aggressive noise reduction algorithms, thereby retaining more original voice features. By summing the squares of the signal amplitude within the frame, not only the local intensity information of the signal is captured, but also the details of the energy fluctuations in the signal time series are retained. The dynamic threshold strategy is used to improve the accuracy of the division between voice segments and silence segments, so that the system still has robust segmentation capabilities under complex backgrounds.

[0066] S2: Extract the frequency domain signal from the aggregate signal, calculate the gain through the filter to reduce the noise of the frequency domain signal, calculate the zero-crossing rate and the short-time energy of the noise reduction to screen the valid speech frames, perform peak detection and calculate the average speaking rate for speech rate normalization, extract the signal value of the sampling point in the frame, calculate the power spectrum and perform filtering analysis on the feature sequence, calculate the conditional probability of the subwords in the BPE word segmentation table, and select the subword sequence with the highest probability;

[0067] Preferably, the signal values of the sampling points in the frame are extracted, the power spectrum is calculated and the characteristic sequence is filtered and analyzed, and the conditional probability of the subwords in the BPE word segmentation table is calculated, including:

[0068] Perform fast Fourier transform (FFT) on each frame of speech signal to obtain the frequency domain signal, and calculate the noise power spectrum density for the silent segment set and non-silent segment frame, which is expressed as:

[0069]

[0070] Among them S nn (ω) represents the value of the noise power spectrum density at frequency ω, M represents the total number of silent segment frames, X j (ω) represents the frequency domain signal of the jth silence frame, U represents the set of silence segments, S xx (ω) represents the value of the power spectrum density of the speech signal at frequency ω, K represents the total number of speech segment frames, X k (ω) represents the frequency domain signal of the kth speech frame;

[0071] For each frequency point of each frame, the gain is calculated according to the noise power spectral density through the Wiener filter, and the gain of the frequency domain signal of each frame is adjusted to obtain the denoised frequency domain signal. The inverse Fourier transform is further performed to obtain the time domain denoised signal, which is expressed as:

[0072]

[0073] Y k (ω)=H k (ω)·X k (ω);

[0074] Among them H k (ω) represents the Wiener filter gain of the kth frame at frequency ω, Y k (ω) represents the frequency domain signal of the kth frame at frequency ω after denoising;

[0075] Re-frame the denoised signal and calculate the denoised short-time energy of each frame, and calculate the zero-crossing rate of each frame, which is expressed as:

[0076]

[0077] where Z k represents the zero-crossing rate of the k-th frame, II(.) represents the indicator function, which is 1 if the condition in the brackets is met, otherwise it is 0, y[kN+i] and y[kN+i-1] represent the amplitudes of the denoised signal at the i-th and i-1-th sampling points in the k-th frame, respectively, and N represents the total number of samples;

[0078] A noise reduction threshold is set based on the sum of the mean and twice the standard deviation of the historical noise reduction short-time energy data, and a zero-crossing rate threshold is set based on the sum of the mean and twice the standard deviation of the historical zero-crossing rate data. A comparison is performed based on each frame of the noise reduction signal. If the noise reduction short-time energy of the noise reduction signal is greater than or equal to the noise reduction threshold, and the zero-crossing rate of the noise reduction signal is greater than or equal to the zero-crossing rate threshold, the corresponding frame is determined to be a valid speech frame, and the output is an index set of valid speech frames;

[0079] The peak value of the short-time energy of noise reduction is detected for the valid speech frame, the number of syllables S is counted, and the effective speech duration is calculated. The average speaking speed is determined based on the ratio of the number of syllables to the effective speech duration, which is expressed as:

[0080]

[0081] Where T represents the effective speech duration, f s Indicates the sampling rate, N v represents the total number of sampling points of the noise reduction signal of the valid speech frame, and R represents the evaluation speech rate;

[0082] The standard speaking speed is determined based on historical experience. If the average speaking speed is higher than the standard speed, the noise reduction signal of the valid speech frame is time-stretched using the WSOLA algorithm to normalize the speaking speed.

[0083] For the effective speech signal after speech rate normalization, the frame length and frame shift are set to calculate the total number of frames. For each frame, the signal value of the sampling point within the frame is extracted, and the signal is multiplied by the Hamming window to perform fast Fourier transform to obtain the frequency domain signal. The power spectrum is calculated and filtered through the Mel filter bank to obtain the Mel energy, which is expressed as:

[0084]

[0085] y vn,t[i] = y vn,t [t·N s +i];

[0086] s t [i] = y vn,t [i]·ω[i];

[0087]

[0088] Where T z represents the total number of frames, N vn represents the total number of signal sampling points, N f represents the number of sampling points in the frame length, N s represents the number of sampling points in the frame shift, y vn,t [i] represents the signal value of the i-th sampling point in the t-th frame, s t [i] represents the t-th frame signal after windowing for the i-th sampling point, ω[i] represents the Hamming window for the i-th sampling point, and the window length is N f , P t [k] represents the power spectrum value of the k-th frequency point in the t-th frame, S t [k] represents the complex frequency spectrum value of the k-th frequency point in the t-th frame, E t [m] represents the output energy of the m-th Mel filter in the t-th frame, H m [k] represents the response value of the m-th Mel filter at the k-th frequency point;

[0089] Take the logarithm of each Mel energy, perform discrete cosine transform on the obtained logarithmic Mel energy to obtain the MFCC feature vector, and form the feature sequence X, denoted as;

[0090]

[0091] Where x t [p] represents the p-th dimensional MFCC feature in the t-th frame, M r represents the total number of filters, L t [m] represents the logarithmic energy of the m-th Mel filter in the t-th frame, and p represents the number of retained MFCC dimensions;

[0092] Adopt a pre-trained end-to-end subword-level deep neural network ASR model. According to the input of the feature sequence, output the conditional probability of the subwords in the BPE segmentation table, and obtain the probability of the overall output sequence, denoted as:

[0093]

[0094] Where P(Y|X) represents the probability of the subword sequence Y being output from the feature sequence X, L represents the total number of subwords, y j represents the j-th subword unit, and y<j represents all the subwords before the position j;

[0095] The beam search algorithm is used to select the subword sequence with the highest probability from the probability distribution output by the model as the final ASR recognition result.

[0096] By performing FFT on the silent segments to extract the noise power spectral density, a frequency domain model of the background noise is effectively established. Subsequent filters can precisely act on the actual frequency band interference areas, avoiding weakening the main speech signal spectrum. This prevents the subsequent Wiener filter from removing all low-energy frequency components in a one-size-fits-all manner, thereby ensuring the integrity and intelligibility of the speech. By dynamically determining the gain of each frequency point in each frame based on the noise power spectrum, the speech spectrum is directly gain-controlled in the frequency domain, enhancing the speech signal-retaining area (high signal-to-noise ratio) while attenuating the noise-dominated area (low signal-to-noise ratio).

[0097] By counting the total number of sampling points and syllables in a valid speech interval, the speech rate is accurately estimated. WSOLA is then used to perform a nonlinear time-scale transformation on fast-speaking audio. This process normalizes the speech rhythm without changing the pitch, resolving the acoustic model mismatch problem caused by high speech rates, thereby improving the speech recognition system's ability to universally recognize speakers of different speaking rates.

[0098] The power spectrum is calculated by FFT after frame windowing and then connected to the Mel filter bank. This can redistribute the energy on the logarithmic Mel scale based on the human ear perception characteristics, compress the redundant dimensions of the speech spectrum, and enhance the discrimination of the main frequency band of the speech, forming a nonlinear spectral feature space that meets the requirements of speech model recognition.

[0099] The filter energy is logarithmized and then subjected to a discrete cosine transform (DCT) to convert from the energy domain to the cepstrum domain. This orthogonalizes the spectral correlation and compresses the features, ultimately forming an MFCC sequence. This process effectively removes inter-frame redundancy, retains the dynamic envelope information and acoustic feature trajectory that are most critical for speech recognition, and provides a well-structured input representation for neural network modeling.

[0100] By retaining multiple maximum probability branches in the candidate path space, beam search avoids the risk of greedy strategies falling into local optimality and effectively integrates the output probabilities of the language model and the acoustic model. This allows the most likely subword sequence to be selected, greatly improving the robustness and accuracy of the overall ASR system, especially for long or polysyllabic words.

[0101] S3 performs semantic recognition on the subword sequence, outputs the corresponding structured instructions, maps them to the DSL commands, performs verification and correction, obtains the output instructions, structures the user-uploaded data through the graph neural network, calculates the goodness of fit of the trend function, and determines the optimal trend;

[0102] Preferably, the corresponding structured instructions are output and mapped to the DSL commands, including:

[0103] Preprocessing the subword sequence includes removing non-linguistic symbols and using the pre-trained BERT large language model for semantic information recognition. Based on the pre-defined labels of the large language model, the maximum likelihood estimation (MLE) is used to determine the probability that the user's intention is the pre-defined label. Based on the results of intent recognition, the model outputs the corresponding structured instructions, which are expressed as follows:

[0104]

[0105] in represents the maximum likelihood estimate, P(L i |X r ) means that given input X r Under the condition of L i probability;

[0106] Each instruction is represented as a set of standardized graph commands. The syntax rules of DSL commands are defined using BNF Backus-Naur Form. The structured instructions are mapped to DSL commands to generate a standardized graph command set.

[0107] A cyclic consistency check mechanism is introduced to perform consistency check between the current instruction and the previous instruction. The similarity value between the current instruction set and the previous instruction set is calculated, which is expressed as:

[0108]

[0109] Where S(X r ,X re ) indicates the current instruction set X r Compared with the previous instruction set X re The cosine similarity value of Indicates the hth instruction;

[0110] Sub-supervised learning method is used for feedback correction to obtain the final output instructions.

[0111] By removing non-verbal symbols, it significantly reduces noise and irrelevant information, ensuring that the model focuses only on valid language content. This prevents non-verbal symbols or punctuation from confusing speech or text input, helping the system focus on the user's actual needs. BERT's pre-training capabilities enable it to accurately extract semantic information from context, especially in complex conversations or multi-round interactions. It can effectively capture the deeper meaning of user utterances, going beyond simple word matching.

[0112] The probability of user intent is calculated through maximum likelihood estimation (MLE). Based on the textual content expressed by the user, this method assigns a corresponding probability to each possible intent, thereby providing a quantitative judgment of user intent. When generating structured instructions, each instruction is mapped according to the DSL command syntax rules defined by standardized diagram commands and the BNF Backus-Naur Form. This ensures the consistency and standardization of the system's instruction output, effectively reduces instruction parsing errors, and prevents the impact of grammatical and logical errors on diagram generation.

[0113] The introduction of a cyclic consistency check mechanism ensures that each generated command is consistent with the previous interaction command. This check not only improves the consistency and coherence of the system, but also effectively prevents deviations or errors in commands caused by multiple rounds of user interaction. The feedback correction mechanism of sub-supervised learning further enhances the flexibility and adaptability of command output. During user interaction, the system can continuously adjust its output based on feedback, optimize recognition results, and ultimately provide a final command that better meets user needs.

[0114] Furthermore, the goodness of fit of the trend function is calculated to determine the optimal trend, including:

[0115] Parse the chart data uploaded by the user, use the graph neural network (GNN) to treat data types (such as data points, axis labels, legends) as nodes, set edges based on the relationship between data attributes, and structure the data table, which can be expressed as follows:

[0116] h i =GNN(v i , E);

[0117] where v i represents the i-th node, E represents the adjacency matrix, which is used to represent the relationship between nodes (such as the relationship between the coordinate axis and the data point), h i represents the embedding representation of the i-th node and extracts structured data;

[0118] Obtain a set of trend functions based on the database, calculate the goodness of fit of the trend functions based on the structured data, and select the trend function with the maximum goodness of fit as the optimal trend, which is expressed as:

[0119]

[0120] Where S(f k (D)) represents the structured data set D under the trend function f k The mean square error indicates the goodness of fit of the model to the data, V represents the total number of structural data points, and f k (d i ) represents the trend function f k For the structural data point di The predicted value of .

[0121] By converting chart data into a graph neural network input structure, setting the elements in the graph (such as data points, axis labels, legend labels, etc.) as nodes of the graph, and constructing edge relationships through their spatial or semantic dependencies in the graph, the model can capture the complex relationships between multidimensional elements in a topological structure. Compared with traditional flat feature extraction methods, this graph structure can effectively retain the structural contextual information implicit in the graph, improve the integrity and accuracy of the conversion from graph data to table data, and avoid the loss of key relationships during the parsing process.

[0122] During the propagation process of graph neural networks, each node exchanges features with adjacent nodes through the edges defined in the adjacency matrix. The model can adaptively aggregate information from adjacent entities to generate an embedded representation of each graph element. The use of a set of trend functions predefined from the database can avoid the problem of underfitting or overfitting when the model is autonomously fitted, ensuring the interpretability and versatility of the trend function. Using the mean square error (MSE) as a consistent standard can not only accurately quantify the degree of fit of the function to the data trend, but also compare and optimize functions through a unified error value. By selecting the trend function with the smallest MSE as the fitting expression of the data trend, the optimal trend description can be automatically obtained after the data is completely restored, avoiding subjective judgment bias in the process of manual intervention, and providing stable input for subsequent data visualization redrawing, predictive analysis or interactive reasoning, forming a closed-loop automated analysis process from chart image input to interpretable trend output.

[0123] S4 generates dynamic charts and provides text and voice explanations, encrypts chart data and backs it up in the cloud;

[0124] Preferably, generating dynamic charts and performing text and voice explanations includes:

[0125] Select the chart type based on the determination of the optimal trend function, draw the trend line in the chart according to the output instructions, and use the Matplotlib tool to generate dynamic charts;

[0126] Based on the changes in data points in the chart, including trends and extreme points, text-to-speech (TTS) method is used to provide text-to-speech explanations.

[0127] By logically inferring chart types based on trend functions, chart displays can better reflect the behavioral characteristics of the data itself, improving users' intuitive understanding of information while avoiding misinterpretation or information ambiguity caused by improper chart type selection. Trend line visualization not only supplements the trend clues of the original data points, but also provides users with clear pattern perception when browsing charts, significantly improving the chart's support for trend analysis. By synthesizing audio from structured descriptive text through a TTS system, not only does it enhance accessibility, allowing visually impaired users to receive chart information, but it also strengthens information memorability and reception efficiency through voice commentary.

[0128] Furthermore, the chart data is encrypted and backed up in the cloud, including:

[0129] The generated chart data is encrypted and stored using the symmetric encryption algorithm AES, a key generation function is used to generate the key, and the data is transmitted to the cloud for data backup through wireless transmission technology, and the storage is performed through key-value mapping.

[0130] The generated chart data is encrypted using the symmetric AES encryption algorithm. While ensuring efficient encryption and decryption, a block cipher structure ensures tamper resistance and confidentiality during data transmission and storage. Cloud storage is implemented using a key-value mapping structure. Each piece of chart data generates a unique identifier (such as a hash value or UUID) based on its metadata. This identifier serves as the key and is mapped to the ciphertext chart data as the value. This structure supports high-concurrency and fast retrieval, simplifying subsequent tasks such as user data synchronization, cross-end loading, and version management. It also avoids index conflicts and data redundancy issues associated with traditional directory-based structures, allowing for efficient storage and management of data even in an encrypted state.

[0131] This embodiment also provides a system for dynamically generating speech interactive charts based on a large language model, including:

[0132] The voice processing module uses a high signal-to-noise ratio microphone array to collect user voice, divides the original voice signal into frames, and calculates the short-time energy of each frame;

[0133] The voice activity detection module reduces noise in the frequency domain of the speech segment based on the noise power spectrum density, performs an inverse transform to restore the time domain signal, calculates the zero-crossing rate and noise reduction short-time energy of each frame, and determines the valid speech frame;

[0134] The speech rate normalization module performs peak detection on valid speech frames, calculates the average speech rate based on the number of syllables and the length of valid frames, and uses the WSOLA algorithm to perform time normalization if the speech rate is abnormal.

[0135] The speech recognition module frames the normalized speech signal, extracts the Mel-frequency cepstral coefficients, uses an end-to-end ASR model to obtain the BPE subword sequence and its conditional probability, and selects the output sequence with the maximum probability.

[0136] The instruction generation module cleans subword sequences, uses the BERT model for semantic recognition, maps the identified structured intent to DSL chart commands, and verifies and corrects the instructions.

[0137] The trend modeling module uses a graph neural network (GNN) to structure the chart data uploaded by users, obtain a set of trend functions, calculate the fitting error of each trend function, and select the optimal trend model;

[0138] The chart display module parses DSL commands to draw charts, draws trend lines based on trend functions, and uses them to realize dynamic playback of charts, synchronizing chart dynamics with data trend changes;

[0139] The intelligent voice explanation module analyzes the key points in the chart, automatically generates corresponding explanation text, and uses TTS technology to synthesize text into speech;

[0140] The cloud storage module encrypts dynamic chart data and completes chart data backup in the cloud through a key-value mapping structure.

[0141] This embodiment also provides a computer device, which is suitable for the case of a method for dynamically generating voice interactive charts based on a large language model, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute computer-executable instructions to implement the method for dynamically generating voice interactive charts based on a large language model as proposed in the above embodiment.

[0142] The computer device may be a terminal, comprising a processor, a memory, a communication interface, a display screen and an input device connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner may be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. The display screen of the computer device may be a liquid crystal display or an electronic ink display screen, and the input device of the computer device may be a touch layer covering the display screen, or a button, trackball or touchpad provided on the housing of the computer device, or an external keyboard, touchpad or mouse.

[0143] This embodiment also provides a storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for dynamically generating a speech interactive chart based on a large language model as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0144] In summary, the present invention effectively establishes a frequency domain model of background noise by extracting the noise power spectral density through FFT on the silent segment. The subsequent filter can accurately act on the actual frequency band interference area to avoid weakening the main speech signal spectrum. The power spectrum is calculated by FFT after frame windowing and then connected to the Mel filter bank. The energy can be redistributed on the logarithmic Mel scale according to the human ear perception characteristics. The probability of user intention is calculated through maximum likelihood estimation. Based on the text content expressed by the user, this method can assign a corresponding probability to each possible intention, thereby providing a quantitative judgment of the user's intention. Using mean square error as the consistency standard, it can not only accurately quantify the degree of fit of the function to the data trend, but also compare and optimize functions through a unified error value. The structured description text is used for audio synthesis through the TTS system, which not only enhances accessibility, allowing visually impaired users to receive chart information, but also enhances the memorability and reception efficiency of information through voice interpretation.

[0145] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A method for dynamically generating speech interactive graphs based on a large language model, characterized in that: include: Collect user voice signals, perform preliminary frame division and calculate short-time energy, and analyze the aggregate signal of silent segments; Extract the frequency domain signal from the collective signal, calculate the gain through the filter to reduce the noise of the frequency domain signal, calculate the zero-crossing rate and the short-time energy of the noise reduction to screen the valid speech frames, perform peak detection and calculate the average speaking rate for speech rate normalization, extract the signal value of the sampling point in the frame, calculate the power spectrum and perform filtering analysis on the feature sequence, calculate the conditional probability of the subwords in the BPE word segmentation table, and select the subword sequence with the highest probability; Perform semantic recognition on subword sequences, output corresponding structured instructions, map them to DSL commands, perform verification and correction, obtain output instructions, structure user-uploaded data through graph neural networks, calculate the goodness of fit of trend functions, and determine the optimal trend; Generate dynamic charts and provide text and voice explanations, encrypt chart data and back it up in the cloud.

2. The method for dynamically generating a speech interactive chart based on a large language model according to claim 1, wherein: The extraction of signal values of sampling points within the frame, calculation of power spectrum and filtering analysis of feature sequences, and calculation of conditional probabilities of subwords in the BPE word segmentation table include: Perform Fast Fourier Transform (FFT) on each frame of speech signal to obtain the frequency domain signal, and calculate the noise power spectrum density for the silent segment set and non-silent segment frames; For each frequency point of each frame, the gain is calculated according to the noise power spectral density through the Wiener filter, and the gain of the frequency domain signal of each frame is adjusted to obtain the denoised frequency domain signal, which is further inverse Fourier transformed to obtain the time domain denoised signal; Re-frame the noise reduction signal and calculate the noise reduction short-time energy of each frame, and calculate the zero-crossing rate of each frame; A noise reduction threshold is set based on the sum of the mean and twice the standard deviation of the historical noise reduction short-time energy data, and a zero-crossing rate threshold is set based on the sum of the mean and twice the standard deviation of the historical zero-crossing rate data. A comparison is performed based on each frame of the noise reduction signal. If the noise reduction short-time energy of the noise reduction signal is greater than or equal to the noise reduction threshold, and the zero-crossing rate of the noise reduction signal is greater than or equal to the zero-crossing rate threshold, the corresponding frame is determined to be a valid speech frame, and the output is an index set of valid speech frames; Perform peak detection of short-time energy of noise reduction on valid speech frames, count the number of syllables S, calculate the effective speech duration, and determine the average speaking speed based on the ratio of the number of syllables to the effective speech duration; The standard speaking speed is determined based on historical experience. If the average speaking speed is higher than the standard speed, the noise reduction signal of the valid speech frame is time-stretched using the WSOLA algorithm to normalize the speaking speed. For the effective speech signal after speech rate normalization, the frame length and frame shift are set to calculate the total number of frames. For each frame, the signal value of the sampling point within the frame is extracted, multiplied by the Hamming window, and fast Fourier transform is performed to obtain the frequency domain signal. The power spectrum is calculated and filtered through the Mel filter bank to obtain the Mel energy. Take the logarithm of each Mel energy, obtain the logarithmic Mel energy, perform discrete cosine transform, and obtain the MFCC feature vector to form the feature sequence X; A pre-trained end-to-end word-level deep neural network ASR model is used to output the conditional probability of the subwords in the BPE word segmentation table based on the input of the feature sequence, and obtain the probability of the entire output sequence; The beam search algorithm is used to select the subword sequence with the highest probability from the probability distribution output by the model as the final ASR recognition result.

3. The method for dynamically generating a speech interactive chart based on a large language model according to claim 2, wherein: The structured instructions corresponding to the output are mapped to DSL commands, including: Preprocessing of subword sequences involves removing non-linguistic symbols and using the pre-trained BERT large language model for semantic information recognition. Based on the pre-defined labels of the large language model, maximum likelihood estimation (MLE) is used to determine the probability that the user's intention is the pre-defined label. Based on the results of intent recognition, the model outputs the corresponding structured instructions. Each instruction is represented as a set of standardized graph commands. The syntax rules of DSL commands are defined using BNF Backus-Naur Form. The structured instructions are mapped to DSL commands to generate a standardized graph command set. A cyclic consistency check mechanism is introduced to perform consistency check between the current instruction and the previous instruction by calculating the similarity value between the current instruction set and the previous instruction set; Sub-supervised learning method is used for feedback correction to obtain the final output instructions.

4. The method for dynamically generating a speech interactive chart based on a large language model according to claim 3, wherein: The calculation of the goodness of fit of the trend function and determination of the optimal trend include: Parse the chart data uploaded by users, use the graph neural network (GNN) to treat data types as nodes, set edges based on the relationship between data attributes, and structure the data table; A trend function set is obtained based on a database, and the goodness of fit of the trend function is calculated based on structured data, and the trend function with the maximum goodness of fit is selected as the optimal trend.

5. The method for dynamically generating a speech interactive chart based on a large language model according to claim 4, wherein: The generation of dynamic charts and text and voice explanations include: Select the chart type based on the determination of the optimal trend function, draw the trend line in the chart according to the output instructions, and use the Matplotlib tool to generate dynamic charts; Based on the changes in data points in the chart, including trends and extreme points, text-to-speech (TTS) method is used to provide text-to-speech explanations.

6. The method for dynamically generating a speech interactive chart based on a large language model according to claim 5, wherein: The user voice signal is collected and preliminarily framed and short-time energy is calculated, and the silent segment aggregate signal is analyzed. include, Use a high signal-to-noise ratio microphone array to collect speech signals and form an original speech signal sequence. Then divide the original speech signal sequence into frames and calculate the short-time energy of each frame. By collecting the short-time energy mean plus twice the standard deviation of the silent segment as the energy threshold, the short-time energy frames less than or equal to the energy threshold are regarded as silent segments, and the short-time energy frames greater than the energy threshold are regarded as speech segments, and the silent segment set is formed according to the frame index of the silent segment.

7. The method for dynamically generating a speech interactive chart based on a large language model according to claim 6, wherein: The encrypted chart data and cloud backup storage include: The generated chart data is encrypted and stored using the symmetric encryption algorithm AES, a key generation function is used to generate the key, and the data is transmitted to the cloud for data backup through wireless transmission technology, and the storage is performed through key-value mapping.

8. A system for dynamically generating interactive speech charts based on a large language model, based on the method for dynamically generating interactive speech charts based on a large language model according to any one of claims 1 to 7, characterized in that: include, The voice processing module uses a high signal-to-noise ratio microphone array to collect user voice, divides the original voice signal into frames, and calculates the short-time energy of each frame; The voice activity detection module reduces noise in the frequency domain of the speech segment based on the noise power spectrum density, performs an inverse transform to restore the time domain signal, calculates the zero-crossing rate and noise reduction short-time energy of each frame, and determines the valid speech frame; The speech rate normalization module performs peak detection on valid speech frames, calculates the average speech rate based on the number of syllables and the length of valid frames, and uses the WSOLA algorithm to perform time normalization if the speech rate is abnormal. The speech recognition module frames the normalized speech signal, extracts the Mel-frequency cepstral coefficients, uses an end-to-end ASR model to obtain the BPE subword sequence and its conditional probability, and selects the output sequence with the maximum probability. The instruction generation module cleans subword sequences, uses the BERT model for semantic recognition, maps the identified structured intent to DSL chart commands, and verifies and corrects the instructions. The trend modeling module uses a graph neural network (GNN) to structure the chart data uploaded by users, obtain a set of trend functions, calculate the fitting error of each trend function, and select the optimal trend model; The chart display module parses DSL commands to draw charts, draws trend lines based on trend functions, and uses them to realize dynamic playback of charts, synchronizing chart dynamics with data trend changes; The intelligent voice explanation module analyzes the key points in the chart, automatically generates corresponding explanation text, and uses TTS technology to synthesize text into speech; The cloud storage module encrypts dynamic chart data and completes chart data backup in the cloud through a key-value mapping structure.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method for dynamically generating a speech interactive chart based on a large language model according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for dynamically generating a speech interactive chart based on a large language model according to any one of claims 1 to 7 are implemented.

Images