Artificial intelligence based natural language interactive data generation method and system

Abstract

The application discloses a natural language interactive data generation method and system based on artificial intelligence, and specifically comprises the following steps: receiving a user voice audio stream; performing spectrum transformation and noise suppression on the voice audio stream to generate a denoised mel spectrum feature; performing incremental fine-tuning on a Whisper speech recognition model by using a business scenario voice corpus, and performing speech transcription on the denoised mel spectrum feature to output a transcribed text sequence; performing feature fusion on the transcribed text sequence and the denoised mel spectrum feature to generate a fusion feature vector; generating a set of structured information units based on the fusion feature vector; generating a form filling result according to the set of structured information units and a business scenario identifier; performing PCard conflict discrimination and projection correction in the model training stage to complete model parameter updating; and performing verification on the form filling result to generate standardized form data. The application combines adaptive noise reduction, business fine-tuning and gradient conflict resolution to realize high-precision speech-driven structured data generation.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence voice interaction data generation technology, and in particular to a method and system for generating natural language interactive data based on artificial intelligence. Background Technology

[0002] With the development of artificial intelligence technology, voice interaction has gradually become one of the important methods for information collection, business processing, and human-computer interaction. In business scenarios such as service ordering, information reporting, and data collection, users typically use natural language to express themselves, and the system completes speech recognition, information parsing, and data entry. However, existing voice interaction systems still face many technical limitations in practical applications.

[0003] On the one hand, user speech is typically characterized by a high degree of colloquialism, inconsistent expression order, inclusion of redundant information, and interference from environmental noise. General speech recognition models are prone to transcription errors in complex business scenarios, which in turn affects the accuracy of subsequent information processing. On the other hand, the transcribed text often lacks clear structural identifiers, and the correspondence between field names and attribute values ​​is implicit in natural language. Existing methods mostly rely on fixed rules or manual configuration for parsing, resulting in limited adaptability.

[0004] Furthermore, in multi-task collaborative processing, tasks such as speech recognition, information extraction, and field mapping typically employ independent modeling or simple concatenation, making it difficult to fully share intermediate representations. This leads to gradient conflicts in model parameters during joint training, affecting model convergence stability and training efficiency. Additionally, existing systems have limited ability to verify data consistency and integrity during form generation, still requiring manual correction.

[0005] Therefore, how to achieve robust speech processing, structured information generation, and multi-task collaborative optimization in complex business speech scenarios remains a pressing technical problem to be solved in the field of natural language interactive data generation. Summary of the Invention

[0006] One objective of this invention is to propose a method and system for generating interactive natural language data based on artificial intelligence. This invention introduces a speech recognition processing mechanism that combines adaptive noise suppression with incremental fine-tuning based on business scenarios. In the multi-task joint training process, a PCard conflict resolution calculation model based on curvature preconditions is introduced to perform collaborative optimization on model parameter updates in the speech transcription, information extraction, and field mapping processes. This constructs an end-to-end interactive natural language data generation process, which has the advantages of strong robustness in speech processing, high consistency in information structure, and good model training stability.

[0007] The artificial intelligence-based natural language interactive data generation method and system according to embodiments of the present invention includes the following steps: Step 1: Receive user voice audio stream; Step 2: Perform short-time Fourier transform on the voice audio stream to obtain spectral features, input the spectral features into the adaptive noise suppression module to generate a noise mask and complete spectral filtering, obtaining denoised Mel spectral features; Collect user voice audio streams generated during system operation, form a business scenario voice corpus according to business scenario identifiers, construct a Whisper speech recognition model and perform incremental fine-tuning, input the denoised Mel spectral features into the incrementally fine-tuned Whisper speech recognition model and output a transcribed text sequence; Step 3: Perform feature fusion on the transcribed text sequence and the denoised Mel spectral features to obtain a fused feature vector; Step 4: Generate a set of structured information units based on the fused feature vector; Step 5: Generate form filling results based on the set of structured information units and business scenario identifiers; Step 6: During the model training phase, construct a multi-task joint model with the Whisper speech recognition model as the backbone, perform PCard conflict discrimination and projection correction operations in the preconditioned gradient space and update the model parameters; Step 7: Validate the form filling results and generate standardized form data.

[0008] Optionally, step 2, inputting the spectral features into the adaptive noise suppression module to generate a noise mask, specifically includes: The audio stream is segmented into frames according to a preset frame length, and windowing is performed on each frame to form a sequence of audio frames. A short-time Fourier transform is performed on the audio frame sequence to generate a complex spectrum matrix. An amplitude spectrum matrix is ​​extracted from the complex spectrum matrix to form spectral features. These spectral features are input sequentially into an adaptive noise suppression module containing a gated recurrent unit. The gated recurrent unit performs a recursive operation along the time steps under the initial hidden state vector condition to generate a sequence of hidden state vectors. A noise mask mapping weight matrix and a noise mask mapping bias vector are input to each hidden state vector in the hidden state vector sequence to generate an intermediate mask vector. A sigmoid operation is performed on the intermediate mask vector to obtain a noise mask matrix. A smoothing operation is performed on the noise mask matrix along the frequency band dimension to form a smoothed noise mask matrix. A numerical clipping operation is performed on the smoothed noise mask matrix to obtain a clipped noise mask matrix. Element-wise multiplication is performed on the amplitude spectrum matrix and the clipped noise mask matrix to generate a filtered amplitude spectrum matrix. The filtered amplitude spectrum matrix is ​​input to the Mel filter bank and a weighted operation is performed to generate the Mel band spectrum matrix. The logarithmic operation is then performed on the Mel band spectrum matrix to obtain the denoised Mel spectrum features.

[0009] Optionally, the construction of the Whisper speech recognition model in step 2 includes the following steps: The process involves: 1) Reading denoised Mel-spectral features and corresponding transcribed text annotations from a business scenario speech corpus, generating a training sample pair set based on sample correspondence; 2) Constructing the model structure parameter set of the Whisper speech recognition model, which includes encoder parameter set, decoder parameter set, general corpus adaptation parameter set, and business-specific vocabulary embedding parameter set; 3) Performing parameter state initialization on the model structure parameter set, keeping the encoder and decoder parameter sets updatable while keeping the general corpus adaptation parameter set frozen; 4) Inputting the denoised Mel-spectral features from the training sample pair set into the encoder of the Whisper speech recognition model in time-step order to generate the encoder output feature sequence; 5) Inputting the encoder output feature sequence into the decoder of the Whisper speech recognition model, shielding the general corpus adaptation parameter set from the decoding calculation, and only introducing the business-specific vocabulary embedding parameter set to participate in the decoding calculation to generate a predicted text sequence; 6) Performing loss calculation on the predicted text sequence and corresponding transcribed text annotations, and performing parameter update operations on the encoder parameter set, decoder parameter set, and business-specific vocabulary embedding parameter set to form the completed Whisper speech recognition model.

[0010] Optionally, the feature fusion performed on the transcribed text sequence and the denoised Mel-spectral features in step 3 specifically includes: The process involves: reading the transcribed text sequence output by the Whisper speech recognition model and mapping it to a text tag sequence based on word order; inputting the text tag sequence into a text semantic encoding model to obtain a text feature vector sequence corresponding to each text tag sequence; reading the denoised Mel-frequency spectroscopic features and performing pooling operations along the time dimension to obtain a speech feature vector that matches the text feature vector sequence in the feature dimension; concatenating the text feature vector sequence and the speech feature vector along the feature dimension to generate a concatenated feature vector sequence; inputting the concatenated feature vector sequence into a channel attention computation structure to generate corresponding feature weight vectors; and performing a channel-wise weighted operation on the concatenated feature vector sequence and the feature weight vector to output a fused feature vector. Optionally, step 4 includes the following steps: Read the fused feature vector and expand it into a feature sequence in chronological order; construct a field label parsing calculation structure, which includes position mapping weight parameters and label mapping bias parameters; input the feature sequence into the field label parsing calculation structure according to its chronological position, and perform field name label calculation on each position in the feature sequence to generate a field name label sequence; during the field name label calculation process, simultaneously perform attribute value calculation on the positions corresponding to the field name labels to generate an attribute value sequence that corresponds one-to-one with the field name; read the calculated score value for each corresponding position in the field name label sequence and attribute value sequence, and perform a normalization operation on the calculated score value to generate the corresponding confidence value; construct structured information units based on the correspondence between field name, attribute value, and confidence, and write all structured information units into a structured information unit set.

[0011] Optionally, step 5 specifically includes: Read the set of structured information units and the corresponding business scenario identifiers; construct a set of field mapping rules based on the business scenario identifiers, where each rule item contains a correspondence between a field name identifier and a form field identifier; input the field names from the set of structured information units into the set of field mapping rules sequentially, perform rule matching operations on each field name to obtain the corresponding form field identifier; establish a field filling correspondence between the matched form field identifiers and the attribute values ​​in the structured information units, generating a set of field filling pairs; generate a form field writing sequence based on the set of field filling pairs, and write the attribute values ​​into the corresponding form field positions according to the writing sequence to form the form filling result; write the generated form filling result into the form filling result cache.

[0012] Optionally, the construction of the multi-task joint model with the Whisper speech recognition model as the backbone in step 6 includes the following steps: A multi-task joint model with the Whisper speech recognition model as the backbone is constructed, integrating the speech transcription, information extraction, and field mapping tasks into the same backbone structure. Model parameters that participate in the computation of all three tasks are selected within the multi-task joint model, forming a shared parameter subset. Task loss functions corresponding to each of the three tasks are constructed separately, forming a multi-task loss function set. Backpropagation is performed on the multi-task loss function set within the same training batch to calculate the multi-task gradient vector set corresponding to the shared parameter subset, and this set is then written to the gradient buffer.

[0013] Optionally, step 6, which involves performing PCard conflict detection and projection correction operations and updating model parameters, includes the following steps: The process involves: 1) Reading the multi-task gradient vector sets recorded in the gradient buffer, where each gradient vector corresponds one-to-one with a shared parameter subset in the parameter dimension; 2) Performing statistical operations on the historical gradient information corresponding to the shared parameter subset, constructing a curvature preconditioning matrix based on the statistical results, and storing the curvature preconditioning matrix in diagonal or block-diagonal form according to the parameter dimension; 3) Inputting each multi-task gradient vector set into the curvature preconditioning matrix to perform preconditioning transformation operations, generating preconditioning gradient vector sets that correspond one-to-one with the multi-task gradient vector sets; 4) Selecting any two preconditioning gradient vectors in the preconditioning gradient space according to the task index order, performing inner product calculation on the selected preconditioning gradient vectors, and recording the inner product calculation result; 5) Performing a sign determination operation on the inner product calculation result, recording a directional conflict relationship between the corresponding task gradient pairs when the inner product result is negative; 6) For the preconditioning gradient vectors recorded as having a directional conflict relationship, selecting one as the corrected gradient vector and the other as the reference gradient vector. A vector projection operation is performed on the corrected gradient vector, projecting it onto the orthogonal subspace of the reference gradient vector to generate a corrected gradient vector after removing conflicting components. Projection correction operations are then performed sequentially on all preconditioned gradient vectors in the multi-task gradient vector group that have conflicting directions, forming a conflict-free preconditioned gradient vector group. Vector synthesis is then performed on the conflict-free preconditioned gradient vector group to generate a single collaboratively updated gradient vector. Based on this single collaboratively updated gradient vector, parameter update operations are performed on the shared parameter subset, and the updated parameters are written back to the Whisper speech recognition model.

[0014] The beneficial effects of this invention are: (1) In the speech processing stage, the present invention introduces an adaptive noise suppression and incremental fine-tuning mechanism for speech corpus in business scenarios, so that the speech recognition process can adapt to different environmental noise, colloquial expressions and differences in business terminology, reduce the impact of speech distortion and transcription deviation on subsequent processing, and ensure the stability and consistency of input data from the source.

[0015] (2) This invention directly generates structured information units containing field names, attribute values ​​and confidence levels by fusing and modeling speech-to-text and speech features. It also completes field mapping and form filling by combining business scenario identifiers, reducing intermediate manual intervention and rule dependence, and ensuring that the data generation process maintains a consistent processing logic and output format in different business scenarios.

[0016] (3) In the model training stage, the present invention constructs a multi-task joint structure with the speech recognition model as the backbone, introduces the PCard conflict discrimination and projection correction mechanism under curvature preconditions, coordinates the gradient update direction of the multi-task, reduces the impact of mutual interference between tasks on the evolution of model parameters, and thus ensures the continuity of the overall training process and the consistency of parameter updates. Attached Figure Description

[0017] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Fig. 1 This is a flowchart of the method and system for generating natural language interactive data based on artificial intelligence proposed in this invention; Fig. 2 This is a schematic diagram illustrating the adaptive noise reduction and business fine-tuning process of the artificial intelligence-based natural language interactive data generation method and system proposed in this invention. Fig. 3 This diagram illustrates the multi-task joint training and PCard optimization process of the artificial intelligence-based natural language interactive data generation method and system proposed in this invention. Detailed Implementation

[0018] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0019] refer to Figs. 1-3 A method and system for generating interactive natural language data based on artificial intelligence, comprising the following steps: Step 1: Receive user voice audio stream; Step 2: Perform short-time Fourier transform on the voice audio stream to obtain spectral features, input the spectral features into the adaptive noise suppression module to generate a noise mask and complete spectral filtering, obtaining denoised Mel spectral features; Collect user voice audio streams generated during system operation, form a business scenario voice corpus according to business scenario identifiers, construct a Whisper speech recognition model and perform incremental fine-tuning, input the denoised Mel spectral features into the incrementally fine-tuned Whisper speech recognition model and output a transcribed text sequence; Step 3: Perform feature fusion on the transcribed text sequence and the denoised Mel spectral features to obtain a fused feature vector; Step 4: Generate a set of structured information units based on the fused feature vector; Step 5: Generate form filling results based on the set of structured information units and business scenario identifiers; Step 6: During the model training phase, construct a multi-task joint model with the Whisper speech recognition model as the backbone, perform PCard conflict discrimination and projection correction operations in the preconditioned gradient space and update the model parameters; Step 7: Validate the form filling results and generate standardized form data.

[0020] In this embodiment, step 2, inputting the spectral features into the adaptive noise suppression module to generate a noise mask, specifically includes: The audio stream is segmented into frames according to a preset frame length, and windowing is performed on each frame to form a sequence of audio frames. A short-time Fourier transform is performed on the audio frame sequence to generate a complex spectrum matrix. An amplitude spectrum matrix is ​​extracted from the complex spectrum matrix to form spectral features. These spectral features are input sequentially into an adaptive noise suppression module containing a gated recurrent unit. The gated recurrent unit performs a recursive operation along the time steps under the initial hidden state vector condition to generate a sequence of hidden state vectors. A noise mask mapping weight matrix and a noise mask mapping bias vector are input to each hidden state vector in the hidden state vector sequence to generate an intermediate mask vector. A sigmoid operation is performed on the intermediate mask vector to obtain a noise mask matrix. A smoothing operation is performed on the noise mask matrix along the frequency band dimension to form a smoothed noise mask matrix. A numerical clipping operation is performed on the smoothed noise mask matrix to obtain a clipped noise mask matrix. Element-wise multiplication is performed on the amplitude spectrum matrix and the clipped noise mask matrix to generate a filtered amplitude spectrum matrix. The filtered amplitude spectrum matrix is ​​input to the Mel filter bank and a weighted operation is performed to generate the Mel band spectrum matrix. The logarithmic operation is then performed on the Mel band spectrum matrix to obtain the denoised Mel spectrum features.

[0021] In this embodiment, step 2, constructing the Whisper speech recognition model, includes the following steps: Read the denoised Mel-spectral features and corresponding transcribed text annotations from the business scenario speech corpus, and generate a training sample pair set according to the sample correspondence; Construct the model structure parameter set of the Whisper speech recognition model, which includes the encoder parameter set, decoder parameter set, general corpus adaptation parameter set, and business-specific vocabulary embedding parameter set; Perform parameter state initialization operation on the model structure parameter set, keeping the encoder and decoder parameter sets in an updatable state and keeping the general corpus adaptation parameter set in a frozen state; Before starting incremental fine-tuning training, assign explicit initial numerical states to the model parameters involved in the update, including loading the encoder and decoder parameter sets into pre-trained weight states, loading the business-specific vocabulary embedding parameter set into preset initial weight states, and fixing the general corpus adaptation parameter set into a non-updatable state. The denoised Mel-spectral features from the training sample pairs are input into the encoder of the Whisper speech recognition model in time-step order to generate the encoder output feature sequence. The encoder output feature sequence is then input into the decoder of the Whisper speech recognition model. During the decoding calculation, the general corpus adaptation parameter set is disabled, and only the business-specific vocabulary embedding parameter set is introduced to participate in the decoding calculation to generate the predicted text sequence. Loss calculation is performed on the predicted text sequence and the corresponding transcribed text annotations, and parameter update operations are performed on the encoder parameter set, decoder parameter set, and business-specific vocabulary embedding parameter set to form the completed Whisper speech recognition model.

[0022] In this embodiment, the freezing of the set in step S2 specifically includes: During the training and recognition process, the set of parameters adapted to the general corpus is excluded from the backpropagation calculation path, so that the set of parameters does not participate in gradient calculation or parameter update, and its value remains unchanged throughout the entire incremental fine-tuning process.

[0023] In this embodiment, the loss calculation operation in step 2 specifically includes: aligning the predicted text sequence with the transcribed text annotations according to word order; reading the predicted probability value of the corresponding tag from the decoder output for each word order position; calculating the negative log probability of the tag at that position and accumulating it across all word order positions to obtain the sequence-level loss value; and performing summation or mean operation on the sequence-level loss value with the training samples to obtain the batch loss value.

[0024] In this embodiment, step 3, which involves performing feature fusion on the transcribed text sequence and the denoised Mel-spectral features, specifically includes: The process involves: reading the transcribed text sequence output by the Whisper speech recognition model and mapping it to a text tag sequence based on word order; inputting the text tag sequence into a text semantic encoding model to obtain a text feature vector sequence corresponding to each text tag sequence; reading the denoised Mel-frequency spectroscopic features and performing pooling operations along the time dimension to obtain speech feature vectors that match the text feature vector sequence in the feature dimension; concatenating the text feature vector sequence and the speech feature vectors along the feature dimension to generate a concatenated feature vector sequence; inputting the concatenated feature vector sequence into a channel attention computation structure to generate corresponding feature weight vectors; and performing a channel-wise weighted operation on the concatenated feature vector sequence and the feature weight vectors to output a fused feature vector.

[0025] In this embodiment, step 4 includes the following steps: Read the fused feature vector and expand it into a feature sequence in chronological order. Construct a field label parsing calculation structure, which includes position mapping weight parameters and label mapping bias parameters. Input the feature sequence into the field label parsing calculation structure according to its time position, and perform field name label calculation on each position in the feature sequence to generate a field name label sequence. During the field name label calculation, perform attribute value calculation operations on the positions corresponding to the field name labels simultaneously to generate an attribute value sequence that corresponds one-to-one with the field name. The specific execution of the attribute value calculation operation includes: reading the field name label result at each time position in the field name label sequence and locating the time position index corresponding to the field name label result; reading the feature vector corresponding to the feature sequence from the same time position index; inputting the feature vector into the attribute value mapping weight parameters and attribute value mapping bias parameters to generate an intermediate attribute value vector; performing a numerical mapping operation on the intermediate attribute value vector to obtain the attribute value representation vector; writing the attribute value representation vector into the attribute value sequence according to the time position index, and establishing a one-to-one correspondence between the field name labels and the attribute value representation vector according to the time position index. Read the calculated score value for each corresponding position in the field name tag sequence and attribute value sequence, and perform a normalization operation on the calculated score value to generate the corresponding confidence value; construct structured information units based on the correspondence between field name, attribute value and confidence value, and write all structured information units into the structured information unit set.

[0026] In this embodiment, step 5 specifically includes: Read the set of structured information units and the corresponding business scenario identifiers; construct a set of field mapping rules based on the business scenario identifiers, where each rule item contains a correspondence between a field name identifier and a form field identifier; input the field names from the set of structured information units into the set of field mapping rules sequentially, perform rule matching operations on each field name to obtain the corresponding form field identifier; establish a field filling correspondence between the matched form field identifiers and the attribute values ​​in the structured information units, generating a set of field filling pairs; generate a form field writing sequence based on the set of field filling pairs, and write the attribute values ​​into the corresponding form field positions according to the writing sequence to form the form filling result; write the generated form filling result into the form filling result cache.

[0027] In this embodiment, step 6, constructing a multi-task joint model with the Whisper speech recognition model as the backbone, includes the following steps: A multi-task joint model with the Whisper speech recognition model as the backbone is constructed, integrating the speech transcription, information extraction, and field mapping tasks into the same backbone structure. Model parameters that participate in the computation of all three tasks are selected within the multi-task joint model, forming a shared parameter subset. Task loss functions corresponding to each of the three tasks are constructed separately, forming a multi-task loss function set. Backpropagation is performed on the multi-task loss function set within the same training batch to calculate the multi-task gradient vector set corresponding to the shared parameter subset, and this set is then written to the gradient buffer.

[0028] In this embodiment, step 6, which involves performing PCard conflict detection and projection correction calculations and updating model parameters, includes the following steps: Read the multi-task gradient vector set recorded in the gradient buffer. Each gradient vector in the multi-task gradient vector set corresponds one-to-one with a shared parameter subset in the parameter dimension. Perform a statistical summation operation on the historical gradient information corresponding to the shared parameter subset, and construct a curvature preconditioning matrix based on the statistical results. The curvature preconditioning matrix is ​​stored in diagonal or block diagonal form according to the parameter dimension. Input the multi-task gradient vector set one by one into the curvature preconditioning matrix to perform preconditioning transformation operation, generating a preconditioning gradient vector set that corresponds one-to-one with the multi-task gradient vector set. The specific preconditioning operation includes: For each task's gradient vector, the gradient components are read in the order of their parameter indices in the shared parameter subset; the curvature preconditioning matrix components that correspond one-to-one with the shared parameter subset in terms of parameter indices are read; for each gradient component in the gradient vector, a numerical scaling operation with the corresponding curvature preconditioning matrix component is performed to obtain the curvature-adjusted gradient component; all curvature-adjusted gradient components are reassembled in the original parameter index order to form the preconditioning gradient vector for the corresponding task; the above operations are performed sequentially on each gradient vector in the multi-task gradient vector group to obtain the preconditioning gradient vector group. In the preconditional gradient space, select any two preconditional gradient vectors in task index order. Perform inner product calculation on the selected preconditional gradient vectors and record the inner product calculation result. Perform sign determination operation on the inner product calculation result. When the inner product result is negative, record that the corresponding task gradient pair has a directional conflict relationship. For the preconditional gradient vectors recorded as having a directional conflict relationship, select one as the correction gradient vector and the other as the reference gradient vector. Perform vector projection operation on the correction gradient vector to project the correction gradient vector onto the orthogonal subspace of the reference gradient vector to generate a correction gradient vector after removing conflicting components. Perform projection correction operation on all preconditional gradient vectors with directional conflicts in the multi-task gradient vector group in sequence to form a conflict-free preconditional gradient vector group. Vector synthesis is performed on the conflict-free preconditioning gradient vector set to generate a single collaboratively updated gradient vector. Specifically, the vector synthesis operation includes: reading the preconditioning gradient vectors corresponding to each task in the conflict-free preconditioning gradient vector set according to parameter index order; performing numerical aggregation on the preconditioning gradient components from different tasks at each parameter index position to obtain the synthesized gradient component corresponding to that parameter index position; arranging the synthesized gradient components obtained at each parameter index position sequentially according to the original parameter index order to form a single collaboratively updated gradient vector; performing parameter update operations on the shared parameter subset based on the single collaboratively updated gradient vector, and writing the updated parameters back to the Whisper speech recognition model. The parameters include the model parameters corresponding to the shared parameter subset in the Whisper speech recognition model, which are updated numerically based on the single collaboratively updated gradient vector. The shared parameter subset includes the encoder parameter components, decoder parameter components, and their corresponding shared intermediate mapping parameter components that participate in forward computation and backward propagation in the speech transcription task, information extraction task, and field mapping task.

[0029] Example 1: To verify the feasibility and stability of this invention in practical applications, it was applied to a scenario of voice-interactive driven business data acquisition and structured generation. In this scenario, users express business needs and key information via voice, and the voice content simultaneously contains environmental noise, colloquial redundancy, and unstructured narration. Traditional voice processing methods are prone to problems such as incomplete transcription, omission of key information, and inconsistent form filling in such scenarios, affecting the continuity of business processing.

[0030] In actual operation, the system first receives continuous voice input, performs spectral analysis and adaptive noise suppression on the voice signal, and generates stable denoised Mel-ray spectral features. Then, it constructs a business scenario voice corpus based on voice samples accumulated during system operation, incrementally fine-tuning the voice recognition model to make it more focused on business-related vocabulary during the decoding stage. On this basis, the system simultaneously integrates voice features and text semantic features to generate a unified feature representation, and uses this to parse field names, attribute values, and corresponding confidence levels, completing the generation of structured information. Finally, the system automatically fills in and validates forms based on the structured information and business scenario rules.

[0031] In the above implementation process, the differences in performance of traditional processing methods and the method of the present invention on several key indicators were compared and statistically analyzed, and the following quantitative results were obtained.

[0032] Table 1: Comparison of Voice Interaction Data Generation Effects

[0033] Table 1 compares the quantitative differences between traditional voice interaction processing methods and the method of this invention in multiple dimensions, including speech transcription, information extraction, form generation, and model training stability, under the same business voice input conditions. The speech transcription completeness rate shows that the traditional method exhibits significant omissions of field-level text in the presence of environmental noise and colloquial expressions, maintaining a completeness rate of 86.4%. In contrast, this invention, after introducing adaptive noise suppression and incremental fine-tuning of the business corpus, improves the transcription completeness rate to 96.8%, indicating that the denoising Mel-spectral features and directional decoding strategy are more effective in preserving core speech information.

[0034] Regarding adaptability to noisy environments, traditional methods yield only 71.2% of effective speech frames for recognition, with numerous spectral regions affected by noise. This invention, through noise masking and frequency band smoothing, increases the effective speech frame rate to 92.5%, providing a more consistent input foundation for subsequent transcription and feature fusion. Furthermore, in terms of business field recognition and attribute value matching, traditional methods achieve field recognition accuracy of 82.1% and attribute value matching accuracy of 79.6%, while this invention, after fusing speech features and text semantic features, achieves these metrics of 95.3% and 94.1%, respectively, significantly enhancing the stability of structured information units.

[0035] Based on actual form generation results, the traditional method averages 2.4 missing fields per interaction, requiring multiple manual additions. In contrast, the method of this invention reduces the number of missing fields to 0.6, increases the form auto-fill completion rate from 68.9% to 93.7%, and achieves a 96.1% pass rate for logical validation on the first attempt, significantly reducing repetitive interactions and corrections. The consistency deviation across multiple runs decreased from ±12.6% to ±3.2%, indicating greater stability of the system's output during continuous operation.

[0036] At the model training level, the gradient conflict rate of shared parameters in traditional multi-task training reaches 31.8%, resulting in a large number of convergence rounds. This invention reduces the gradient conflict rate to 4.7% by performing PCard conflict discrimination and projection correction in the precondition gradient space. The number of rounds required for the model to reach the stable loss interval is reduced to about 27 rounds, the parameter update process is more stable, and reliable support is provided for the long-term operation of the overall system.

[0037] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method and system for generating interactive natural language data based on artificial intelligence, characterized in that, Includes the following steps: Step 1: Receive the user's voice audio stream; Step 2: Perform a short-time Fourier transform on the voice audio stream to obtain spectral features, input the spectral features into the adaptive noise suppression module to generate a noise mask and complete spectral filtering to obtain denoised Mel spectral features; The system collects user voice audio streams generated during operation, forms a business scenario voice corpus according to business scenario identifiers, constructs a Whisper speech recognition model and performs incremental fine-tuning, inputs the denoised Mel-spectral features into the incrementally fine-tuned Whisper speech recognition model, and outputs a transcribed text sequence; Step 3: Perform feature fusion on the transcribed text sequence and the denoised Mel-spectral features to obtain a fused feature vector; Steps Step 4: Generate a set of structured information units based on the fused feature vectors; Step 5: Generate form filling results based on the set of structured information units and business scenario identifiers; Step 6: During the model training phase, construct a multi-task joint model with the Whisper speech recognition model as the backbone, perform PCard conflict discrimination and projection correction operations in the preconditioned gradient space, and update the model parameters; Step 7: Validate the form filling results and generate standardized form data.

2. The method and system for generating interactive natural language data based on artificial intelligence according to claim 1, characterized in that, Step 2, which involves inputting the spectral features into the adaptive noise suppression module to generate a noise mask, specifically includes: The audio stream is segmented into frames according to a preset frame length. A windowing operation is performed on each frame to form a sequence of audio frames. A short-time Fourier transform is performed on the audio frame sequence to generate a complex spectrum matrix. An amplitude spectrum matrix is ​​extracted from the complex spectrum matrix to form spectral features. These spectral features are input sequentially into an adaptive noise suppression module containing a gated recurrent unit. The gated recurrent unit performs a recursive operation along the time steps under the initial hidden state vector condition, generating a sequence of hidden state vectors. A noise mask mapping weight matrix and a noise mask mapping bias vector are input to each hidden state vector in the hidden state vector sequence to generate an intermediate mask vector. A sigmoid operation is performed on the intermediate mask vector to obtain a noise mask matrix. A smoothing operation is performed on the noise mask matrix along the frequency band dimension to form a smoothed noise mask matrix. A numerical clipping operation is performed on the smoothed noise mask matrix to obtain a clipped noise mask matrix. Element-wise multiplication is performed on the amplitude spectrum matrix and the clipped noise mask matrix to generate a filtered amplitude spectrum matrix. The filtered amplitude spectrum matrix is ​​input to the Mel filter bank and a weighted operation is performed to generate the Mel band spectrum matrix. The logarithmic operation is then performed on the Mel band spectrum matrix to obtain the denoised Mel spectrum features.

3. The method and system for generating interactive natural language data based on artificial intelligence according to claim 2, characterized in that, The construction of the Whisper speech recognition model in step 2 includes the following steps: The process involves: 1) Reading denoised Mel-spectral features and corresponding transcribed text annotations from a business scenario speech corpus, generating a training sample pair set based on sample correspondence; 2) Constructing the model structure parameter set of the Whisper speech recognition model, which includes encoder parameter set, decoder parameter set, general corpus adaptation parameter set, and business-specific vocabulary embedding parameter set; 3) Performing parameter state initialization on the model structure parameter set, keeping the encoder and decoder parameter sets updatable while keeping the general corpus adaptation parameter set frozen; 4) Inputting the denoised Mel-spectral features from the training sample pair set into the encoder of the Whisper speech recognition model in time-step order to generate the encoder output feature sequence; 5) Inputting the encoder output feature sequence into the decoder of the Whisper speech recognition model, shielding the general corpus adaptation parameter set from the decoding calculation, and only introducing the business-specific vocabulary embedding parameter set to participate in the decoding calculation to generate a predicted text sequence; 6) Performing loss calculation on the predicted text sequence and corresponding transcribed text annotations, and performing parameter update operations on the encoder parameter set, decoder parameter set, and business-specific vocabulary embedding parameter set to form the completed Whisper speech recognition model.

4. The method and system for generating interactive natural language data based on artificial intelligence according to claim 3, characterized in that, The feature fusion process in step 3, which involves combining the transcribed text sequence with the denoised Mel-spectral features, specifically includes: The process involves: reading the transcribed text sequence output by the Whisper speech recognition model and mapping it to a text tag sequence based on word order; inputting the text tag sequence into a text semantic encoding model to obtain a text feature vector sequence corresponding to each text tag sequence; reading the denoised Mel-frequency spectroscopic features and performing pooling operations along the time dimension to obtain speech feature vectors that match the text feature vector sequence in the feature dimension; concatenating the text feature vector sequence and the speech feature vectors along the feature dimension to generate a concatenated feature vector sequence; inputting the concatenated feature vector sequence into a channel attention computation structure to generate corresponding feature weight vectors; and performing a channel-wise weighted operation on the concatenated feature vector sequence and the feature weight vectors to output a fused feature vector.

5. The method and system for generating interactive natural language data based on artificial intelligence according to claim 4, characterized in that, Step 4 includes the following steps: Read the fused feature vector and expand it into a feature sequence in chronological order; construct a field label parsing calculation structure, which includes position mapping weight parameters and label mapping bias parameters; input the feature sequence into the field label parsing calculation structure according to its chronological position, and perform field name label calculation on each position in the feature sequence to generate a field name label sequence; during the field name label calculation process, simultaneously perform attribute value calculation on the positions corresponding to the field name labels to generate an attribute value sequence that corresponds one-to-one with the field name; read the calculated score value for each corresponding position in the field name label sequence and attribute value sequence, and perform a normalization operation on the calculated score value to generate the corresponding confidence value; construct structured information units based on the correspondence between field name, attribute value, and confidence, and write all structured information units into a structured information unit set.

6. The method and system for generating interactive natural language data based on artificial intelligence according to claim 5, characterized in that, Step 5 specifically includes: Read the set of structured information units and the corresponding business scenario identifiers; construct a set of field mapping rules based on the business scenario identifiers, where each rule item contains a correspondence between a field name identifier and a form field identifier; input the field names from the set of structured information units into the set of field mapping rules sequentially, perform rule matching operations on each field name to obtain the corresponding form field identifier; establish a field filling correspondence between the matched form field identifiers and the attribute values ​​in the structured information units, generating a set of field filling pairs; generate a form field writing sequence based on the set of field filling pairs, and write the attribute values ​​into the corresponding form field positions according to the writing sequence to form the form filling result; write the generated form filling result into the form filling result cache.

7. The method and system for generating interactive natural language data based on artificial intelligence according to claim 6, characterized in that, Step 6, constructing a multi-task joint model with the Whisper speech recognition model as the backbone, includes the following steps: A multi-task joint model with the Whisper speech recognition model as the backbone is constructed, integrating the speech transcription, information extraction, and field mapping tasks into the same backbone structure. Model parameters that participate in the computation of all three tasks are selected within the multi-task joint model, forming a shared parameter subset. Task loss functions corresponding to each of the three tasks are constructed separately, forming a multi-task loss function set. Backpropagation is performed on the multi-task loss function set within the same training batch to calculate the multi-task gradient vector set corresponding to the shared parameter subset, and this set is then written to the gradient buffer.

8. The method and system for generating interactive natural language data based on artificial intelligence according to claim 7, characterized in that, Step 6, which involves performing PCard conflict detection and projection correction operations and updating model parameters, includes the following steps: The process involves several steps: First, reading the multi-task gradient vector sets recorded in the gradient buffer. Each gradient vector in the multi-task gradient vector set corresponds one-to-one with a shared parameter subset in the parameter dimension. Second, performing statistical operations on the historical gradient information corresponding to the shared parameter subset, and constructing a curvature preconditioning matrix based on the statistical results. This curvature preconditioning matrix is ​​stored diagonally or in block diagonal form along the parameter dimension. Third, inputting each multi-task gradient vector set into the curvature preconditioning matrix to perform preconditioning transformation operations, generating a preconditioning gradient vector set that corresponds one-to-one with the multi-task gradient vector sets. Fourth, selecting any two preconditioning gradient vectors in the preconditioning gradient space according to the task index order, performing an inner product calculation on the selected preconditioning gradient vectors, and recording the inner product calculation result. Fifth, performing a sign determination operation on the inner product calculation result; if the inner product result is negative, recording that the corresponding task gradient pair has a directional conflict relationship. Sixth, for the preconditioning gradient vectors recorded as having a directional conflict relationship, selecting one as the corrected gradient vector and the other as the reference gradient vector. Seventh, performing a vector projection operation on the corrected gradient vector, projecting the corrected gradient vector onto the orthogonal subspace of the reference gradient vector, generating a corrected gradient vector after removing conflicting components. For all preconditioned gradient vectors with conflicting directions in the multi-task gradient vector group, projection correction operation is performed sequentially to form a conflict-free preconditioned gradient vector group; vector synthesis operation is performed on the conflict-free preconditioned gradient vector group to generate a single collaborative update gradient vector; parameter update operation is performed on the shared parameter subset based on the single collaborative update gradient vector, and the updated parameters are written back to the Whisper speech recognition model.

Images