Computer and large model interaction system based on manual dictation command

Abstract

The invention discloses a computer and large model interaction system based on an artificial dictation command, and the system comprises a control unit which is used for controlling and managing the whole information collection and conversion process; the acquisition module is responsible for capturing voice signals, recognizing the voice signals and converting the voice signals into readable text data for a computer; the analysis module is used for analyzing the text data and converting the text data into a readable instruction format for a large model; the interface module is used for receiving and processing an instruction format and realizing a corresponding function; and the feedback module is used for adjusting and optimizing the accuracy of the voice recognition system and the instruction analysis module according to the feedback result of the user. According to the method, through a deep learning algorithm based on a large model technology, commands dictated by a user are accurately recognized, texts are analyzed and key information is extracted through a large model, and the texts are converted into executable instruction formats, so that seamless joint of man-machine interaction is realized.

Description

Technical Field

[0001] The present invention belongs to the technical field of human-computer interaction, and more specifically, particularly relates to a computer and large model interaction system based on artificial oral commands. Background Art

[0002] Human-Computer Interaction Techniques refers to the technology of realizing the dialogue between people and computers in an effective way through computer input and output devices. Human-Computer Interaction Techniques include machines providing people with a large amount of relevant information and prompts through output or display devices, and people inputting relevant information to machines through input devices, answering questions and prompts. Human-Computer Interaction Technology is one of the fastest-growing fields in user interface research, and all countries attach great importance to it. In the national key technologies, the United States lists the human-computer interface as one of the six key technologies in information technology, along with software and computers, and calls it "of outstanding importance to the computer industry and also very important to other industries." In the key technologies of the United States' defense, the human-computer interface is not only one of the important contents of software technology, but also one of the 11 key technologies along with computer and software technology. The European Community's European Strategic Plan for Research and Development of Information Technology (ESPRIT) has also set up a user interface technology project, including a multi-channel human-computer interaction interface.

[0003] However, there are some problems with the existing technology: the user's spoken commands may be ambiguous and polysemous, and different users may use different expressions to express the same intention, resulting in the system being unable to accurately recognize and understand. Therefore, we propose a computer-large model interaction system based on artificial spoken commands. Summary of the invention

[0004] In view of the problems existing in the prior art, the purpose of the present invention is to provide a computer and big model interaction system based on manual oral commands. Through a deep learning algorithm based on big model technology, the user's oral commands are accurately identified, and then the text is parsed and key information is extracted through the understanding ability of the big model, and converted into an instruction format that can be executed by the big model, thereby achieving seamless human-computer interaction.

[0005] To achieve the above object, the present invention provides the following technical solution: a computer and large model interaction system based on manual oral commands, comprising:

[0006] The control unit, which includes the central processing unit CPU and related software, is used to control and manage the entire process of information collection and conversion, and transmit data to the back-end device through the network;

[0007] The acquisition module, which includes a microphone, a sound sensor, an analog-to-digital converter, and an audio interface, is responsible for capturing and recognizing voice signals and converting them into readable text data for computers;

[0008] A parsing module, used for parsing text data and converting the text data into a readable instruction format for a large model;

[0009] The interface module receives and processes the instruction format to implement the corresponding function;

[0010] The feedback module is used to adjust and optimize the accuracy of the speech recognition system and the instruction parsing module according to the user's feedback results.

[0011] Optionally, the collection module workflow is:

[0012] S1, uses a microphone to capture sound waves;

[0013] S2, converting the analog sound signal into a digital signal through an analog-to-digital converter;

[0014] S3, performing data preprocessing on the digital signal to filter out unnecessary frequencies or noise;

[0015] S4, extracting feature parameters useful for identification from the processed audio signal;

[0016] S5, running the extracted features through acoustic models and language models to identify the speech content;

[0017] S6, outputting the recognition result into a computer-readable format.

[0018] Optionally, in step S3, the data preprocessing is specifically:

[0019] Data point elimination: Delete abnormal data points, which include measurement errors and equipment failures. The elimination criteria include threshold elimination and statistical testing.

[0020] Data filtering: According to the characteristics of the signal and noise, a filter is selected to eliminate high-frequency noise in the signal. The filter includes a low-pass filter, a high-pass filter or a band-pass filter;

[0021] Smoothing and denoising: Smoothing techniques are used to reduce random fluctuations and outliers in the signal, including moving average and weighted average methods, and noise components in the signal are removed through noise reduction algorithms, including wavelet transform and Fourier transform;

[0022] Detrending: Detrending is used to eliminate trends and offsets and is used to analyze the essential characteristics of signals.

[0023] Optionally, the data culling process includes identifying and deleting outliers or erroneous data in the data set, and the specific processing method is:

[0024] Threshold elimination, identifying outliers by setting upper and lower thresholds and eliminating data points that exceed the thresholds;

[0025] Statistical tests, which use statistical methods to determine whether data points differ significantly from the mean, thereby identifying outliers

[0026] Model-based methods build statistical models of data and check whether data points meet the predictions of the model. Data points that do not meet the predictions are considered outliers.

[0027] Cluster analysis, which divides the data into different groups and examines isolated points that are significantly different from other data points;

[0028] Sliding window method, for time series data, uses a sliding window to calculate the statistical characteristics of each data point and identify outliers based on the characteristics.

[0029] Optionally, the Fourier transform is performed by converting the time domain signal into the frequency domain, and the specific formula is expressed as:

[0030]

[0031] Where F(k) is a function in the frequency domain;

[0032] f(x) is a function in the time domain;

[0033] e is the base of natural logarithms;

[0034] i is the imaginary unit;

[0035] k is the frequency variable;

[0036] The inverse transform of Fourier transform can restore the frequency domain signal to the time domain. The specific formula is:

[0037]

[0038] By multiplying all frequency components F(k) by the corresponding complex exponential function and integrating, the original time domain signal f(x) is recovered. When processing audio, the inverse transform can be used to reconstruct the original audio signal from the spectral data.

[0039] Optionally, the parsing module deploys a large language model, and its workflow is:

[0040] Receive input: The module receives text format commands from the user;

[0041] Syntax parsing: Use lexical analysis and syntax analysis techniques to decompose text commands into meaningful vocabulary units and construct the grammatical structure of the command;

[0042] Semantic analysis: Identify and understand the intent of the command and the actions to be performed based on predefined rules;

[0043] Instruction formatting: convert the parsed commands into a specific instruction format that the large model can understand and execute;

[0044] Verification and execution: Verify the formatted instructions to ensure they are correct, and then pass them to the large model for execution.

[0045] Optionally, the grammatical analysis includes lexical analysis, grammatical analysis, context-free grammar, parsing algorithm and semantic analysis;

[0046] The semantic analysis further analyzes the constructed grammatical structure to ensure that the vocabulary combination in the sentence is meaningful and conforms to the semantic rules of the language, including:

[0047] Data augmentation: By providing more high-quality and diverse training data, especially data containing rich semantic information, we can leverage the model’s learning ability to help it understand and learn the nuances of language.

[0048] Contextual understanding: By introducing more contextual information, including conversation history and document context, the model can understand the meaning of language more accurately;

[0049] Pre-training and fine-tuning: Use a pre-trained large language model as a starting point, and then fine-tune it on a specific task to improve the model's semantic understanding ability in a specific domain or task;

[0050] Cross-language training: The model learns the semantic correspondence between different languages ​​and improves its adaptability to multilingual environments;

[0051] Model pruning and quantization: Reduce the complexity of the model through model pruning and quantization techniques, thereby improving the efficiency and generalization ability of the model.

[0052] Optionally, the pre-training is used for the model to learn common features and patterns and improve the performance of the model on new tasks, including:

[0053] Data collection: collect a large amount of unlabeled data, which must cover the fields and task types that the model needs to handle.

[0054] Task selection: determine that the pre-training task is natural language processing, which is used to process the data text converted by the speech acquisition module;

[0055] Model training, using unlabeled data to pre-train the model, including:

[0056] 1) Randomly initialize the model parameters;

[0057] 2) Define a loss function to measure the difference between the model prediction and the real data.

[0058] Optionally, the model training adopts a stochastic gradient descent method;

[0059] 1) Initialize model parameters to random values;

[0060] 2) Determine a learning rate, which is used to define how much the model parameters are adjusted in the direction indicated by the gradient of the loss function to minimize the loss during the iteration process;

[0061] 3) Randomly select a sample from the training data set;

[0062] 4) Calculate the gradient of the loss function with respect to each parameter based on the current parameters and sampled data;

[0063] 5) Update each parameter using the learning rate multiplied by the gradient;

[0064] 6) Repeat the above process multiple times until the convergence condition or other stopping criteria are met.

[0065] The present invention uses Fourier transform in the noise reduction algorithm to distinguish the characteristics of signals and noise in the frequency domain, making it easier to identify and process noise components. In addition, Fourier transform can greatly improve the calculation speed, making real-time or near real-time signal processing possible, which not only improves the quality of the noise reduction effect, but also improves the efficiency of the processing process.

[0066] Secondly, by building an instruction parsing system based on the big model and utilizing the deep learning ability, accuracy and self-improvement ability of the big model, the language patterns learned by the big model training data can better understand the meaning of the text and generate coherent text. The big model can also improve itself during the training process and use feedback loops to continuously optimize its own structure and parameters to improve the parsing system's ability to understand and execute instructions, thereby improving the accuracy of human-computer interaction execution instructions.

[0067] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 It is a diagram of the computer and large model interaction system architecture based on manual oral commands of the present invention;

[0069] Figure 2 is a flow chart of the system architecture in an embodiment of the present invention;

[0070] Figure 3 It is a system implementation flow chart in an embodiment of the present invention. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0072] like Figures 1 to 3 As shown, an embodiment of the present invention provides a computer and large model interaction system based on manual oral commands, including:

[0073] The control unit, which includes the central processing unit CPU and related software, is used to control and manage the entire process of information collection and conversion, and transmit data to the back-end device through the network;

[0074] The acquisition module, which includes a microphone, a sound sensor, an analog-to-digital converter, and an audio interface, is responsible for capturing and recognizing voice signals and converting them into readable text data for computers;

[0075] A parsing module, used for parsing text data and converting the text data into a readable instruction format for a large model;

[0076] The interface module receives and processes the instruction format to implement the corresponding function;

[0077] The feedback module is used to adjust and optimize the accuracy of the speech recognition system and the instruction parsing module according to the user's feedback results.

[0078] It should be noted that in the existing technology, speech recognition technology may be affected by various factors such as accent, speaking speed, background noise, etc., resulting in reduced recognition accuracy. In the professional terms or complex instructions in specific fields, the existing speech recognition system may not be able to accurately recognize and understand;

[0079] Different users may use different expressions to express the same intention, and the user's spoken commands may be ambiguous or polysemous, resulting in the instruction parsing module being unable to accurately extract key information.

[0080] Furthermore, the present invention first needs to construct a complete speech recognition system that can accurately recognize the user's spoken commands and convert them into a text format that can be understood by the computer;

[0081] Secondly, it is necessary to design an instruction parsing module that is compatible with the big model, which can parse the commands in the above text format and convert them into an instruction format that can be executed by the big model;

[0082] That is, by collecting data from microphones, sound sensors, analog-to-digital converters, audio interfaces and other devices, the sound data collected by microphones and other devices are converted into electrical signals, which are transmitted to the computer through the audio interface. The processor in the control unit deploys a speech recognition system to convert the electrical signals into text data, which is then converted into an executable instruction format by the big model. Finally, the big model executes the command, which improves the convenience of human-computer interaction, and users can interact with the big model without entering text.

[0083] Specifically, the collection module workflow is as follows:

[0084] S1, uses a microphone to capture sound waves;

[0085] S2, converting the analog sound signal into a digital signal through an analog-to-digital converter;

[0086] S3, performing data preprocessing on the digital signal to filter out unnecessary frequencies or noise;

[0087] S4, extracting feature parameters useful for identification from the processed audio signal;

[0088] S5, running the extracted features through acoustic models and language models to identify the speech content;

[0089] S6, outputting the recognition result into a computer-readable format.

[0090] Furthermore, when the acquisition module performs work, the analog-to-digital converter measures the instantaneous value of the analog signal at fixed time intervals; converts the continuous signal value into a discrete digital value, and finally converts the quantized value into a binary code, and then performs data preprocessing on the converted binary code;

[0091] In step S3, the data preprocessing is specifically as follows:

[0092] Data point elimination: Delete abnormal data points, which include measurement errors and equipment failures. The elimination criteria include threshold elimination and statistical testing.

[0093] Data point elimination is an important step in data analysis and signal processing. It is used to help improve data quality and the reliability of analysis results. Abnormal data points are also called outliers, which will have a negative impact on statistical analysis and model prediction. Therefore, effective methods are needed to identify and eliminate them. They can be eliminated in the following ways:

[0094] Threshold rejection: Identify abnormal data points by setting one or more thresholds. Data points exceeding these thresholds are considered abnormal and will be rejected.

[0095] Statistical test: This method is based on statistical principles and identifies outliers by calculating the probability distribution of data points. Common statistical tests include Z-score (standard score) and Grubbs' test. Z-score determines whether a data point is abnormal by measuring its distance from the mean, while Grubbs' test is specifically used to detect a single extreme value in a data set.

[0096] In general, data point elimination is one of the key steps to ensure the accuracy of data analysis. By using methods such as threshold elimination and statistical testing, abnormal data points caused by measurement errors or equipment failures can be effectively identified and eliminated, thereby improving the quality and reliability of the data.

[0097] Data filtering: According to the characteristics of the signal and noise, a filter is selected to eliminate high-frequency noise in the signal. The filter includes a low-pass filter, a high-pass filter or a band-pass filter;

[0098] Data filtering is a technology used for signal smoothing and noise removal. Its purpose is to remove noise or interference components from data and retain valuable signal information through various algorithms and technical means.

[0099] Smoothing and denoising: Smoothing techniques are used to reduce random fluctuations and outliers in the signal, including moving average and weighted average methods, and noise components in the signal are removed through noise reduction algorithms, including wavelet transform and Fourier transform;

[0100] Moving average smoothing: Reduces random noise by calculating the average value within a certain time window while retaining the overall trend. Simple moving average (SMA) averages the data within a fixed window and is suitable for stable data. Exponentially weighted moving average (EMA) gives more weight to recent data and is suitable for tracking rapidly changing data;

[0101] Weighted average method: By multiplying each data point by the corresponding weight, then calculating the sum and dividing it by the sum of the weights, a comprehensive average is obtained.

[0102] Suppose there are n data points X1, X2, ..., Xn and corresponding weights W1, W2, ..., Wn, then the weighted average W A The calculation formula is:

[0103]

[0104] Detrending: Detrending is used to eliminate trends and offsets and is used to analyze the essential characteristics of signals.

[0105] The data culling process includes identifying and deleting outliers or erroneous data in the data set. The specific processing method is as follows:

[0106] Threshold elimination, identifying outliers by setting upper and lower thresholds and eliminating data points that exceed the thresholds;

[0107] Statistical tests, which use statistical methods to determine whether data points differ significantly from the mean, thereby identifying outliers

[0108] Model-based methods build statistical models of data and check whether data points meet the predictions of the model. Data points that do not meet the predictions are considered outliers.

[0109] Cluster analysis, which divides the data into different groups and examines isolated points that are significantly different from other data points;

[0110] Sliding window method, for time series data, uses a sliding window to calculate the statistical characteristics of each data point and identify outliers based on the characteristics.

[0111] The Fourier transform converts the time domain signal into the frequency domain, and the specific formula is expressed as:

[0112]

[0113] Where F(k) is a function in the frequency domain;

[0114] f(x) is a function in the time domain;

[0115] e is the base of natural logarithms;

[0116] i is the imaginary unit;

[0117] k is the frequency variable;

[0118] The inverse transform of Fourier transform can restore the frequency domain signal to the time domain. The specific formula is:

[0119]

[0120] By multiplying all frequency components F(k) by the corresponding complex exponential function and integrating, the original time domain signal f(x) is recovered. When processing audio, the inverse transform can be used to reconstruct the original audio signal from the spectral data.

[0121] For example, suppose we have a time domain signal (f(t)) consisting of two simple sine waves:

[0122] (f(t))=sin(2πt)+sin(4pit)

[0123] The frequency components of this signal are 1Hz and 2Hz;

[0124] Apply the definition of the Continuous Fourier Transform (CFT) to find its frequency domain representation F(w):

[0125]

[0126] Since (f(t)) is composed of two sine waves of known frequencies, we can use the Dirac delta function to represent its spectrum:

[0127]

[0128] F(omega)=δ(w-1)-δ(w-2)

[0129] Here, δ(w-1) and δ(w-2) represent the impulse responses at frequencies of 1 Hz and 2 Hz, respectively;

[0130] Apply the inverse Fourier transform to restore the time domain signal. By comparing the original time domain signal and the signal obtained by the inverse transform, we can see that they are the same, which verifies the correctness of the Fourier transform and the inverse transform.

[0131] Specifically, the parsing module deploys a large language model, and its workflow is as follows:

[0132] Receive input: The module receives text format commands from the user;

[0133] Syntax parsing: Use lexical analysis and syntax analysis techniques to decompose text commands into meaningful vocabulary units and construct the grammatical structure of the command;

[0134] Semantic analysis: Identify and understand the intent of the command and the actions to be performed based on predefined rules;

[0135] Instruction formatting: convert the parsed commands into a specific instruction format that the large model can understand and execute;

[0136] Verification and execution: Verify the formatted instructions to ensure they are correct, and then pass them to the large model for execution.

[0137] The grammatical analysis includes lexical analysis, grammatical analysis, context-free grammar, parsing algorithm and semantic analysis;

[0138] The semantic analysis further analyzes the constructed grammatical structure to ensure that the vocabulary combination in the sentence is meaningful and conforms to the semantic rules of the language, including:

[0139] Data augmentation: By providing more high-quality and diverse training data, especially data containing rich semantic information, we can leverage the model’s learning ability to help it understand and learn the nuances of language.

[0140] It should be noted that data augmentation is a commonly used technique in machine learning and deep learning, which improves the performance of the model by increasing the diversity of training samples. In the field of natural language processing (NLP), data augmentation is particularly important because the complexity and diversity of text data often require the model to understand and process the nuances of various language expressions;

[0141] Furthermore, data enhancement techniques are used:

[0142] Back translation: Translate text from a source language into another language and then back to the source language to generate new, slightly changed training samples. This reveals semantic differences between different languages ​​and helps the model learn more general language representations;

[0143] Randomly insert, delete or replace words: New training samples can be created by randomly inserting, deleting or replacing words in sentences. This helps the model learn the importance of word order and context and improves its robustness to input changes.

[0144] Robustness is the property of a system that can maintain its functionality and performance in the face of internal uncertainty or external disturbances. In the field of machine learning and artificial intelligence, a robust model is one that can make accurate predictions or decisions even when there is noise, outliers, or unseen situations in the input data.

[0145] Synonym replacement: By replacing words in a sentence with synonyms, we increase the richness of the vocabulary and help the model understand the similarities between different words;

[0146] Sentence conversion: changing the structure of a sentence, such as switching from active voice to passive voice, or adjusting word order, to help the model learn different grammatical structures and sentence patterns;

[0147] Use noise: artificially add some noise to the text data, such as spelling errors, typos, or special characters, to train the model to have a certain tolerance for such noise;

[0148] Contextual embedding: embedding additional information in the original text, such as sentiment, topic, or other metadata, which can help the model learn more contextual information;

[0149] Multi-task learning: training multiple related tasks simultaneously to help the model learn common features across tasks, thereby improving generalization capabilities;

[0150] Generative Adversarial Networks: Generative adversarial networks are used to generate new training samples that introduce new variations while maintaining the original data distribution.

[0151] Contextual understanding: By introducing more contextual information, including conversation history and document context, the model can understand the meaning of language more accurately;

[0152] By introducing more contextual information, machine learning and natural language processing models can more accurately understand and predict the meaning and intent of language;

[0153] 1) When processing conversation or chat data, using previous conversation turns as part of the input can help the model understand the context of the current statement.

[0154] 2) In long text analysis, considering the structure of the entire document rather than just a single sentence can help the model capture the semantic coherence across paragraphs or chapters.

[0155] 3) Using information provided by external knowledge bases or knowledge graphs can enhance the model's understanding of specific fields.

[0156] 4) Multimodal context: Combining other modal information besides text, such as image, audio, or video data, can provide the model with richer contextual information.

[0157] 5) In specific contexts, the introduction of factors such as weather, location, or user emotions can help the model better understand the intent of the language.

[0158] 6) Through interactive dialogue with the user, the model can gradually obtain more information and adjust its understanding.

[0159] 7) Using networks with external memory components, such as Neural Turing Machines (NTMs) or Differentiable Neural Computers (DNCs), can help models store and retrieve previous information to support the learning of long-term dependencies.

[0160] 8) Use pre-trained word embeddings such as BERT or ELMO, which have been trained on a large amount of text and can capture rich contextual information.

[0161] 9) Applying the attention mechanism allows the model to dynamically focus on the pieces of information most relevant to the current task when processing input, thereby improving understanding of the context.

[0162] Pre-training and fine-tuning: Use a pre-trained large language model as a starting point, and then fine-tune it on a specific task to improve the model's semantic understanding ability in a specific domain or task;

[0163] Cross-language training: The model learns the semantic correspondence between different languages ​​and improves its adaptability to multilingual environments;

[0164] Model pruning and quantization: Reduce the complexity of the model through model pruning and quantization techniques, thereby improving the efficiency and generalization ability of the model.

[0165] Model pruning is the process of reducing the size of a model by removing unimportant parts of the model, such as weights, neurons, or layers. Pruning can be divided into the following types:

[0166] Weight pruning: Remove some weights by setting them to 0.

[0167] Neuron pruning: Evaluate the importance of neurons during training and remove those that have no significant impact on the model output.

[0168] The pruned model needs to be retrained to restore or optimize performance. Pruning can reduce the complexity of the model, reduce the risk of overfitting, and reduce the consumption of computing resources.

[0169] Quantization is the process of converting the values ​​in the model from high precision (such as 32-bit floating point numbers) to low precision (such as 8-bit integers). Quantization can reduce the storage size of the model and speed up the inference process because processing low-precision values ​​is usually faster and more energy-efficient. Quantization techniques include:

[0170] Static quantization: quantization is applied after the model is trained, without changing the structure of the model.

[0171] Dynamic quantization: Quantization is applied dynamically during model inference, and the quantization parameters can be adjusted according to the actual input data.

[0172] It should be noted that quantization may cause some accuracy loss, but this loss can be minimized through designed quantization strategies and post-quantization training.

[0173] The pre-training is used for the model to learn common features and patterns and improve the performance of the model on new tasks, including:

[0174] Data collection: collect a large amount of unlabeled data, which must cover the fields and task types that the model needs to handle.

[0175] Task selection: determine that the pre-training task is natural language processing, which is used to process the data text converted by the speech acquisition module;

[0176] Model training, using unlabeled data to pre-train the model, including:

[0177] 1) Randomly initialize the model parameters;

[0178] 2) Define a loss function to measure the difference between the model prediction and the real data.

[0179] During pre-training, the model parameters are randomly initialized to ensure that the model is not affected by any specific bias at the beginning of training, so that it can learn the true distribution and characteristics of the data. Random initialization usually involves assigning a random value to each parameter in the model, such as weights and biases. These values ​​are usually generated based on a specific distribution, such as uniform distribution or normal distribution.

[0180] The model training adopts the stochastic gradient descent method;

[0181] 1) Initialize model parameters to random values;

[0182] 2) Determine a learning rate, which is used to define how much the model parameters are adjusted in the direction indicated by the gradient of the loss function to minimize the loss during the iteration process;

[0183] 3) Randomly select a sample from the training data set;

[0184] 4) Calculate the gradient of the loss function with respect to each parameter based on the current parameters and sampled data;

[0185] 5) Update each parameter using the learning rate multiplied by the gradient;

[0186] 6) Repeat the above process multiple times until the convergence condition or other stopping criteria are met.

[0187] Furthermore, the stochastic gradient descent algorithm can perform adaptive learning rate adjustment to accelerate the convergence process, and has lower requirements on the objective function, and can find a good local minimum even for non-convex problems;

[0188] The large model uses the stochastic gradient descent method to calculate the gradient of each sample and update the weights when processing large-scale data sets. Each iteration is computationally efficient and is updated immediately when new data is received. It also helps explore different areas of the loss function, making it possible to find a better solution.

[0189] The present invention uses Fourier transform in the noise reduction algorithm to distinguish the characteristics of signal and noise in the frequency domain, making it easier to identify and process noise components. In addition, Fourier transform can greatly improve the calculation speed, making real-time or near real-time signal processing possible, which not only improves the quality of the noise reduction effect, but also improves the efficiency of the processing process.

[0190] Secondly, by building an instruction parsing system based on the big model and utilizing the deep learning ability, accuracy and self-improvement ability of the big model, the language patterns learned by the big model training data can better understand the meaning of the text and generate coherent text. The big model can also improve itself during the training process, and use feedback loops to continuously optimize its own structure and parameters to improve the parsing system's ability to understand and execute instructions, thereby improving the accuracy of human-computer interaction execution instructions and the convenience of human-computer interaction, allowing users to interact with the big model without entering text, expanding the application scope of the big model, allowing it to play a role in more scenarios, and providing new ideas and methods for the technological development of related fields.

[0191] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A computer and large model interaction system based on manual oral commands, characterized in that: include: The control unit, which includes the central processing unit CPU and related software, is used to control and manage the entire process of information collection and conversion, and transmit data to the back-end device through the network; The acquisition module, which includes a microphone, a sound sensor, an analog-to-digital converter, and an audio interface, is responsible for capturing and recognizing voice signals and converting them into readable text data for computers; A parsing module, used for parsing text data and converting the text data into a readable instruction format for a large model; The interface module receives and processes the instruction format to implement the corresponding function; The feedback module is used to adjust and optimize the accuracy of the speech recognition system and the instruction parsing module according to the user's feedback results.

2. The computer-large-model interactive system based on manual oral commands according to claim 1, characterized in that: The acquisition module workflow is as follows: S1, uses a microphone to capture sound waves; S2, converting the analog sound signal into a digital signal through an analog-to-digital converter; S3, performing data preprocessing on the digital signal to filter out unnecessary frequencies or noise; S4, extracting feature parameters useful for identification from the processed audio signal; S5, running the extracted features through acoustic models and language models to identify the speech content; S6, outputting the recognition result into a computer-readable format.

3. The computer-large-model interactive system based on manual oral commands according to claim 2, characterized in that: In step S3, the data preprocessing is specifically as follows: Data point elimination: Delete abnormal data points, which include measurement errors and equipment failures. The elimination criteria include threshold elimination and statistical testing. Data filtering: According to the characteristics of the signal and noise, a filter is selected to eliminate high-frequency noise in the signal. The filter includes a low-pass filter, a high-pass filter or a band-pass filter; Smoothing and denoising: Smoothing techniques are used to reduce random fluctuations and outliers in the signal, including moving average and weighted average methods, and noise components in the signal are removed through noise reduction algorithms, including wavelet transform and Fourier transform; Detrending: Detrending is used to eliminate trends and offsets and is used to analyze the essential characteristics of signals.

4. The computer-large-model interactive system based on manual oral commands according to claim 3, characterized in that: The data culling process includes identifying and deleting outliers or erroneous data in the data set. The specific processing method is as follows: Threshold elimination, identifying outliers by setting upper and lower thresholds and eliminating data points that exceed the thresholds; Statistical tests, which use statistical methods to determine whether data points differ significantly from the mean, thereby identifying outliers Model-based methods build statistical models of data and check whether data points meet the predictions of the model. Data points that do not meet the predictions are considered outliers. Cluster analysis, which divides the data into different groups and examines isolated points that are significantly different from other data points; Sliding window method, for time series data, uses a sliding window to calculate the statistical characteristics of each data point and identify outliers based on the characteristics.

5. The computer-large-model interactive system based on manual oral commands according to claim 3 is characterized in that: The Fourier transform converts the time domain signal into the frequency domain, and the specific formula is expressed as: Where F(k) is a function in the frequency domain; f(x) is a function in the time domain; e is the base of natural logarithms; i is the imaginary unit; k is the frequency variable; The inverse transform of Fourier transform can restore the frequency domain signal to the time domain. The specific formula is: By multiplying all frequency components F(k) by the corresponding complex exponential function and integrating, the original time domain signal f(x) is recovered. When processing audio, the inverse transform can be used to reconstruct the original audio signal from the spectral data.

6. The computer-large-model interactive system based on manual oral commands according to claim 1, characterized in that: The parsing module deploys a large language model, and its workflow is as follows: Receive input: The module receives text format commands from the user; Syntax parsing: Use lexical analysis and syntax analysis techniques to decompose text commands into meaningful vocabulary units and construct the grammatical structure of the command; Semantic analysis: Identify and understand the intent of the command and the actions to be performed based on predefined rules; Instruction formatting: convert the parsed commands into a specific instruction format that the large model can understand and execute; Verification and execution: Verify the formatted instructions to ensure they are correct, and then pass them to the large model for execution.

7. The computer-large-model interactive system based on manual oral commands according to claim 6, characterized in that: The grammatical analysis includes lexical analysis, grammatical analysis, context-free grammar, parsing algorithm and semantic analysis; The semantic analysis further analyzes the constructed grammatical structure to ensure that the vocabulary combination in the sentence is meaningful and conforms to the semantic rules of the language, including: Data augmentation: By providing more high-quality and diverse training data, especially data containing rich semantic information, we can leverage the model’s learning ability to help it understand and learn the nuances of language. Contextual understanding: By introducing more contextual information, including conversation history and document context, the model can understand the meaning of language more accurately; Pre-training and fine-tuning: Use a pre-trained large language model as a starting point, and then fine-tune it on a specific task to improve the model's semantic understanding ability in a specific domain or task; Cross-language training: The model learns the semantic correspondence between different languages ​​and improves its adaptability to multilingual environments; Model pruning and quantization: Reduce the complexity of the model through model pruning and quantization techniques, thereby improving the efficiency and generalization ability of the model.

8. The computer-large-model interactive system based on manual oral commands according to claim 7, characterized in that: The pre-training is used for the model to learn common features and patterns and improve the performance of the model on new tasks, including: Data collection: collect a large amount of unlabeled data, which must cover the fields and task types that the model needs to handle. Task selection: determine that the pre-training task is natural language processing, which is used to process the data text converted by the speech acquisition module; Model training, using unlabeled data to pre-train the model, including: 1) Randomly initialize the model parameters; 2) Define a loss function to measure the difference between the model prediction and the real data.

9. The computer-large-model interactive system based on manual oral commands according to claim 8, characterized in that: The model training adopts the stochastic gradient descent method; 1) Initialize model parameters to random values; 2) Determine a learning rate, which is used to define how much the model parameters are adjusted in the direction indicated by the gradient of the loss function to minimize the loss during the iteration process; 3) Randomly select a sample from the training data set; 4) Calculate the gradient of the loss function with respect to each parameter based on the current parameters and sampled data; 5) Update each parameter using the learning rate multiplied by the gradient; 6) Repeat the above process multiple times until the convergence condition or other stopping criteria are met.

Images