A data-driven based natural language processing system and method

Abstract

This invention discloses a data-driven natural language processing system and method, relating to the field of natural language processing technology. The invention extracts response time and data features from each historical natural language processing instance, calculates the correlation between response time and data features, and filters out complex features affecting the complexity of the original data. It then calculates the complexity of the real-time original data using feature functions of all complex features. The invention analyzes the computational load of the model to obtain node partitioning, constructs multi-level processing paths, and builds natural language processing models for each multi-level processing path. It establishes a mapping relationship between computational complexity and computational load, sets a complexity threshold, and uses reinforcement learning to train and optimize the complexity threshold using a validation set, setting a training objective. A dynamic routing gating function is constructed to select the real-time processing path. Finally, a gradient collaborative training mechanism is constructed, using multiple processing paths combined for training to optimize the complexity threshold.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, specifically to a data-driven natural language processing system and method. Background Technology

[0002] In the early stages of natural language processing development, rule-based methods were primarily employed. For example, based on Chomsky's generative grammar theory, formal rules were used to describe human language, and sentences were parsed and generated by manually writing a large number of grammatical rules and dictionaries. This method could achieve certain results when dealing with some relatively simple language segments with well-defined rules, but when faced with the complexity, diversity, and ambiguity of natural language, such as different language habits, newly emerging vocabulary, and expressions, the cost of writing and maintaining rules was extremely high, and it was difficult to cover all situations, resulting in poor adaptability.

[0003] With the popularization of the internet and the rapid development of information technology, the amount of data has exploded, providing abundant material for data-driven natural language processing. Large amounts of text data, such as web page text, social media content, news reports, and academic literature, have become valuable resources for training natural language processing models. This data covers various fields and topics, enabling models to learn rich linguistic expressions and semantic information. However, in current data-driven natural language processing, most models use fixed structures for different processing tasks, significantly delaying the system's inference efficiency. Therefore, how to flexibly handle tasks by selecting different processing paths based on the difficulty characteristics of the task is crucial for processing efficiency. Summary of the Invention

[0004] The purpose of this invention is to provide a data-driven natural language processing system and method to solve the problems raised in the prior art.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A data-driven natural language processing method, the method comprising the following steps:

[0007] S100. Collect all raw data from historical natural language processing, extract the response time and data features from each natural language processing session in history, calculate the correlation between response time and data features, and filter out complex features that affect the complexity of the raw data.

[0008] Furthermore, the specific steps for filtering out the complex features that affect the complexity of the original data are as follows:

[0009] S101. Extract all raw data from historical natural language processing (NLP) processes from the NLP system logs, remove duplicate data, standardize the raw data, extract the task start and end timestamps for each NLP process, and calculate the response time for each NLP process using the following formula:

[0010] ;

[0011] In the formula, T i t represents the response time during the i-th natural language processing iteration in history. end，i t represents the end timestamp of the i-th natural language processing task in history. start，i Represents the start timestamp of the i-th natural language processing task in history;

[0012] S102. Extract data features from the system logs of the original data in each natural language processing task. Data features include, but are not limited to, data length and data semantics. Calculate the response time and correlation of different data features for each historical natural language processing task. Standardize the response time and data features using the following formula:

[0013] ;

[0014] In the formula, ρ(X) j (T) represents the correlation between the j-th data feature and the response time, X j，i Xp represents the j-th data feature value in the i-th natural language processing task. j T represents the average value of the j-th data feature in n natural language processing tasks. i Tp represents the response time of the i-th natural language processing task, Tp represents the average response time of n natural language processing tasks, and n represents the total number of historical natural language processing tasks collected.

[0015] S103. Sort all the data features and response time correlations from smallest to largest, calculate the difference between adjacent correlations, select the largest difference as the boundary, set the maximum value of the two correlations corresponding to the boundary as the correlation threshold, use the correlation threshold to judge all correlations, when the correlation is greater than the correlation threshold, the corresponding data feature is regarded as a complex feature, and a complex feature set is obtained.

[0016] Extracting response time and data features and calculating correlations allows for the precise identification of key features that significantly impact the complexity of the original data, avoiding interference from irrelevant features and improving the efficiency and focus of subsequent processing. Deduplication and standardization of the original data ensure data quality and consistency, laying a solid foundation for subsequent analysis and modeling.

[0017] S200. Extract the corresponding feature functions based on the selected complex features, and use the feature functions of all complex features to calculate the complexity of the real-time raw data.

[0018] Furthermore, the specific steps for calculating the complexity of real-time raw data using the feature functions of all complex features are as follows:

[0019] S201. Extract the corresponding feature functions based on the selected complex features, set fusion weights for each feature function, and use the fusion weights to fuse the feature functions to construct a complexity calculation model. The model is as follows:

[0020] ;

[0021] In the formula, Com represents the complexity of the original data, and w h Let g(s) represent the fusion weight of the h-th feature function. h Let H represent the h-th characteristic function, and H represent the total number of characteristic functions;

[0022] S202. In a natural language processing system, deep learning algorithms are used to train, optimize, and update the fusion weights of each feature function.

[0023] By extracting feature functions from the selected complex features and constructing a complexity calculation model, the complexity of real-time raw data can be accurately quantified, providing an important basis for subsequent processing path selection. Deep learning algorithms are used to train and optimize the fusion weights, enabling the complexity calculation model to continuously adapt to data changes, thus improving the model's accuracy and generalization ability.

[0024] S300: Collect the computational cost of models in all historical natural language processing tasks, analyze the computational cost of models to obtain the partition nodes, divide the total computational cost of natural language processing models, construct multi-level processing paths, and construct natural language processing models for each multi-level processing path.

[0025] Furthermore, the specific steps for constructing a multi-level processing path are as follows:

[0026] S301. Collect the computational load of natural language processing systems in all historical natural language processing tasks, sort the computational load of natural language processing systems from smallest to largest and plot the computational load curve, differentiate the computational load curve to obtain the change curve, calculate the standard deviation and mean of the change curve, set the change threshold by adding the mean and standard deviation, and take the computational load curve points in the change curve that are greater than the change threshold as the partitioning nodes, and use the partitioning nodes to obtain the m-level processing path.

[0027] S302. Set up a natural language processing model for each processing path. Let the structure of the natural language processing model be {G: f(s) = Q-layer transformer, computational cost}; where G represents the processing path number, transformer represents the natural language processing algorithm, and the value of Q is set by professional knowledge in combination with computational cost.

[0028] By analyzing the computational load of historical models and constructing multi-level processing paths, corresponding processing paths can be provided according to different computational load requirements, thus realizing the rational allocation and utilization of resources.

[0029] Natural language processing models are built for each level of the processing path, enabling the models to better adapt to data processing needs of varying complexity, thereby improving processing efficiency and effectiveness.

[0030] S400, the mapping relationship between computational complexity and computational cost, setting a complexity threshold, using reinforcement learning to train and optimize the complexity threshold through the validation set, and setting a training objective;

[0031] Furthermore, the specific steps for setting training objectives are as follows:

[0032] S401. Collect the computational cost and original data complexity of natural language processing systems in the past when processing tasks. Use the linear regression algorithm to generate a mapping function between computational cost and complexity. Input the computational cost of the division nodes of the m-level processing path into the mapping function to calculate the complexity threshold of each level of processing path.

[0033] S402. Utilize reinforcement learning to train and optimize the complexity threshold using the validation set, setting the training objective as follows:

[0034] ;

[0035] In the formula, Total represents the total loss. task λ represents the task loss, FLOPs represents the system computational efficiency, and λ represents the computational efficiency weight; the result with the minimum total loss is selected during training.

[0036] The mapping relationship between computational complexity and computational cost provides a scientific basis for setting the complexity threshold, enabling the threshold to accurately reflect the relationship between data complexity and computational cost.

[0037] By using reinforcement learning to train and optimize the complexity threshold, the threshold can be continuously adjusted based on the actual processing effect, thereby improving the system's adaptability and performance.

[0038] S500: Construct a dynamic routing gate function to determine the complexity of real-time raw data and select the real-time processing path;

[0039] Furthermore, the specific steps for selecting the real-time processing path are as follows:

[0040] S501. Set the routing gating function using a computational complexity threshold, the formula is:

[0041] ;

[0042] In the formula, Gate(s) represents the routing gate function, f1(s), f2(s), ..., f m Com(s) represents the natural language processing model for the 1st, 2nd, ..., mth level processing paths, Com(s) represents the complexity of the real-time raw data, and α1, α2, ..., αm are the parameters of the processing path. m-1 This represents the 1st, 2nd, ..., m-1th complexity thresholds;

[0043] S502. Input the real-time raw data into the routing threshold function, determine the complexity threshold range of the real-time raw data, and select the natural language model corresponding to the processing path for real-time task processing.

[0044] By constructing a dynamic routing gating function, the appropriate processing path can be automatically selected based on the complexity of the real-time raw data, thereby realizing the intelligence and automation of the processing process and improving the efficiency and accuracy of processing.

[0045] By selecting the processing path based on the threshold range of real-time data complexity, the choice of processing path is ensured to be targeted and reasonable, and can better meet the processing needs of different data.

[0046] S600. Construct a gradient collaborative training mechanism. When training on the complexity threshold, use multiple processing paths to combine training and optimize the complexity threshold.

[0047] Furthermore, the specific steps for optimizing the complexity threshold by combining multiple processing paths with training are as follows:

[0048] S601. When training the complexity threshold, the deviation parameters of each processing path are obtained through reinforcement learning, the temperature coefficient during the training process is extracted, and the gating probability of each processing path in the routing gating function during training is calculated. The formula is as follows:

[0049] ;

[0050] In the formula, p k Let b represent the gating probability of the k-th processing path. k The deviation parameter for the k-th processing path is represented by τ, where τ represents the temperature coefficient.

[0051] S602. Utilize the probability of each processing path to combine all processing paths during training to handle the natural language processing task, constructing a gradient co-training mechanism, the formula of which is:

[0052] ;

[0053] In the formula, y represents the natural language task processing result during training, and f k (s) represents the natural language processing model for the k-th processing path; it is trained using a gradient co-training mechanism, and the complexity threshold is trained, optimized, and updated.

[0054] By constructing a gradient collaborative training mechanism and combining multiple processing paths to optimize the training complexity threshold, the advantages of different processing paths can be fully utilized, thereby improving the efficiency and accuracy of threshold training.

[0055] By calculating the gating probability of each processing path and combining it with the processing task, the training process becomes more scientific and reasonable, and can better adapt to changes in data and processing needs.

[0056] A data-driven natural language processing system includes a data acquisition module, a complex feature module, a complexity calculation module, a processing path hierarchical module, a complexity threshold module, a routing gating judgment module, and a training and update module.

[0057] The data acquisition module is used to collect all raw data, response times, and data features from historical natural language processing.

[0058] The complex feature module is used to calculate the correlation between response time and data features, and to filter out complex features that affect the complexity of the original data.

[0059] The complexity calculation module is used to extract the corresponding feature functions based on the selected complex features, and to calculate the complexity of the real-time raw data using the feature functions of all complex features.

[0060] The processing path hierarchical module is used to analyze the computational load of the model to obtain the partition nodes, divide the total computational load of the natural language processing model, and construct a multi-level processing path.

[0061] The complexity threshold module is used to calculate the mapping relationship between complexity and computational cost, set a complexity threshold, use reinforcement learning to train and optimize the complexity threshold through a validation set, and set a training objective.

[0062] The routing gating judgment module is used to construct a dynamic routing gating function, judge the complexity of real-time raw data, and select the real-time processing path;

[0063] The training update module is used to construct a gradient collaborative training mechanism. When training on the complexity threshold, it utilizes multiple processing paths in combination to optimize the complexity threshold.

[0064] The routing gating and judgment module includes a routing gating function unit and a judgment unit;

[0065] The routing gating function unit is used to set the routing gating function using a computational complexity threshold;

[0066] The judgment unit is used to input real-time raw data into the routing threshold function, determine the complexity threshold range of the real-time raw data, and select the natural language model corresponding to the processing path for real-time task processing.

[0067] The training update module includes a gated probability unit and a gradient co-training mechanism unit;

[0068] The gating probability unit is used to calculate the gating probability of each processing path using the deviation parameter and the temperature coefficient;

[0069] The gradient collaborative training mechanism unit is used to combine all processing paths during training by utilizing the probability of each processing path to process the natural language processing task during training, and to construct a gradient collaborative training mechanism to update the complexity threshold.

[0070] Compared with the prior art, the beneficial effects of the present invention are:

[0071] 1. By accurately assessing data complexity and selecting dynamic routing gating functions, this invention enables the system to automatically match the optimal processing path for data of different complexities, avoiding waste of resources and unnecessary computation, thereby significantly improving the efficiency of natural language processing.

[0072] 2. By filtering complex features, accurately calculating complexity, and optimizing the gradient collaborative training mechanism, the system can process natural language data more accurately, thus improving the quality and accuracy of natural language processing.

[0073] 3. This invention uses a gradient collaborative training mechanism to train and update the complexity threshold, which solves the problem of non-differentiable processing paths and ensures end-to-end training. Attached Figure Description

[0074] Figure 1 This is a module distribution diagram of a data-driven natural language processing system according to the present invention.

[0075] Figure 2 This is a schematic diagram illustrating the steps of a data-driven natural language processing method according to the present invention. Detailed Implementation

[0076] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0077] Example: Figures 1-2 As shown, the present invention provides a technical solution.

[0078] A data-driven natural language processing method, the method comprising the following steps:

[0079] S100. Collect all raw data from historical natural language processing, extract the response time and data features from each natural language processing session in history, calculate the correlation between response time and data features, and filter out complex features that affect the complexity of the raw data.

[0080] The specific steps for filtering out complex features that affect the complexity of the original data are as follows:

[0081] S101. Extract all raw data from historical natural language processing (NLP) processes from the NLP system logs, remove duplicate data, standardize the raw data, extract the task start and end timestamps for each NLP process, and calculate the response time for each NLP process using the following formula:

[0082] ;

[0083] In the formula, T i t represents the response time during the i-th natural language processing iteration in history. end，i t represents the end timestamp of the i-th natural language processing task in history. start，i Represents the start timestamp of the i-th natural language processing task in history;

[0084] S102. Extract data features from the system logs of the original data in each natural language processing task. Data features include, but are not limited to, data length and data semantics. Calculate the response time and correlation of different data features for each historical natural language processing task. Standardize the response time and data features using the following formula:

[0085] ;

[0086] In the formula, ρ(X) j (T) represents the correlation between the j-th data feature and the response time, X j，i Xp represents the j-th data feature value in the i-th natural language processing task.j T represents the average value of the j-th data feature in n natural language processing tasks. i Tp represents the response time of the i-th natural language processing task, Tp represents the average response time of n natural language processing tasks, and n represents the total number of historical natural language processing tasks collected.

[0087] S103. Sort all the data features and response time correlations from smallest to largest, calculate the difference between adjacent correlations, select the largest difference as the boundary, set the maximum value of the two correlations corresponding to the boundary as the correlation threshold, use the correlation threshold to judge all correlations, when the correlation is greater than the correlation threshold, the corresponding data feature is regarded as a complex feature, and a complex feature set is obtained.

[0088] Extracting response time and data features and calculating correlations allows for the precise identification of key features that significantly impact the complexity of the original data, avoiding interference from irrelevant features and improving the efficiency and focus of subsequent processing. Deduplication and standardization of the original data ensure data quality and consistency, laying a solid foundation for subsequent analysis and modeling.

[0089] S200. Extract the corresponding feature functions based on the selected complex features, and use the feature functions of all complex features to calculate the complexity of the real-time raw data.

[0090] The specific steps for calculating the complexity of real-time raw data using the feature functions of all complex features are as follows:

[0091] S201. Extract the corresponding feature functions based on the selected complex features, set fusion weights for each feature function, and use the fusion weights to fuse the feature functions to construct a complexity calculation model. The model is as follows:

[0092] ;

[0093] In the formula, Com represents the complexity of the original data, and w h Let g(s) represent the fusion weight of the h-th feature function. h Let H represent the h-th characteristic function, and H represent the total number of characteristic functions;

[0094] S202. In a natural language processing system, deep learning algorithms are used to train, optimize, and update the fusion weights of each feature function.

[0095] By extracting feature functions from the selected complex features and constructing a complexity calculation model, the complexity of real-time raw data can be accurately quantified, providing an important basis for subsequent processing path selection. Deep learning algorithms are used to train and optimize the fusion weights, enabling the complexity calculation model to continuously adapt to data changes, thus improving the model's accuracy and generalization ability.

[0096] S300: Collect the computational cost of models in all historical natural language processing tasks, analyze the computational cost of models to obtain the partition nodes, divide the total computational cost of natural language processing models, construct multi-level processing paths, and construct natural language processing models for each multi-level processing path.

[0097] The specific steps for constructing a multi-level processing path are as follows:

[0098] S301. Collect the computational load of natural language processing systems in all historical natural language processing tasks, sort the computational load of natural language processing systems from smallest to largest and plot the computational load curve, differentiate the computational load curve to obtain the change curve, calculate the standard deviation and mean of the change curve, set the change threshold by adding the mean and standard deviation, and take the computational load curve points in the change curve that are greater than the change threshold as the partitioning nodes, and use the partitioning nodes to obtain the m-level processing path.

[0099] S302. Set up a natural language processing model for each processing path. Let the structure of the natural language processing model be {G: f(s) = Q-layer transformer, computational cost}; where G represents the processing path number, transformer represents the natural language processing algorithm, and the value of Q is set by professional knowledge in combination with computational cost.

[0100] By analyzing the computational load of historical models and constructing multi-level processing paths, corresponding processing paths can be provided according to different computational load requirements, thus realizing the rational allocation and utilization of resources.

[0101] Natural language processing models are built for each level of the processing path, enabling the models to better adapt to data processing needs of varying complexity, thereby improving processing efficiency and effectiveness.

[0102] S400, the mapping relationship between computational complexity and computational cost, setting a complexity threshold, using reinforcement learning to train and optimize the complexity threshold through the validation set, and setting a training objective;

[0103] The specific steps for setting training objectives are as follows:

[0104] S401. Collect the computational cost and original data complexity of natural language processing systems in the past when processing tasks. Use the linear regression algorithm to generate a mapping function between computational cost and complexity. Input the computational cost of the division nodes of the m-level processing path into the mapping function to calculate the complexity threshold of each level of processing path.

[0105] S402. Utilize reinforcement learning to train and optimize the complexity threshold using the validation set, setting the training objective as follows:

[0106] ;

[0107] In the formula, Total represents the total loss. task λ represents the task loss, FLOPs represents the system computational efficiency, and λ represents the computational efficiency weight; the result with the minimum total loss is selected during training.

[0108] The mapping relationship between computational complexity and computational cost provides a scientific basis for setting the complexity threshold, enabling the threshold to accurately reflect the relationship between data complexity and computational cost.

[0109] By using reinforcement learning to train and optimize the complexity threshold, the threshold can be continuously adjusted based on the actual processing effect, thereby improving the system's adaptability and performance.

[0110] S500: Construct a dynamic routing gate function to determine the complexity of real-time raw data and select the real-time processing path;

[0111] The specific steps for selecting a real-time processing path are as follows:

[0112] S501. Set the routing gating function using a computational complexity threshold, the formula is:

[0113] ;

[0114] In the formula, Gate(s) represents the routing gate function, f1(s), f2(s), ..., f m Com(s) represents the natural language processing model for the 1st, 2nd, ..., mth level processing paths, Com(s) represents the complexity of the real-time raw data, and α1, α2, ..., αm are the parameters of the processing path. m-1 This represents the 1st, 2nd, ..., m-1th complexity thresholds;

[0115] S502. Input the real-time raw data into the routing threshold function, determine the complexity threshold range of the real-time raw data, and select the natural language model corresponding to the processing path for real-time task processing.

[0116] By constructing a dynamic routing gating function, the appropriate processing path can be automatically selected based on the complexity of the real-time raw data, thereby realizing the intelligence and automation of the processing process and improving the efficiency and accuracy of processing.

[0117] By selecting the processing path based on the threshold range of real-time data complexity, the choice of processing path is ensured to be targeted and reasonable, and can better meet the processing needs of different data.

[0118] S600. Construct a gradient collaborative training mechanism. When training on the complexity threshold, use multiple processing paths to combine training and optimize the complexity threshold.

[0119] The specific steps for optimizing the complexity threshold by combining multiple processing paths during training are as follows:

[0120] S601. When training the complexity threshold, the deviation parameters of each processing path are obtained through reinforcement learning, the temperature coefficient during the training process is extracted, and the gating probability of each processing path in the routing gating function during training is calculated. The formula is as follows:

[0121] ;

[0122] In the formula, p k Let b represent the gating probability of the k-th processing path. k The deviation parameter for the k-th processing path is represented by τ, where τ represents the temperature coefficient.

[0123] S602. Utilize the probability of each processing path to combine all processing paths during training to handle the natural language processing task, constructing a gradient co-training mechanism, the formula of which is:

[0124] ;

[0125] In the formula, y represents the natural language task processing result during training, and f k (s) represents the natural language processing model for the k-th processing path; it is trained using a gradient co-training mechanism, and the complexity threshold is trained, optimized, and updated.

[0126] By constructing a gradient collaborative training mechanism and combining multiple processing paths to optimize the training complexity threshold, the advantages of different processing paths can be fully utilized, thereby improving the efficiency and accuracy of threshold training.

[0127] By calculating the gating probability of each processing path and combining it with the processing task, the training process becomes more scientific and reasonable, and can better adapt to changes in data and processing needs.

[0128] A data-driven natural language processing system includes a data acquisition module, a complex feature module, a complexity calculation module, a processing path hierarchical module, a complexity threshold module, a routing gating judgment module, and a training and update module.

[0129] The data acquisition module is used to collect all raw data, response times, and data features from historical natural language processing.

[0130] The complex feature module is used to calculate the correlation between response time and data features, and to filter out complex features that affect the complexity of the original data.

[0131] The complexity calculation module is used to extract the corresponding feature functions based on the selected complex features, and to calculate the complexity of the real-time raw data using the feature functions of all complex features.

[0132] The processing path hierarchical module is used to analyze the computational load of the model to obtain the partition nodes, divide the total computational load of the natural language processing model, and construct a multi-level processing path.

[0133] The complexity threshold module is used to calculate the mapping relationship between complexity and computational cost, set a complexity threshold, use reinforcement learning to train and optimize the complexity threshold through a validation set, and set a training objective.

[0134] The routing gating judgment module is used to construct a dynamic routing gating function, judge the complexity of real-time raw data, and select the real-time processing path;

[0135] The training update module is used to construct a gradient collaborative training mechanism. When training on the complexity threshold, it utilizes multiple processing paths in combination to optimize the complexity threshold.

[0136] The routing gating and judgment module includes a routing gating function unit and a judgment unit;

[0137] The routing gating function unit is used to set the routing gating function using a computational complexity threshold;

[0138] The judgment unit is used to input real-time raw data into the routing threshold function, determine the complexity threshold range of the real-time raw data, and select the natural language model corresponding to the processing path for real-time task processing.

[0139] The training update module includes a gated probability unit and a gradient co-training mechanism unit;

[0140] The gating probability unit is used to calculate the gating probability of each processing path using the deviation parameter and the temperature coefficient;

[0141] The gradient collaborative training mechanism unit is used to combine all processing paths during training by utilizing the probability of each processing path to process the natural language processing task during training, and to construct a gradient collaborative training mechanism to update the complexity threshold.

[0142] Example 1: Taking short text sentiment analysis as an example, this example classifies the sentiment polarity (positive / negative / neutral) of social media comments. The data processed includes historical user comments, response time, and model computational cost.

[0143] 100,000 historical comments were extracted from social media logs, and after removing duplicate text, they were standardized into a uniform format. The start and end timestamps of each comment were extracted. Data features included: text length (X1), number of negative words (X2), sentiment word density (X3), and syntactic complexity (X4). The correlation coefficients between these data features and response time were calculated to be 0.72, 0.58, 0.65, and 0.81, respectively.

[0144] The relevance scores are sorted as [0.58, 0.65, 0.72, 0.81]. The adjacent differences are calculated (0.07, 0.07, 0.09). The relevance threshold corresponding to the largest difference of 0.09 is 0.72. Features with relevance scores > 0.72 are selected: text length (X1) and syntactic complexity (X4), forming a complex feature set.

[0145] Example 2: Collect FLOPs of models from historical tasks, sort them, plot curves and calculate derivatives. Set the mean plus standard deviation as the change threshold, determine 3 partitioning nodes, and construct a 3-level processing path (m=3):

[0146] Level 1 path (lightweight): 2-layer Transformer, computation cost 1.2 GFLOPs, used for simple text.

[0147] Level 2 path (medium): 4 layers of Transformer, 3.5 GFLOPs computation, for medium-complexity text.

[0148] Level 3 Path (Heavy): 6-layer Transformer, 7.8 GFLOPs computation, used for complex text.

[0149] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

Claims

1. A data-driven natural language processing method, characterized in that: The method includes the following steps: S100. Collect all raw data from historical natural language processing, extract the response time and data features from each natural language processing session in history, calculate the correlation between response time and data features, and filter out complex features that affect the complexity of the raw data. S200. Extract the corresponding feature functions based on the selected complex features, and use the feature functions of all complex features to calculate the complexity of the real-time raw data. S300: Collect the computational cost of models in all historical natural language processing tasks, analyze the computational cost of models to obtain the partition nodes, divide the total computational cost of natural language processing models, construct multi-level processing paths, and construct natural language processing models for each multi-level processing path. S400, the mapping relationship between computational complexity and computational cost, setting a complexity threshold, using reinforcement learning to train and optimize the complexity threshold through the validation set, and setting a training objective; S500: Construct a dynamic routing gate function to determine the complexity of real-time raw data and select the real-time processing path; S600. Construct a gradient collaborative training mechanism. When training on the complexity threshold, use multiple processing paths to combine training and optimize the complexity threshold.

2. The data-driven natural language processing method according to claim 1, characterized in that: The specific steps in S100 to filter out complex features that affect the complexity of the original data are as follows: S101. Extract all raw data from historical natural language processing (NLP) processes from the NLP system logs, remove duplicate data, standardize the raw data, extract the task start and end timestamps for each NLP process, and calculate the response time for each NLP process using the following formula: ； In the formula, T i t represents the response time during the i-th natural language processing iteration in history. end，i t represents the end timestamp of the i-th natural language processing task in history. start，i Represents the start timestamp of the i-th natural language processing task in history; S102. Extract data features from the system logs of the original data in each natural language processing task. Data features include, but are not limited to, data length and data semantics. Calculate the response time and correlation of different data features for each historical natural language processing task. Standardize the response time and data features using the following formula: ； In the formula, ρ(X) j (T) represents the correlation between the j-th data feature and the response time, X j，i Xp represents the j-th data feature value in the i-th natural language processing task. j T represents the average value of the j-th data feature in n natural language processing tasks. i Tp represents the response time of the i-th natural language processing task, Tp represents the average response time of n natural language processing tasks, and n represents the total number of historical natural language processing tasks collected. S103. Sort all the data features and response time correlations from smallest to largest, calculate the difference between adjacent correlations, select the largest difference as the boundary, set the maximum value of the two correlations corresponding to the boundary as the correlation threshold, use the correlation threshold to judge all correlations, when the correlation is greater than the correlation threshold, the corresponding data feature is regarded as a complex feature, and a complex feature set is obtained.

3. The data-driven natural language processing method according to claim 2, characterized in that: The specific steps in S200 for calculating the complexity of real-time raw data using the feature functions of all complex features are as follows: S201. Extract the corresponding feature functions based on the selected complex features, set fusion weights for each feature function, and use the fusion weights to fuse the feature functions to construct a complexity calculation model. The model is as follows: ； In the formula, Com represents the complexity of the original data, and w h Let g(s) represent the fusion weight of the h-th feature function. h Let H represent the h-th characteristic function, and H represent the total number of characteristic functions; S202. In a natural language processing system, deep learning algorithms are used to train, optimize, and update the fusion weights of each feature function.

4. The data-driven natural language processing method according to claim 1, characterized in that: The specific steps for constructing a multi-level processing path in S300 are as follows: S301. Collect the computational load of natural language processing systems in all historical natural language processing tasks, sort the computational load of natural language processing systems from smallest to largest and plot the computational load curve, differentiate the computational load curve to obtain the change curve, calculate the standard deviation and mean of the change curve, set the change threshold by adding the mean and standard deviation, and take the computational load curve points in the change curve that are greater than the change threshold as the partitioning nodes, and use the partitioning nodes to obtain the m-level processing path. S302. Set up a natural language processing model for each processing path. Let the structure of the natural language processing model be {G: f(s) = Q-layer transformer, computational cost}; where G represents the processing path number, transformer represents the natural language processing algorithm, and the value of Q is set by professional knowledge in combination with computational cost.

5. The data-driven natural language processing method according to claim 1, characterized in that: The specific steps for setting the training objective in S400 are as follows: S401. Collect the computational cost and original data complexity of natural language processing systems in the past when processing tasks. Use the linear regression algorithm to generate a mapping function between computational cost and complexity. Input the computational cost of the division nodes of the m-level processing path into the mapping function to calculate the complexity threshold of each level of processing path. S402. Utilize reinforcement learning to train and optimize the complexity threshold using the validation set, setting the training objective as follows: ； In the formula, Total represents the total loss. task λ represents task loss, FLOPs represents system computational efficiency, and λ represents computational efficiency weight. The result with the minimum total loss is selected during training.

6. The data-driven natural language processing method according to claim 5, characterized in that: The specific steps for selecting the real-time processing path in S500 are as follows: S501. Set the routing gating function using a computational complexity threshold, the formula is: ； In the formula, Gate(s) represents the routing gate function, f1(s), f2(s), ..., f m Com(s) represents the natural language processing model for the 1st, 2nd, ..., mth level processing paths, Com(s) represents the complexity of the real-time raw data, and α1, α2, ..., αm are the parameters of the processing path. m-1 This represents the 1st, 2nd, ..., m-1th complexity thresholds; S502. Input the real-time raw data into the routing threshold function, determine the complexity threshold range of the real-time raw data, and select the natural language model corresponding to the processing path for real-time task processing.

7. The data-driven natural language processing method according to claim 6, characterized in that: The specific steps in S600 that utilize multiple processing paths combined with training to optimize the complexity threshold are as follows: S601. When training the complexity threshold, the deviation parameters of each processing path are obtained through reinforcement learning, the temperature coefficient during the training process is extracted, and the gating probability of each processing path in the routing gating function during training is calculated. The formula is as follows: ； In the formula, p k Let b represent the gating probability of the k-th processing path. k The deviation parameter for the k-th processing path is represented by τ, where τ represents the temperature coefficient. S602. Utilize the probability of each processing path to combine all processing paths during training to handle the natural language processing task, constructing a gradient co-training mechanism, the formula of which is: ； In the formula, y represents the natural language task processing result during training, and f k (s) represents the natural language processing model for the k-th processing path; it is trained using a gradient co-training mechanism, and the complexity threshold is trained, optimized, and updated.

8. A data-driven natural language processing system, characterized in that: The natural language processing system includes a data acquisition module, a complex feature module, a complexity calculation module, a processing path hierarchical module, a complexity threshold module, a route gating judgment module, and a training and update module. The data acquisition module is used to collect all raw data, response times, and data features from historical natural language processing. The complex feature module is used to calculate the correlation between response time and data features, and to filter out complex features that affect the complexity of the original data. The complexity calculation module is used to extract the corresponding feature functions based on the selected complex features, and to calculate the complexity of the real-time raw data using the feature functions of all complex features. The processing path hierarchical module is used to analyze the computational load of the model to obtain the partition nodes, divide the total computational load of the natural language processing model, and construct a multi-level processing path. The complexity threshold module is used to calculate the mapping relationship between complexity and computational cost, set a complexity threshold, use reinforcement learning to train and optimize the complexity threshold through a validation set, and set a training objective. The routing gating judgment module is used to construct a dynamic routing gating function, judge the complexity of real-time raw data, and select the real-time processing path; The training update module is used to construct a gradient collaborative training mechanism. When training on the complexity threshold, it utilizes multiple processing paths in combination to optimize the complexity threshold.

9. A data-driven natural language processing system according to claim 8, characterized in that: The routing gating judgment module includes a routing gating function unit and a judgment unit; The routing gating function unit is used to set the routing gating function using a computational complexity threshold; The judgment unit is used to input real-time raw data into the routing threshold function, determine the complexity threshold range of the real-time raw data, and select the natural language model corresponding to the processing path for real-time task processing.

10. A data-driven natural language processing system according to claim 8, characterized in that: The training update module includes a gated probability unit and a gradient collaborative training mechanism unit; The gating probability unit is used to calculate the gating probability of each processing path using the deviation parameter and the temperature coefficient; The gradient collaborative training mechanism unit is used to combine all processing paths during training by utilizing the probability of each processing path to process the natural language processing task during training, and to construct a gradient collaborative training mechanism to update the complexity threshold.

Images