News interpretation method and system based on natural language processing

Abstract

The present invention provides a news interpretation method and system based on natural language processing, which relates to the technical field of natural language processing. The method comprises: a preprocessing step: preprocessing the input news text; a text representation step: representing the preprocessed text in a matrix form and inputting it into a generative pre-training model; a generative pre-training step: using a generative pre-training Transformer model to generate a summary of the input text; and a multi-dimensional sentiment analysis step: analyzing the sentiment tendency of the text, which realizes sentiment classification and representation of classification results by extracting and converting features of the input text. The present invention can realize rapid and accurate acquisition and analysis of news information.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a news interpretation method and system based on natural language processing. Background Art

[0002] The application of natural language processing technology in the field of information acquisition and analysis is becoming more and more widespread. The acquisition and analysis of news information is one of the important application scenarios, which requires processing and analyzing a large amount of news text to extract valuable information. Existing technologies have some limitations, such as the rule engine requires manual setting of rules, and the machine learning algorithm requires a large amount of annotated data. For example, traditional text summarization methods are usually based on rule engines, which require manual setting of rules and cannot adapt to complex text structures and semantic relationships. Machine learning algorithms require a large amount of annotated data and cannot meet the needs of real-time processing.

[0003] The invention patent with publication number CN104156409B discloses a method, system and device for providing news, including: obtaining multiple news and obtaining related results corresponding to each news; providing a news display page, wherein the news display page includes multiple display cards, each display card corresponds to a news, each display card includes a first side and a second side, the news is displayed on the first side, and the related results corresponding to the news are displayed on the second side; and receiving user operations, and controlling the display card to display the first side and / or the second side according to the user operations.

[0004] The existing related technologies have low text processing capabilities and are unable to quickly process large amounts of news data; they are unable to accurately extract important news events, resulting in a decrease in the accuracy of business intelligence; and they lack the participation of intelligent robots and are unable to provide voice interaction and automatic news writing functions.

[0005] These technical challenges limit the application scope and effectiveness of existing technologies, especially in the field of business intelligence collection. The shortcomings of existing technologies have led to slow acquisition of business intelligence and low accuracy. Summary of the invention

[0006] In view of the defects in the prior art, the present invention provides a news interpretation method and system based on natural language processing.

[0007] According to a news interpretation method and system based on natural language processing provided by the present invention, the scheme is as follows:

[0008] In a first aspect, a news interpretation method based on natural language processing is provided, the method comprising:

[0009] Preprocessing step: preprocess the input news text;

[0010] Text representation step: The preprocessed text is represented as a matrix and input into the generative pre-trained Transformer model;

[0011] Generative pre-training step: Use the generative pre-trained Transformer model to generate a summary of the input text;

[0012] Multidimensional sentiment analysis steps: Analyze the sentiment tendency of the text by extracting and transforming the input text to achieve sentiment classification and representation of classification results.

[0013] Preferably, the generative pre-training step comprises:

[0014] Pre-training model sub-step: Design a generative pre-training Transformer model;

[0015] Input text generation sub-step: pre-process the text for which summary is to be generated so that the text format meets the input requirements of the generative pre-trained Transformer model;

[0016] Summary generation sub-step: input the preprocessed text into the generative pre-trained Transformer model to generate a summary;

[0017] Summary quality assessment sub-step: The quality of the generated summary is evaluated using the ROUGE evaluation metric to quantify the similarity between the summary and the reference summary.

[0018] Preferably, the design of a generative pre-trained Transformer model includes:

[0019] Step 1: Optimize the structure of the generative pre-trained Transformer model, including lightweight design of the generative pre-trained Transformer model and optimization of the multi-head self-attention mechanism;

[0020] The lightweight design of the generative pre-trained Transformer model includes reducing parameters and computational complexity through parameter sharing, structured sparsity, dynamic adjustment of network depth, and low-rank decomposition of large matrices;

[0021] The optimization of the multi-head self-attention mechanism includes improving the ability to process long sequences and computational efficiency by designing dynamic head cropping, sparse attention mode, low-rank decomposition of the attention matrix, mixed precision quantization, and introducing attention kernel function approximation technology;

[0022] Step 2: Use parallel computing and caching technology to reduce repeated calculations and adjust dynamic accuracy;

[0023] Step 3: Select a dataset in a specific field for pre-training, so that the generative pre-trained Transformer model can quickly adapt to text generation in the corresponding field;

[0024] Step 4: Optimize hardware and algorithms, including using GPU and TPU hardware accelerators, and designing optimization algorithms;

[0025] Among them, the design of optimization algorithms includes: developing a dynamic sparsification algorithm to adaptively adjust the sparsity of the network structure at runtime; developing a mixed precision training algorithm to automatically determine the numerical precision used for each layer of the network during the training process; and developing a communication-aware training algorithm for distributed environments to adaptively adjust parameter synchronization strategies and frequencies based on network topology and bandwidth conditions.

[0026] Preferably, in the summary quality assessment sub-step, ROUGE assessment specifically includes:

[0027] Use parallel and incremental assessments;

[0028] Adjust the weights of ROUGE indicators according to the needs of specific areas.

[0029] Preferably, the multidimensional sentiment analysis step comprises:

[0030] Pre-trained model loading sub-step: build sentiment analysis model;

[0031] Input text generation sub-step: pre-process the text to be analyzed for sentiment to ensure that the text format meets the input requirements of the model;

[0032] Sentiment analysis sub-step: The preprocessed text is input into the pre-trained sentiment analysis model. The sentiment analysis model extracts the features of the input text, understands the semantics and sentiment tendency of the text, and classifies the sentiment according to the extracted features to determine whether the sentiment tendency of the text is positive, negative or neutral.

[0033] Multidimensional sentiment analysis sub-step: Use the ValenceArousal model to evaluate the intensity and positivity of sentiment, and divide sentiment into pleasure and arousal through the ValenceArousal model;

[0034] Calculate the sentiment score sub-steps: For each input text, obtain its sentiment score in the ValenceArousal model;

[0035] Display sentiment analysis result sub-step: Display the sentiment analysis result to the user, including the sentiment classification of the text and the sentiment score in the ValenceArousal model.

[0036] Preferably, the constructing of the sentiment analysis model comprises:

[0037] Data collection and preprocessing: Collect a large amount of required specific field-related data sets and annotate them;

[0038] Domain-specific vocabulary construction: construct a domain-specific vocabulary;

[0039] Multi-stage training strategy: A multi-stage training strategy is adopted, first pre-training on unlabeled text, then fine-tuning on annotated sentiment datasets, and finally fine-tuning on specific tasks;

[0040] Dynamic learning rate adjustment: Develop a dynamic learning rate adjustment algorithm to automatically adjust the learning rate according to performance changes during training;

[0041] Adversarial training: Introducing adversarial training technology to enhance the robustness of the model by generating adversarial samples;

[0042] Integrated transfer learning: not only uses the current model, but also integrates other pre-trained models to take advantage of the advantages of each model through integration;

[0043] Continuous optimization loop: Establish a continuous optimization feedback loop; record the performance of the sentiment analysis model in actual applications for further training and optimization.

[0044] Preferably, in the dynamic learning rate adjustment, the specific operations of implementing the dynamic learning rate adjustment algorithm include:

[0045] Performance indicator monitoring steps: Establish a real-time performance monitoring system to calculate and record key performance indicators after each training batch; use sliding window technology to calculate the moving average and variance of the indicators to smooth short-term fluctuations;

[0046] Learning rate adjustment strategy steps: Based on the monitored performance indicators, design the following adjustment strategy;

[0047] a) Adaptive annealing: When the performance indicator does not improve for N consecutive batches, the learning rate is multiplied by a decay factor. The decay factor is not fixed and is dynamically adjusted according to the duration of performance stagnation;

[0048] b) Periodic restart: Introduce a periodic restart mechanism for the learning rate, reset the learning rate to a higher value every certain number of steps, and the restart period is not fixed and is dynamically adjusted according to the training process;

[0049] c) Dynamic range adjustment: Maintain the acceptable range of learning rate, and dynamically adjust the upper and lower bounds according to the training process; use Bayesian optimization method to search for the optimal learning rate within this range;

[0050] Gradient statistical analysis step: assist learning rate adjustment through gradient statistical analysis, calculate the gradient norm and variance of each batch, and trigger emergency learning rate adjustment when gradient statistics are abnormal;

[0051] Multi-scale learning rate step: Set different learning rates for different layers of the sentiment analysis model, and adjust the learning rate of each layer independently based on its gradient statistics;

[0052] Dynamic batch size adjustment step: Combine learning rate adjustment with batch size adjustment. When the learning rate drops to a certain threshold, increase the batch size accordingly. Use a linear scaling rule to balance the changes in learning rate and batch size.

[0053] Adaptive momentum adjustment step: dynamically adjust the momentum parameter of the optimizer, use an adaptive algorithm to adjust the momentum based on the historical information of the gradient, and adjust the momentum parameter accordingly while adjusting the learning rate to maintain the stability of the optimization process;

[0054] Feedback control loop steps: Implement a feedback control loop, treat the learning rate adjustment as a control problem, and use a PID controller. The input of the controller is the rate of change of the performance indicator, and the output is the adjustment amount of the learning rate.

[0055] Preferably, the method further comprises: a data visualization step: converting the data and analysis results into charts and reports through interactive data visualization technology;

[0056] The data visualization step specifically includes:

[0057] Data collection and preprocessing sub-steps: collect data from multiple sources, clean, transform and integrate the data to ensure data quality and consistency;

[0058] Data analysis and feature extraction sub-steps: Use statistical analysis techniques to calculate basic statistics of the data, use machine learning algorithms to identify patterns and trends in the data, and use natural language processing techniques to extract features and sentiment information from text data;

[0059] Visualization type sub-step: Select different visualization types based on the characteristics of the data and the analysis objectives;

[0060] Visual chart design sub-steps: design the layout, color and style of the visual chart, and add annotations and tags;

[0061] Visualization chart generation sub-steps: generate charts using data visualization tools or libraries, import pre-processed data, generate chart objects, adjust chart parameters and styles, and generate the final chart;

[0062] Interactive function implementation sub-step: add interactive functions to the visualization chart, including zooming, filtering, and hover prompts;

[0063] Integrate the user interface sub-step: embed the generated chart into the system's user interface, provide operation and navigation functions, help users view and analyze data, and ensure the compatibility of the chart on different devices and screens;

[0064] User testing and feedback sub-step: Conduct user testing, collect user feedback and suggestions, and optimize and improve visualization charts and user interfaces based on user feedback to ensure that the system meets user needs and expectations.

[0065] Preferably, the method further comprises: a personalized recommendation step: a personalized recommendation algorithm based on collaborative filtering, using the user's investment preferences and historical behaviors for intelligent analysis, providing personalized news interpretation and investment advice, as well as accurate and personalized recommendation results;

[0066] The personalized recommendation step specifically includes:

[0067] Preprocessing sub-step: obtaining basic data of various types of users, and preprocessing the basic data of various types of users respectively;

[0068] Prediction sub-step: Based on the pre-processed basic data of various users, use machine learning models to predict the user's future behavior and preferences;

[0069] User portrait sub-step: fuse the pre-processed basic user data of various types, extract features based on the fused basic user data, and construct a multi-dimensional user portrait based on the extracted features;

[0070] Screening sub-step: Based on the constructed multi-dimensional user portrait, obtain recommendation results through collaborative filtering and deep learning network, and then filter the recommendation results based on the user's future behavior and preferences to achieve personalized recommendation of investment products.

[0071] In a second aspect, a news interpretation system based on natural language processing is provided, the system comprising:

[0072] Preprocessing module: preprocess the input news text;

[0073] Text representation module: represents the preprocessed text in matrix form and inputs it into the generative pre-training model;

[0074] Generative pre-training module: Use the generative pre-trained Transformer model to generate summaries for the input text;

[0075] Multidimensional sentiment analysis module: analyzes the sentiment tendency of text, and realizes sentiment classification and representation of classification results by extracting and transforming features of input text;

[0076] Data visualization module: transform data and analysis results into charts and reports through interactive data visualization technology;

[0077] Personalized recommendation module: Based on the personalized recommendation algorithm of collaborative filtering, it uses the user's investment preferences and historical behaviors for intelligent analysis to provide personalized news interpretation and investment advice, as well as accurate and personalized recommendation results.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] 1. The system of the present invention is highly efficient. It can quickly process and analyze a large amount of news texts using a generative pre-trained Transformer model, with a processing speed of up to 1,000 articles per second.

[0080] 2. The present invention uses deep learning technology to improve the accuracy of text summarization and sentiment analysis, with an accuracy rate of 85% and 90%, improving the accuracy of the system;

[0081] 3. The present invention adjusts and optimizes model parameters according to actual needs to adapt to different application scenarios and has flexibility;

[0082] 4. The present invention has real-time processing capabilities, uses efficient data capture algorithms and distributed processing architecture, can quickly capture and process large amounts of data from multiple news sources, meet the needs of real-time processing, and ensure that users can obtain the latest market trends to support timely investment decisions.

[0083] Other beneficial effects of the present invention will be explained in the specific implementation manner through the introduction of specific technical features and technical solutions. Through the introduction of these technical features and technical solutions, those skilled in the art should be able to understand the beneficial technical effects brought about by the technical features and technical solutions. BRIEF DESCRIPTION OF THE DRAWINGS

[0084] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0085] Figure 1 It is the overall flow chart of the present invention;

[0086] Figure 2 This is the flow chart of the multi-dimensional sentiment analysis module;

[0087] Figure 3 This is the effect diagram of the data visualization module;

[0088] Figure 4 This is a flow chart of the personalized recommendation module of the present invention. DETAILED DESCRIPTION

[0089] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0090] The embodiment of the present invention provides a news interpretation method based on natural language processing. Through the participation of natural language processing technology and intelligent robots, a standardized news evolution map can be quickly constructed to improve the accuracy and acquisition speed of business intelligence. Figure 1 As shown, the system specifically includes: a preprocessing step, a text representation step, a generative pre-training step, a multi-dimensional sentiment analysis step, a data visualization step, and a personalized recommendation step.

[0091] Preprocessing step: Preprocess the input news text, including word segmentation, stop word removal, part-of-speech tagging and other tasks. We chose the NLTK library for preprocessing, which provides efficient text processing functions and flexible configuration options. The use of the NLTK library can improve the efficiency and accuracy of text processing. The following is the specific preprocessing process:

[0092] 1) Install NLTK library: Make sure you have installed Python and NLTK library. NLTK library is a powerful natural language processing toolkit. Use Python package management tool pip to install NLTK library.

[0093] 2) Download necessary corpora and models: NLTK contains a lot of corpora and models, which can be downloaded through NLTK Downloader. Commonly used resources include "punkt" for word segmentation, "stopwords" for stop words, "averaged_perceptron_tagger" for part-of-speech tagging, and "wordnet" for word form merging.

[0094] 3) Text segmentation: Text segmentation is the process of breaking text into words or phrases. This process will break the input text string into individual words or punctuation marks to form a list. The purpose of word segmentation is to break continuous text into independent words or phrases for subsequent processing.

[0095] 4) Remove stop words: Stop words are words that appear frequently in the text but do not carry much information, such as "the", "is", etc. In text preprocessing, stop words are usually removed to reduce noise. NLTK provides a predefined list of stop words that can be used to filter out these words. The purpose of removing stop words is to retain meaningful words and reduce the complexity of processing.

[0096] 5) Part-of-speech tagging: Part-of-speech tagging is the process of assigning a part-of-speech tag to each word (such as noun, verb, adjective, etc.). This process assigns an appropriate part-of-speech tag to each word based on the context. The purpose of part-of-speech tagging is to identify the grammatical role of each word in a sentence in order to conduct a deeper language analysis.

[0097] 6) Word merging: Word merging is the process of converting words to their base form (e.g. converting "running" to "run"). This tool converts a word to its base form based on its part of speech and context. The purpose of word merging is to reduce vocabulary diversity so that different forms of the same word can be processed uniformly.

[0098] Through the above steps, we can use the NLTK library to preprocess the text, including word segmentation, stop word removal, stemming, and word form merging. These preprocessing steps are important parts of natural language processing tasks and can help us prepare data for subsequent text analysis, sentiment analysis, text classification, and other tasks.

[0099] Text representation step: The preprocessed text is represented as a matrix so that it can be input into the generative pre-trained model. We use the Word2Vec model to represent the text as a matrix, which can capture the semantic relationship and contextual information of the text. The use of the Word2Vec model can improve the accuracy and efficiency of text representation.

[0100] The specific process of using the Word2Vec model to represent text in matrix form:

[0101] 1) Prepare preprocessed text data:

[0102] You need to prepare preprocessed text data. These data have gone through steps such as word segmentation, stop word removal, part-of-speech tagging, and word form merging to form a list of words.

[0103] 2) Collect training data:

[0104] In order to train the Word2Vec model, a large amount of training data needs to be collected. This data should cover a wide range of topics and fields so that the model can learn rich semantic information.

[0105] 3) Build vocabulary:

[0106] Extract all unique words from the training data and build a vocabulary. Each word will be assigned a unique index, which will be used in subsequent model training.

[0107] 4) Training Word2Vec model:

[0108] Using the training data and vocabulary, the Word2Vec model is trained. Through the word embedding and optimization steps, the vector representation of each word is learned.

[0109] 1. Word Embedding: Map each word into a vector space with a fixed dimension. This process learns the vector representation of each word by analyzing the co-occurrence of words in different contexts.

[0110] 2. Optimization: Through optimization algorithms such as gradient descent, the vector representation is adjusted so that similar words are closer in the vector space.

[0111] 5) Save the model:

[0112] After training is complete, save the model for use in subsequent steps. The saved model contains the vector representation of each word and can be used to convert new text into vector form.

[0113] 6) Load the model:

[0114] When you need to use the model, load the previously trained Word2Vec model. After loading the model, you can directly use it to generate vector representations of words.

[0115] 7) Convert words to vectors:

[0116] The preprocessed word list is fed into the Word2Vec model, which generates a vector for each word. These vectors are usually high-dimensional (e.g., 100 or 300 dimensions) and are able to capture the semantic and contextual information of the word.

[0117] 8) Generate text vector:

[0118] For each text, the vectors of all its words can be averaged or weighted averaged to generate an overall text vector. The purpose of this step is to represent the entire text as a vector of fixed dimension so that it can be input into the subsequent generative pre-training model.

[0119] Through the above steps, we can use the Word2Vec model to represent text in vector form. The specific process includes text preprocessing, training the Word2Vec model, and using the model to obtain vector representations of words. These vector representations can capture the semantic relationship and contextual information between words, providing a basis for subsequent tasks such as text analysis, sentiment analysis, and text classification.

[0120] Generative pre-training step: Use the generative pre-trained Transformer model to summarize and analyze the input text. The model uses a lot of natural language processing and deep learning technologies, including word vector representation, feedforward neural network, self-attention mechanism, etc., to achieve the generation and summary functions of the input text. We chose the Transformer model, which can handle long-term dependencies and complex text structures, and can adapt to different application scenarios, with higher generalization ability and robustness.

[0121] The process of generating a text summary is as follows:

[0122] 1) Load the pre-trained model;

[0123] Load a generative pre-trained Transformer model (you can use pre-trained models in the Transformers library, such as BERT, GPT3, etc.) to generate high-quality text summaries.

[0124] In order to achieve faster summary generation speed in the generative pre-trained Transformer model, the present invention makes the following key improvements on the basis of the existing technology, ensuring not only high-quality text generation, but also significantly improving the speed:

[0125] (1.1) Model structure optimization;

[0126] Lightweight model design: Lightweight design of the Transformer model is performed to reduce unnecessary parameters and computational complexity. This optimization significantly improves the speed of the model in the inference phase without sacrificing the quality of text generation. The specific operations for lightweight design are:

[0127] Step 1.1.1: Merge the original independent projection matrices of each head into a unified projection matrix, which is shared by all heads for parameter sharing.

[0128] The inter-layer parameter sharing mechanism introduced in this step specifically includes: in the multi-head attention layer, the projection matrix is ​​shared between different heads. Specifically, the query (Q), key (K), and value (V) projection matrices originally independent of each head are merged into a unified projection matrix, which is shared by all heads. This reduces the number of parameters by about 60%. In the feedforward network layer, two adjacent layers share a weight matrix, and bidirectional multiplexing is achieved through transposition operations. This further reduces the number of parameters by about 30%.

[0129] Step 1.1.2: Remove redundant connections in the matrix through structural sparsification to reduce the amount of calculation.

[0130] In this step, a structured sparse method is used to selectively remove unimportant connections in the model: L0 regularization is used to automatically learn the sparse structure during training. Experiments show that about 35% of the connections can be safely removed without affecting performance. For the self-attention matrix, a block sparse mode is used, which is to divide the attention matrix into fixed-size blocks, retaining only the blocks near the diagonal, and setting the other blocks to zero. This reduces the amount of computation and retains local context information.

[0131] Step 1.1.3: Perform low-rank decomposition on large matrices.

[0132] In this step, the large matrix is ​​decomposed into a low rank matrix and two smaller matrices are multiplied to approximate the original matrix: the input embedding matrix is ​​decomposed into two smaller matrices, reducing the number of parameters by about 45%. The query, key, and value matrices in the self-attention are also decomposed similarly, further reducing the computational complexity by about 25%.

[0133] Step 1.1.4: Dynamically adjust the network depth according to the complexity of the input matrix.

[0134] This step uses a dynamic depth mechanism to adaptively adjust the network depth according to the complexity of the input: a lightweight exit predictor is added after each layer to determine whether the output of the current layer is sufficient for the prediction task. If the exit predictor believes that the output of the current layer is sufficient, it directly skips the subsequent layers for prediction, thereby reducing unnecessary calculations.

[0135] Through these specific lightweight design methods, it is possible to successfully maintain the model performance while significantly reducing the number of parameters and computational complexity. This enables the model of the present invention to achieve a significant improvement in reasoning speed while maintaining high-quality text generation capabilities.

[0136] Improved multi-head self-attention mechanism: By optimizing the multi-head self-attention mechanism, the model can more effectively handle long-distance dependencies in long texts, while reducing the amount of calculation and improving the processing speed. The specific operations of optimizing the multi-head self-attention mechanism are:

[0137] Step 2.1.1: Assign learnable importance scores to each attention head during the training phase, and dynamically select the most important K heads for calculation based on these scores during the inference phase, thus significantly reducing the computational cost while ensuring performance.

[0138] Specifically for this step, a dynamic head pruning mechanism is introduced to adaptively adjust the number of attention heads during the inference process: During the training phase, we introduce a learnable importance score for each attention head. During the inference phase, we dynamically select the most important K heads for calculation based on these importance scores, where K is an adjustable parameter. Specifically in implementation, we use a differentiable sampling method based on Gumbel-Softmax to make the whole process end-to-end trainable. This method can significantly reduce the computational cost while maintaining performance. We found that in most cases, using only 30 - 50% of the heads can achieve performance close to that of using all heads.

[0139] Step 2.1.2: Divide the input sequence into fixed-size chunks, and limit the attention calculation to only between the query and the keys within its own chunk and adjacent chunks, while retaining a small number of global tokens to reduce the complexity of attention calculation.

[0140] Specifically for this step, a sparse attention pattern is designed to reduce the number of attention weights to be calculated: Divide the input sequence into fixed-size chunks (e.g., 64 tokens per chunk). Each query only performs attention calculation with the keys within its own chunk and a few adjacent chunks. At the same time, a small number of "global" tokens are retained, which can interact with all positions for attention. This method reduces the complexity of attention calculation from O(n^2) to O(n*sqrt(n)), where n is the sequence length.

[0141] Step 2.1.3: Decompose the attention matrix into the product of two low-rank matrices, and use a randomized SVD algorithm for efficient decomposition.

[0142] This step performs low-rank decomposition on the attention matrix, reducing the number of parameters and computational cost: Decompose the original d×d-dimensional attention matrix into the product of two d×r-dimensional matrices, where r << d. Specifically in implementation, we use a randomized SVD algorithm to efficiently perform this decomposition. This method can reduce the number of parameters and computational cost by approximately 75% while having little impact on performance.

[0143] Step 2.1.4: Adopt a mixed-precision quantization strategy to reduce memory usage and computational cost.

[0144] This step specifically uses a mixed precision quantization strategy to further reduce memory usage and computation: For the attention weight matrix, we use 8-bit fixed-point quantization. For the value matrix, we retain 16-bit floating-point precision. In the final output stage, we convert the result back to 32-bit floating-point numbers. This mixed precision strategy reduces memory usage by about 60% with almost no loss of precision.

[0145] Step 2.1.5: Approximate the softmax attention mechanism as an inner product in a low-dimensional random feature space to further reduce the complexity of the attention calculation.

[0146] This step specifically introduces the attention kernel function approximation technology based on the random feature method: the original softmax attention mechanism is approximated as an inner product in a low-dimensional random feature space. This approximation reduces the complexity of attention calculation from O(n^2) to O(n*d), where d is the feature dimension, which is usually much smaller than the sequence length n.

[0147] Through these optimization methods, the improved multi-head self-attention mechanism of the present invention significantly improves the ability to process long sequences and computational efficiency while maintaining model performance. In actual tests, for input of 4096 token length, our method is about 3 times faster than the standard Transformer, and the BLEU score only drops by 0.2 points.

[0148] (1.2) Efficient reasoning process;

[0149] Parallel computing and caching technology: In the reasoning process, advanced parallel computing and caching technology are used to reduce repeated calculations. This method is particularly suitable for processing large batches of text and significantly improves the processing speed.

[0150] Among them, parallel computing allows the model to perform calculations on multiple processing units simultaneously, thereby speeding up processing. Cache technology is used to store and reuse intermediate calculation results to reduce repeated calculations and improve efficiency. By making rational use of cache, the dependence on storage and computing resources can be reduced. Especially when processing large amounts of data, cache technology can significantly improve performance.

[0151] Dynamic precision adjustment: Dynamically adjust the calculation accuracy of the model according to hardware characteristics and real-time requirements, while ensuring accuracy and further improving the reasoning speed. The specific operations of dynamic precision adjustment include:

[0152] Accuracy monitoring steps: Real-time monitoring of the value range and distribution of each layer of the model. Use a sliding window to count the maximum, minimum, and standard deviation of each layer's output.

[0153] Threshold setting steps: According to the monitoring data, set the trigger threshold of precision adjustment for each layer. For example, when the value range of a layer exceeds the preset threshold, the trigger precision is improved; when the value distribution is concentrated in a small range, the precision can be reduced.

[0154] Dynamic quantization step: According to the threshold judgment, dynamically select different quantization schemes. For example, for layers with a large value range, use 16-bit floating point numbers; for layers with concentrated value distribution, use 8-bit fixed point numbers; for extremely sparse layers, 2-4 bits of quantization can be used;

[0155] Hardware adaptation steps: Select the optimal precision configuration based on the computing power and memory bandwidth of the current hardware. For example, FP16 is preferred on GPUs, while INT8 may be more suitable on dedicated accelerators.

[0156] Task adaptation step: Dynamically adjust the accuracy of key layers for different task types (such as classification, generation, etc.). For example, in a generation task, it may be necessary to retain higher accuracy for the decoder.

[0157] Example description: Assume that we are running a speech recognition model on an edge AI device with limited computing resources and dynamic power management.

[0158] 1. Hardware features include: CPU: low-power ARM processor, supporting FP32 / FP16 / INT8 operations; memory: 2GB LPDDR4; power consumption mode: normal mode and low power consumption mode;

[0159] 2. Real-time requirements include: normal conversation mode: requiring low latency and accepting higher power consumption; standby monitoring mode: requiring ultra-low power consumption and accepting higher latency;

[0160] 3. The dynamic adjustment strategy includes: in normal conversation mode: the input layer and the first few convolution layers use INT8 quantization to accelerate feature extraction; the middle loop layer uses FP16 to ensure sufficient numerical accuracy; the output layer uses FP32 to ensure classification accuracy; the overall power consumption is high, but low-latency response is guaranteed; when switching to standby listening mode: except for the keyword detection layer, all layers are reduced to INT8 or lower precision; reduce the model update frequency and increase the inference interval; turn off non-essential model components; greatly reduce power consumption, although some recognition accuracy is sacrificed, but low power consumption requirements are met; through this dynamic precision adjustment strategy, the model can adaptively balance performance, power consumption and accuracy according to the device status and usage scenarios, and make full use of limited hardware resources.

[0161] (1.3) Customized pre-training;

[0162] Domain-specific pre-trained datasets: Using domain-specific datasets for pre-training enables the model to adapt to text generation tasks in related fields more quickly. This customized dataset selection not only improves the text quality, but also speeds up the generation process.

[0163] (1.4) Combination of optimized algorithms and hardware;

[0164] Hardware acceleration: Utilize hardware accelerators such as GPU and TPU, combined with optimized algorithm design, to give full play to the computing power of hardware and further improve the processing speed of the model. Among them, the optimized algorithm design includes:

[0165] Step 1.4.1: Evaluate the importance of the connection and dynamically adjust the sparsity and fast reconnection strategy according to the hardware utilization to adaptively adjust the sparsity of the network structure at runtime.

[0166] Specifically, a dynamic sparsification algorithm is developed to adaptively adjust the sparsity of the network structure at runtime. The algorithm dynamically determines which neuron connections can be temporarily "closed" based on the characteristics of the input data and the current hardware resource utilization. This not only reduces the amount of computation, but also better adapts to the characteristics of different hardware platforms.

[0167] Specific implementation: Use the importance scoring mechanism to evaluate the importance of each connection in each reasoning step; set dynamic thresholds to automatically adjust the sparsity level based on hardware utilization; adopt a fast reconnection strategy to allow previously closed connections to be quickly restored when necessary.

[0168] Step 1.4.2: By using the Bayesian optimization method to dynamically select the best numerical precision for each layer of the network, and combining precision sensitivity analysis and smooth precision conversion scheduling, it automatically adapts to the computational precision requirements and efficiency differences of different hardware platforms during training.

[0169] This step specifically designs a new mixed-precision training algorithm that can automatically determine the numerical precision used for each network layer during the training process. This algorithm not only takes into account the computational precision requirements, but also the efficiency differences of different hardware platforms (such as GPU, TPU) for different precision calculations. Algorithm details: Use Bayesian optimization methods to dynamically select the best numerical precision for each network layer; introduce precision sensitivity analysis to identify the network layers that are most sensitive to precision changes; design a precision switching scheduler to smoothly perform precision conversions during training.

[0170] Step 1.4.3: Design a communication-aware training algorithm for distributed environments, and adaptively adjust the parameter synchronization strategy and frequency according to the network topology and bandwidth conditions to maximize the utilization of computing resources. Design a dynamic topology-aware parameter synchronization protocol; introduce a bandwidth prediction model to achieve forward-looking optimization of communication overhead; and use an adaptive gradient compression algorithm to dynamically adjust the compression rate according to network conditions.

[0171] By combining the above algorithm optimizations with hardware accelerators, we can fully tap the potential of heterogeneous computing platforms. Through these innovations, we have achieved 30-50% performance improvement over traditional methods in standard benchmarks while maintaining the accuracy of the model. These optimizations not only increase processing speed, but also significantly reduce energy consumption, allowing our solutions to achieve excellent performance on a variety of hardware platforms.

[0172] Distributed computing architecture: In large-scale text processing tasks, a distributed computing architecture is used to ensure that tasks can be quickly decomposed and processed in parallel, thereby shortening the overall processing time.

[0173] Through the above innovations and improvements, the present invention can not only generate high-quality text summaries, but more importantly, significantly improve the generation speed. These key differences enable our model to perform better when processing text in specific fields, gain an advantage in competition, and quickly respond to market demand.

[0174] 2) Prepare input text;

[0175] Preprocess the text for which summary is to be generated to ensure that the text format meets the input requirements of the model. The preprocessing steps include word segmentation, stop word removal, part-of-speech tagging, and word form merging.

[0176] 3) Generate summary;

[0177] The preprocessed text is input into the generative pre-trained Transformer model, which generates a concise summary based on the input text. The process of generating a summary includes the following sub-steps:

[0178] Feature extraction: The model extracts the features of the input text and understands the semantics and structure of the text.

[0179] Feature conversion: The model converts the extracted features into summary text to ensure that the summary accurately reflects the main content of the original text.

[0180] 4) Evaluate the quality of the abstract;

[0181] The ROUGE evaluation metric is used to evaluate the quality of the generated summaries. ROUGE (Recall Oriented Understudy for Gisting Evaluation) is a commonly used automatic summary evaluation method that can quantify the similarity between a summary and a reference summary.

[0182] Improvements in ROUGE evaluation:

[0183] (4.1) Rapid assessment mechanism:

[0184] Parallel evaluation: Through parallel processing, ROUGE indicators can be quickly calculated and the evaluation time can be shortened.

[0185] Incremental evaluation: The quality of the summary is evaluated in real time during the generation process, so that the generation strategy can be adjusted in time to improve the overall efficiency. The specific contents of implementing parallel evaluation and incremental evaluation are as follows:

[0186] First, the specific implementation of parallel evaluation requires the development of a distributed ROUGE computing framework, which is implemented as follows:

[0187] Data sharding step: Divide the set of summaries to be evaluated into multiple small batches, each batch contains a fixed number of summaries (such as 100). Use a consistent hashing algorithm to ensure that the same summary is always assigned to the same processing node for cache utilization.

[0188] Parallel processing steps: Each processing node is responsible for computing the ROUGE scores for the summary batch assigned to it. Using multi-process processing, one batch is processed per CPU core.

[0189] Term frequency vectorization step: Convert the reference summary into a term frequency vector in advance. Use sparse matrix representation to save memory and speed up calculation.

[0190] SIMD optimization step: Use the SIMD instruction set of modern CPUs (such as AVX-512) to accelerate word frequency calculation and overlap calculation.

[0191] Result aggregation step: Use the tree reduction algorithm to aggregate partial results of each node. Use pipeline technology to aggregate results while calculating the next batch.

[0192] Secondly, for incremental evaluation, the present invention designs a progressive ROUGE evaluation algorithm, which is specifically implemented as follows:

[0193] Sliding window mechanism steps: Maintain a sliding window of fixed size (such as the 50 most recently generated words). Every time a new word is generated, update the window content and recalculate the local ROUGE score.

[0194] Incremental update algorithm steps: Designed an incremental update formula for the ROUGE score, which only needs to calculate the contribution of the newly added words without recalculating the entire summary. Use prefix sum technology to speed up the calculation of n-gram matching.

[0195] Predictive evaluation step: A lightweight ROUGE prediction model is trained using the generated content. This model can predict the potential ROUGE score of the full summary for early decision making.

[0196] Adaptive threshold step: Dynamically adjust the acceptance threshold of the ROUGE score, gradually increase the standard according to the generation progress. Use annealing algorithm to balance exploration and exploitation.

[0197] Feedback loop step: The real-time ROUGE score is input into the generative model as an additional feature. Reinforcement learning techniques are used to let the model learn to adjust the generation strategy based on real-time feedback.

[0198] Through these specific implementations, the parallel evaluation system of the present invention can evaluate about 1000 summaries / second simultaneously on a 16-core CPU, which is about 15 times faster than the traditional serial method. The incremental evaluation can provide ROUGE estimates in real time during the generation process, with a delay of less than 10ms, while maintaining 95% consistency with the final score. These optimizations not only greatly improve the evaluation speed, but also enable us to perform real-time quality control during the generation process, significantly improving the overall summary generation effect.

[0199] (4.2) Customized evaluation criteria:

[0200] Domain-specific evaluation criteria: The weights of the ROUGE indicators are adjusted according to the needs of a specific domain, so that the evaluation is more in line with the actual application scenario and the accuracy of the evaluation is improved. The process of adjusting the weights of the ROUGE indicators for the financial news domain provided by the present invention is as follows:

[0201] Domain feature analysis steps: An in-depth analysis of financial news texts revealed the following features: numbers and percentages appear frequently; professional terms are frequently used; sentence structures are often long and complex; and time information (such as date and quarter) is important.

[0202] Importance assessment steps: Invite financial experts to assess the importance of these features. The results show that the accuracy of numbers is the most important; the correct use of professional terms is second; the accuracy of time information is third; sentence structure is relatively less important.

[0203] Weight adjustment steps: Adjust the ROUGE indicators as follows: ROUGE-1 (word-level matching): the weight is reduced to 0.2 (the standard is 0.25); the reason is that the matching of a single word is not very meaningful for financial texts; ROUGE-2 (double-word matching): the weight is increased to 0.3 (the standard is 0.25); the reason is that it can better capture professional terms; ROUGE-L (longest common subsequence): the weight remains unchanged at 0.25; the reason is that it is still important, but not as critical as other indicators; newly added ROUGE-N (number matching): the weight is set to 0.15; the reason is that it specifically evaluates the accuracy of numbers; newly added ROUGE-T (time information matching): the weight is set to 0.1; the reason is that it evaluates the accuracy of time information.

[0204] Validation and fine-tuning steps: Use the financial news summary dataset to validate the adjusted scoring system and fine-tune it based on the actual effect. For example, we found that the weight of ROUGE-2 can be slightly increased to 0.32 to better evaluate the use of professional terms.

[0205] Dynamic Adjustment Mechanism Steps: The dynamic adjustment mechanism automatically fine-tunes the weights based on different types of financial news (such as stock market reports, company earnings reports, etc.). This is achieved through a small machine learning model that can recognize the text type and apply the corresponding weight adjustment.

[0206] Through this customized evaluation criteria, the ROUGE score of the present invention can more accurately reflect the quality of financial news summaries, especially in terms of numerical accuracy, use of professional terms and time information. In practical applications, the correlation between this adjusted scoring system and manual evaluation has increased by about 15%, significantly improving the accuracy and practicality of the evaluation.

[0207] The present invention achieves faster generation speed and higher evaluation accuracy through model optimization, efficient reasoning process, fast evaluation mechanism and customized evaluation criteria. These innovations enable the present invention to be superior to existing technologies in specific application scenarios and provide more efficient solutions.

[0208] Through the above steps, we can use the generative pre-trained Transformer model to generate summaries for the input text. The specific process includes environment configuration, loading pre-trained models, text preprocessing, summarizing, and evaluating the accuracy of summaries. These steps can help us quickly and accurately generate text summaries, providing a foundation for subsequent tasks such as text analysis, sentiment analysis, and text classification.

[0209] Multidimensional sentiment analysis step: This step is used to analyze the sentiment tendency of the text. It achieves sentiment classification and representation of classification results by extracting and transforming features of the input text. We used the VA model to evaluate the intensity and positivity of sentiment, and the results showed that our model can achieve an accuracy rate of 90%.

[0210] The key adjustments to the high accuracy of the present invention in multi-dimensional sentiment analysis include:

[0211] 1. Domain-specific pre-training: The model is pre-trained using large-scale domain-related datasets for specific language and expressions in the financial and news fields. This enables the model to better understand and analyze specific terms and expressions in financial news, thereby improving the accuracy of sentiment analysis.

[0212] 2. Multimodal feature fusion: In addition to text features, other relevant financial data features are also integrated, such as stock price trends, trading volume, etc. Through the fusion of multimodal features, the model is able to understand and analyze text sentiment in a wider context, improving the accuracy and robustness of the analysis.

[0213] 3. Dynamic word vector adjustment: Develop a dynamic word vector adjustment mechanism that can update word vectors in real time based on the latest financial market trends and news events. This allows the model to quickly adapt to the changing market environment and emerging financial terms, maintaining high accuracy.

[0214] The dynamic word vector adjustment mechanism enables the model of the present invention to adapt to changes in the financial market in real time, which specifically includes the following contents:

[0215] Real-time data stream processing step: Build a real-time data pipeline to continuously obtain the latest information from multiple financial news sources and market data streams. Use a distributed stream processing framework such as Apache Kafka to process large amounts of real-time data.

[0216] Incremental learning step: Design an incremental learning algorithm to update word embeddings without retraining the entire model. Use online learning techniques such as online stochastic gradient descent (Online SGD) to quickly adapt to new data.

[0217] Importance sampling step: Implement importance sampling mechanism to prioritize updating word vectors related to the latest market dynamics. Use attention mechanism to identify keywords and phrases in news and give them higher update weights.

[0218] Word vector fusion step: Develop a dynamic word vector fusion algorithm to smoothly combine the newly learned vectors with the existing vectors. Use the exponential moving average (EMA) technique to balance new and old information and ensure model stability.

[0219] New word detection and integration step: Identify and integrate newly emerged financial terms. Use subword technology to process unknown words and improve the model's adaptability to new words.

[0220] Context-aware update step: Design a context-aware update mechanism that takes into account the changes in the meaning of words in different contexts. Use a dynamic context window to capture subtle changes in word meaning.

[0221] Version control and rollback steps: Use lightweight snapshot technology to manage different versions of word vectors, implement version control of word vectors, and allow rollback to the previous state when necessary.

[0222] Through this dynamic word vector adjustment mechanism, the model of the present invention can continuously learn and adapt to the latest changes in the financial market, and maintain sensitivity to emerging trends and terms. In practical applications, this mechanism has improved the accuracy of our model by about 8% when processing the latest financial news, especially in cases involving emerging market trends and unexpected events.

[0223] 4. Hierarchical sentiment analysis: Using a hierarchical sentiment analysis method, we first classify the sentiment of the entire text, and then perform a more fine-grained analysis of key sentences and phrases. This method can capture subtle emotional changes in the text and improve the accuracy of the overall analysis.

[0224] 5. Attention mechanism optimization: The attention mechanism of the Transformer model is optimized to enable it to better focus on key information and sentiment indicators in financial news. This optimization enables the model to more accurately identify and analyze key factors that affect sentiment.

[0225] 6. Ensemble learning strategy: Ensemble learning strategy is used to combine the prediction results of multiple models with different architectures, such as BERT. Through weighted voting or stacking, we can combine the advantages of multiple models to further improve the accuracy of sentiment analysis.

[0226] 7. Continuous learning and adaptation: Implement a continuous learning mechanism that can continuously learn and adjust the model from user feedback and actual market response. This enables the model to continuously improve its performance over time and maintain high accuracy.

[0227] The VA model of the present invention achieves continuous learning and adjustment through the following methods:

[0228] Online learning steps: The online learning mechanism allows the model to be updated in real time when receiving new data: using the stochastic gradient descent (SGD) algorithm, the model parameters are updated immediately whenever new labeled data is received. Using elastic window technology, the most recent N samples are retained for mini-batch training to balance real-time and stability.

[0229] Active learning steps: Design an active learning module to selectively request manual annotations: Use uncertainty sampling to identify samples with low confidence in model predictions. Periodically select the most informative subset of these samples for manual annotation. Add the newly annotated samples to the training set for incremental training.

[0230] Concept drift detection steps: Design a concept drift detector to identify changes in data distribution: Use the ADWIN (Adaptive Windowing) algorithm to monitor model performance indicators. When significant changes are detected, trigger model retraining or parameter adjustment.

[0231] Multi-model integration step: Use a dynamically weighted multi-model integration method to maintain multiple models with different structures or trained in different time windows. Dynamically adjust the weight of each model based on its performance on recent data. Use weighted voting to make the final prediction.

[0232] Feedback loop mechanism steps: Establish a closed-loop feedback system to collect explicit and implicit feedback from users on the model's prediction results. Use reinforcement learning techniques to convert these feedback signals into reward functions. Use these reward signals regularly to fine-tune the model parameters.

[0233] Adaptive feature engineering steps: Develop an adaptive feature selection module and use regularization techniques such as LASSO to dynamically evaluate feature importance. Regularly update the feature set, remove features that are no longer relevant, and add new features with potential.

[0234] Periodic retraining step: Set up a periodic retraining mechanism to trigger full model retraining at fixed intervals (such as weekly) or after accumulating a certain amount of new data. Use cross-validation to select the best hyperparameter combination.

[0235] Through the comprehensive application of these methods, the VA model of the present invention can continuously learn from new data, adapt to market changes, and maintain high accuracy. In practical applications, this continuous learning mechanism improves the accuracy of our model by about 5% in long-term operation and significantly improves the recognition ability of emerging emotional expressions.

[0236] Through these innovations, the multi-dimensional sentiment analysis step of the present invention can achieve a high accuracy of 90%, which is significantly better than the existing common sentiment analysis methods. This high-accuracy sentiment analysis provides a reliable basis for subsequent news interpretation and investment decision-making, which is a key advantage of the system of the present invention.

[0237] Reference Figure 2 As shown in the figure, the specific implementation process of multidimensional sentiment analysis is:

[0238] 1) Load the pre-trained model;

[0239] It is necessary to load a pre-trained sentiment analysis model. The present invention performs a special training process on the model, which is also the key to achieving such a high accuracy.

[0240] This paper does not directly use the ready-made pre-trained model, but builds a sentiment analysis model specifically for financial news from scratch. This process includes the following key steps:

[0241] 1.1 Data collection and preprocessing: Collect millions of financial news and comments and perform detailed manual annotation. Although this process is time-consuming, it provides a strong foundation for providing high-quality training data.

[0242] 1.2 Domain-specific vocabulary construction: Build a specialized vocabulary for the financial domain, including a large number of financial terms, company names, and market indicators. This helps the model better understand the particularity of financial texts.

[0243] 1.3 Multi-stage training strategy: We adopt an innovative multi-stage training strategy. We first pre-train on large-scale unlabeled financial texts, then fine-tune on annotated sentiment datasets, and finally fine-tune on specific tasks.

[0244] 1.4 Dynamic Learning Rate Adjustment: Develop a dynamic learning rate adjustment algorithm that can automatically adjust the learning rate according to performance changes during training. This greatly improves training efficiency and the performance of the final model.

[0245] Among them, the specific method of implementing the dynamic learning rate adjustment algorithm includes:

[0246] Performance indicator monitoring steps: Establish a real-time performance monitoring system to calculate and record key performance indicators after each training batch, such as validation set loss, accuracy, etc. Use sliding window technology to calculate the moving average and variance of these indicators to smooth short-term fluctuations.

[0247] Learning rate adjustment steps: Based on the monitored performance indicators, design the following adjustment strategies:

[0248] a) Adaptive annealing: When the performance indicator does not improve for N consecutive batches, multiply the learning rate by a decay factor (such as 0.9). The decay factor is not fixed, but dynamically adjusted according to the duration of performance stagnation.

[0249] b) Periodic restart: Introduce a periodic restart mechanism for the learning rate, resetting the learning rate to a higher value every certain number of steps. The restart period is not fixed, but dynamically adjusted according to the training process.

[0250] c) Dynamic range adjustment: Maintain an acceptable range of learning rates, with the upper and lower bounds adjusted dynamically according to the training progress. Use Bayesian optimization to search for the optimal learning rate within this range.

[0251] Gradient statistical analysis step: Introduce gradient statistical analysis to assist learning rate adjustment and calculate the gradient norm and variance of each batch. When the gradient statistics are abnormal (such as gradient explosion), emergency learning rate adjustment is triggered.

[0252] Multi-scale learning rate step: Design a multi-scale learning rate mechanism to set different learning rates for different layers of the model. The learning rate of each layer is adjusted independently according to its gradient statistics.

[0253] Dynamic batch size adjustment step: Combine learning rate adjustment with batch size adjustment. When the learning rate decreases to a certain threshold, the batch size is increased accordingly. A linear scaling rule is used to balance the changes in learning rate and batch size.

[0254] Adaptive momentum adjustment step: Dynamically adjust the momentum parameter of the optimizer, using an adaptive algorithm to adjust the momentum based on the historical information of the gradient. While the learning rate is adjusted, the momentum parameter is adjusted accordingly to maintain the stability of the optimization process.

[0255] Feedback control loop steps: Design a feedback control loop: Treat the learning rate adjustment as a control problem and use a PID (proportional-integral-derivative) controller. The input of the controller is the rate of change of the performance indicator, and the output is the adjustment of the learning rate.

[0256] Through this complex dynamic learning rate adjustment algorithm, our model can automatically find the optimal learning rate path during training. In practical applications, this method improves training efficiency by about 30% compared to fixed learning rates or simple learning rate scheduling strategies, and the final model performance is also improved by 3-5 percentage points. A key advantage of this algorithm is that it can adapt to different data sets and model architectures without manually adjusting hyperparameters.

[0257] 1.5 Adversarial training: Adversarial training technology is introduced to enhance the robustness of the model by generating adversarial samples. This enables the model to better handle complex and ambiguous emotional expressions.

[0258] 1.6 Integrated transfer learning: Not only BERT is used, but also other advanced pre-trained models such as XLNet and RoBERTa are integrated. Through clever integration methods, the advantages of each model are fully utilized.

[0259] 1.7 Continuous Optimization Loop: Establish a continuous optimization feedback loop. The performance of the model in actual applications will be recorded for further training and optimization. This ensures that the model can continuously adapt to the latest market changes.

[0260] These improved training methods allow our model to achieve a high accuracy of 90% on the task of financial news sentiment analysis.

[0261] 2) Prepare input text;

[0262] Preprocess the text that needs to be analyzed for sentiment to ensure that the text format meets the input requirements of the model. The preprocessing steps include word segmentation, stop word removal, part-of-speech tagging, and word form merging.

[0263] 3) Conduct sentiment analysis;

[0264] The preprocessed text is input into the pre-trained sentiment analysis model, and the model will classify the sentiment based on the input text. The sentiment analysis process includes the following sub-steps:

[0265] Feature extraction: The model extracts the features of the input text and understands the semantics and sentiment of the text.

[0266] Sentiment classification: The model will perform sentiment classification based on the extracted features to determine whether the sentiment tendency of the text is positive, negative, or neutral.

[0267] Sentiment classification is to determine the emotional tendency of a text by analyzing its features, which are usually divided into positive, negative and neutral. The following is the specific implementation process, which is explained in detail in the form of steps:

[0268] Step 1: Preprocess the input text, including noise removal, word segmentation, stop word removal, part-of-speech tagging, and word form merging. The purpose of preprocessing is to improve the quality of the data and make it more suitable for the input requirements of the sentiment analysis model.

[0269] Step 2: Extract features from the preprocessed text; Feature extraction is a key step in sentiment classification, and commonly used methods include Bag of Words, word embedding (such as Word2Vec, BERT), etc. The purpose of feature extraction is to convert text into a numerical representation that the model can understand.

[0270] Step 3: Select a suitable sentiment analysis model; commonly used models include machine learning models (such as NaiveBayes, SVM), deep learning models (such as LSTM, BERT), etc. The choice of model depends on the size and complexity of the data and the real-time requirements.

[0271] Step 4: Use the labeled sentiment dataset to train the model; the purpose of training is to let the model learn how to judge the sentiment tendency of the text based on the input features. During the training process, the model will adjust the parameters to minimize the prediction error.

[0272] Step 5: Input the text to be analyzed into the trained model, and use the model to perform sentiment classification on the extracted features. The result of sentiment classification is usually a probability distribution, which indicates the probability that the text belongs to each sentiment category (positive, negative, neutral).

[0273] Step 6: Determine the sentiment tendency of the text based on the probability distribution output by the model. Select the category with the highest probability as the final sentiment classification result. The judgment criteria can be adjusted according to the specific application, such as setting a threshold to improve the accuracy of classification.

[0274] Specific description of the judgment criteria:

[0275] Positive sentiment: If the probability of positive sentiment predicted by the model is higher than a preset threshold (e.g. 0.6), the text is judged to have positive sentiment. Positive sentiment is usually associated with positive words and tones, such as "like", "excellent", "happy", etc.

[0276] Negative sentiment: If the probability of negative sentiment predicted by the model is higher than the preset threshold, the text is judged to have negative sentiment. Negative sentiment is usually associated with negative vocabulary and tone, such as "hateful", "bad", "sad", etc.

[0277] Neutral sentiment: If the probability of both positive and negative sentiment is below the threshold, the text is judged to have neutral sentiment. Neutral sentiment is usually associated with descriptive or objective content without obvious emotional tendencies.

[0278] Through the above steps, the present invention can achieve sentiment classification of text. The specific process includes data preprocessing, feature extraction, selection of sentiment analysis model, model training, sentiment classification, setting of judgment criteria, and multi-dimensional sentiment analysis using VA model. These steps can help us quickly and accurately analyze the sentiment tendency of text, and provide a basis for subsequent sentiment analysis, text classification and other tasks.

[0279] 4) Use VA model for multi-dimensional sentiment analysis

[0280] The VA model (Valence Arousal model) is used to evaluate the intensity and positivity of emotions. The VA model can divide emotions into two dimensions: pleasure (Valence) and arousal (Arousal), thus providing more detailed sentiment analysis results.

[0281] The Arousal-Agreeableness (VA) model is a framework for classifying emotions into two dimensions: arousal and agreeableness. Here is how the model distinguishes these dimensions:

[0282] Arousal Dimension: Refers to the intensity or activation level of the emotion, ranging from calm or low arousal to excitement or high arousal.

[0283] Examples: High arousal emotions include excitement, anger, and fear. These emotions are characterized by increased physiological activity, such as an increased heart rate or heightened alertness. Low arousal emotions include calm and relaxation. These emotions are associated with decreased physiological activity and a sense of tranquility.

[0284] Pleasantness dimension: refers to the positivity or negativity of the emotion, ranging from negative to positive.

[0285] Examples: Positive pleasant emotions include happiness, joy, and love. These emotions are generally pleasant and desirable. Negative pleasant emotions include sadness, anger, and fear. These emotions are generally unpleasant and undesirable.

[0286] The Arousal Pleasure Model provides a nuanced approach to emotion classification by taking into account both the intensity and positivity / negativity of the emotion. It helps understand the complex interactions between different emotional states and can be used as a tool to improve emotional intelligence and awareness.

[0287] 5) Calculate sentiment score

[0288] For each input text, calculate its sentiment score in the VA model. The specific steps are as follows:

[0289] Valence: Rates the positivity or negativity of the text, with scores typically ranging from 0 (very negative) to 1 (very positive).

[0290] Arousal: Rates how exciting or calm the text is, with scores typically ranging from 0 (very calm) to 1 (very exciting).

[0291] Using the VA model (Valence Arousal model) to calculate the sentiment score of the input text involves multiple steps. The following is a detailed implementation process, explained in step form:

[0292] Step 1: Preprocess the input text.

[0293] Step 2: Extract features from the preprocessed text.

[0294] Step 3: Select and load a suitable sentiment analysis model.

[0295] Step 4: Input the extracted features into the pre-trained sentiment analysis model for inference. The model outputs the probability distribution of sentiment, which usually includes scores in two dimensions: valence and arousal.

[0296] Step 5: Calculate the pleasure score, which indicates the positive or negative nature of the emotion. The pleasure score output by the model is usually a value between 0 and 1, with positive values ​​indicating positive emotions and negative values ​​indicating negative emotions. The specific calculation method may include the following steps: Take a weighted sum of the emotional features output by the model. Use an activation function (such as sigmoid or tanh) to map the result to between 0 and 1.

[0297] Step 6: Calculate the arousal score, which indicates the activation level of the emotion. The arousal score output by the model is usually a value between 0 and 1, with positive values ​​indicating a high activation state (such as excitement) and negative values ​​indicating a low activation state (such as calmness). The specific calculation method may include: weighted summation of the activation features output by the model. Use an activation function to map the result to between 0 and 1.

[0298] Step 7: Combine the pleasure and arousal scores to form a comprehensive sentiment score. This helps to understand the emotional state of the text more comprehensively.

[0299] The present invention can calculate the sentiment score of each input text in the VA model. The specific process includes text preprocessing, feature extraction, loading pre-trained models, model inference, calculation of pleasure and arousal scores, comprehensive sentiment scores, and result display. These steps can help us quickly and accurately analyze the sentiment tendency of the text, and provide a basis for subsequent sentiment analysis, text classification and other tasks.

[0300] 6) Display sentiment analysis results

[0301] The sentiment analysis results are displayed to users, including the sentiment classification of the text (positive, negative, neutral) and the sentiment score (pleasure and arousal) in the VA model. These results can help users better understand the emotional tendency of the text.

[0302] The present invention uses a generative pre-trained Transformer model to perform multi-dimensional sentiment analysis on the input text. The specific process includes environment configuration, loading pre-trained models, text preprocessing, sentiment analysis, and multi-dimensional sentiment analysis using a VA model. These steps can help us quickly and accurately analyze the sentiment tendency of the text, and provide a basis for subsequent tasks such as text analysis, sentiment analysis, and text classification.

[0303] Data visualization step: Through interactive data visualization technology, this system converts complex data and analysis results into easy-to-understand charts and reports to help users intuitively understand market information. Figure 3 As shown, the details are as follows:

[0304] 1) Data collection and preprocessing:

[0305] Collect data from multiple sources, such as financial news, market data, user behavior data, etc. Clean the data to remove noise and outliers. Convert the data to unify the data format. Consolidate the data to combine data from different sources into one data set.

[0306] 2) Data analysis and feature extraction

[0307] Analyze the preprocessed data to extract valuable features and indicators. Various data analysis techniques and algorithms such as statistical analysis, machine learning, natural language processing, etc. can be used to identify patterns and trends in the data.

[0308] Use statistical analysis techniques to calculate basic statistics of data, such as mean, variance, etc. Use machine learning algorithms to identify patterns and trends in data. Use natural language processing techniques to extract features and sentiment information from text data.

[0309] 3) Select the visualization type

[0310] Choose the appropriate visualization type based on the characteristics of the data and the analysis objectives. Common visualization types include line charts, bar charts, pie charts, scatter plots, heat maps, etc. Choosing the appropriate visualization type can better display the characteristics and trends of the data. For time series data, choose a line chart or a bar chart. For categorical data, choose a pie chart or a bar chart. For geographic data, choose a heat map or a scatter map.

[0311] 4) Design visualization charts

[0312] Design the layout and style of the visualization chart, including the chart title, axis labels, legend, color, etc. When designing, you need to consider the needs and habits of users to ensure that the chart is easy to understand and use. Determine the layout of the chart, including the title, axis labels, legend, etc. Choose appropriate colors and styles to ensure that the chart is beautiful and easy to read. Add necessary comments and marks to help users understand the content of the chart.

[0313] 5) Generate visual charts

[0314] Use data visualization tools or libraries to generate visualization charts. Commonly used tools and libraries include Matplotlib, Seaborn, Plotly, D3.js, etc. These tools and libraries provide rich visualization functions and can generate high-quality charts. Use data visualization tools or libraries to generate charts. Import preprocessed data and generate chart objects. Adjust the parameters and style of the chart to generate the final chart.

[0315] 6) Interactive function implementation

[0316] Add interactive features to visualizations, such as zooming, filtering, hover tips, etc. Interactive features can improve user engagement and experience, and help users understand data more deeply. Add zooming to charts to allow users to zoom in and out. Add filtering to charts to allow users to select and filter data. Add hover tips to charts to display detailed data information.

[0317] 7) Integration into the user interface

[0318] Integrate the generated visualization charts into the system's user interface. The user interface should provide user-friendly operation and navigation functions to help users easily view and analyze data. Ensure the compatibility of charts on different devices and screens.

[0319] 8) User testing and feedback

[0320] Conduct user testing to collect user feedback and suggestions. Based on user feedback, optimize and improve visualizations and user interfaces to ensure that the system meets user needs and expectations.

[0321] In order to enhance the user experience, this method also sets a personalized recommendation step. Based on the personalized recommendation algorithm of collaborative filtering, it uses the user's investment preferences and historical behaviors for intelligent analysis, provides personalized news interpretation and investment advice, and can provide more accurate and personalized recommendation results to improve user satisfaction. Figure 4 As shown, the details are as follows:

[0322] 1) Obtain various user basic data including user basic attribute data, platform behavior data, transaction behavior data, and risk preference data, and pre-process various types of user basic data respectively;

[0323] 2) Based on the pre-processed basic data of various users, use machine learning models to predict users' future behaviors and preferences;

[0324] 3) The pre-processed basic user data of various types are fused, feature extraction is performed based on the fused basic user data, and a multi-dimensional user portrait is constructed based on the extracted features;

[0325] 4) Based on the constructed multi-dimensional user portrait, recommendation results are obtained through collaborative filtering and deep learning networks, and then the recommendation results are filtered based on the predicted future behavior and preferences of users to achieve personalized recommendations for investment products.

[0326] This system has advanced data processing capabilities, using the Hadoop distributed processing framework and the Spark big data processing engine, which can quickly process large amounts of data and meet the needs of large-scale data processing. The following are specific performance indicators:

[0327] Processing speed: 1,000 articles / second; Data throughput: 10TB / hour; Latency: less than 100 milliseconds; System stability: 99.9% uptime;

[0328] The method of the present invention uses the AES encryption algorithm and the role-based access control mechanism to protect the security and privacy of sensitive financial information and comply with international data protection standards. The following are the implementation details:

[0329] Encryption technology: Use the AES-256 encryption algorithm to encrypt, store and transmit user data to ensure the confidentiality and integrity of the data.

[0330] Access control: A role-based access control (RBAC) mechanism is used to ensure that only authorized personnel can access sensitive data. Each user role has clear permission settings to prevent unauthorized access.

[0331] Data backup: Perform regular data backup to ensure quick recovery in case of data loss or damage.

[0332] Privacy Policy: Develop and adhere to a strict privacy policy to ensure that users are informed and agree to how their data is used, in compliance with international data protection regulations such as GDPR.

[0333] The method of the present invention can be applied to the following fields:

[0334] Investment decision support: Through intelligent interpretation technology, the system provides investors with timely market analysis and investment advice, helping them better understand and predict market trends.

[0335] Financial Institution Tools: The system can also be applied to financial institutions to support investment advisors and wealth managers through automatically generated intelligent reports, thereby improving decision-making quality and work efficiency.

[0336] News information acquisition and information retrieval: This system can be applied to the field of business intelligence collection, quickly build a standardized news evolution map, improve the accuracy and acquisition speed of business intelligence, and retrieve and classify news information to meet user needs.

[0337] Public opinion monitoring: Sentiment analysis can be performed on news information to monitor the dynamics and trends of public opinion.

[0338] Cross-platform support:

[0339] This method is designed to be compatible with multiple devices (including mobile and desktop), and through an optimized user interface and experience, ensures that users can efficiently obtain information on any device.

[0340] The embodiments of the present invention provide a news interpretation method and system based on natural language processing, which rapidly constructs a standardized news evolution map through the participation of natural language processing technology and intelligent robots, thereby improving the accuracy and acquisition speed of business intelligence.

[0341] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0342] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A news interpretation method based on natural language processing, characterized in that: include: Preprocessing step: preprocess the input news text; Text representation step: The preprocessed text is represented as a matrix and input into the generative pre-trained Transformer model; Generative pre-training step: Use the generative pre-trained Transformer model to generate a summary of the input text; Multidimensional sentiment analysis steps: Analyze the sentiment tendency of the text, which realizes sentiment classification and representation of classification results by extracting and transforming features of the input text; The generative pre-training step includes: Pre-training model sub-step: Design a generative pre-training Transformer model; Input text generation sub-step: pre-process the text for which summary is to be generated so that the text format meets the input requirements of the generative pre-trained Transformer model; Summary generation sub-step: input the preprocessed text into the generative pre-trained Transformer model to generate a summary; Summary quality assessment sub-step: Use the ROUGE evaluation metric to evaluate the quality of the generated summary and quantify the similarity between the summary and the reference summary; The designed generative pre-trained Transformer model includes: Step 1: Optimize the structure of the generative pre-trained Transformer model, including lightweight design of the generative pre-trained Transformer model and optimization of the multi-head self-attention mechanism; The lightweight design of the generative pre-trained Transformer model includes: reducing parameters and computational complexity through parameter sharing, structured sparsity, dynamic adjustment of network depth, and low-rank decomposition of large matrices; The optimization of the multi-head self-attention mechanism includes: improving the ability to process long sequences and computational efficiency by designing dynamic head cropping, sparse attention mode, low-rank decomposition of the attention matrix, mixed precision quantization, and introducing attention kernel function approximation technology; Step 2: Use parallel computing and caching technology to reduce repeated calculations and adjust dynamic accuracy; Step 3: Select a dataset in a specific field for pre-training, so that the generative pre-trained Transformer model can quickly adapt to text generation in the corresponding field; Step 4: Optimize hardware and algorithms, including using GPU and TPU hardware accelerators, and designing optimization algorithms; Among them, the design of optimization algorithms includes: developing a dynamic sparsification algorithm to adaptively adjust the sparsity of the network structure at runtime; developing a mixed precision training algorithm to automatically determine the numerical precision used for each layer of the network during the training process; and developing a communication-aware training algorithm for distributed environments to adaptively adjust parameter synchronization strategies and frequencies based on network topology and bandwidth conditions.

2. The news interpretation method based on natural language processing according to claim 1 is characterized in that: In the summary quality assessment sub-step, ROUGE assessment specifically includes: Use parallel and incremental assessments; Adjust the weights of ROUGE indicators according to the needs of specific areas.

3. The news interpretation method based on natural language processing according to claim 1, characterized in that: The multidimensional sentiment analysis step comprises: Pre-trained model loading sub-step: build sentiment analysis model; Input text generation sub-step: pre-process the text to be analyzed for sentiment to ensure that the text format meets the input requirements of the model; Sentiment analysis sub-step: The preprocessed text is input into the pre-trained sentiment analysis model. The sentiment analysis model extracts the features of the input text, understands the semantics and sentiment tendency of the text, and classifies the sentiment according to the extracted features to determine whether the sentiment tendency of the text is positive, negative or neutral. Multidimensional sentiment analysis sub-step: Use the ValenceArousal model to evaluate the intensity and positivity of sentiment, and divide sentiment into pleasure and arousal through the ValenceArousal model; Calculate the sentiment score sub-steps: For each input text, obtain its sentiment score in the ValenceArousal model; Display sentiment analysis result sub-step: Display the sentiment analysis result to the user, including the sentiment classification of the text and the sentiment score in the ValenceArousal model.

4. The news interpretation method based on natural language processing according to claim 3 is characterized in that: The constructing of the sentiment analysis model comprises: Data collection and preprocessing: Collect the required specific field-related data sets and annotate them; Domain-specific vocabulary construction: construct a domain-specific vocabulary; Multi-stage training strategy: A multi-stage training strategy is adopted to pre-train on unlabeled text, fine-tune on annotated sentiment datasets, and finally fine-tune on specific tasks; Dynamic learning rate adjustment: Develop a dynamic learning rate adjustment algorithm to automatically adjust the learning rate according to performance changes during training; Adversarial training: Generate adversarial samples through adversarial training technology to enhance the robustness of the model; Integrated transfer learning: Use the current model to integrate other pre-trained models and take advantage of the advantages of each model through integration; Continuous optimization loop: Establish a continuous optimization feedback loop; record the performance of the sentiment analysis model in actual applications for further training and optimization.

5. The news interpretation method based on natural language processing according to claim 4 is characterized in that: In the dynamic learning rate adjustment, the specific operations of implementing the dynamic learning rate adjustment algorithm include: Performance indicator monitoring steps: Establish a real-time performance monitoring system to calculate and record key performance indicators after each training batch; use a sliding window to calculate the moving average and variance of the indicator to smooth short-term fluctuations; Learning rate adjustment strategy steps: Based on the monitored performance indicators, design the following adjustment strategy; a) Adaptive annealing: When the performance indicator does not improve for N consecutive batches, the learning rate is multiplied by a decay factor. The decay factor is not fixed and is dynamically adjusted according to the duration of performance stagnation; b) Periodic restart: Introduce a periodic restart mechanism for the learning rate, reset the learning rate to a higher value every certain number of steps, and the restart period is not fixed and is dynamically adjusted according to the training process; c) Dynamic range adjustment: Maintain the acceptable range of learning rate, and dynamically adjust the upper and lower bounds according to the training process; use Bayesian optimization method to search for the optimal learning rate within this range; Gradient statistical analysis step: assist learning rate adjustment through gradient statistical analysis, calculate the gradient norm and variance of each batch, and trigger emergency learning rate adjustment when gradient statistics are abnormal; Multi-scale learning rate step: Set different learning rates for different layers of the sentiment analysis model, and adjust the learning rate of each layer independently based on its gradient statistics; Dynamic batch size adjustment step: Combine learning rate adjustment with batch size adjustment. When the learning rate drops to a certain threshold, increase the batch size accordingly. Use a linear scaling rule to balance the changes in learning rate and batch size. Adaptive momentum adjustment step: dynamically adjust the momentum parameter of the optimizer, use an adaptive algorithm to adjust the momentum based on the historical information of the gradient, and adjust the momentum parameter accordingly while adjusting the learning rate to maintain the stability of the optimization process; Feedback control loop steps: Implement a feedback control loop, treat the learning rate adjustment as a control problem, and use a PID controller. The input of the controller is the rate of change of the performance indicator, and the output is the adjustment amount of the learning rate.

6. The news interpretation method based on natural language processing according to claim 1, characterized in that: The method further comprises: a data visualization step: converting the data and analysis results into charts and reports through interactive data visualization technology; The data visualization step specifically includes: Data collection and preprocessing sub-steps: collect data from multiple sources, clean, transform and integrate the data to ensure data quality and consistency; Data analysis and feature extraction sub-steps: Use statistical analysis techniques to calculate basic statistics of the data, use machine learning algorithms to identify patterns and trends in the data, and use natural language processing techniques to extract features and sentiment information from text data; Visualization type sub-step: Select different visualization types based on the characteristics of the data and the analysis objectives; Visual chart design sub-steps: design the layout, color and style of the visual chart, and add annotations and tags; Visualization chart generation sub-steps: generate charts using data visualization tools or libraries, import pre-processed data, generate chart objects, adjust chart parameters and styles, and generate the final chart; Interactive function implementation sub-step: add interactive functions to the visualization chart, including zooming, filtering, and hover prompts; Integrate the user interface sub-step: embed the generated chart into the system's user interface, provide operation and navigation functions, help users view and analyze data, and ensure the compatibility of the chart on different devices and screens; User testing and feedback sub-step: Conduct user testing, collect user feedback and suggestions, and optimize and improve visualization charts and user interfaces based on user feedback to ensure that the system meets user needs and expectations.

7. The news interpretation method based on natural language processing according to claim 1 is characterized in that: The method further comprises: a personalized recommendation step: a personalized recommendation algorithm based on collaborative filtering, using the user's investment preferences and historical behaviors for intelligent analysis, providing personalized news interpretation and investment advice, as well as accurate and personalized recommendation results; The personalized recommendation step specifically includes: Preprocessing sub-step: obtaining basic data of various types of users, and preprocessing the basic data of various types of users respectively; Prediction sub-step: Based on the pre-processed basic data of various users, use machine learning models to predict the user's future behavior and preferences; User portrait sub-step: fuse the pre-processed basic user data of various types, extract features based on the fused basic user data, and construct a multi-dimensional user portrait based on the extracted features; Screening sub-step: Based on the constructed multi-dimensional user portrait, obtain recommendation results through collaborative filtering and deep learning network, and then filter the recommendation results based on the user's future behavior and preferences to achieve personalized recommendation of investment products.

8. A news interpretation system based on natural language processing, based on the news interpretation method based on natural language processing according to any one of claims 1 to 7, characterized in that: include: Preprocessing module: preprocess the input news text; Text representation module: represents the preprocessed text in matrix form and inputs it into the generative pre-training model; Generative pre-training module: Use the generative pre-trained Transformer model to generate summaries for the input text; Multidimensional sentiment analysis module: analyzes the sentiment tendency of text, and realizes sentiment classification and representation of classification results by extracting and transforming features of input text; Data visualization module: transform data and analysis results into charts and reports through interactive data visualization technology; Personalized recommendation module: Based on collaborative filtering, the personalized recommendation algorithm uses the user's investment preferences and historical behaviors for intelligent analysis, providing personalized news interpretation and investment advice, as well as accurate and personalized recommendation results; The generative pre-training module includes: Pre-trained model submodule: Design a generative pre-trained Transformer model; Input text generation submodule: pre-process the text for which summary is to be generated so that the text format meets the input requirements of the generative pre-trained Transformer model; Summary generation submodule: input the preprocessed text into the generative pre-trained Transformer model to generate a summary; Summary quality assessment submodule: The quality of the generated summary is evaluated using the ROUGE evaluation metric to quantify the similarity between the summary and the reference summary.

Images