A network information trend prediction method and system based on deep learning

Abstract

The application relates to the technical field of artificial intelligence and information dissemination analysis, and discloses a network information trend prediction method and system based on deep learning, wherein the network information trend prediction method based on deep learning comprises the following steps: automatically discovering optimal neural network structures suitable for different types of network information through neural architecture search combining an evolutionary algorithm and reinforcement learning; realizing dynamic reconstruction of a model structure through an environment perception and event triggering mechanism; adopting an importance perception neuron adaptive pruning technology according to resource constraints of a deployment environment; specially extracting and enhancing turning point features in network information dissemination; realizing knowledge migration from a large high-precision model to a lightweight model; and adopting an online learning and continuous optimization mechanism to prevent catastrophic forgetting; the application can accurately predict key turning points of network information dissemination, improves the prediction accuracy, prolongs the early warning time, and reduces the consumption of computing resources.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and information dissemination analysis, more specifically, it relates to a network information trend prediction method and system based on deep learning. BACKGROUND

[0002] With the rapid development of the Internet and social media, network information presents an explosive growth and complex and changeable dissemination characteristics. Accurate prediction of network information dissemination trends, especially key turning points, has important value for decision-making of government departments, enterprise institutions and social organizations. Currently, network information trend prediction is mainly based on statistical models and machine learning methods, but faces the following technical challenges:

[0003] Existing prediction methods mostly use fixed neural network architectures. These architectures are optimized for specific information types when designed, and are difficult to adapt to the diversity and dynamics of network information. When facing different types of information dissemination patterns, the prediction accuracy decreases;

[0004] The network information environment has highly dynamic change characteristics. Once the traditional model is trained, the structure is fixed and cannot be dynamically adjusted according to environmental changes. When the information environment mutates, the model performance often drops sharply and needs to be retrained to recover, which cannot meet the real-time response demand;

[0005] High-precision prediction models usually have high computational complexity and large parameter quantity, making it difficult to efficiently deploy in resource-constrained environments. Although the model can be simplified to reduce resource consumption, the simplified model's accuracy drops significantly on complex tasks such as turning point prediction, making it difficult to balance prediction accuracy and resource efficiency;

[0006] The turning point features in network information dissemination are often overwhelmed by a large amount of ordinary time series data. Existing methods lack a feature extraction and enhancement mechanism specifically for turning points, resulting in low turning point prediction accuracy, insufficient early warning time, and difficulty in providing sufficient preparation time for decision-making;

[0007] The network information environment continues to change, and new data is constantly generated. Traditional models are difficult to effectively utilize new data for continuous learning after initial training, and are prone to the "catastrophic forgetting" problem, i.e., losing memory of learned knowledge when learning new data, resulting in a decline in prediction performance over time;

[0008] Therefore, there is a need for a network information trend prediction method that can adapt to the diversity and dynamics of network information, is resource-efficient, and has continuous learning capabilities. In particular, there is a need to improve the prediction accuracy and lead time of key turning points to provide more accurate and timely decision support for various application scenarios. SUMMARY

[0009] The application provides a network information trend prediction method and system based on deep learning, and solves the technical problems of prediction accuracy reduction, inability to meet real-time response requirements, difficulty in balancing prediction accuracy and resource efficiency, and difficulty in providing sufficient preparation time for decision-making in the related art.

[0010] The application provides a network information trend prediction method based on deep learning, comprising the following steps:

[0011] Based on historical network information propagation data, an optimal neural network structure is automatically searched by combining an evolutionary algorithm and reinforcement learning;

[0012] Based on the optimal neural network structure, model structure dynamic reconstruction is triggered when a change in information propagation mode is detected by monitoring changes in the network information environment;

[0013] For the reconstructed neural network, importance-aware neuron adaptive pruning is performed according to the resource constraints of the deployment environment;

[0014] The turning point features in network information propagation are extracted and enhanced, and the knowledge of the large high-precision model searched is migrated to the lightweight model after pruning;

[0015] The lightweight model after knowledge migration is deployed, and online learning and continuous optimization are performed, and the experience replay mechanism is used to prevent catastrophic forgetting.

[0016] In a preferred embodiment, the step of automatically searching for an optimal neural network structure by combining an evolutionary algorithm and reinforcement learning comprises:

[0017] A neural architecture search space is constructed, including network layer types, connection methods, activation functions, and attention mechanisms;

[0018] A multi-objective evaluation function is defined, considering prediction accuracy, model complexity, and inference time;

[0019] An initial candidate architecture is randomly generated to form an initial population;

[0020] The multi-objective evaluation function value is calculated for each candidate architecture;

[0021] Based on the evaluation function value, an excellent architecture is selected as the parent;

[0022] The selected parent architecture is applied to generate new candidate architectures through crossover and mutation operations;

[0023] The mutation strategy is optimized using reinforcement learning; the evaluation, selection, evolution, and optimization steps are repeatedly executed until convergence, and the architecture with the highest evaluation function value in the final population is selected as the optimal architecture.

[0024] In a preferred embodiment, the step of triggering dynamic reconstruction of model structure when detecting changes in information propagation patterns comprises:

[0025] Construct an information environment change detection module to monitor information flow rate, topic distribution deviation, sentiment polarity change and user interaction mode change;

[0026] Pre-construct and maintain a candidate architecture library, where each architecture is optimized for different types of network information;

[0027] Establish a performance index table for each architecture in the candidate architecture library, recording its predictive performance on different information categories;

[0028] When detecting changes in the information environment, analyze the current information flow characteristics and identify the category to which they belong;

[0029] Select the architecture with the best performance in the category from the candidate architecture library;

[0030] Migrate the key parameters of the current model to the new architecture;

[0031] Fine-tune the new model using recently collected data.

[0032] In a preferred embodiment, the step of importance-aware neuron adaptive pruning comprises:

[0033] Obtain resource constraint parameters of the deployment environment, including computing power, memory limit, energy consumption requirement and delay requirement;

[0034] Determine the target compression ratio of the model based on the resource constraint parameters;

[0035] Calculate the importance score of each neuron in the neural network, including activation importance, gradient importance and turning point sensitivity;

[0036] Define a resource-accuracy balance optimization function to guide the pruning process;

[0037] Iteratively perform the steps of neuron importance sorting, pruning, evaluation and decision-making, gradually remove neurons with low importance, until the target compression ratio is reached or the performance reduction exceeds the threshold.

[0038] In a preferred embodiment, the step of online learning and continuous optimization comprises:

[0039] Construct an incremental learning module, configure an immediate buffer and a long-term buffer to store recently collected data and historical key data;

[0040] Design a priority sampling strategy to preferentially retain turning point samples, high loss samples and low frequency pattern samples;

[0041] Incremental update is performed by using elastic weight consolidation algorithm to balance new task learning and old knowledge preservation.

[0042] Configure concept drift detection component to judge data distribution changes through distribution shift metrics and performance monitoring.

[0043] Dynamically adjust learning rate based on data characteristics and model performance.

[0044] Select representative historical samples from long-term buffer to construct mixed training batch.

[0045] Set different loss weights for new data and historical samples to prevent catastrophic forgetting.

[0046] In a preferred embodiment, the step of extracting and enhancing turning point features in network information propagation comprises:

[0047] Decompose original time series data into trend term, periodic term and residual term.

[0048] Calculate first and second derivatives of time series data to capture change rate and acceleration of data.

[0049] Calculate Z-score to identify data points significantly deviating from normal range.

[0050] Calculate cumulative distribution features to capture long-term accumulation effects.

[0051] Combine original features with enhanced features to form enhanced feature vector.

[0052] In a preferred embodiment, the step of automatically searching for optimal neural network structure by combining evolutionary algorithm and reinforcement learning further comprises:

[0053] Construct multi-level architecture search space, including macro-level overall architecture pattern and micro-level specific module design.

[0054] On the macro level, define different types of information processing modules, including time series feature extraction module, attention enhancement module, multi-modal fusion module and prediction output module.

[0055] On the micro level, define optional internal structure and parameter configuration for each module.

[0056] Use hierarchical search strategy to first determine macro architecture and then optimize micro structure of each module.

[0057] Introduce adaptive resource constraint mechanism to dynamically adjust search space boundaries according to target deployment environment.

[0058] Design special evaluation indicators for network information propagation characteristics, including turning point prediction accuracy, early warning time and environmental adaptability score;

[0059] In the evolution process, the reinforcement learning agent is used to dynamically adjust the mutation probability and crossover strategy, and the search efficiency is optimized according to the historical search trajectory.

[0060] In a preferred embodiment, the step of extracting and enhancing the turning point features in network information propagation further comprises:

[0061] A multi-scale time window analysis module is constructed to capture short-term fluctuations, medium-term changes and long-term trends;

[0062] An adaptive threshold detection algorithm is designed to dynamically adjust the turning point judgment criteria according to the propagation characteristics of different types of network information;

[0063] Network topology feature analysis is introduced to calculate the structural entropy, centrality distribution and community evolution indicators of the information propagation network;

[0064] A user influence weighted model is constructed to weight the information propagation features according to the influence of user nodes;

[0065] A feature importance self-learning mechanism is designed to automatically discover the most predictive turning point features in different scenarios through meta-learning methods;

[0066] Feature time series correlation analysis is implemented to capture the time series dependence and interaction patterns between features;

[0067] A turning point feature enhancement network is constructed to enhance the feature representation of turning point samples through contrastive learning methods, improving the discrimination in the prediction process.

[0068] In a preferred embodiment, the step of online learning and continuous optimization further comprises:

[0069] Set the sample retention strategy, preferentially retain the turning point samples, high loss samples and low frequency pattern samples;

[0070] Configure the sample decay factor to control the decrease of historical sample weight with time;

[0071] Use the elastic weight merging algorithm for incremental update to balance new knowledge learning and old knowledge preservation;

[0072] Parameter importance evaluation based on Fisher information matrix estimation;

[0073] Set the incremental update frequency and batch size to reasonably allocate the proportion of new data and historical samples;

[0074] Configure the concept drift detection mechanism to determine the change in data distribution through divergence calculation and performance monitoring.

[0075] Set the continuous monitoring step and the reference distribution update period.

[0076] Implement an adaptive learning rate adjustment mechanism to dynamically adjust the learning rate based on gradient changes and performance.

[0077] In a preferred embodiment, a deep learning-based network information trend prediction system is used to implement a deep learning-based network information trend prediction method, comprising:

[0078] A neural architecture automatic search module is used to automatically search for an optimal neural network structure based on historical network information propagation data using a combination of evolutionary algorithms and reinforcement learning.

[0079] An environment perception and model reconstruction module is used to monitor changes in the network information environment and trigger model structure reconstruction, while performing adaptive pruning of the neural network based on resource constraints in the deployment environment.

[0080] A turning point feature enhancement module is used to extract and enhance turning point features in network information propagation, capturing key change nodes in information propagation.

[0081] A knowledge transfer and distillation module is used to transfer knowledge from large, high-precision models to lightweight models, achieving model compression and performance preservation.

[0082] An online learning and continuous optimization module is used to continuously learn from new data and optimize model performance after deployment, while preventing catastrophic forgetting through experience replay, achieving accurate prediction of network information propagation turning points and trends.

[0083] The beneficial effects of the present application are:

[0084] The present application can automatically find the optimal neural network architecture for different types of network information through evolutionary reinforcement learning architecture search mechanism, greatly improving the adaptability of the model to diversified information types.

[0085] The event-triggered model structure dynamic reconstruction mechanism constructed by the present application can monitor changes in the network information environment in real time, and automatically select the most suitable architecture or combined architecture from the candidate architecture library when significant changes are detected, achieving rapid adaptation of the model to environmental changes.

[0086] The adaptive model compression technology used by the present application combines neuron importance evaluation, structured pruning and knowledge distillation, etc., which can greatly reduce the model calculation complexity and memory occupation without significantly reducing the prediction accuracy.

[0087] The turning point feature enhancement module specially designed by the application can effectively extract and amplify the turning point signals in the network information flow through multi-scale time sequence comparison, attention mechanism and feature importance weighting technologies. BRIEF DESCRIPTION OF DRAWINGS

[0088] Figure 1 is a flowchart of a network information trend prediction method based on deep learning of the application;

[0089] Figure 2 is a pie chart of the importance weight of different information environment change indicators of the application;

[0090] Figure 3 is a column chart of the prediction performance comparison before and after the turning point feature enhancement of the application;

[0091] Figure 4 is a radar chart of the performance change of different types of models in the online learning process of the application. DETAILED DESCRIPTION

[0092] The subject matter described herein will now be discussed with reference to example implementations. It should be understood that discussions of these implementations are merely provided to enable those skilled in the art to better understand and thus implement the subject matter described herein, and are not intended to limit the scope of the disclosure. Various procedures or components can be omitted, altered, or supplemented in various examples. In addition, features described in some examples can be combined in other examples.

[0093] A network information trend prediction method based on deep learning is disclosed in at least one embodiment of the application, as shown in Figure 1 includes the following steps:

[0094] Step 1, based on historical network information propagation data, automatically search for the optimal neural network structure by combining evolutionary algorithm and reinforcement learning;

[0095] Specifically, the following steps are included:

[0096] Step 1.1, neural architecture search space construction;

[0097] First, the neural architecture search space is constructed, and a set of searchable network components is defined.

[0098] The search space includes:

[0099] Network layer type: convolutional layer, recurrent layer, attention layer, residual connection, etc.

[0100] Connection mode: serial connection, jump connection, parallel connection, etc.

[0101] Activation functions: ReLU, Sigmoid, Tanh, LeakyReLU, etc.

[0102] Attention mechanisms: self-attention, cross-attention, temporal attention, etc.

[0103] Number of layers: 2 to 20 layers;

[0104] Number of hidden units: 16 to 1024.

[0105] The index space can be represented as:

[0106] ;

[0107] where, represents the index space; represents the set of network layer types; represents the set of connection methods; represents the set of activation functions; represents the set of attention mechanisms; represents the range of the number of layers; represents the range of the number of hidden units.

[0108] Step 1.2, multi-objective evaluation function definition;

[0109] Define a multi-objective evaluation function considering three dimensions: prediction accuracy, model complexity, and inference time:

[0110] ;

[0111] where, represents the neural network model to be evaluated; represents the comprehensive evaluation score of the model , the higher the score, the better the model; represents the prediction accuracy of the model ; represents the complexity of the model ; represents the inference time of the model , the larger the value, the slower the model inference speed; 、 、 respectively represent the weight coefficients of prediction accuracy, model complexity, and inference time.

[0112] Step 1.3, architecture search algorithm of evolutionary reinforcement learning;

[0113] This embodiment adopts a method combining evolutionary reinforcement learning for architecture search:

[0114] Initialization: randomly generate Several candidate architectures constitute the initial population. ,in, This represents the initial population, which is the set of all candidate architectures; , , They represent the first , , One candidate neural network architecture; This indicates the number of candidate neural network architectures.

[0115] Evaluation: For each candidate architecture Calculate its multi-objective evaluation function value on the validation set. ;

[0116] Selection: Based on the evaluation function value, a tournament selection method is used to select the winner. A superior architecture serves as the parent generation;

[0117] Evolutionary operations: Apply crossover and mutation operations to the selected parent architecture to generate new candidate architectures.

[0118] Cross: Randomly select components from each layer of two parent architectures to form a sub-architecture;

[0119] Mutation: in terms of probability Randomly change certain components in the architecture;

[0120] Reinforcement learning optimization: Using reinforcement learning to guide mutation operations and optimizing mutation strategies based on historical search experience.

[0121] ;

[0122] in, Indicates the current architecture state Selective mutation operation The probability, This describes the current state of the neural network architecture. Indicates possible mutation operations. Represents the set of all possible mutation operations. The state-action value function represents the state-action value. Execute the following operation The expected cumulative reward, The function represents the natural exponential function, and the denominator represents the normalization factor to ensure that the sum of the probabilities of all operations is 1.

[0123] Updated via TD (Time Differential) learning:

[0124] ;

[0125] where, represents the value function of taking action in state ; represents the first learning rate, controlling the step size of Q value update; represents the immediate reward obtained by the current mutation operation; represents the new state transferred after performing action ; is the maximum Q value among all possible actions in the new state , representing the estimation of the maximum return that can be obtained in the future; is the time difference error.

[0126] Iteration: repeat the steps until the preset number of iterations is reached or the evaluation function value converges;

[0127] Output: select the architecture with the highest evaluation function value in the final population as the optimal architecture.

[0128] Step 2, based on the optimal neural network structure, by monitoring the network information environment changes, when detecting information propagation mode changes, trigger model structure dynamic reconstruction;

[0129] Specifically, the following steps are included:

[0130] Step 2.1, information environment change detection;

[0131] Construct an information environment change detection module to monitor changes in the network information environment through the following indicators:

[0132] Information flow change rate:

[0133] ;

[0134] where, represents the information flow change rate of time relative to the previous time window ; represents the information flow at time ; represents the information flow in the previous time window ; represents the size of the time window; when the value is large, it indicates that the information flow has surged, which may indicate the occurrence of a major event or the formation of a hot topic.

[0135] Topic distribution offset degree: , using the KL divergence to measure the topic distribution at the current time and the topic distribution in the previous time window Topic distribution The differences between them; when A larger value indicates a significant change in the distribution of topics, possibly the emergence of new hot topics or a substantial change in the popularity of existing topics.

[0136] Changes in emotional polarity:

[0137] ;

[0138] in, Indicates time Compared to the previous time window The absolute value of the change in emotional polarity; Indicates time Overall emotional polarity value; Indicates the previous time window Overall emotional polarity value; when A large value indicates a significant change in the emotional tone of online information, which may foreshadow a turning point in public sentiment.

[0139] Changes in user interaction patterns: Using graph structure similarity Measuring the current user interaction network Interaction network with users in the previous time window Structural changes between them.

[0140] When any of the above indicators exceeds a preset threshold, or when the weighted combination of indicators exceeds a comprehensive threshold, an event detection signal is triggered.

[0141] ;

[0142] in, Indicates time Whether to trigger the event detection signal, 1 indicates that it is triggered, 0 indicates that it is not triggered; , , , These represent the weighting coefficients for information flow change rate, topic distribution shift, sentiment polarity change, and user interaction pattern change, respectively. Indicates time Information flow change rate; Indicates the current time KL divergence between the topic distribution and the previous time window; Indicates time The magnitude of change in emotional polarity; This indicates the graph structure similarity between the current user interaction network and the previous time window; This represents the overall threshold, which determines the sensitivity of event detection; the smaller the value, the more sensitive the event.

[0143] Step 2.2, candidate architecture library management;

[0144] In the embodiments of the present application, a candidate architecture library is constructed and maintained in advance , wherein, represents a set of candidate neural network architectures; , , , and represent the first, second, , and represent the first, second, , and represent the first, second,

[0145] "Different types of network information": refers to different categories of network information data such as various social media information streams, news dissemination, public events, etc.

[0146] "Good performance": refers to achieving high levels in accuracy, recall rate, prediction lead time, computational efficiency, etc.

[0147] The candidate architecture library is formed and updated in the following ways:

[0148] Initialization: use neural architecture search algorithm to search for optimal architectures on different types of historical network information data, respectively, to form an initial candidate library;

[0149] Performance index: for each candidate architecture establish a performance index table , record the prediction performance of the architecture on different information categories ;

[0150] Dynamic update: periodically evaluate the performance of each candidate architecture using newly collected data, and update the performance index table; when a significantly degraded architecture is found, generate a new architecture using neural architecture search to replace it.

[0151] Step 2.3, optimal architecture selection and structure reconstruction;

[0152] When changes in the information environment are detected , trigger the model structure reconstruction process:

[0153] Information category identification: analyze the characteristics of the current information stream and identify its category ;

[0154] Architecture matching: based on the performance index table, select the architecture with the best performance in that category:

[0155] ;

[0156] , wherein, represents the selected optimal neural network architecture; This indicates that the maximum value of the variable in the following expression is taken; Indicates candidate architecture library The first in One candidate neural network architecture; Represents a set of candidate neural network architectures; Presentation Architecture In information category On the performance index values; This indicates the currently identified category of network information.

[0157] Parameter migration: Migrate key parameters of the current model to the new architecture while preserving the model's memory of historical data.

[0158] ;

[0159] in, This represents the optimal neural network architecture selected. The set of parameters shared with the current architecture; This represents the set of parameters shared between the current neural network architecture and the new architecture. This indicates a parameter migration operation;

[0160] "Shared parameters" refer to network layer parameters that have similar functions or the same structure in two different architectures, such as embedding layer weights, initial convolutional layer parameters, or feature extraction layer parameters.

[0161] Quick fine-tuning: Use recently collected data to briefly fine-tune the new model to better adapt it to the current environment.

[0162] ;

[0163] in, Represents the optimal architecture The set of model parameters; This indicates a parameter update operation; Indicates the second learning rate; Indicates about parameters The gradient operator; Represents the loss function; This represents the most recently collected dataset, containing the latest network information samples and their corresponding labels.

[0164] The entire reconstruction process was completed within 30 seconds, ensuring that the model could quickly recover its predictive capabilities after changes in the information environment.

[0165] Step 3: For the reconstructed neural network, perform importance-aware adaptive pruning of neurons based on the resource constraints of the deployment environment;

[0166] Specifically comprising the following steps:

[0167] Step 3.1, resource constraint awareness;

[0168] First, obtain the resource constraint parameters of the deployment environment, including:

[0169] Computing capacity: , representing the maximum computing capacity (floating point operations per second) of the target device;

[0170] Memory limit: , representing the available memory size;

[0171] Energy consumption requirement: , representing the maximum allowed energy consumption level;

[0172] Delay requirement: , representing the maximum allowed inference delay.

[0173] Based on the above constraint parameters, determine the target proportion of model compression :

[0174] ;

[0175] Wherein, represents the target proportion of model compression; represents the maximum computing capacity of the target device; represents the computing demand of the original model; represents the available memory size; represents the minimum value in the parentheses; represents the memory demand of the original model; represents the maximum allowed energy consumption level; represents the energy consumption demand of the original model; represents the maximum allowed inference delay; represents the inference delay of the original model.

[0176] Step 3.2, neuron importance evaluation;

[0177] For each neuron in the neural network , calculate its importance score .

[0178] This embodiment adopts a weighted combination of three importance evaluation criteria:

[0179] Activation importance: statistically calculate the average activation amplitude of the neuron on the training set:

[0180] ;

[0181] Gradient importance: Compute the gradient of the loss function with respect to the neuron's output:

[0182] ;

[0183] Turning point sensitivity: Evaluate the neuron's contribution to the turning point prediction:

[0184] ;

[0185] where, denotes the activation importance score of neuron ; denotes the gradient importance score of neuron ; denotes the turning point sensitivity score of neuron ; represents the entire training sample set; is the loss function; denotes the partial derivative of the loss function with respect to the neuron's activation value; denotes the size of the sample set containing the turning point; denotes an input sample in the training set; represents the sample set containing the turning point; denotes the turning point prediction output; is the activation value of neuron for input sample ; denotes the partial derivative of the turning point prediction output with respect to the neuron's activation value.

[0186] The comprehensive importance score is calculated as:

[0187] ;

[0188] where, denotes the comprehensive importance score of neuron ; denotes the activation importance of the neuron; denotes the gradient importance of the neuron; denotes the turning point sensitivity of the neuron; , , are the weight coefficients of activation importance, gradient importance and turning point sensitivity, respectively.

[0189] Step 3.3, resource accuracy balance optimization;

[0190] Define the resource accuracy balance optimization function to guide the pruning process:

[0191] ;

[0192] wherein, represents the value of the resource-accuracy balance optimization function; represents the pruned model; represents the prediction accuracy of the model; represents the computational complexity of the model; represents the energy consumption of the model; is a balance parameter.

[0193] Step 3.4, iterative pruning algorithm;

[0194] This embodiment adopts an iterative pruning algorithm to gradually remove neurons with low importance:

[0195] Initialization: set the original model as the current model ;

[0196] Importance ranking: calculate the importance scores of all neurons in the current model and rank them from low to high;

[0197] Pruning: remove a batch of neurons with the lowest importance to obtain a pruned model ;

[0198] Evaluation: calculate the resource-accuracy balance optimization function value of the pruned model ;

[0199] Decision:

[0200] If and the compression ratio has not reached the target ratio , accept the pruning result, update the current model , and return to importance ranking for iteration;

[0201] Otherwise, terminate the iteration and return the current model as the final pruning result.

[0202] In some embodiments, the pruning process can adopt the following variants:

[0203] Layer-by-layer pruning: prune layer by layer in order, first prune deep layers, then prune shallow layers, or adopt a pruning order from shallow to deep;

[0204] Structured pruning: not only prune individual neurons, but also prune entire convolution kernels, attention heads, or layer structures, etc.

[0205] Soft pruning: instead of directly removing neurons, regularize part of the weights to approach zero;

[0206] Dynamic growth pruning: dynamically remove unimportant connections and grow new connections during training, allowing the network structure to evolve.

[0207] Step 4, extract and enhance turning point features in network information dissemination, and migrate the knowledge of large high-precision models found by search to pruned lightweight models;

[0208] Specifically, the following steps are included:

[0209] Step 4.1, turning point feature extraction;

[0210] Construct a turning point feature extraction module to enhance the features of the original network information time series data and highlight the key patterns that may indicate turning points:

[0211] Time series decomposition:

[0212] ;

[0213] Where, represents the original time series data at time point ; represents the long-term trend item of the time series; represents the periodic fluctuation item of the time series, capturing the cyclic change pattern of the data; represents the residual item of the time series, containing random fluctuations and changes that are difficult to explain through trends or cycles.

[0214] Change rate calculation:

[0215] ;

[0216] Where, represents the instantaneous change speed of the data at time point ; represents the change speed of the data change rate; represents the original time series data value at time point ; represents the original time series data value at time point ; represents the first derivative value at time point ; represents the time interval between adjacent time points.

[0217] Abrupt change detection: calculate Z-score , identify data points that significantly deviate from the normal range:

[0218] ;

[0219] Where, represents the Z-score at time point ; represents the original time series data value at time point ; represents the mean value of all data points within a sliding window centered at time point ; represents the standard deviation of all data points within a sliding window centered at time point ;

[0220] The size of the sliding window determines the range of data points used for calculating the mean and standard deviation, typically chosen to reflect a normal fluctuation range.

[0221] Cumulative feature: Calculate cumulative distribution feature , capturing long-term accumulation effects:

[0222] ;

[0223] where, represents the cumulative distribution feature value at time point ; represents the original time series data value at time point ; represents the summation operation from the initial time point to the current time point ; represents a time decay function that decreases as the time interval increases; is the decay coefficient.

[0224] Feature fusion: Combine the original features with the enhanced features to form an enhanced feature vector :

[0225] ;

[0226] where, represents the enhanced feature vector at time point ; represents the original time series data at time point ; represents the trend item obtained from time series decomposition; represents the periodic item obtained from time series decomposition; represents the residual item obtained from time series decomposition; represents the rate of change of data at time point ; represents the rate of change of data at time point ; represents the Z-score of time point ; represents the cumulative distribution feature value at time point , used to capture long-term accumulation effects.

[0227] After feature enhancement, the feature pattern of the turning point is obviously strengthened, which is beneficial for the model to accurately identify and predict the turning point.

[0228] Step 4.2, knowledge distillation model construction;

[0229] Apply knowledge distillation technology to migrate the knowledge of large high-precision teacher model (original model or unpruned model) to the pruned lightweight student model:

[0230] Teacher model preparation: select a high-precision model as the teacher model , which performs well in prediction accuracy but may have high computational complexity;

[0231] Student model preparation: use a lightweight model pruned by self-adaptive pruning as the student model , which has low computational resource demand but may have decreased prediction accuracy;

[0232] Distillation loss function configuration: according to the embodiments of the present application, define a comprehensive loss function , including hard target loss and soft target loss:

[0233] ;

[0234] Wherein, represents the total loss function of knowledge distillation; is the loss between the prediction of the student model and the real label; is the loss between the student model and the soft label of the teacher model; represents the real label of the training data; represents the prediction output of the student model for input ; represents the softened output of the teacher model for input under the temperature parameter ; represents the softened output of the student model for input under the temperature parameter ; is a balance coefficient, which controls the relative importance of hard loss and soft loss, and the value range is ; is a temperature parameter.

[0235] When approaches 1, the model pays more attention to matching the real label (hard loss) and ignores the knowledge of the teacher model;

[0236] When Close to 0, the model pays more attention to learning the knowledge of the teacher model (soft loss) and less attention to the real label; the typical setting is 0.5, indicating equal attention to hard loss and soft loss;

[0237] Higher values (such as 8 or 10) will produce smoother probability distributions, so that the information of non-maximum probability categories can also be passed to the student model, helping to pass the subtle differences and uncertainty information in the teacher model;

[0238] And lower values (close to 1) will preserve the sharp distribution of the teacher model output, mainly passing the knowledge of high-confidence predictions.

[0239] Turning point sensitive knowledge transfer: during knowledge distillation, pay special attention to the knowledge transfer of turning point samples, increase the weight of turning point samples:

[0240] ;

[0241] where, represents the total loss function of knowledge distillation; represents the knowledge distillation loss on ordinary samples; represents the knowledge distillation loss on turning point samples; is the weight coefficient of turning point samples.

[0242] Distillation training: train the student model using a dataset containing rich turning points, and minimize the distillation loss:

[0243] ;

[0244] where, represents the optimal parameters of the student model after knowledge distillation training; represents the student model parameters that make the following expression minimum ; represents the total loss function of knowledge distillation.

[0245] Step 5, deploy the lightweight model after knowledge transfer, and perform online learning and continuous optimization, using the experience replay mechanism to prevent catastrophic forgetting;

[0246] Specifically, the following steps are included:

[0247] Step 5.1, incremental learning module construction;

[0248] Build an incremental learning module to enable the model to learn from new data without retraining:

[0249] Data buffer configuration: create two levels of data buffer:

[0250] Real-time buffer: Stores recently collected data for real-time incremental updates;

[0251] Long-term buffer: Stores key historical data, especially samples containing turning points, to prevent catastrophic forgetting.

[0252] Priority sampling strategy: Assign priorities to samples in the buffer, and retain those that are prioritized.

[0253] Turning point samples: data containing turning points in the dissemination of online information;

[0254] High-loss samples: Samples with large model prediction errors;

[0255] Low-frequency pattern samples: Samples representing rare propagation patterns.

[0256] Incremental Update Algorithm: It should be understood that this embodiment uses the Elastic Weight Consolidation (EWC) algorithm to balance learning new tasks with retaining old knowledge.

[0257] ;

[0258] in, This represents the total loss function of the elastic weight merging algorithm; It is the model's predictive loss on newly collected data; Represents the complete set of parameters of the model; It is the first in the current model parameter set. Several parameters are currently being optimized. It is the first time the model saves data after learning from the old task. Each parameter value represents prior knowledge; It is the first of the Fisher information matrix. The diagonal elements quantify the parameters. The higher the value, the more important the parameter is to the performance of legacy tasks. It is the regularization coefficient, which controls the balance between learning new and old tasks. The larger the value, the more emphasis is placed on retaining old knowledge. This indicates summing over all model parameters; It is a regularization term that prevents catastrophic forgetting by penalizing large changes in important parameters.

[0259] Progressive Networks: Add new network branches for each new task while maintaining the network structure for older tasks, transferring knowledge through lateral connections.

[0260] ;

[0261] where, is the th hidden state of the th layer of the th task; is the th hidden state of the th layer of the th task; is the weight matrix of the th layer of the th task; is the lateral connection weight from the th task to the th task; is the th hidden state of the th layer of the th task, representing the features of early tasks;

[0262] represents the weighted sum of features of all previous tasks; is the activation function.

[0263] Knowledge distillation assisted incremental learning: when learning a new task, use the old model as the teacher model to guide the new model to retain old knowledge:

[0264] where, represents the total loss function of knowledge distillation; is the prediction loss of the model on newly collected data; represents the full parameter set of the model; is the distillation weight coefficient; represents the output probability distribution of the old model (teacher model) on the input data; represents the output probability distribution of the new model (student model) on the same input data; is the KL divergence, which measures the difference between two probability distributions, the smaller the value, the closer the two distributions.

[0265] Step 5.2, concept drift detection;

[0266] Configure the concept drift detection component to monitor changes in data distribution:

[0267] Distribution shift measure: calculate the JS divergence between the current data distribution and the baseline distribution :

[0268] ;

[0269] where, represents the JS divergence between the current data distribution and the baseline distribution; P(t) represents the probability distribution of data collected within the current time window; P(ref) represents the probability distribution of reference data; P(avg) represents the average distribution of the current distribution and the reference distribution; D_KL represents the KL divergence.

[0270] Performance monitoring: track the performance trend of the model on the validation set:

[0271] ;

[0272] where, P(t) represents the accuracy change at time point ; P(t) represents the predicted accuracy at the current time point ; P(t) represents the predicted accuracy at the previous time point ; P(t) represents the time interval parameter.

[0273] Drift detection decision: trigger a concept drift alert when the distribution shift metric exceeds a threshold or the performance continues to decline beyond a threshold ;

[0274] ;

[0275] where, I(t) represents the binary indicator function of whether concept drift is detected at time point , with a value of 1 indicating that drift is detected and a value of 0 indicating that drift is not detected; D_JS represents the JS divergence between the current data distribution and the reference distribution; P(t) represents the probability distribution of data collected within the current time window; P(ref) represents the probability distribution of reference data; is a threshold parameter for distribution change; P(t) represents the model accuracy change at time point ; is a threshold parameter for performance decline; is a step parameter for continuous detection, requiring performance to decline for consecutive time steps to trigger a drift alert, used to filter short-term fluctuations; "consecutive steps" represents consecutive time steps, ensuring that only sustained performance decline is considered as a true concept drift signal.

[0276] Gradual drift detection: use time series analysis techniques to detect slow concept drift, such as using linear regression or ARIMA models to fit the performance trend, and detect significant downward slope.

[0277] Integrated-based drift detection: maintain multiple models trained at different time periods, when the prediction disagreement among models increases significantly, consider it as a concept drift signal:

[0278] ;

[0279] where, denotes the diversity measure of the model ensemble at time point ; denotes the total number of models in the ensemble; denotes the total number of possible model pairs; denotes the sum over all different model pairs ; ensures not comparing a model with itself; and denote the th and the th different model in the ensemble, respectively; denotes the input data sample collected at time point ; and denote the prediction results of model and on input ; denotes a measure function to measure the difference between two prediction results.

[0280] Step 5.3, adaptive learning rate adjustment;

[0281] Dynamic adjustment of learning rate based on data characteristics and model performance:

[0282] Based on gradient change: calculate the change amount of gradient, judge the stability of parameter update:

[0283] ;

[0284] where, denotes the exponentially moving average value of the gradient change at the current time step t; denotes the exponentially moving average value of the gradient change at the previous time step t-1; is the decay coefficient; denotes the gradient of the loss function on the model parameters at the current time step t; denotes the gradient of the loss function on the model parameters at the previous time step t-1; denotes the squared norm of the gradient difference between the current time step and the previous time step; is the weight coefficient of the new gradient change information.

[0285] Based on performance change: adjust learning rate according to model performance change:

[0286] ;

[0287] where, denotes the learning rate at the current time step ; denotes the learning rate at the previous time step ; denotes the model performance metric at the current time step ; denotes the model performance metric at the previous time step ; denotes the increase coefficient of learning rate, and ; denotes the decrease coefficient of learning rate, and ; denotes the threshold for determining whether the performance change is significant enough to trigger learning rate adjustment.

[0288] Comprehensive learning rate calculation:

[0289] ;

[0290] where, denotes the final learning rate applied to model update; denotes the preliminary learning rate determined after performance evaluation at the current time step; denotes the exponentially moving average of gradient change at the current time step; denotes the square root of gradient change; is a small constant added to prevent the denominator from being zero, ensuring the stability of numerical calculation.

[0291] Step 5.4, experience replay mechanism;

[0292] Configure experience replay component to prevent catastrophic forgetting:

[0293] Sample selection: select representative historical samples from long-term buffer, especially turning point samples;

[0294] Mixed batch construction: mix newly collected data with historical samples to construct training batch:

[0295] ;

[0296] where, denotes the mixed training batch constructed at time step ; , , denote the newly collected first 、 、 data samples; 、 、 represents the i-th data sample selected from the experience replay buffer; 、 、 represents the i-th historical sample; represents the union operation of the set; represents the number of new data samples; represents the number of samples drawn from historical data.

[0297] Balanced loss function: Set different loss weights for new data and historical samples:

[0298] ;

[0299] where, represents the total loss function used in model training; represents the loss function value calculated based on newly collected data; represents the loss function value calculated based on historical data sampled from the experience replay buffer; is the weight coefficient of the historical sample loss.

[0300] In some embodiments, the experience replay mechanism can employ the following enhancement methods:

[0301] Core set extraction: Extract a representative core sample set from historical data, maximize sample diversity and coverage, and store more representative samples using a smaller buffer:

[0302] ;

[0303] where, represents the selected representative core sample set; represents the parameter that returns the maximum value of the following expression, i.e., find the core set that maximizes the objective function ; represents is a subset of the complete data set ; represents the size of the core set ; , i.e., only samples are selected; represents the remaining samples in the complete data set excluding the core set ; represents finding the minimum value in the remaining sample set ; Find the minimum value among the selected core set ; Denote the sample and the sample distance metric function between samples, used to calculate the similarity or difference between samples.

[0304] Generative playback: use a generative model (such as a variational autoencoder or a generative adversarial network) to learn the distribution of historical data, then generate synthetic samples for playback, reducing the storage requirements for original historical data:

[0305] ;

[0306] Where, denotes the generated synthetic sample, used to replace the original historical data for experience playback; denotes the trained generator network, responsible for mapping random noise to meaningful data samples; denotes the random noise vector input to the generator, as a random seed for the generation process; denotes the standard normal distribution, with mean 0 and covariance matrix I, which is the sampling distribution of random noise ; denotes the mathematical symbol "subject to" a certain distribution, indicating that the noise vector is randomly sampled from the standard normal distribution.

[0307] Dynamic playback weight: dynamically adjust the weight of the playback sample according to its scarcity, importance and age:

[0308] ;

[0309] Where, denotes the total weight value of the sample , which determines the probability of being selected in the playback process; denotes the importance score of the sample , usually based on the impact of the sample on model performance, for example, can be quantified by gradient amplitude or loss value size; denotes the rarity score of the sample , measures the rarity of the sample in the feature space, usually determined by calculating the average distance or density estimation with other samples; denotes the timeliness score of the sample , evaluates the new and old degree of the sample, usually calculated based on the difference between the sample's timestamp and the current time; , , denote the weight coefficients of importance, rarity and timeliness, respectively.

[0310] Technical effects of the embodiment:

[0311] According to the embodiment of the present application, the deep learning-based network information trend prediction method provided has the following significant technical effects:

[0312] Significant improvement in prediction performance:

[0313] Through multi-objective optimization of neural architecture automatic search and turning point feature enhancement technology, the method of the present embodiment has made a significant breakthrough in network information turning point prediction:

[0314] The turning point prediction accuracy is improved by 28.3%, far exceeding traditional fixed architecture models;

[0315] The prediction time advance rate increases by 15.7%, providing more sufficient preparation time for emergency response;

[0316] Strong adaptability to different types of network information, can automatically find the optimal model structure for different information types.

[0317] Quick adaptation to environmental changes:

[0318] Based on the event-triggered model structure dynamic reconstruction technology, the method provided by the present application performs well when facing network information environmental changes:

[0319] Within 30 seconds after the information environment changes, the model reconstruction is completed and the prediction ability is restored;

[0320] No need for complete retraining, retain the model's memory of historical data through parameter migration;

[0321] Strong adaptability to changes in emergency information propagation patterns, with stable prediction performance.

[0322] Significant reduction in resource consumption:

[0323] With the help of importance-aware neuron adaptive pruning and knowledge distillation technology, the method provided by the present application significantly reduces resource consumption while maintaining prediction accuracy:

[0324] Model parameter quantity is reduced by 32.5%, significantly reducing storage requirements;

[0325] Computing resource demand is reduced by 41.2%, reducing processing delay;

[0326] Energy consumption is reduced by 35.8%, prolonging the battery life of the device;

[0327] Can flexibly adjust the model size according to the resource constraints of different deployment environments.

[0328] Continuous learning and adaptation:

[0329] According to one embodiment of the present application, the method has the ability of continuous evolution through online learning and continuous optimization technology:

[0330] Constantly learn from new data, and the prediction performance improves over time;

[0331] Effectively prevent catastrophic forgetting, and retain the memory of historical propagation patterns;

[0332] Can detect and adapt to the concept drift of network information distribution;

[0333] The model performance steadily improves over time without frequent offline retraining.

[0334] Wide applicability:

[0335] The method of the present application has wide application prospects and is suitable for various network information trend prediction scenarios:

[0336] Monitoring and early warning of public opinion on sudden public events;

[0337] Analysis of social hot topic propagation trend;

[0338] Prediction of changes in commercial brand reputation;

[0339] Evaluation of the propagation effect of policy interpretation information;

[0340] Suitable for various deployment environments from high-performance servers to resource-constrained edge devices.

[0341] Figures 2 to 4 Importance weights of different information environment change indicators; comparison of prediction performance before and after turning point feature enhancement; performance changes of different types of models during online learning.

[0342] As can be seen, the method provided by the present application has significant advantages in prediction performance, environmental adaptability, resource efficiency and continuous learning ability, and can effectively solve the core technical challenges faced by existing network information trend prediction methods.

[0343] The embodiments of the present application are described above, but the embodiments are not limited to the specific implementation described above, and the specific implementation described above is only illustrative and not limiting. Those skilled in the art can make more forms of equivalent embodiments under the inspiration of the embodiments, which are all within the protection scope of the embodiments.

Claims

1.A deep learning-based network information trend prediction method, characterized by, The method comprises the following steps: Based on historical network information propagation data, the optimal neural network structure is automatically searched by combining evolutionary algorithm and reinforcement learning; Based on the optimal neural network structure, the model structure is dynamically reconstructed when the change of information propagation mode is detected by monitoring the change of network information environment; For the reconstructed neural network, importance-aware neuron adaptive pruning is performed according to the resource constraints of the deployment environment; The turning point features in network information propagation are extracted and enhanced, and the knowledge of the large and high-precision model searched is migrated to the lightweight model after pruning; The step of extracting and enhancing the turning point features in network information propagation comprises: The original time series data is decomposed into trend item, periodic item and residual item; The first derivative and the second derivative of the time series data are calculated to capture the change rate and acceleration of the data; The Z-score is calculated to identify data points that significantly deviate from the normal range; The cumulative distribution feature is calculated to capture the long-term accumulation effect; The original features and the enhanced features are combined to form an enhanced feature vector; The lightweight model after knowledge migration is deployed, and online learning and continuous optimization are performed, and the experience replay mechanism is used to prevent catastrophic forgetting. 2.The network information trend prediction method based on deep learning according to claim 1, characterized in that, The step of automatically searching the optimal neural network structure by combining evolutionary algorithm and reinforcement learning comprises: A neural architecture search space is constructed, including network layer type, connection mode, activation function and attention mechanism; A multi-objective evaluation function is defined, considering prediction accuracy, model complexity and inference time; An initial candidate architecture is randomly generated to form an initial population; The multi-objective evaluation function value of each candidate architecture is calculated; The selected excellent architectures are used as parents; The selected parent architectures are applied to generate new candidate architectures through crossover and mutation operations; The mutation strategy is optimized using reinforcement learning; the evaluation, selection, evolution and optimization steps are repeated until convergence, and the architecture with the highest evaluation function value in the final population is selected as the optimal architecture. 3.The network information trend prediction method based on deep learning according to claim 1, characterized in that, The step of triggering dynamic reconstruction of the model structure when the change of information propagation mode is detected comprises: An information environment change detection module is constructed to monitor the information flow change rate, topic distribution deviation degree, sentiment polarity change and user interaction mode change; A candidate architecture library is constructed and maintained in advance, wherein each architecture is optimized for different types of network information; A performance index table is established for each architecture in the candidate architecture library to record its prediction performance on different information categories; When the information environment change is detected, the current information flow characteristics are analyzed and the category to which it belongs is identified; The architecture with the best performance in the category is selected from the candidate architecture library; The key parameters of the current model are migrated to the new architecture; The newly collected data is used to quickly fine-tune the new model. 4.The network information trend prediction method based on deep learning according to claim 1, characterized in that, The step of importance-aware neuron adaptive pruning comprises: Obtain the resource constraint parameters of the deployment environment, including computing power, memory limit, energy consumption requirement and delay requirement; Determine the target proportion of model compression based on the resource constraint parameters; Calculate the importance score of each neuron in the neural network, including activation importance, gradient importance and turning point sensitivity; Define a resource-precision balance optimization function to guide the pruning process; The steps of iterative execution of neuron importance ranking, pruning, evaluation and decision include gradually removing neurons with low importance until the target compression ratio is reached or the performance reduction exceeds the threshold. 5.The network information trend prediction method based on deep learning according to claim 1, characterized in that, The steps of online learning and continuous optimization include: Building an incremental learning module, configuring an instant buffer and a long-term buffer for storing recently collected data and historical key data; Designing a priority sampling strategy to preferentially retain turning point samples, high loss samples and low frequency pattern samples; Using an elastic weight merging algorithm for incremental update to balance new task learning and old knowledge retention; Configuring a concept drift detection component to determine data distribution changes through distribution offset metrics and performance monitoring; Dynamically adjusting the learning rate based on data characteristics and model performance; Selecting representative historical samples from the long-term buffer to construct a mixed training batch; Setting different loss weights for new data and historical samples to prevent catastrophic forgetting. 6.The network information trend prediction method based on deep learning according to claim 1, characterized in that, The steps of automatically searching for the optimal neural network structure using the combination of evolutionary algorithms and reinforcement learning further include: Building a multi-level architecture search space, including macro-level overall architecture patterns and micro-level specific module designs; On the macro level, define different types of information processing modules, including time series feature extraction modules, attention enhancement modules, multi-modal fusion modules and prediction output modules; On the micro level, define the optional internal structure and parameter configuration for each module; Using a hierarchical search strategy, first determine the macro architecture, and then optimize the micro structure of each module; Introducing an adaptive resource constraint mechanism to dynamically adjust the search space boundaries according to the target deployment environment; Designing special evaluation indicators for network information propagation characteristics, including turning point prediction accuracy, early warning time and environmental adaptability score; During the evolution process, use a reinforcement learning agent to dynamically adjust mutation probability and crossover strategy to optimize search efficiency based on historical search trajectories. 7.The network information trend prediction method based on deep learning according to claim 1, characterized in that, The steps of extracting and enhancing turning point features in network information propagation further include: Building a multi-scale time window analysis module to capture short-term fluctuations, medium-term changes and long-term trends; Designing an adaptive threshold detection algorithm to dynamically adjust the turning point judgment criteria based on the propagation characteristics of different types of network information; Introducing network topology feature analysis to calculate the structural entropy, centrality distribution and community evolution indicators of the information propagation network; Building a user influence weighted model to weight the information propagation features based on the influence of user nodes; Designing a feature importance self-learning mechanism to automatically discover the most predictive turning point features in different scenarios through meta-learning methods; Implementing feature temporal correlation analysis to capture temporal dependency relationships and interaction patterns between features; Building a turning point feature enhancement network to enhance the feature representation of turning point samples through contrastive learning methods, improving discrimination in the prediction process. 8.The network information trend prediction method based on deep learning according to claim 1, characterized in that, The steps of online learning and continuous optimization further include: Setting a sample retention strategy to preferentially retain turning point samples, high loss samples and low frequency pattern samples; Configuring a sample decay factor to control the decrease of historical sample weights over time; Using an elastic weight merging algorithm for incremental update to balance new knowledge learning and old knowledge retention; Parameter importance evaluation based on Fisher information matrix estimation; Set the incremental update frequency and batch size, and reasonably allocate the proportion of new data and historical samples; Configure the concept drift detection mechanism to judge the change of data distribution through divergence calculation and performance monitoring; Set the continuous monitoring step number and the benchmark distribution update period; Implement the adaptive learning rate adjustment mechanism to dynamically adjust the learning rate according to the gradient change and performance. 9.A deep learning based network information trend prediction system configured to perform the deep learning based network information trend prediction method according to any one of claims 1-8. Comprise: A neural architecture automatic search module for automatically searching the optimal neural network structure based on historical network information propagation data by combining evolutionary algorithms and reinforcement learning; An environment perception and model reconstruction module for monitoring network information environment changes and triggering model structure reconstruction, and for performing adaptive pruning on the neural network according to the resource constraints of the deployment environment; A turning point feature enhancement module for extracting and enhancing the turning point features in network information propagation, and capturing the key change nodes of information propagation; A knowledge transfer and distillation module for transferring the knowledge of large and high-precision models to lightweight models, and realizing model compression and performance maintenance; An online learning and continuous optimization module for continuously learning and optimizing model performance from new data after deployment, and for preventing catastrophic forgetting through experience replay, and for realizing accurate prediction of network information propagation turning points and trends.

Images