DDPG-based text relation graph dynamic optimization control method and system

Abstract

The application provides a text relation graph dynamic optimization control method driven by a deep deterministic policy gradient (DDPG) as a core, which combines a pre-training BERT, a graph convolution network and a deep reinforcement learning DDPG to realize collaborative optimization of text semantic modeling, relation graph construction and multi-label prediction. The method comprises the following steps: performing word segmentation and BERT coding on the text, extracting word-level and sentence-level semantic information to generate an initial relation graph; inputting the text features, the graph structure features, the model training state and the resource state into a DDPG policy network to dynamically adjust the relation graph structure and the label prediction sequence, and completing multi-label prediction through graph convolution and BERT fusion. Compared with the prior art, the application can update the graph structure and the prediction strategy in real time according to the task state, improve the adaptability, real-time performance and accuracy of multi-label prediction, and is suitable for text processing scenes with complex relations and high labeling requirements.

Description

Technical Field

[0001] This invention falls within the interdisciplinary research field of natural language processing and deep learning, specifically involving a dynamic optimization control method and supporting system for text relationship graphs based on Deep Deterministic Policy Gradient (DDPG). Background Technology

[0002] Text classification and multi-label prediction are common tasks in natural language processing. Pre-trained language models, such as BERT, have been widely used for various text understanding tasks due to their ability to capture contextual semantic information.

[0003] However, these pre-trained language models also have shortcomings. For example, their ability to explicitly model the structured relationships between words is relatively limited, their training and inference costs are high, and there is still room for improvement in terms of knowledge representation stability, logical reasoning ability, generalization performance, and interpretability. Graph Convolutional Networks (GCNs) transform text into a graph structure for information transmission, supplementing local structural information and long-range dependencies. Therefore, the hybrid architecture combining BERT and GCNs has been gradually used for tasks such as text classification and has achieved some success.

[0004] However, existing BERT plus GCN solutions still have several limitations. First, the graph structure is usually statically constructed during the preprocessing stage, often in the form of fixed graphs, dual graphs, or labeled graphs. When the input text changes, the graph topology is difficult to adapt in a timely manner. Second, the inference process lacks flexible control; texts of different complexities often use the same number of GCN layers and graph parameters, easily leading to redundant computations. Third, text relationship graph adjustment and inference process control are mostly separate designs, lacking collaborative optimization between the two. Fourth, in multi-label scenarios, fixed classification thresholds and simple ranking strategies cannot simultaneously consider label hierarchy constraints, error costs, prediction accuracy, and computational costs. Summary of the Invention

[0005] To achieve the above objectives, this invention provides a dynamic optimization control method for text relationship graphs based on DDPG, the main steps of which are as follows:

[0006] Step S1: Perform word segmentation and encoding on the input text sequence to obtain the corresponding word sequence and attention mask.

[0007] Step S2: Feed the word sequence into the pre-trained BERT encoder to obtain a word-level hidden layer representation matrix and a sentence-level convergence vector with context awareness.

[0008] Step S3: Initialize the dynamic text relation graph based on text semantic association, word co-occurrence relationship or syntactic dependency relationship. The dynamic text relation graph includes node feature vectors, initial edge connection relationship, and graph structure parameters such as initial edge weight, sparse threshold, neighbor sampling rate, subgraph expansion hop count and subgraph budget. In order to address the problem that the graph structure cannot dynamically adapt to text features, this step adopts the DDPG continuous control strategy to replace the traditional static graph construction scheme.

[0009] Step S4: Construct a continuous action optimization network with DDPG as its core. The network consists of an Actor policy network and a Critic value network. The state input includes text features, text relationship graph structure complexity, model inference confidence, and computational resource consumption. The action space output includes three main types of parameters: text relationship graph structure adjustment parameters, GCN inference control parameters, and multi-label decision parameters. The text relationship graph structure adjustment parameters include edge weight adjustment coefficient, neighbor sampling rate, sparsity threshold, subgraph expansion hop count, and subgraph budget. The GCN inference control parameters include inference layer depth selection, inference path switching instructions, and early exit decision threshold. The multi-label decision parameters include label classification threshold, candidate pruning ratio, and prediction priority order. All action parameters are mapped to continuous real values ​​in the interval [0,1], excluding discrete classification or binary decision. The label prediction priority order is achieved by continuously weighting and sorting the label classification scores. The inference path switching instructions are represented by continuous real numbers in the interval [0,1], and then mapped to the actual executable path selection by the threshold set by the system.

[0010] Step S5: Using the BERT hidden layer representation matrix as node features, the graph structure information is propagated through a multi-layer edge type-aware graph convolutional network. Each graph convolutional layer uses an independent weight matrix to aggregate information for different types of edges and then performs a weighted summation. It also includes residual connections and layer normalization. During the graph convolution process, the action parameters of the DDPG network are received in real time to dynamically adjust the text relationship graph structure.

[0011] Step S6: Perform weighted pooling on the graph convolution output using a learnable attention mechanism to obtain sequence-level graph feature vectors.

[0012] Step S7: Concatenate and fuse the graph feature vector and the BERT sentence-level convergence vector, and output the category prediction or multi-label prediction result through a multi-layer fully connected classification head.

[0013] Step S8: During the inference phase, adaptive computation and dynamic control of the text relation graph are performed. The DDPG continuous action control network perceives the current state of the text relation graph and the usage status of computing resources in real time, and outputs continuous action parameters to dynamically adjust the edge weight distribution, neighbor sampling range, sparsity, subgraph expansion hop count and budget allocation of the text relation graph. At the same time, it can also effectively control the GCN inference layer call depth, inference path switching timing and inference process early exit decision based on the current inference confidence and text complexity.

[0014] Step S9: For multi-label long text scenarios, perform continuous label decision control. The DDPG continuous action control network outputs continuous label classification threshold, candidate label pruning ratio and label prediction priority order, and incorporates label mislabeling cost, label level constraints, classification accuracy, inference latency, memory usage and power consumption into the reward function to jointly optimize the entire label decision process.

[0015] Step S10: During the training phase, reward feedback and policy network iterative updates are implemented. Based on the text relationship graph classification accuracy, computational resource consumption, label prediction accuracy, mislabeling cost, and hierarchical constraint satisfaction after steps S8 and S9, a comprehensive reward value is calculated. The comprehensive reward value is input into the Critic value network, and the parameters of the Actor policy network and Critic value network are updated through backpropagation. This forms a closed-loop adaptive control system that coordinates text relationship graph structure optimization, inference process control, and label decision optimization. The above reward feedback and parameter updates are only effective during the training phase. During the inference phase, the DDPG network only executes action outputs and does not perform any parameter updates.

[0016] Furthermore, this invention also provides a dynamic optimization control system for text relationship graphs based on DDPG, used to implement the above-mentioned method. This system includes a text encoding module, a BERT representation module, a dynamic text relationship graph construction module, a DDPG continuous control module, an edge type-aware graph convolution module, an attention pooling module, a feature fusion classification module, an inference adaptive control module, a label continuous optimization module, and a reward calculation module.

[0017] The beneficial effects of this invention are as follows:

[0018] 1. Continuous adjustment of the text relationship graph structure. After adopting DDPG, the system adjusts the graph topology, edge weight distribution, and neighbor sampling strategy according to the text characteristics, alleviating the problem that traditional static graphs are difficult to adapt to different inputs, and making the text relationship graph configuration closer to the actual text features.

[0019] 2. The inference computation process is adjusted according to the state. The DDPG continuous action control network can adjust the GCN inference layer depth, model path, and early exit threshold by combining inference confidence and resource usage, thereby reducing inference latency, memory usage, and device power consumption while maintaining classification performance.

[0020] 3. Text relationship graph structure and inference process are collaboratively optimized under the same reward framework. By integrating the reward function, graph structure adjustment, inference process control, and label decision-making are incorporated into a unified DDPG framework, reducing the performance loss caused by the fragmentation of each module.

[0021] 4. More flexible decision-making in multi-label scenarios. DDPG can output continuous label classification thresholds, candidate label selection ratios, and label prediction order, thus allowing for a more nuanced trade-off between label error costs, hierarchical constraints, and computational costs. Attached Figure Description

[0022] Figure 1 This is a technical roadmap for the dynamic optimization control method for text relationship graphs based on DDPG proposed in this invention. Detailed Implementation

[0023] The technical solution of the present invention will be further described below with reference to the embodiments and accompanying drawings.

[0024] like Figure 1 As shown, the dynamic optimization control method and system for text relationship graphs based on DDPG of the present invention can be implemented according to the following steps:

[0025] Step S1: Perform word segmentation and encoding on the input text sequence. In actual implementation, WordPiece or BPE word segmentation algorithm is used to segment the original text into word sequence and generate corresponding attention mask. The mask value of the effective word position is set to 1, and the filling position is set to 0. When the word sequence length exceeds the preset maximum value, a truncation strategy is used for processing.

[0026] Step S2: Input the word sequence into the pre-trained BERT encoder. After context feature encoding through a multi-layer Transformer structure, the corresponding word-level hidden layer representation matrix is ​​obtained. With sentence-level convergence vectors In the formula Represents the length of the word sequence. Representing the hidden layer dimension, all parameters inside the BERT encoder participate in the end-to-end gradient update process. The learning rate adopts the AdamW optimization scheme of 2e-5, and is paired with an early stopping mechanism to avoid overfitting of the model.

[0027] Step S3: Create a dynamic text relationship graph based on semantic associations, word co-occurrence relationships, or syntactic dependency relationships within the text. The initial construction, in which Represents a set of nodes. Denotes the set of edges. The adjacency matrix is ​​represented, and the overall construction process consists of three parts:

[0028] Step S301, Syntactic Dependency Edge Construction: Call the Chinese syntactic analyzer to extract dependency relation triples from the text, establish directed dependency edges between the word pairs involved in the triples, with the edge direction following the dependency tree structure, and the initial edge weight determined by the dependency relation strength.

[0029] Step S302, Sliding window neighbor edge construction: Slide the window on the word sequence with a fixed window size, and establish undirected neighbor edges between any two words in the window as a supplement to the local context relationship. Different learnable fusion weights are assigned to the two types of edges.

[0030] Step S303, optional knowledge graph semantic edge expansion: For each lexical unit in the sequence, retrieve the corresponding concept entity from the external Chinese knowledge graph, extract the semantic relationship between any two lexical entities, and establish a bidirectional knowledge graph semantic edge set accordingly. Skip this step for lexical units not included in the graph. Perform symmetric normalization on the three types of edges respectively, and assign independent learnable fusion weight coefficients to each. After weighted summation, generate a three-channel heterogeneous adjacency matrix. The core structural parameters of the dynamic text relationship graph include the initial edge weights. sparse threshold Neighbor sampling rate Subgraph expansion jump count Subgraph Budget During the inference phase, these parameters are all adjusted in real time by the DDPG continuous action control network.

[0031] Step S4: The DDPG network adopts an Actor-Critic dual-network architecture, specifically designed as follows: The state input space is defined as: ,in This is the normalized value for text length. The structural complexity of the text relation graph is a weighted combination of the number of nodes and the number of edges. As the current confidence level of the inference, This represents the percentage of video memory already used. The reasoning time already used, This represents the current GCN inference layer number. This represents the percentage of processed text length. The label hierarchy constraint satisfaction. The continuous action space is defined as: All parameters are mapped to continuous real values ​​in the interval [0,1], and do not include any discrete classification or binary decisions. This refers to the adjustment coefficients for the edge weights of the text relationship graph. For neighbor sampling rate, To set the threshold for graph sparsity, Expand the hop count for the subgraph. Budget for the subgraph, For label classification threshold, The cropping ratio for candidate tags. Predict the priority order for the labels. For GCN inference layer depth, This is a command to switch inference paths. To set an early exit threshold for decision-making in the reasoning process.

[0032] The Actor policy network consists of a three-layer fully connected network. It takes a state vector as input and outputs a continuous action vector. The output is then mapped to [-1,1] by a tanh activation function and linearly transformed to the [0,1] interval. The Critic value network has the same structure, receiving a joint input of state-action pairs and outputting a Q-value estimate. Both networks use the Adam optimizer to update parameters, with the target network using soft-update coefficients. This ensures the stability of the training process.

[0033] Step S5: The BERT hidden layer representation matrix H is used as the graph node feature and fed into a multi-layer edge-type-aware graph convolutional network (Edge-Type-AwareGCN) to achieve information propagation and updating. The specific operation process of the l-th layer graph convolution can be divided into three parts:

[0034] Step S501: First, for R different types of edges, use independent learnable parameter matrices respectively. Perform weighted summation on the neighbor aggregation information transmitted by various edges;

[0035] Step S502: Next, the features of the current layer itself are superimposed to form a residual connection structure. After layer normalization, the node features H^(l+1) of the next layer are obtained.

[0036] Step S503: Finally, the GELU activation function is used to implement nonlinear transformation after each layer output. When the input and output dimensions are consistent, the residuals can be added directly to ensure the stability of the overall feature propagation process.

[0037] The graph convolution operation receives action parameters from DDPG in real time, among which... Used to drive the adjustment of edge weight distribution. Controlling the range of neighbor sampling To achieve graph sparsity processing, and The size of the subgraph is jointly constrained, and multiple types of parameters work together to achieve online adaptive adjustment of the text relationship graph structure. Each parameter matrix is ​​generated randomly with a uniform distribution during initialization, and the bias term is uniformly set to zero. The parameter matrices corresponding to different types of edges are initialized independently, and the multi-channel fusion coefficients are assigned equal weights in the initial stage.

[0038] Step S6: Convolve the graph and output the result. Perform learnable attention-weighted pooling to obtain sequence-level graph feature vectors. The specific process is as follows: using two learnable parameter matrices and For each word, perform bilinear scoring to obtain an importance score e_i; assign extremely negative values ​​to the scores of the filling positions to ensure that their weights approach zero after Softmax normalization; perform Softmax normalization on the scores of all valid words to obtain the attention weight α_i for each word; sum the features of each word according to the attention weights. This yields the graphical feature vectors of the entire sequence.

[0039] Step S7: Combine the graph feature vector g with the BERT sentence-level convergence vector By splicing, a merged representation is obtained. The classification head consists of three linear transformation layers stacked sequentially. The first layer will... Upgrade to The output is after batch normalization, GELU activation, and Dropout regularization. The second segment is compressed to the middle dimension. After GELU activation and Dropout regularization, the output is completed. The third segment maps the intermediate dimension to the number of categories C or the number of labels K, and outputs the final predicted score.

[0040] Step S8: Inference Phase. The DDPG continuous action control network continuously perceives the state and outputs control actions, executing the following strategy: High-complexity text processing: When the text complexity is higher than a preset threshold and the inference confidence is lower than a preset threshold, the DDPG output increases. ,reduce Expansion and increase The action parameters enhance the feature extraction capability of the text relationship graph, while improving... To ensure inference accuracy, a deep, high-parameter GCN is employed. For low-complexity text processing: when text complexity is below a preset threshold and inference confidence is above a preset threshold, DDPG output is reduced. ,improve ,reduce Action parameters, and based on Determine whether to terminate the reasoning prematurely, and reduce Switching to a shallow, low-parameter GCN enables fast inference. Dynamic inference path switching: Following the dynamic changes in text complexity and inference confidence, through... Real-time switching between shallow, low-parameter GCNs and deep, high-parameter GCNs improves computational efficiency while maintaining accuracy.

[0041] Step S9: For multi-label long text scenarios, the DDPG continuous action control network outputs the following continuous control parameters: continuous label classification threshold. Dynamically filter candidate labels whose classification scores exceed a threshold, replacing the fixed threshold strategy. Candidate label pruning ratio. To control the number of candidate labels, the top-K candidate labels with the highest classification scores are retained according to a pruning ratio, reducing subsequent computational costs. Label prediction priority order. The label prediction results are output according to priority, and hierarchical constraint violation penalties are applied to labels with hierarchical relationships. The multi-label reward function incorporates the following core reward items: mislabeling cost weight. Hierarchical constraint violation penalty value Inference accuracy gain value Inference latency reduction Memory usage reduction Power consumption reduction value The weighted combination constitutes a complete multi-label reward signal.

[0042] Step S10: The expression for the comprehensive reward function is as follows: ,in To normalize classification accuracy, To normalize inference delay, To normalize video memory usage, , , This is the balancing coefficient. The training mechanism of the DDPG network includes several key steps. The experience replay buffer stores historical interaction quadruples, including the state... ,action ,award Next state In each training round, a small batch of samples is randomly drawn from the buffer to update parameters. The Actor and Critic networks each have an online network and a target network, forming a total of four network replicas. The target network uses a soft update strategy to synchronize parameters, with the soft update coefficient ρ set to 0.005. During the exploration phase, Ornstein-Uhlenbeck noise is superimposed on the action output to ensure sufficient exploration scope. DDPG graph hyperparameter control and dynamic label decision control use independent experience replay buffers and the target network. Both the Actor and Critic networks in these two subsystems use the Adam optimizer for parameter updates. Reward feedback and parameter updates only take effect during the training phase. During the inference phase, all parameters of the DDPG network are frozen, and only the forward action output is executed without any parameter adjustments, thus maintaining the stability and efficiency of the inference process.

[0043] The text relationship graph dynamic optimization control system based on DDPG provided by this invention has the following functions and interaction relationships among its modules:

[0044] Text encoding module: Performs word segmentation and token encoding on the input text, and generates corresponding attention masks to provide standard-compliant input data for the subsequent BERT encoding process.

[0045] BERT Representation Module: Calls a pre-trained BERT model to generate a word-level context representation matrix and a sentence-level convergence vector. All parameters in the network participate in the end-to-end gradient update process.

[0046] Dynamic text relation graph construction module: Based on the semantic association, syntactic dependency relationship or word co-occurrence relationship of the text, the module initializes the dynamic text relation graph. It can optionally connect to the semantic edges of the external Chinese knowledge graph. It constructs a symmetric normalized heterogeneous adjacency matrix with edge type annotation based on the independent and learnable fusion coefficients of various edges, and receives DDPG control commands in real time to dynamically adjust the relevant parameters of the graph structure.

[0047] The DDPG continuous control module contains an Actor policy network and a Critic value network. It can receive various state inputs and output continuous action parameters corresponding to text relationship graph structure adjustment, inference flow control and label decision. During the training phase, it maintains an independent experience playback buffer and target network to realize iterative updates of policy parameters. During the inference phase, it only performs action output operations.

[0048] Edge type-aware graph convolution module: contains multi-layer graph convolution units, each unit assigns an independent weight matrix to different types of edges, includes residual connections, layer normalization and GELU activation, receives the graph structure output from the dynamic text relationship graph construction module, and performs graph feature propagation.

[0049] Attention pooling module: It performs soft weighted aggregation on the graph convolution output using a learnable bilinear scoring function to generate corresponding sequence-level graph feature vectors.

[0050] Feature fusion classification module: The spliced ​​image feature vector and BERT sentence-level convergence vector are passed through a three-layer fully connected network to output classification results or multi-label prediction results.

[0051] The inference adaptive control module maintains communication with the DDPG continuous control module, the dynamic text relationship graph construction module, and the edge type-aware graph convolution module. It executes the inference layer depth control, model path switching, and early exit decision-making of the DDPG output to achieve dynamic and reasonable allocation of computing resources.

[0052] Label Continuous Optimization Module: It maintains connection with the DDPG Continuous Control Module and Feature Fusion Classification Module, receives label class control parameters output by DDPG, realizes label threshold screening, candidate label pruning and label prediction order arrangement, and calculates the label mislabeling cost and hierarchical constraint satisfaction, and feeds the results back to the reward calculation module.

[0053] The reward calculation module is connected to the inference adaptive control module, the label continuous optimization module, and the DDPG continuous control module. Based on the text relationship graph classification accuracy, computational resource consumption, label prediction accuracy, mislabeling cost, and hierarchical constraint satisfaction, it calculates the comprehensive reward value. During the training phase, the reward value is input into the DDPG continuous control module to realize the iterative update of the parameters of the Actor policy network and the Critic value network.

Claims

1. A deep deterministic policy gradient (DDPG) based dynamic optimization control method for text relation graph, characterized in that, The method includes the following steps: Step S1: Perform word segmentation and encoding on the input text sequence, and output the corresponding word sequence and attention mask; Step S2: Feed the word sequence into the pre-trained BERT encoder to obtain a word-level hidden layer representation matrix and a sentence-level convergence vector with context awareness; Step S3: Based on the semantic relationships and syntactic dependencies of the text, initialize and construct the dynamic text relationship graph; Step S4: Construct a continuous action optimization network with DDPG as the core. The network consists of two parts: the Actor policy network and the Critic value network. The network's state input includes text features, text relationship graph structure complexity, model inference confidence, and computational resource consumption status. The output of the continuous action space includes three types of parameters: text relationship graph structure adjustment parameters, GCN inference control parameters, and multi-label decision parameters. Step S5: Using the BERT hidden layer representation matrix as node features, the graph structure information is propagated through a multi-layer edge type-aware graph convolutional network, and the action parameters output by the DDPG network are received in real time to dynamically adjust the text relationship graph structure. Step S6: Perform weighted pooling on the graph convolution output using a learnable attention mechanism to generate sequence-level graph feature vectors; S7: The graph feature vector and the BERT sentence-level convergence vector are concatenated and fused, and then passed through a multi-layer fully connected classification head to obtain the category prediction or multi-label prediction results.

2. The method according to claim 1, characterized in that, Step S8 also includes an adaptive computation and dynamic control mechanism for the text relationship graph during the inference phase. The DDPG continuous action control network continuously senses the current running status, outputs action parameters in real time to adjust the structure of the text relationship graph, and dynamically controls the GCN inference depth, inference path switching timing, and early exit decision of the inference process based on inference confidence and text complexity.

3. The method according to claim 1, characterized in that, Step S9 also includes a continuous decision control method for multi-label scenarios. The DDPG continuous action control network outputs a continuous label classification threshold, candidate label pruning ratio, and label prediction priority order. The label mislabeling cost, label level constraints, classification accuracy, inference latency, memory usage, and power consumption are all incorporated into the reward function to jointly optimize the label decision process.

4. The method according to claim 1, characterized in that, Step S10 also includes implementing reward feedback and iterative updates of the policy network during the training phase. The reward value is calculated based on the comprehensive index after the control action is executed in steps S8 and S9. The parameters of the Actor and Critic networks are updated through backpropagation to construct a closed-loop adaptive control system that coordinates text relationship graph structure optimization, inference process control, and label decision optimization.

5. The method according to claim 1, characterized in that, The dynamic text relationship graph in step S3 is a text semantic graph under the BERT and GCN fusion architecture. The nodes correspond to the word-level, sentence-level or paragraph-level feature vectors of the text. The initial values ​​of the edge weights are selected from the semantic similarity between nodes or the attention weights. This type of dynamic text relationship graph is not limited to fixed graphs, dual graphs or label graphs. The graph structure can be adjusted in real time according to the action parameters output by DDPG. During the initialization process, the dependency triplet extracted by the Chinese syntactic parser can also be combined to construct syntactic dependency edges. At the same time, the sliding window neighbor edges are retained as a supplement to the local context, and different learnable edge weights are assigned to the two types of edges.

6. The method according to claim 1, characterized in that, The status input for step S4 also includes the percentage of processed text length, the current GCN inference layer number (the specific number of GCN layers ranges from 3 to 6), the percentage of GPU memory consumed, the inference time already used, and the satisfaction of tag level constraints. Specific parameters of the continuous action space include the text relation graph edge weight adjustment coefficient. Neighbor sampling rate sparse threshold Subgraph expansion jump count Subgraph Budget Tag classification threshold Candidate tag cropping ratio Tag prediction priority order GCN inference layer depth Inference path switching command Early exit threshold for reasoning decisions ,in This is achieved by continuously weighting the label classification scores, replacing the discrete arrangement method with continuous real number weights. The path switching tendency is represented by consecutive real numbers in [0,1], and the actual executable path selection result is mapped by the threshold set by the system.

7. The method according to claim 1, characterized in that, In step S5, the operation method of each layer of the edge type-aware graph convolutional network is as follows: for different types of edges, independent learnable parameter matrices are used to aggregate the neighbor information transmitted by each type of edge separately and then perform weighted summation. The features of the current layer are superimposed as the residual structure. After layer normalization, the node features of the next layer are obtained. After each layer output, the non-linear transformation is achieved through the GELU activation function. When the input and output dimensions are the same, the residuals can be directly added to ensure the stability of feature propagation.

8. The method according to claim 1, characterized in that, The specific method of attention-weighted pooling in step S6 is as follows: first, the hidden representation of each word is scored using two learnable parameter matrices, with the specific formula being... The importance score of the current word is obtained, and then a very large negative value is assigned to the filling position to ensure that its weight approaches zero after Softmax normalization. Softmax normalization is then performed on the scores of all valid words to obtain the attention weight corresponding to each word. Finally, the features of each word are summed according to their weights to obtain the graph feature vector corresponding to the entire sequence. .

9. The method according to claim 1, characterized in that, The classification head in step S7 consists of three linear transformation layers stacked sequentially. The first layer performs dimensionality upscaling on the concatenated fusion features, and outputs the results after batch normalization, GELU activation, and Dropout regularization. The second layer further compresses the features to an intermediate dimension, and outputs the results after GELU activation and Dropout regularization. The third layer maps the intermediate dimension to the number of categories or labels, and outputs the classification prediction score or label prediction score.

10. The method according to claim 1, characterized in that, In step S8, the adaptive computation and dynamic control of the text relationship graph during the inference phase are divided into two scenarios. When the text complexity exceeds the preset threshold and the model inference confidence is lower than the preset threshold, the DDPG continuous action control network outputs action parameters that increase the neighbor sampling rate, decrease the sparsity threshold, expand the subgraph hop count, and increase the subgraph budget to enhance the feature extraction capability of the text relationship graph. At the same time, a deep, high-parameter GCN is enabled to ensure inference accuracy. When the text complexity is lower than the preset threshold and the model inference confidence is higher than the preset threshold, the DDPG continuous action control network outputs action parameters that decrease the neighbor sampling rate, increase the sparsity threshold, and reduce the subgraph budget. Based on the exit threshold, it is determined whether to terminate the inference early and instead enable a shallow, low-parameter GCN to achieve fast inference.

11. The method according to claim 1, characterized in that, In step S9, the continuous decision control method for the label graph in the multi-label long text scenario is as follows: the DDPG continuous action control network outputs a continuous label classification threshold to dynamically filter candidate labels that meet the standard, outputs the candidate label pruning ratio, controls the number of candidate labels to reduce computational cost, outputs the label prediction priority order, and gives the prediction results in order of priority. Hierarchical constraint violation penalties are applied to labels with hierarchical relationships. The mislabeling cost weight, hierarchical constraint violation penalty value, inference accuracy gain value, inference latency reduction value, memory usage reduction value, and power consumption reduction value are weighted and combined to form the core reward item in the reward function.

12. The method according to claim 1, characterized in that, The expression for the comprehensive reward function in step S10 is as follows: ,in To normalize classification accuracy, To normalize inference delay, To normalize video memory usage, As a balancing coefficient, this function can achieve joint optimization of computational cost while ensuring accuracy.

13. The method according to claim 1, characterized in that, The training mechanism of the DDPG continuous action control network in step S10 is as follows: the historical interaction quadruple data is stored in the experience replay buffer, which includes state, action, reward and next state. In each round of training, a small batch of samples is randomly drawn from the buffer to update the parameters. The Actor network and Critic network are set up as online network and target network respectively, forming a total of four network copies. The target network adopts a soft update strategy to synchronize parameters. During the exploration phase, the model will add Ornstein-Uhlenbeck noise to the action output to ensure a sufficient exploration range. The DDPG graph hyperparameter control and dynamic label decision control use independent experience replay buffers and target networks respectively. The Actor and Critic networks of both subsystems use the Adam optimizer to update parameters.

14. The method according to claim 1, characterized in that, In step S3, semantic relationship edges from an external Chinese knowledge graph can be further introduced on top of syntactic dependency edges and sliding window edges to form a three-channel heterogeneous graph. Specifically, when constructing the semantic edges of the knowledge graph, for each word in the word sequence, the corresponding concept entity in the external Chinese knowledge graph is retrieved, and the semantic relationship between any two words and their corresponding entities is extracted, covering hierarchical relationships, synonym relationships, and attribute relationships. Based on this, a bidirectional set of semantic edges of the knowledge graph is established. For words not included in the knowledge graph, the semantic edge construction step for that word is skipped. When generating the three-channel heterogeneous adjacency matrix, the three types of edges are symmetrically normalized, and each type of edge is assigned an independent learnable fusion weight coefficient. After weighted summation, they are merged into a three-channel heterogeneous adjacency matrix. The three fusion weight coefficients are automatically adjusted through end-to-end gradients during training. When expanding the edge type-aware graph convolution, an additional set of independent learnable parameter matrices is assigned to the semantic edges of the knowledge graph. Each layer of graph convolution aggregates the three types of edges separately and then performs a weighted summation. The DDPG network can synchronously adjust the relevant control parameters of the three types of edges.

15. A text relationship graph dynamic optimization control system based on DDPG, used to implement the method described in any one of claims 1 to 14, characterized in that, The system includes the following modules: Text encoding module: used for word segmentation, lexical encoding, and attention mask generation of input text; BERT Representation Module: Calls a pre-trained BERT model to generate word-level context representations and sentence-level convergence vectors; Dynamic text relation graph construction module: Initializes dynamic text relation graph based on text semantic association, syntactic dependency relationship or word co-occurrence relationship, supports optional access to semantic edges of external Chinese knowledge graph, constructs symmetric normalized heterogeneous adjacency matrix with edge type labeling through independent learnable fusion coefficients of various edges, and receives DDPG control commands in real time to adjust graph structure parameters; DDPG Continuous Control Module: Includes Actor policy network and Critic value network. It receives state input and outputs continuous action parameters for text relationship graph structure adjustment, inference flow control and label decision. During the training phase, it maintains an independent experience replay buffer and target network and is responsible for iterative updates of policy parameters. During the inference phase, it only executes action output. Edge type-aware graph convolution module: contains multi-layer graph convolution units, each equipped with an independent weight matrix for different edge types, including residual connections, layer normalization and GELU activation, receives the graph structure output by the dynamic text relation graph construction module and performs graph feature propagation; Attention pooling module: performs soft weighted aggregation on the graph convolution output using a learnable bilinear scoring function to generate sequence-level graph feature vectors; Feature fusion classification module: The spliced ​​image feature vector and BERT sentence-level convergence vector are passed through a three-layer fully connected network to output classification results or multi-label prediction results; Inference Adaptive Control Module: Establishes connections with DDPG Continuous Control Module, Dynamic Text Relationship Graph Construction Module, and Edge Type Aware Graph Convolution Module; executes inference layer depth control, model path switching, and early exit decisions from DDPG output; and dynamically allocates computing resources. Label Continuous Optimization Module: Establishes connections with the DDPG Continuous Control Module and Feature Fusion Classification Module, receives label-related control parameters output by DDPG, realizes label threshold screening, candidate label pruning and label prediction order arrangement, calculates label mislabeling cost and hierarchical constraint satisfaction and feeds it back to the reward calculation module; Reward Calculation Module: It establishes connections with the inference adaptive control module, label continuous optimization module, and DDPG continuous control module respectively. It calculates the reward value based on the comprehensive index. During the training phase, the reward value is input into the DDPG continuous control module to drive the iterative update of the parameters of the Actor policy network and the Critic value network.

16. The system according to claim 12, characterized in that, In the edge type-aware graph convolution module, the parameter matrices of each layer of graph convolution units are randomly initialized using a uniform distribution method, the initial value of the bias term is set to zero, the independent parameter matrices corresponding to different types of edges are initialized separately, and the multi-channel fusion coefficients are given equal weights in the initial stage and are automatically adjusted during training.

17. The system according to claim 12, characterized in that, BERT indicates that all parameters in the module participate in end-to-end gradient updates, using the AdamW optimization scheme with a learning rate of 2e-5 and enabling early stopping to avoid overfitting. The Actor and Critic networks in the DDPG continuous control module both use the Adam optimizer to update parameters, and the target network softly updates its coefficients. The value is set to 0.005, and the experience replay buffer adopts a first-in-first-out strategy to ensure sample diversity.

Images