Deep reinforcement learning optimization algorithm for big data analysis

Abstract

The invention relates to the technical field of learning optimization, in particular to a big data analysis-oriented deep reinforcement learning optimization algorithm, which comprises the steps of dynamic feature topological graph generation: performing time sequence correlation analysis on a big data stream input in real time to generate a dynamic feature topological graph; an enhanced state space is reconstructed, wherein the enhanced state space with space-time correlation characteristics is reconstructed through potential path mining; hierarchical decoupling training: asynchronous parameter updating is carried out by adopting a gradient isolation mechanism, and a strategy gradient flow and a value gradient flow are generated; multi-dimensional decision fusion: generating a multi-objective optimization decision through dynamic weight fusion; and feedback driving adjustment: according to the actual execution effect of the multi-objective optimization decision, reversely adjusting the construction threshold of the dynamic feature topological graph and the dimension parameter of the enhanced state space. According to the method, by constructing the dynamic feature topological graph, the time sequence dependency relationship in the big data flow can be captured in real time, the dynamic evolution process between the features is effectively reflected, and the flexibility and precision of feature modeling are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of learning optimization, and particularly to a deep reinforcement learning optimization algorithm for big data analysis. Background Art

[0002] With the rapid growth of data scale and the complexity of data types, traditional data analysis and decision optimization methods have gradually revealed various limitations. In the face of dynamically changing and real-time flowing big data streams, existing deep reinforcement learning algorithms usually adopt static feature modeling methods, which cannot fully characterize the time-dependent relationships and multi-dimensional dynamic feature evolution processes in the data streams, resulting in a lag in the model's response to data changes and difficulty in ensuring decision accuracy. At the same time, redundant features and weakly associated paths in the data streams are not effectively screened and dynamically adjusted during the static modeling process, further exacerbating the dimensionality disaster problem of the state space, significantly increasing the computational resource consumption and model complexity during the training process, and affecting the system's real-time decision-making ability and overall operation efficiency.

[0003] In addition, existing deep reinforcement learning methods generally have problems of low training efficiency and insufficient decision fusion ability. On the one hand, mainstream algorithms usually adopt a coupled policy and value joint training framework, lacking effective gradient flow isolation and asynchronous update mechanisms, resulting in obvious gradient interference phenomena, slow model convergence speed, and easy to fall into local optima. On the other hand, existing methods mostly rely on fixed-weight strategies for multi-objective decision-making, ignoring the dynamic change characteristics of policy gradients and value gradients in different training stages, making it difficult to achieve dynamic decision weight adjustment, resulting in poor multi-objective optimization effects. At the same time, the lack of an execution feedback closed-loop mechanism cannot reverse-adjust the feature topology construction and state space parameters according to the actual decision-making effects, restricting the model's adaptive adjustment ability and long-term optimization potential. Summary of the Invention

[0004] The present invention provides a deep reinforcement learning optimization algorithm for big data analysis. By constructing a dynamic feature topology graph, it can timely reflect the temporal dependence relationships in big data streams, and based on potential path mining, it realizes enhanced state space reconstruction, effectively improving the state representation ability.

[0005] The deep reinforcement learning optimization algorithm for big data analysis includes the following steps: S1, generation of a dynamic feature topology graph: performing temporal correlation analysis on the real-time input big data stream to generate a dynamic feature topology graph including timestamp dependence relationships; S2, enhanced state space reconstruction: based on the structural characteristics of the dynamic feature topology graph, reconstructing an enhanced state space with spatio-temporal correlation characteristics through potential path mining; S3, Hierarchical Decoupling Training: Input the enhanced state space into the policy network branch and the value network branch respectively, and use the gradient isolation mechanism to perform asynchronous parameter updates to generate the policy gradient flow and the value gradient flow; S4, Multi-dimensional Decision Fusion: Based on the distribution characteristics of the policy gradient flow and the value gradient flow, generate multi-objective optimization decisions through dynamic weight fusion; S5, Feedback-driven Adjustment: According to the actual execution effect of the multi-objective optimization decision, reversely adjust the construction threshold of the dynamic feature topology graph and the dimension parameters of the enhanced state space.

[0006] Optionally, the S1 includes: S11, Data Preprocessing and Timestamp Standardization: Perform timestamp normalization processing on the real-time input large data stream; S12, Temporal Correlation Calculation between Features: Use a sliding time window to slice the data stream and calculate the temporal correlation coefficient between any two features and ; S13, Constructing the Timestamp Dependent Edge Weight Matrix: Based on the calculated temporal correlation coefficients, construct an edge weight matrix; S14, Generating the Dynamic Feature Topology Graph: According to the edge weight matrix, construct a dynamic graph to obtain a sequence of feature topology graphs that change dynamically with the sliding of the time window.

[0007] Optionally, the S2 includes: S21, Generating the Candidate Set of Potential Paths: In the sequence of dynamic feature topology graphs, use the depth-first traversal algorithm to enumerate all possible paths within the current time window; S22, Calculating the Path Association Strength: For any path, calculate its spatio-temporal association strength; S23, Path Screening and Feature Mapping: According to the association strength threshold , screen the set of potential paths that meet the conditions, and map the screened paths to enhanced state space feature vectors; S24, Constructing the Enhanced State Space: Serialize the enhanced feature vectors within all time windows to construct the enhanced state space.

[0008] Optionally, the S3 includes: S31, Input of the Enhanced State Space and Construction of Network Loss: Input the enhanced state space into the policy network and the value network respectively. The policy network generates an action probability distribution through feedforward mapping and the Softmax function, and the value network outputs a state value evaluation. Respectively construct the loss function of the policy network; S32, Gradient Isolation Calculation and Asynchronous Update: Adopt a gradient isolation mechanism to calculate the gradient flows of the policy network and the value network separately, and use their respective learning rates for asynchronous parameter update.

[0009] Optionally, the S31 includes: S311, Policy Network Generates Action Probability Distribution: Input the state vector in the augmented state space into the policy network branch. First, perform feature extraction through a feedforward mapping function, and then calculate the probability distribution of each possible action through the Softmax function; S312, Value Network Outputs State Value Evaluation: Input the same state vector into the value network branch, calculate the value estimation result of the current state through a feedforward mapping function, and output the state value evaluation; S313, Construct Policy Network Loss Function: Based on the output result of the policy network, construct a policy loss function and calculate the product of the action log probability and the advantage function; S314, Construct Value Network Loss Function: According to the error between the state value estimation output by the value network and the actual cumulative reward, construct a value network loss function.

[0010] Optionally, the S32 includes: S321, Policy Gradient Flow Calculation: Perform backpropagation on the policy network loss function to calculate the gradient flow of the policy network; S322, Value Gradient Flow Calculation: Perform backpropagation on the loss function of the value network to calculate the gradient flow of the value network; S323, Asynchronous Parameter Update: Respectively use the respective learning rates of the policy network and the value network, and based on their respective gradient flows, asynchronously update the parameters of the two networks.

[0011] Optionally, the S4 includes: S41, Extract Gradient Flow Features and Calculate Fusion Weights: Extract the mean and variance of the policy gradient flow and the value gradient flow, and dynamically calculate the fusion weights according to the variance size to reduce the influence of the gradient with large volatility; S42, Weighted Fusion Generates Optimization Decision: Use dynamic weights to perform weighted fusion on the two gradient flows to generate a multi-objective optimization decision vector.

[0012] Optionally, the S41 includes: S411, Policy Gradient Flow and Value Gradient Flow Distribution Feature Extraction: Extract the distribution features of the policy gradient flow and the value gradient flow, and calculate the mean and variance respectively to describe the change trend of the gradient flow; S412, Dynamic Weight Calculation: Dynamically calculate the fusion weights according to the distribution features, and use normalization to keep the sum of the weights as 1.

[0013] Optionally, S5 includes: S51, calculating the actual execution effect index: According to the multi-objective optimization decision vector, the cumulative reward within the current period is statistically calculated to obtain the comprehensive execution effect index; S52, adjusting the construction threshold of the dynamic feature topology graph: Comparing the actual execution effect with the preset target effect, dynamically adjusting the construction threshold of the feature topology graph. When the execution effect is low, the threshold is increased to enhance the feature screening intensity; otherwise, the threshold is decreased to enrich the topological structure; S53, adjusting the dimension parameter of the enhanced state space: Dynamically adjusting the dimension parameter of the state space according to the actual execution effect.

[0014] Optionally, S53 dynamically adjusts the dimension parameter of the state space according to the actual execution effect. When the effect is good, the dimension is appropriately increased to enrich the state description; when the effect is poor, the dimension is appropriately decreased to reduce redundancy and improve the training efficiency.

[0015] Advantages of the present invention: By constructing a dynamic feature topology graph, the present invention can capture the temporal dependence relationship in the large data stream in real time, effectively reflect the dynamic evolution process between features, and significantly improve the flexibility and accuracy of feature modeling; combined with the potential path mining technology, the enhanced state space is reconstructed to fully explore the deep associations hidden between multi-dimensional features, effectively alleviating the state space dimensionality disaster problem existing in traditional methods, and improving the state expression ability and data utilization efficiency; at the same time, the construction threshold of the dynamic feature topology graph and the dimension parameter of the state space can be adaptively optimized through feedback adjustment, further enhancing the adaptability of the model to dynamic data changes, and significantly improving the real-time response ability and modeling accuracy of the system in a complex data environment.

[0016] Through the hierarchical decoupling training strategy, the present invention realizes the independent optimization of the policy network and the value network, adopts the gradient isolation and asynchronous parameter update methods, effectively avoids the gradient interference problem, speeds up the model convergence speed, and improves the training efficiency; further, combined with the multi-dimensional decision fusion method, the fusion weight of the policy gradient flow and the value gradient flow is dynamically calculated, fully utilizing the gradient flow change characteristics in different training stages to realize the adaptive optimization of multi-objective decision-making, and improving the decision-making accuracy and system stability; at the same time, a feedback-driven adjustment mechanism is constructed to dynamically optimize the model structure parameters according to the actual execution effect of multi-objective decision-making, forming a complete closed-loop self-optimization process, significantly enhancing the long-term optimization ability and environmental adaptability of the model, and meeting the high-efficiency and robustness requirements of multi-objective decision-making in complex and changeable scenarios. Description of the Drawings

[0017] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only those of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0018] Figure 1 It is the flowchart of the method of the embodiment of the present invention; Figure 2 It is the spatial reconstruction diagram of the embodiment of the present invention. Detailed implementation manners

[0019] The present invention will be described in detail below in conjunction with the drawings and specific embodiments. At the same time, it should be noted here that in order to make the embodiments more detailed, the following embodiments are the best and preferred embodiments. For some well-known technologies, those skilled in the art can also adopt other alternative ways for implementation; moreover, the drawings are only for more specific description of the embodiments and are not intended to specifically limit the present invention.

[0020] It should be pointed out that in the specification, when referring to "an embodiment", "embodiments", "exemplary embodiments", "some embodiments", etc., it indicates that the described embodiments may include specific features, structures or characteristics, but not necessarily every embodiment includes such specific features, structures or characteristics. In addition, when combining embodiments to describe specific features, structures or characteristics, implementing such features, structures or characteristics in combination with other embodiments (whether explicitly described or not) should be within the knowledge of those skilled in the relevant art.

[0021] Generally, the terms can be understood at least in part from their use in the context. For example, at least in part depending on the context, the term "one or more" used herein can be used to describe any feature, structure or characteristic in a singular sense, or can be used to describe a combination of features, structures or characteristics in a plural sense. In addition, the term "based on" can be understood as not necessarily intended to convey a set of exclusive factors, but instead, at least in part depending on the context, allowing the existence of other factors that may not be explicitly described.

[0022] As Figure 1 - Figure 2 shown, the deep reinforcement learning optimization algorithm for big data analysis includes the following steps: S1, dynamic feature topology graph generation: performing temporal correlation analysis on the real-time input big data stream to generate a dynamic feature topology graph including timestamp dependency relationships; S2, enhanced state space reconstruction: based on the structural characteristics of the dynamic feature topology graph, reconstructing an enhanced state space with spatio-temporal correlation characteristics through latent path mining; S3, Hierarchical decoupled training: Input the enhanced state space into the policy network branch and the value network branch respectively, and use the gradient isolation mechanism to perform asynchronous parameter updates to generate the policy gradient flow and the value gradient flow; S4, Multi-dimensional decision fusion: Based on the distribution characteristics of the policy gradient flow and the value gradient flow, generate multi-objective optimization decisions through dynamic weight fusion; S5, Feedback-driven adjustment: According to the actual execution effect of the multi-objective optimization decision, reversely adjust the construction threshold of the dynamic feature topology graph and the dimension parameters of the enhanced state space.

[0023] S1 includes: S11, Data preprocessing and timestamp standardization: Perform timestamp normalization on the real-time input large data stream which is expressed as: ; where represents the feature vector of the th data, represents the timestamp of the th data, and represent the minimum timestamp and the maximum timestamp in the data stream respectively; S12, Calculation of temporal correlation between features: Use a sliding time window to slice the data stream and calculate the temporal correlation coefficient between any two features , which is expressed as: ; where represents the value of feature in sample , represents the mean value of feature within window ; S13, Construction of timestamp-dependent edge weight matrix: Based on the calculated temporal correlation coefficient, construct the edge weight matrix , which is expressed as: [[ID=6)]]; where represents the average timestamp difference between features and , and are the adjustment coefficients of the correlation term and the time difference term respectively; S14, Generation of dynamic feature topology graph: According to the edge weight matrix construct the dynamic graph , a sequence of characteristic topology maps that change dynamically as the time window slides, is represented as: ; Among them, represents the set of characteristic nodes, represents the set of edges whose edge weights satisfy the threshold condition within the current time window, is a dynamic threshold, which is dynamically updated according to the feedback-driven adjustment step.

[0024] S2 includes: S21, generation of the candidate set of potential paths: In the sequence of dynamic characteristic topology maps , using the depth-first search (DFS) algorithm, enumerate all possible paths within the current time window, and the path length is ; S22, calculation of path correlation strength: For any path , calculate its spatio-temporal correlation strength , which is represented as: ; Among them, is the spatio-temporal correlation strength of the path, is the adjacent node pair in the path, is the edge weight, is the time decay factor, is the time decay coefficient, which is a positive real number and controls the degree of time dependence, is the average timestamp difference between nodes; S23, path screening and feature mapping: According to the correlation strength threshold , ( ), screen the candidate set of potential paths that meet the conditions, and map the screened paths to the enhanced state space feature vector , which is represented as: ; ; Among them, is the path feature aggregation mapping function, which is implemented through pooling operations (Max / Mean Pooling), is the dimension of the enhanced state space, which is used as a control variable for feedback-driven adjustment; S24, construction of the enhanced state space: Serialize the enhanced feature vectors within all time windows to construct the enhanced state space , which is represented as: ; Among them, is the constructed enhanced state space, is the time of the enhanced state vector.

[0025] S3 includes: S31, constructing the enhanced state space input and network loss: Input the enhanced state space into the policy network and the value network respectively. The policy network generates an action probability distribution through feed-forward mapping and the Softmax function, and the value network outputs a state value evaluation. Construct the policy network loss function to optimize action selection, and the value network loss function to optimize state value estimation; S32, gradient isolation calculation and asynchronous update: Adopt a gradient isolation mechanism to calculate the gradient flows of the policy network and the value network respectively, and perform asynchronous parameter updates using their respective learning rates.

[0026] S31 includes: S311, the policy network generates an action probability distribution: Input the state vector in the enhanced state space into the policy network branch. First, perform feature extraction through the feed-forward mapping function, and then calculate the probability distribution of each possible action through the Softmax function, expressed as: ; where is the policy network branch, is the policy network parameter, is the feed-forward mapping function; S312, the value network outputs a state value evaluation: Input the same state vector into the value network branch, calculate the value estimation result of the current state through the feed-forward mapping function, and output the state value evaluation, expressed as: ; where is the value network branch, is the value network parameter, is the mapping function; S313, constructing the policy network loss function: Based on the output result of the policy network, construct a policy loss function, and calculate the product of the action log probability and the advantage function, expressed as: ; ; where is the advantage function, is the policy loss function, is the time of the immediate reward, is the discount factor, is the expected value on the state-action pair; S314. Construct the value network loss function: Based on the error between the state value estimate output by the value network and the actual cumulative reward, construct the value network loss function, expressed as: ; ; Among them, is the discounted cumulative reward, is the value network loss function, is the expected value on the state.

[0027] S32 includes: S321. Calculate the policy gradient flow: Perform backpropagation on the policy network loss function to calculate the gradient flow of the policy network, expressed as: ; Among them, is the policy gradient flow; S322. Calculate the value gradient flow: Perform backpropagation on the loss function of the value network to calculate the gradient flow of the value network, expressed as: ; Among them, is the value gradient flow; S323. Asynchronously update the parameters: Respectively use the learning rates of the policy network and the value network, and based on their respective gradient flows, asynchronously update the parameters of the two networks, expressed as: ; ; Among them, and are the learning rates of the policy network and the value network respectively, is the set of policy network parameters, is the set of value network parameters.

[0028] S4 includes: S41. Extract the gradient flow features and calculate the fusion weights: Extract the mean and variance of the policy gradient flow and the value gradient flow, and dynamically calculate the fusion weights according to the variance size to reduce the influence of the gradient with large volatility; S42. Generate the optimized decision by weighted fusion: Use the dynamic weights to perform weighted fusion on the two gradient flows to generate the multi-objective optimized decision vector, expressed as: ; Among them, is the multi-objective optimized decision vector.

[0029] S41 includes: S411, Extraction of distribution characteristics of policy gradient flow and value gradient flow: Extract the distribution characteristics of the policy gradient flow and the value gradient flow , calculate the mean and variance respectively, which are used to describe the change trend of the gradient flow, and are expressed as: ; ; where are the means of the policy gradient flow and the value gradient flow respectively, are the variances of the policy gradient flow and the value gradient flow respectively; S412, Dynamic weight calculation: Dynamically calculate the fusion weights according to the distribution characteristics, and use normalization processing to keep the sum of the weights as 1, which is expressed as: ; where is the fusion weight of the policy gradient flow, is the fusion weight of the value gradient flow.

[0030] S5 includes: S51, Calculate the actual execution effect index: According to the multi-objective optimization decision vector , count the cumulative rewards within the current period, and calculate the comprehensive execution effect index , which is expressed as: ; where is the number of time steps within the current evaluation period, is the discounted cumulative reward at time , is the comprehensive execution effect index; S52, Adjust the construction threshold of the dynamic feature topology map: Compare the actual execution effect with the preset target effect, and dynamically adjust the construction threshold of the feature topology map. When the execution effect is low, increase the threshold to enhance the feature screening intensity, and vice versa, decrease the threshold to enrich the topological structure, which is expressed as: ; where is the construction threshold at the -th iteration, is the threshold adjustment step size, a positive real number, is the preset target execution effect value. When the actual effect is lower than the expectation, increase the construction threshold to enhance the feature screening strictness; when the effect is better than the expectation, decrease the threshold to enrich the feature connection; S53, Adjust the dimension parameter of the enhanced state space: Dynamically adjust the dimension parameter of the state space according to the actual execution effect.

[0031] S53 dynamically adjusts the dimensional parameter of the state space according to the actual execution effect . When the effect is excellent, appropriately increase the dimension to enrich the state description. When the effect is poor, appropriately reduce the dimension to reduce redundancy and improve the training efficiency, which is expressed as: ; wherein, is the dimensionality of the state space at the -th iteration, and is the dimensional adjustment step size, a positive real number.

[0032] The present invention covers any substitutions, modifications, equivalent methods and solutions made on the essence and scope of the present invention. For the public to have a thorough understanding of the present invention, specific details are described in detail in the following preferred embodiments of the present invention. However, those skilled in the art can fully understand the present invention without the description of these details. In addition, well-known methods, processes, procedures, components and circuits are not described in detail to avoid unnecessary confusion to the essence of the present invention.

[0033] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. A deep reinforcement learning optimization algorithm for big data analysis, characterized in that It includes the following steps: S1, Dynamic Feature Topology Map Generation: Perform temporal correlation analysis on the real-time input big data stream to generate a dynamic feature topology map including timestamp dependency relationships; S2, Enhanced State Space Reconstruction: Based on the structural characteristics of the dynamic feature topology map, reconstruct an enhanced state space with spatio-temporal correlation characteristics through potential path mining; S3, Hierarchical Decoupling Training: Input the enhanced state space into the policy network branch and the value network branch respectively, and use the gradient isolation mechanism to perform asynchronous parameter updates to generate a policy gradient flow and a value gradient flow; S4, Multi-dimensional Decision Fusion: Based on the distribution characteristics of the policy gradient flow and the value gradient flow, generate a multi-objective optimization decision through dynamic weight fusion; S5, Feedback-driven Adjustment: According to the actual execution effect of the multi-objective optimization decision, reversely adjust the construction threshold of the dynamic feature topology map and the dimension parameters of the enhanced state space.

2. The deep reinforcement learning optimization algorithm for big data analysis according to claim 1, wherein The S1 includes: S11, Data Preprocessing and Timestamp Standardization: Perform timestamp normalization processing on the real-time input big data stream; S12, Calculation of the temporal correlation between features: Use a sliding time window Slice the data stream and calculate the temporal correlation coefficient between any two features and ; S13, Construct Timestamp Dependency Edge Weight Matrix: Based on the calculated temporal correlation coefficients, construct an edge weight matrix; S14, Dynamic Feature Topology Map Generation: Construct a dynamic graph according to the edge weight matrix to obtain a sequence of feature topology maps that change dynamically as the time window slides.

3. The deep reinforcement learning optimization algorithm for big data analysis according to claim 2, characterized in that The S2 includes: S21, Potential Path Candidate Set Generation: In the sequence of dynamic feature topology maps, use the depth-first traversal algorithm to enumerate all possible paths within the current time window; S22, Path Association Strength Calculation: For any path, calculate its spatio-temporal association strength; S23, Path Screening and Feature Mapping: Based on the association strength threshold , screen the set of potential paths that meet the conditions, and map the screened paths into enhanced state space feature vectors; S24, Enhanced State Space Construction: Serialize the enhanced feature vectors within all time windows to construct an enhanced state space.

4. The deep reinforcement learning optimization algorithm for big data analysis according to claim 3, characterized in that The S3 includes: S31, Enhanced State Space Input and Network Loss Construction: Input the enhanced state space into the policy network and the value network respectively. The policy network generates an action probability distribution through feedforward mapping and the Softmax function, and the value network outputs a state value evaluation. Respectively construct the policy network loss function; S32, Gradient Isolation Calculation and Asynchronous Update: Use the gradient isolation mechanism to calculate the gradient flows of the policy network and the value network respectively, and perform asynchronous parameter updates using their respective learning rates.

5. The deep reinforcement learning optimization algorithm for big data analysis according to claim 4, wherein The S31 includes: S311, Policy Network Generates Action Probability Distribution: Input the state vector in the enhanced state space into the policy network branch. First, perform feature extraction through the feedforward mapping function, and then calculate the probability distribution of each possible action through the Softmax function; S312, Value Network Outputs State Value Evaluation: Input the same state vector into the value network branch, calculate the value estimation result of the current state through the feedforward mapping function, and output the state value evaluation; S313, Construct Policy Network Loss Function: Based on the output result of the policy network, construct a policy loss function and calculate the product of the action log probability and the advantage function; S314, Construct Value Network Loss Function: According to the error between the state value estimation output by the value network and the actual cumulative reward, construct a value network loss function.

6. The deep reinforcement learning optimization algorithm for big data analysis according to claim 4, characterized in that The S32 includes: S321, Policy Gradient Flow Calculation: Perform backpropagation on the loss function of the policy network to calculate the gradient flow of the policy network; S322, Value Gradient Flow Calculation: Perform backpropagation on the loss function of the value network to calculate the gradient flow of the value network; S323, Asynchronous Parameter Update: Respectively use the learning rates of the policy network and the value network, and based on their respective gradient flows, asynchronously update the parameters of the two networks.

7. The deep reinforcement learning optimization algorithm for big data analysis according to claim 6, characterized in that The S4 includes: S41, Extract Gradient Flow Features and Calculate Fusion Weights: Extract the mean and variance of the policy gradient flow and the value gradient flow, and dynamically calculate the fusion weights according to the variance size to reduce the influence of the gradient with large volatility; S42, Weighted Fusion to Generate Optimization Decision: Use dynamic weights to perform weighted fusion on the two gradient flows to generate a multi-objective optimization decision vector.

8. The deep reinforcement learning optimization algorithm for big data analysis according to claim 7, characterized in that The S41 includes: S411, Distribution Feature Extraction of Policy Gradient Flow and Value Gradient Flow: Extract the distribution features of the policy gradient flow and the value gradient flow, and calculate the mean and variance respectively to describe the change trend of the gradient flow; S412, Dynamic Weight Calculation: Dynamically calculate the fusion weights according to the distribution features, and use normalization to keep the sum of the weights as 1.

9. The deep reinforcement learning optimization algorithm for big data analysis according to claim 7, characterized in that The S5 includes: S51, Calculate the Actual Execution Effect Index: According to the multi-objective optimization decision vector, count the cumulative rewards within the current period and calculate the comprehensive execution effect index; S52, Adjust the Construction Threshold of the Dynamic Feature Topological Map: Compare the actual execution effect with the preset target effect, and dynamically adjust the construction threshold of the feature topological map. When the execution effect is low, increase the threshold to enhance the feature screening intensity, and vice versa, reduce the threshold to enrich the topological structure; S53, Adjust the Dimension Parameter of the Enhanced State Space: Dynamically adjust the dimension parameter of the state space according to the actual execution effect.

10. The deep reinforcement learning optimization algorithm for big data analysis according to claim 9, wherein The S53 dynamically adjusts the dimension parameter of the state space according to the actual execution effect. When the effect is good, appropriately increase the dimension to enrich the state description. When the effect is poor, appropriately reduce the dimension to reduce redundancy and improve the training efficiency.

Images