Database adaptive data flow acquisition optimization method and system based on reinforcement learning

Abstract

The invention provides a reinforcement learning-based database adaptive data flow acquisition optimization method and system, and relates to the technical field of databases, and the method comprises the steps of constructing a distributed acquisition proxy network, obtaining database data flow information, and carrying out multi-modal feature decomposition and fusion. And calculating an optimal initial acquisition parameter combination by using the fusion feature vector and combining a genetic algorithm, constructing a double-layer deep Q learning network structure, performing hierarchical training, and generating a hierarchical acquisition strategy model. The model adaptively adjusts a global acquisition strategy and parameters according to a characteristic drift degree and a performance index so as to realize continuous optimization of the acquisition strategy. According to the method, through multi-modal feature fusion, reinforcement learning and a self-adaptive tuning mechanism, the data flow collection efficiency is effectively improved, the collection delay is reduced, and the resource utilization rate is optimized.

Description

Technical Field

[0001] The present invention relates to database technology, and in particular to a database adaptive data stream acquisition optimization method and system based on reinforcement learning. Background Art

[0002] With the rapid development of big data technology, data stream collection in databases has become an important research direction. How to efficiently and in real time collect data streams from databases and adaptively adjust collection strategies according to the dynamic changes of data streams is crucial to ensure the accuracy and timeliness of data analysis.

[0003] Existing data stream collection methods mainly adopt the method of statically configuring collection parameters, which cannot adapt to the dynamic changes of data streams. In addition, traditional collection methods usually only focus on a single indicator, such as collection delay or throughput, and lack comprehensive optimization of multiple objectives. Moreover, existing methods rarely consider the changes in data stream characteristics, resulting in a decrease in the effectiveness of collection strategies and difficulty in meeting the needs of practical applications. Summary of the invention

[0004] The embodiments of the present invention provide a database adaptive data stream acquisition optimization method and system based on reinforcement learning, which can solve the problems in the prior art.

[0005] According to a first aspect of the embodiments of the present invention, Provides a database adaptive data stream collection optimization method based on reinforcement learning, including: Construct a distributed data acquisition agent network, obtain data flow information from the database in real time, and generate an initial feature set; perform multimodal decomposition on the initial feature set, extract data flow fluctuation spectrum eigenvalues, fine-grained type distribution eigenvalues, multi-scale time series eigenvalues ​​and graph structure association eigenvalues, and form a multimodal feature matrix; based on the multimodal feature matrix, calculate the feature weight coefficient through an improved attention mechanism, adaptively fuse different modal features, and output a fused feature vector; use the fused feature vector and combine it with a genetic algorithm to calculate the optimal initial acquisition parameter combination, including concurrent acquisition threshold, cache capacity threshold and batch size threshold, and generate an initial state space vector; Based on the initial state space vector and the acquisition parameter combination, a two-layer deep Q learning network structure is constructed, the input of the first network is set to the fused feature vector, and a global acquisition strategy is output; the input of the second network is set to the initial state space vector, and a specific parameter adjustment plan is output; a hierarchical reward function is designed, the feature weight coefficient is used as a balancing factor of the reward function, and a multi-objective evaluation system including acquisition delay rate, throughput efficiency and resource utilization is established; a hierarchical training process is performed based on the multi-objective evaluation system, the global acquisition strategy output by the first network is used as a constraint condition for the second network, and the network parameters are iteratively updated through a priority experience replay mechanism to generate a hierarchical acquisition strategy model; The hierarchical acquisition strategy model is deployed to the production environment, data stream features are continuously collected, and similarity is calculated between real-time features and the fused feature vector to evaluate the degree of feature drift; when the degree of feature drift exceeds a preset feature threshold, the first network is triggered to re-plan the global acquisition strategy based on the current fused feature vector, and the updated global acquisition strategy is input into the second network for parameter fine-tuning; the acquisition performance index is monitored in real time, and when the performance index is lower than the preset target threshold, the progressive tuning mechanism is started, the newly collected state transition sequence is input into the training cache pool, and incremental learning is performed based on historical tuning experience to achieve continuous optimization of the hierarchical acquisition strategy model.

[0006] Based on the multimodal feature matrix, the feature weight coefficient is calculated through the improved attention mechanism, the features of different modalities are adaptively fused, and the fused feature vector is output; the fused feature vector is used to calculate the optimal initial acquisition parameter combination in combination with the genetic algorithm, including the concurrent acquisition threshold, the cache capacity threshold and the batch size threshold, and the initial state space vector is generated, including: The multimodal feature matrix includes data flow fluctuation spectrum feature values ​​reflecting the dynamic changes of the data flow, fine-grained type distribution feature values ​​reflecting the data distribution law, and multi-scale time series feature values ​​reflecting the time series fluctuation law. A data multi-dimensional representation feature matrix is ​​constructed based on the data flow fluctuation spectrum feature values, the fine-grained type distribution feature values, and the multi-scale time series feature values; Calculate the statistical distribution characteristics of the multidimensional characterization feature matrix of the data, select a corresponding normalization strategy according to the statistical distribution characteristics, use the maximum and minimum value normalization for the data flow fluctuation spectrum feature value to maintain the peak and trough relationship, use softmax normalization for the fine-grained type distribution feature value to highlight the main type, and use z-score normalization for the multi-scale time series feature value to eliminate the dimension effect, and generate a standardized feature matrix; Analyze the information entropy distribution of each modal feature in the standardized feature matrix, use the information entropy distribution as a modal importance indicator, construct a modal adaptive factor based on the information entropy distribution, dynamically adjust the modal adaptive factor with the information entropy distribution to adaptively capture information volume features, and integrate the modal adaptive factor into the attention mechanism to calculate the dynamic attention weight; A two-layer feature fusion network is constructed using the dynamic attention weights. In the first layer, a self-attention operation is performed within the modality to extract the internal correlation of the modality. In the second layer, a cross-attention operation between modalities is performed to model the complementary relationship of the modalities. The outputs of the two layers of attention are fused through a residual structure and a normalization layer to obtain a fused feature vector containing multi-modal complementary information. Component analysis is performed on the fused feature vector, and the concurrent acquisition threshold boundary is determined by using the period and peak characteristics of the data stream fluctuation spectrum feature value, the cache capacity threshold boundary is determined by using the entropy value and skewness characteristics of the fine-grained type distribution feature value, and the batch processing threshold boundary is determined by using the autocorrelation and stationary characteristics of the multi-scale time series feature value, and the three threshold boundaries constitute a parameter search space; Extracting feature importance based on the principal component analysis result of the fused feature vector, mapping the feature importance into a weight coefficient, constructing a multi-objective fitness function including system throughput, processing delay and resource utilization, and dynamically weighting the three threshold boundaries according to the weight coefficient; Initializing a genetic algorithm population in the parameter search space, using the multi-objective fitness function to evaluate the quality of individuals in the population, adaptively adjusting the crossover probability based on the population diversity level to balance local and global searches, adaptively adjusting the mutation probability based on the optimization convergence degree to prevent premature convergence, and obtaining the optimal parameter combination through iterative optimization; The optimal parameter combination and the fused feature vector are deeply embedded to generate a state space vector of coupling parameter information and feature representation.

[0007] Based on the initial state space vector and the acquisition parameter combination, a double-layer deep Q learning network structure is constructed, the input of the first network is set to the fused feature vector, and a global acquisition strategy is output; the input of the second network is set to the initial state space vector, and a specific parameter adjustment scheme is output, including: Based on the initial state space vector and the acquisition parameter combination, a double-layer deep Q learning network structure is constructed, the fused feature vector is set as the input of the first network, and the initial state space vector is set as the input of the second network, and the first network and the second network adopt a double network structure in which the target network and the evaluation network are separated; A dual-stream feature extraction channel is constructed in the first network, the first channel extracts static features in the fused feature vector through a multi-layer fully connected network to obtain a static feature representation, and the second channel extracts dynamic features in the fused feature vector through a bidirectional LSTM network to obtain a dynamic feature representation, and the static feature representation and the dynamic feature representation are adaptively fused through a multi-head attention mechanism to output a global acquisition strategy, wherein the global acquisition strategy includes an adjustment direction, an adjustment step and an adjustment priority, wherein the adjustment direction is used to indicate a parameter increase or decrease, the adjustment step is used to control a parameter change amplitude, and the adjustment priority is used to determine a multi-parameter adjustment order; In the second network, based on the initial state space vector, the Dueling architecture is used to decompose the Q value into a state value function and an advantage function. The state value function evaluates the parameter configuration performance through a three-layer feedforward neural network to obtain a performance evaluation value. The advantage function evaluates and adjusts the action advantage through a deep residual network to obtain an action advantage value. The performance evaluation value and the action advantage value are weightedly combined to output a specific parameter adjustment plan.

[0008] Deploy the hierarchical collection strategy model to the production environment, continuously collect data stream features, calculate the similarity between the real-time feature vector and the fused feature vector, and evaluate the degree of feature drift; when the degree of feature drift exceeds a preset feature threshold, trigger the first network to re-plan the collection strategy based on the current fused feature vector, and input the updated global strategy into the second network for parameter fine-tuning, including: The hierarchical collection strategy model is deployed to the production environment to continuously collect data stream features to obtain real-time feature vectors, and the directional deviation degree of the real-time feature vector and the fused feature vector on the continuous feature is calculated by cosine similarity to obtain continuous feature similarity, and the distribution change amplitude of the real-time feature vector and the fused feature vector on the discrete feature is calculated by KL divergence to obtain discrete feature similarity; The continuous feature similarity and the discrete feature similarity are normalized and weighted to obtain a feature drift comprehensive index, and a multi-scale feature drift detection mechanism is constructed based on the feature drift comprehensive index; within a first time window, the feature drift comprehensive index is smoothed by an exponentially weighted moving average method to obtain a feature trend, within a second time window, a mutation point detection algorithm is used to identify the jump position of the feature drift comprehensive index to obtain a feature change point, and within a third time window, a trend analysis method is used to capture the gradual change process of the feature drift comprehensive index to obtain a feature trend; Input the feature trend, the feature change point and the feature trend into a decision fusion module, compare the feature drift index at each time scale with the corresponding preset feature threshold, and generate a feature drift degree assessment result; When the feature drift degree evaluation result indicates that the feature drift index at any time scale exceeds the corresponding preset feature threshold, recalculating the feature importance weight matrix based on the current fused feature vector, and inputting the feature importance weight matrix into the first network; The first network replans the acquisition strategy according to the feature importance weight matrix, updates the feature extraction scheme to obtain the feature weight update strategy, adjusts the parameter change direction and change step size based on the feature weight update strategy to obtain the global strategy update scheme, and inputs the global strategy update scheme into the second network for parameter fine-tuning.

[0009] Real-time monitoring of acquisition performance indicators. When the performance indicators are lower than the preset target threshold, the progressive tuning mechanism is started, the newly acquired state transition sequence is input into the training cache pool, and incremental learning is performed based on historical tuning experience to achieve continuous optimization of the hierarchical acquisition strategy model, including: Construct a multi-dimensional performance monitoring model, collect data throughput, data delay time, and resource utilization in real time to obtain efficiency index data, collect data integrity, data accuracy, and data timeliness to obtain quality index data, and collect computing overhead, storage overhead, and bandwidth overhead to obtain cost index data; calculate the respective index weights for the efficiency index data, the quality index data, and the cost index data, and perform a weighted combination of the index weights and the corresponding index data to obtain a comprehensive performance score; compare the comprehensive performance score with a preset target threshold, and trigger a parameter optimization process when the comprehensive performance score is lower than the preset target threshold; In the parameter optimization process, the parameter space is randomly sampled based on the Thompson sampling strategy to obtain a parameter sampling set, and the uncertainty of each parameter interval in the parameter sampling set is calculated to obtain the parameter exploration priority; the parameter sampling set is prioritized according to the parameter exploration priority to obtain a parameter tuning sequence, and the estimated mean and estimated standard deviation of each parameter in the parameter tuning sequence are calculated; the state transition data generated during the optimization process is collected, and the state transition data is stored in the first cache layer, the second cache layer and the third cache layer respectively according to the data timestamp to construct a hierarchical training cache pool; Calculate the timeliness weight and value weight of the experience data in the hierarchical training cache pool, multiply the timeliness weight and the value weight to obtain the experience sampling weight; sample the training data set from the hierarchical training cache pool according to the experience sampling weight, analyze the common features in the training data set to obtain the transferable knowledge representation; use the transferable knowledge representation to initialize the policy model parameters to obtain the initial policy model, and compress the initial policy model into a lightweight reasoning model through the knowledge distillation method; deploy the lightweight reasoning model and start performance evaluation, calculate the performance difference before and after optimization to obtain the optimization improvement, and when the optimization improvement is lower than the preset improvement threshold for multiple consecutive rounds, save the current tuning state to obtain a tuning snapshot and stop the optimization process.

[0010] The transferable knowledge representation is used to initialize the policy model parameters to obtain an initial policy model, and the initial policy model is compressed into a lightweight reasoning model by a knowledge distillation method; the lightweight reasoning model is deployed and performance evaluation is started, and the performance difference before and after optimization is calculated to obtain the optimization improvement. When the optimization improvement is lower than a preset improvement threshold for multiple consecutive rounds, the current tuning state is saved to obtain a tuning snapshot and the optimization process is stopped. The transferable knowledge representation is used to initialize the parameters of the corresponding network layer in the policy model to obtain an initial policy model; the initial policy model is used as a teacher model, the policy distribution output and value function prediction of the teacher model are extracted as soft label information, and a student model with fewer layers and a parameter sharing mechanism is constructed; Constructing a knowledge distillation loss function based on the soft label information and the output of the student model, wherein the knowledge distillation loss function is obtained by linearly combining the strategy distribution KL divergence, the value function prediction mean square error, and the task objective function through weight coefficients; optimizing the parameters of the student model using the knowledge distillation loss function to obtain a lightweight reasoning model, deploying the lightweight reasoning model and starting performance evaluation; During the performance evaluation process, the matching degree between the output action of the lightweight inference model and the optimal action is calculated to obtain the strategy accuracy, and the deviation between the predicted value function and the true value function is calculated to obtain the value function error; the strategy accuracy and the value function error are calculated according to the preset weights to obtain a comprehensive performance index, and the optimization improvement is calculated based on the comprehensive performance index of the current round and the previous round; A sliding time window is set. When the optimization improvement in multiple consecutive rounds within the sliding time window is lower than a preset improvement threshold, the network structure parameters, optimizer status and performance indicators of the lightweight inference model are saved as a tuning snapshot, the optimization process is stopped and the tuning snapshot is output.

[0011] Prioritizing the parameter sampling set according to the parameter exploration priority to obtain a parameter tuning sequence, calculating the estimated mean and the estimated standard deviation of each parameter in the parameter tuning sequence; constructing a confidence boundary adjustment strategy based on the estimated mean and the estimated standard deviation, and determining the parameter optimization direction by adaptively adjusting the learning rate and the confidence coefficient to obtain an optimized parameter solution, including: Prioritizing the parameter sampling set according to the parameter exploration priority to obtain a parameter tuning sequence, calculating the correlation between adjacent parameters in the parameter tuning sequence, and combining the parameters whose correlation is greater than a preset correlation threshold into a parameter group; Establishing a historical data sampling window of the parameter tuning sequence, applying a time decay weight to the data in the historical data sampling window to obtain a weighted historical sample set; in the weighted historical sample set, calculating an estimated mean and an estimated standard deviation of each parameter in the parameter tuning sequence, and calculating the joint distribution characteristics of the parameters in the parameter group, wherein the estimated mean and the estimated standard deviation reflect the distribution characteristics of the parameters; A confidence boundary adjustment strategy is constructed based on the estimated mean and the estimated standard deviation, and the upper and lower boundaries of the parameter confidence interval are determined in combination with the joint distribution characteristics, wherein the confidence boundary adjustment strategy includes a single parameter confidence interval and a parameter group confidence interval; the degree of deviation between the current parameter value and the parameter confidence interval is calculated, and the combined deviation between the current value of the parameter group and the parameter group confidence interval is calculated, and a reference direction for parameter adjustment is determined based on the degree of deviation and the combined deviation; Analyze the performance change trend during the historical parameter optimization process, calculate the performance improvement sequence of the most recent optimizations, adaptively adjust the learning rate according to the fluctuation characteristics of the performance improvement sequence, and use the ratio of the learning rate to the performance improvement amplitude as the confidence coefficient; extract several optimization directions with the most significant performance improvement in the historical parameter optimization path, perform weighted combination of the several optimization directions and the parameter adjustment reference direction, correct the combined direction according to the confidence coefficient to obtain the parameter optimization direction, and multiply the parameter optimization direction by the learning rate to obtain the optimization parameter solution.

[0012] According to a second aspect of the embodiments of the present invention, Provides a database adaptive data stream collection optimization system based on reinforcement learning, including: The first unit is used to build a distributed data acquisition agent network, obtain data flow information of the database in real time, and generate an initial feature set; perform multimodal decomposition on the initial feature set, extract data flow fluctuation spectrum eigenvalues, fine-grained type distribution eigenvalues, multi-scale time series eigenvalues ​​and graph structure association eigenvalues, and form a multimodal feature matrix; based on the multimodal feature matrix, calculate the feature weight coefficient through an improved attention mechanism, adaptively fuse different modal features, and output a fused feature vector; use the fused feature vector, combined with a genetic algorithm to calculate the optimal initial acquisition parameter combination, including a concurrent acquisition threshold, a cache capacity threshold and a batch size threshold, and generate an initial state space vector; The second unit is used to construct a two-layer deep Q learning network structure based on the initial state space vector and the acquisition parameter combination, set the input of the high-level network to the fused feature vector, and output a global acquisition strategy; set the input of the low-level network to the initial state space vector, and output a specific parameter adjustment plan; design a hierarchical reward function, use the feature weight coefficient as a balancing factor of the reward function, and establish a multi-objective evaluation system including acquisition delay rate, data consistency, throughput efficiency and resource utilization; perform a hierarchical training process based on the multi-objective evaluation system, use the global acquisition strategy output by the high-level network as a constraint condition of the low-level network, iteratively update the network parameters through a priority experience replay mechanism, and generate a hierarchical acquisition strategy model; The third unit is used to deploy the hierarchical collection strategy model to the production environment, continuously collect data stream features, calculate the similarity between the real-time features and the fused feature vector, and evaluate the degree of feature drift; when the degree of feature drift exceeds the preset feature threshold, trigger the high-level network to re-plan the collection strategy based on the current fused feature vector, and input the updated global strategy into the low-level network for parameter fine-tuning; monitor the collection performance index in real time, and when the performance index is lower than the preset target threshold, start the progressive tuning mechanism, input the newly collected state transition sequence into the training cache pool, and perform incremental learning based on historical tuning experience to achieve continuous optimization of the hierarchical collection strategy model.

[0013] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0014] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0015] The beneficial effects of this application are as follows: 1. The adaptive optimization of database data stream collection is realized. Through multimodal feature extraction and adaptive fusion, as well as a two-layer Q learning network based on reinforcement learning, the collection parameters can be dynamically adjusted according to the changes in data stream characteristics to improve the collection efficiency.

[0016] 2. Improved the efficiency and stability of data stream collection. The design of the layered reward function and multi-objective evaluation system, combined with the priority experience playback mechanism, can effectively balance the collection delay rate, throughput efficiency and resource utilization, and ensure the stability of collection performance.

[0017] 3. Reduced manual intervention costs and achieved continuous optimization of collection strategies. Feature drift detection and progressive tuning mechanisms can automatically identify changes in data stream features and make corresponding parameter adjustments, reducing the need for manual intervention and achieving continuous optimization of collection strategies to adapt to the ever-changing data stream environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 A schematic diagram of a process flow of a database adaptive data stream acquisition optimization method based on reinforcement learning according to an embodiment of the present invention; Figure 2 It is a structural diagram of a database adaptive data stream acquisition optimization system based on reinforcement learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0019] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0021] Figure 1 FIG. 1 is a flow chart of a method for optimizing database adaptive data flow collection based on reinforcement learning according to an embodiment of the present invention. Figure 1 As shown, the method includes: S11. Construct a distributed data acquisition agent network, obtain data flow information of the database in real time, and generate an initial feature set; perform multimodal decomposition on the initial feature set, extract data flow fluctuation spectrum eigenvalues, fine-grained type distribution eigenvalues, multi-scale time series eigenvalues ​​and graph structure association eigenvalues, and form a multimodal feature matrix; based on the multimodal feature matrix, calculate the feature weight coefficient through an improved attention mechanism, adaptively fuse different modal features, and output a fused feature vector; use the fused feature vector, combined with a genetic algorithm to calculate the optimal initial acquisition parameter combination, including a concurrent acquisition threshold, a cache capacity threshold and a batch size threshold, and generate an initial state space vector; S12. Based on the initial state space vector and the acquisition parameter combination, a two-layer deep Q learning network structure is constructed, the input of the first network is set to the fused feature vector, and a global acquisition strategy is output; the input of the second network is set to the initial state space vector, and a specific parameter adjustment plan is output; a hierarchical reward function is designed, the feature weight coefficient is used as a balancing factor of the reward function, and a multi-objective evaluation system including acquisition delay rate, throughput efficiency and resource utilization is established; a hierarchical training process is performed based on the multi-objective evaluation system, the global acquisition strategy output by the first network is used as a constraint condition for the second network, and the network parameters are iteratively updated through a priority experience playback mechanism to generate a hierarchical acquisition strategy model; S13. Deploy the hierarchical acquisition strategy model to the production environment, continuously collect data stream features, calculate the similarity between the real-time features and the fused feature vector, and evaluate the degree of feature drift; when the degree of feature drift exceeds the preset feature threshold, trigger the first network to re-plan the global acquisition strategy based on the current fused feature vector, and input the updated global acquisition strategy into the second network for parameter fine-tuning; monitor the acquisition performance index in real time, and when the performance index is lower than the preset target threshold, start the progressive tuning mechanism, input the newly collected state transition sequence into the training cache pool, and perform incremental learning based on historical tuning experience to achieve continuous optimization of the hierarchical acquisition strategy model.

[0022] In an optional implementation, based on the multimodal feature matrix, feature weight coefficients are calculated through an improved attention mechanism, and different modal features are adaptively fused to output a fused feature vector; the fused feature vector is used in combination with a genetic algorithm to calculate an optimal initial acquisition parameter combination, including a concurrent acquisition threshold, a cache capacity threshold, and a batch size threshold, and generating an initial state space vector includes: The multimodal feature matrix includes data flow fluctuation spectrum feature values ​​reflecting the dynamic changes of the data flow, fine-grained type distribution feature values ​​reflecting the data distribution law, and multi-scale time series feature values ​​reflecting the time series fluctuation law. A data multi-dimensional representation feature matrix is ​​constructed based on the data flow fluctuation spectrum feature values, the fine-grained type distribution feature values, and the multi-scale time series feature values; Calculate the statistical distribution characteristics of the multidimensional characterization feature matrix of the data, select a corresponding normalization strategy according to the statistical distribution characteristics, use the maximum and minimum value normalization for the data flow fluctuation spectrum feature value to maintain the peak and trough relationship, use softmax normalization for the fine-grained type distribution feature value to highlight the main type, and use z-score normalization for the multi-scale time series feature value to eliminate the dimension effect, and generate a standardized feature matrix; Analyze the information entropy distribution of each modal feature in the standardized feature matrix, use the information entropy distribution as a modal importance indicator, construct a modal adaptive factor based on the information entropy distribution, dynamically adjust the modal adaptive factor with the information entropy distribution to adaptively capture information volume features, and integrate the modal adaptive factor into the attention mechanism to calculate the dynamic attention weight; A two-layer feature fusion network is constructed using the dynamic attention weights. In the first layer, a self-attention operation is performed within the modality to extract the internal correlation of the modality. In the second layer, a cross-attention operation between modalities is performed to model the complementary relationship of the modalities. The outputs of the two layers of attention are fused through a residual structure and a normalization layer to obtain a fused feature vector containing multi-modal complementary information. Component analysis is performed on the fused feature vector, and the concurrent acquisition threshold boundary is determined by using the period and peak characteristics of the data stream fluctuation spectrum feature value, the cache capacity threshold boundary is determined by using the entropy value and skewness characteristics of the fine-grained type distribution feature value, and the batch processing threshold boundary is determined by using the autocorrelation and stationary characteristics of the multi-scale time series feature value, and the three threshold boundaries constitute a parameter search space; Extracting feature importance based on the principal component analysis result of the fused feature vector, mapping the feature importance into a weight coefficient, constructing a multi-objective fitness function including system throughput, processing delay and resource utilization, and dynamically weighting the three threshold boundaries according to the weight coefficient; Initializing a genetic algorithm population in the parameter search space, using the multi-objective fitness function to evaluate the quality of individuals in the population, adaptively adjusting the crossover probability based on the population diversity level to balance local and global searches, adaptively adjusting the mutation probability based on the optimization convergence degree to prevent premature convergence, and obtaining the optimal parameter combination through iterative optimization; The optimal parameter combination and the fused feature vector are deeply embedded to generate a state space vector of coupling parameter information and feature representation.

[0023] First, construct a multi-dimensional characterization feature matrix of the data. This matrix contains three key features: data flow fluctuation spectrum feature values, which are used to reflect the frequency and amplitude information of the dynamic changes of the data flow; fine-grained type distribution feature values, which are used to reflect the distribution of data types; multi-scale time series feature values, which are used to reflect the changing patterns of data at different time scales. For example, for a network traffic data flow, the number of packets per second can be extracted as the data flow fluctuation spectrum feature value, the proportion of packets of different protocol types as the fine-grained type distribution feature value, and the change trend of the number of packets per minute and hour as the multi-scale time series feature value.

[0024] Next, the constructed multi-dimensional characterization feature matrix is ​​standardized. Different normalization strategies are used for different feature types: the maximum and minimum values ​​are normalized for the data flow fluctuation spectrum feature values, for example, the number of packets per second is scaled to between 0 and 1 to maintain the relative relationship between peaks and troughs; softmax normalization is used for fine-grained type distribution feature values, for example, the proportion of various protocol types is converted to probability distribution to highlight the main data types; z-score normalization is used for multi-scale time series feature values, for example, the number of packets per minute and hour is converted to a standard normal distribution with a mean of 0 and a standard deviation of 1 to eliminate the dimensional effects of different time scales. Assuming that the data flow fluctuation spectrum feature value is [100, 200, 150], the maximum value is 200, and the minimum value is 100, the normalized result is [0, 1, 0.5].

[0025] Then, the information entropy distribution of each modal feature in the standardized feature matrix is ​​analyzed and used as a modal importance indicator. The higher the information entropy, the richer the information contained in the modal feature. A modal adaptive factor is constructed based on the information entropy distribution, which is dynamically adjusted with the information entropy distribution to adaptively capture the information volume characteristics. For example, if the information entropy of the eigenvalue of the data stream fluctuation spectrum is high, the corresponding modal adaptive factor will also be high. The modal adaptive factor is integrated into the attention mechanism to calculate the dynamic attention weight.

[0026] A two-layer feature fusion network is constructed using dynamic attention weights. The first layer performs intra-modal self-attention operations to extract correlations within the modality, such as analyzing the relationship between different frequency components within the eigenvalues ​​of the data stream fluctuation spectrum. The second layer performs inter-modal cross-attention operations to model the complementary relationship between modalities, such as analyzing the mutual influence between the eigenvalues ​​of the data stream fluctuation spectrum and the eigenvalues ​​of the fine-grained type distribution. The outputs of the two layers of attention are fused through a residual structure and a normalization layer to obtain a fused feature vector containing multi-modal complementary information.

[0027] Perform component analysis on the fused feature vector to determine the search space for acquisition parameters. Use the period and peak characteristics of the data stream fluctuation spectrum feature values ​​to determine the concurrent acquisition threshold boundary, for example, determine the frequency range of concurrent acquisition based on the time interval when the peak number of data packets occurs. Use the entropy and skewness characteristics of the fine-grained type distribution feature values ​​to determine the cache capacity threshold boundary, for example, determine the size range of the cache capacity based on the distribution of different data types. Use the autocorrelation and stationary characteristics of the multi-scale time series feature values ​​to determine the batch processing threshold boundary, for example, determine the size range of the batch processing based on the change pattern of the data stream at different time scales. These three threshold boundaries together constitute the parameter search space.

[0028] The feature importance is extracted based on the principal component analysis results of the fused feature vector and mapped to a weight coefficient. A multi-objective fitness function is constructed that includes system throughput, processing delay, and resource utilization. The function dynamically weights the three threshold boundaries according to the weight coefficients. For example, if the importance of system throughput is high, it will be given a higher weight in the fitness function.

[0029] Initialize the genetic algorithm population in the parameter search space, and use the multi-objective fitness function to evaluate the quality of the individuals in the population. Adaptively adjust the crossover probability based on the population diversity level to balance local and global search. Adaptively adjust the mutation probability based on the degree of optimization convergence to prevent premature convergence. Through iterative optimization, the optimal parameter combination is finally obtained, such as the concurrent acquisition threshold is set to 1000, the cache capacity threshold is set to 10MB, and the batch size threshold is set to 1024.

[0030] Finally, the optimal parameter combination and fusion feature vector are deeply embedded to generate a state space vector of coupling parameter information and feature representation. This vector can be used for subsequent data acquisition control and performance optimization.

[0031] The solution of this application can: Improve data collection efficiency: By dynamically adjusting the collection parameters, data can be collected adaptively according to the real-time characteristics of the data stream, avoiding the collection of redundant data, thereby improving data collection efficiency. Improve system performance: By optimizing the collection parameters, the system throughput, processing delay and resource utilization can be effectively balanced, thereby improving the overall system performance. Enhance system adaptability: This method can dynamically adjust the collection parameters according to the characteristics of different data streams, enhancing the system's adaptability to different data streams.

[0032] In an optional implementation, based on the initial state space vector and the acquisition parameter combination, a two-layer deep Q learning network structure is constructed, the input of the first network is set to the fused feature vector, and a global acquisition strategy is output; the input of the second network is set to the initial state space vector, and a specific parameter adjustment scheme is output, including: Based on the initial state space vector and the acquisition parameter combination, a double-layer deep Q learning network structure is constructed, the fused feature vector is set as the input of the first network, and the initial state space vector is set as the input of the second network, and the first network and the second network adopt a double network structure in which the target network and the evaluation network are separated; A dual-stream feature extraction channel is constructed in the first network, the first channel extracts static features in the fused feature vector through a multi-layer fully connected network to obtain a static feature representation, and the second channel extracts dynamic features in the fused feature vector through a bidirectional LSTM network to obtain a dynamic feature representation, and the static feature representation and the dynamic feature representation are adaptively fused through a multi-head attention mechanism to output a global acquisition strategy, wherein the global acquisition strategy includes an adjustment direction, an adjustment step and an adjustment priority, wherein the adjustment direction is used to indicate a parameter increase or decrease, the adjustment step is used to control a parameter change amplitude, and the adjustment priority is used to determine a multi-parameter adjustment order; In the second network, based on the initial state space vector, the Dueling architecture is used to decompose the Q value into a state value function and an advantage function. The state value function evaluates the parameter configuration performance through a three-layer feedforward neural network to obtain a performance evaluation value. The advantage function evaluates and adjusts the action advantage through a deep residual network to obtain an action advantage value. The performance evaluation value and the action advantage value are weightedly combined to output a specific parameter adjustment plan.

[0033] First, obtain the initial state space vector and acquisition parameter combination of the system. The initial state space vector is used to describe the initial state of the system, and the acquisition parameter combination contains the parameters to be adjusted and their initial values. For example, the initial state space vector of a control system may include temperature, pressure, flow rate, etc., and the acquisition parameter combination may include the proportional coefficient, integral coefficient, and differential coefficient of the PID controller. Assume that the initial temperature is 25°C, the pressure is 1 standard atmosphere, the flow rate is 10m / s, the proportional coefficient is 1, the integral coefficient is 0.1, and the differential coefficient is 0.01.

[0034] Then, a fused feature vector is constructed. The fused feature vector integrates the static and dynamic features of the system. Static features describe the inherent properties of the system, such as device model, parameter range, etc. Dynamic features describe the real-time status of the system, such as current performance indicators, historical adjustment records, etc. Assume that the device model is type A, the parameter range is 0 to 10, the current performance indicator is 80%, and the past 5 parameter adjustment records are increase by 0.1, decrease by 0.05, increase by 0.2, unchanged, and decrease by 0.1. Integrate this information into a fused feature vector.

[0035] Next, a two-layer deep Q-learning network structure is constructed. The input of the first network is the fused feature vector, and the output is the global acquisition strategy, including the adjustment direction (increase or decrease), adjustment step size, and adjustment priority. The input of the second network is the initial state space vector, and the output is the specific parameter adjustment plan. Both networks adopt a dual network structure with the target network and the evaluation network separated to improve training stability.

[0036] In the first network, a dual-stream feature extraction channel is constructed. The first channel uses a multi-layer fully connected network to extract static features in the fused feature vector to obtain a static feature representation. The second channel uses a bidirectional LSTM network to extract dynamic features in the fused feature vector to obtain a dynamic feature representation. Then, a multi-head attention mechanism is used to adaptively fuse the static feature representation and the dynamic feature representation, and finally output a global acquisition strategy. For example, the global acquisition strategy that may be output by the first network is: first adjust the proportional coefficient by increasing it by 0.2; second adjust the integral coefficient by decreasing it by 0.01; and finally adjust the differential coefficient by increasing it by 0.005.

[0037] In the second network, based on the initial state space vector, the Dueling architecture is used to decompose the Q value into a state value function and an advantage function. The state value function uses a three-layer feedforward neural network to evaluate the parameter configuration performance and obtain a performance evaluation value. The advantage function uses a deep residual network to evaluate the action advantage and obtain an action advantage value. Then, the performance evaluation value and the action advantage value are weighted and combined to output a specific parameter adjustment plan. For example, the specific parameter adjustment plan that the second network may output is: adjust the proportional coefficient to 1.2, the integral coefficient to 0.09, and the differential coefficient to 0.015.

[0038] Finally, according to the global acquisition strategy and specific parameter adjustment plan, adjust the system parameters and observe the changes in system performance. Feedback the new state space vector and adjustment results to the network for the next round of learning and adjustment. Repeat this process until the system performance reaches the preset target or the maximum number of iterations is reached.

[0039] The solution of this application can: Improve parameter adjustment efficiency: By jointly optimizing the global acquisition strategy and specific parameter adjustment scheme, invalid adjustment attempts can be reduced and the speed of parameter optimization can be accelerated. Improve parameter adjustment accuracy: The application of the two-layer network structure and deep learning algorithm can more accurately capture the complex relationship between system status and parameters, thereby achieving more refined parameter adjustment. Enhance the robustness of parameter adjustment: The fusion of static and dynamic features, and the combination of the target network and the evaluation network can improve the stability and generalization ability of the algorithm, so that it can maintain good performance in different environments and working conditions.

[0040] In an optional implementation, the hierarchical acquisition strategy model is deployed to a production environment, data stream features are continuously acquired, similarity is calculated between real-time feature vectors and fused feature vectors, and the degree of feature drift is evaluated; when the degree of feature drift exceeds a preset feature threshold, the first network is triggered to re-plan the acquisition strategy based on the current fused feature vector, and the updated global strategy is input into the second network for parameter fine-tuning, including: The hierarchical collection strategy model is deployed to the production environment to continuously collect data stream features to obtain real-time feature vectors, and the directional deviation degree of the real-time feature vector and the fused feature vector on the continuous feature is calculated by cosine similarity to obtain continuous feature similarity, and the distribution change amplitude of the real-time feature vector and the fused feature vector on the discrete feature is calculated by KL divergence to obtain discrete feature similarity; The continuous feature similarity and the discrete feature similarity are normalized and weighted to obtain a feature drift comprehensive index, and a multi-scale feature drift detection mechanism is constructed based on the feature drift comprehensive index; within a first time window, the feature drift comprehensive index is smoothed by an exponentially weighted moving average method to obtain a feature trend, within a second time window, a mutation point detection algorithm is used to identify the jump position of the feature drift comprehensive index to obtain a feature change point, and within a third time window, a trend analysis method is used to capture the gradual change process of the feature drift comprehensive index to obtain a feature trend; Input the feature trend, the feature change point and the feature trend into a decision fusion module, compare the feature drift index at each time scale with the corresponding preset feature threshold, and generate a feature drift degree assessment result; When the feature drift degree evaluation result indicates that the feature drift index at any time scale exceeds the corresponding preset feature threshold, recalculating the feature importance weight matrix based on the current fused feature vector, and inputting the feature importance weight matrix into the first network; The first network replans the acquisition strategy according to the feature importance weight matrix, updates the feature extraction scheme to obtain the feature weight update strategy, adjusts the parameter change direction and change step size based on the feature weight update strategy to obtain the global strategy update scheme, and inputs the global strategy update scheme into the second network for parameter fine-tuning.

[0041] First, the trained hierarchical collection strategy model is deployed to the production environment. The model consists of two sub-networks: the first network is responsible for planning the collection strategy based on feature importance, and the second network is responsible for feature extraction and parameter fine-tuning. The initial collection strategy and feature extraction scheme are obtained by offline training.

[0042] Next, the model starts to continuously collect data stream features. Assume that the data stream contains features such as user age, number of purchases, and product category. The collected data is preprocessed and feature engineered to be converted into a real-time feature vector. For example, the user feature vector collected at a certain moment is: [25, 3, 'electronic products'].

[0043] At the same time, the system maintains a fused feature vector, which represents the overall characteristics of the data distribution over a period of time. The fused feature vector is updated regularly, for example, once an hour, to incorporate the newly collected data features. The fused feature vector can be updated using methods such as sliding window averaging or exponentially weighted moving average. Assume that the current fused feature vector is: [28, 2, 'clothing'].

[0044] In order to evaluate the degree of feature drift, it is necessary to calculate the similarity between the real-time feature vector and the fused feature vector. For continuous features, such as user age and number of purchases, cosine similarity is used to measure the degree of directional deviation. For example, the cosine similarity calculation result of user age is 0.95. For discrete features, such as product categories, KL divergence is used to measure the magnitude of distribution change. For example, the KL divergence calculation result of product categories is 0.2.

[0045] The continuous feature similarity and discrete feature similarity are normalized and weighted to obtain the feature drift comprehensive index. For example, assuming that the continuous feature weight is 0.7 and the discrete feature weight is 0.3, the feature drift comprehensive index is 0.7 *0.95 + 0.3 * 0.2 = 0.725.

[0046] In order to detect feature drift more comprehensively, a multi-scale feature drift detection mechanism is constructed. In the first shorter time window, such as the past hour, the exponentially weighted moving average method is used to smooth the feature drift comprehensive index to obtain the feature trend in order to filter short-term fluctuations. In the second medium time window, such as the past day, the mutation point detection algorithm is used to identify the jump position of the feature drift comprehensive index and obtain the feature change point in order to capture sudden drift. In the third longer time window, such as the past week, the trend analysis method is used to capture the gradual change process of the feature drift comprehensive index and obtain the feature trend in order to discover long-term slow drift.

[0047] The feature trend, feature change point, and feature trend are input into the decision fusion module, and the feature drift index at each time scale is compared with the corresponding preset feature threshold. For example, assume that the short-term feature trend threshold is 0.1, the medium-term feature change point threshold is 0.2, and the long-term feature trend threshold is 0.3. If the feature drift index at any time scale exceeds the corresponding preset threshold, it is considered that significant feature drift has occurred.

[0048] When the feature drift evaluation result shows that the feature drift index at any time scale exceeds the corresponding preset feature threshold, the system will recalculate the feature importance weight matrix based on the current fused feature vector. For example, if the drift of the user's age is large, its importance weight will be increased accordingly. Then, the feature importance weight matrix is ​​input into the first network.

[0049] The first network replans the collection strategy according to the feature importance weight matrix and updates the feature extraction scheme to obtain the feature weight update strategy. For example, if the importance weight of the user's age increases, the collection strategy may be more inclined to collect age-related data, and the feature extraction scheme may pay more attention to the processing of age features. Based on the feature weight update strategy, the change direction and change step size of the model parameters are adjusted to obtain the global strategy update scheme.

[0050] Finally, the global policy update scheme is input into the second network for parameter fine-tuning to adapt to the new data distribution.

[0051] The solution of this application can: Improve data collection efficiency: By dynamically adjusting the collection strategy according to the importance of features, it is possible to avoid collecting redundant or useless data, thereby improving data collection efficiency and reducing storage and computing costs. Enhance model robustness: Through real-time monitoring and adaptive adjustment, the model can respond to changes in data distribution in a timely manner, maintain the predictive performance and stability of the model, and enhance the robustness of the model. Realize automated operation and maintenance: This method realizes the automation of feature drift detection and model parameter adjustment, reduces the need for manual intervention, reduces operation and maintenance costs, and improves system efficiency.

[0052] In an optional implementation, real-time monitoring of the acquisition performance index, when the performance index is lower than a preset target threshold, starting a progressive tuning mechanism, inputting the newly acquired state transition sequence into a training cache pool, and performing incremental learning based on historical tuning experience to achieve continuous optimization of the hierarchical acquisition strategy model includes: Construct a multi-dimensional performance monitoring model, collect data throughput, data delay time, and resource utilization in real time to obtain efficiency index data, collect data integrity, data accuracy, and data timeliness to obtain quality index data, and collect computing overhead, storage overhead, and bandwidth overhead to obtain cost index data; calculate the respective index weights for the efficiency index data, the quality index data, and the cost index data, and perform a weighted combination of the index weights and the corresponding index data to obtain a comprehensive performance score; compare the comprehensive performance score with a preset target threshold, and trigger a parameter optimization process when the comprehensive performance score is lower than the preset target threshold; In the parameter optimization process, the parameter space is randomly sampled based on the Thompson sampling strategy to obtain a parameter sampling set, and the uncertainty of each parameter interval in the parameter sampling set is calculated to obtain the parameter exploration priority; the parameter sampling set is prioritized according to the parameter exploration priority to obtain a parameter tuning sequence, and the estimated mean and estimated standard deviation of each parameter in the parameter tuning sequence are calculated; the state transition data generated during the optimization process is collected, and the state transition data is stored in the first cache layer, the second cache layer and the third cache layer respectively according to the data timestamp to construct a hierarchical training cache pool; Calculate the timeliness weight and value weight of the experience data in the hierarchical training cache pool, multiply the timeliness weight and the value weight to obtain the experience sampling weight; sample the training data set from the hierarchical training cache pool according to the experience sampling weight, analyze the common features in the training data set to obtain the transferable knowledge representation; use the transferable knowledge representation to initialize the policy model parameters to obtain the initial policy model, and compress the initial policy model into a lightweight reasoning model through the knowledge distillation method; deploy the lightweight reasoning model and start performance evaluation, calculate the performance difference before and after optimization to obtain the optimization improvement, and when the optimization improvement is lower than the preset improvement threshold for multiple consecutive rounds, save the current tuning state to obtain a tuning snapshot and stop the optimization process.

[0053] First, build a multi-dimensional performance monitoring model. This model collects three-dimensional indicator data in real time: efficiency indicator data includes data throughput, data delay time, and resource utilization; quality indicator data includes data integrity, data accuracy, and data timeliness; cost indicator data includes computing overhead, storage overhead, and bandwidth overhead. For example, set the collection period to 1 minute, and record the data of these nine indicators once a minute. Assume that the data throughput collected in a certain minute is 1000 pieces / second, the data delay time is 0.5 seconds, the resource utilization rate is 60%, the data integrity is 99%, the data accuracy is 98%, the data timeliness is 5 minutes, the computing overhead is 10 CPU units, the storage overhead is 1GB, and the bandwidth overhead is 10Mbps.

[0054] Next, calculate the comprehensive performance score. Calculate the indicator weights of the efficiency indicator, quality indicator, and cost indicator respectively. The weights can be determined based on expert experience, hierarchical analysis method, and other methods. For example, set the efficiency indicator weight to 0.4, the quality indicator weight to 0.5, and the cost indicator weight to 0.1. The indicator weights are weighted and combined with the corresponding indicator data to obtain a comprehensive performance score. Taking the above case as an example, the comprehensive performance score = 0.4 * (throughput standardized value + delay time standardized value + resource utilization standardized value) + 0.5 * (completeness standardized value + accuracy standardized value + timeliness standardized value) + 0.1 * (computing overhead standardized value + storage overhead standardized value + bandwidth overhead standardized value). Among them, each indicator needs to be standardized, such as min-max standardization, to map the indicator value to between 0 and 1.

[0055] Then, the comprehensive performance score is compared with the preset target threshold. If the comprehensive performance score is lower than the preset target threshold, the parameter optimization process is triggered. For example, if the preset target threshold is 0.8 and the calculated comprehensive performance score is 0.7, the parameter optimization process is triggered.

[0056] In the parameter optimization process, the parameter space is first randomly sampled based on the Thompson sampling strategy to obtain a parameter sampling set. For example, assuming that the parameters to be adjusted are the acquisition frequency and the cache size, random sampling is performed within the preset parameter range to obtain multiple combinations of acquisition frequencies and cache sizes. Then, the uncertainty of each parameter interval in the parameter sampling set is calculated to obtain the parameter exploration priority. Uncertainty can be expressed by the variance of the parameter interval or the width of the confidence interval. The parameter sampling set is prioritized according to the parameter exploration priority to obtain a parameter tuning sequence. For example, parameter combinations with high uncertainty are ranked first and tested first. Next, the estimated mean and estimated standard deviation of each parameter in the parameter tuning sequence are calculated. The estimated mean and estimated standard deviation can be calculated based on historical tuning data or the Bayesian method.

[0057] Collect state transition data generated during the optimization process, such as acquisition parameters, performance indicators, and environmental information. Store the state transition data in the first cache layer, the second cache layer, and the third cache layer according to the data timestamp to build a hierarchical training cache pool. For example, store the data of the last hour in the first cache layer, the data of the last day in the second cache layer, and the data of the last month in the third cache layer.

[0058] Calculate the timeliness weight and value weight of the experience data in the hierarchical training cache pool. The timeliness weight is calculated based on the timestamp of the data. The closer the time is, the higher the weight is. The value weight is calculated based on the performance index corresponding to the data. The better the performance index is, the higher the weight is. For example, the timeliness weight can be calculated using an exponential decay function, and the value weight can be calculated using the standardized value of the performance index. Multiply the timeliness weight and the value weight to get the experience sampling weight.

[0059] The training data set is sampled from the hierarchical training cache pool according to the empirical sampling weights. For example, a weighted random sampling method can be used to sample data from different cache layers according to the empirical sampling weights. The common features in the training data set are analyzed to obtain transferable knowledge representations. For example, data distribution features, parameter change rules, etc. can be extracted as transferable knowledge representations.

[0060] The transferable knowledge representation is used to initialize the policy model parameters to obtain the initial policy model. For example, the transfer learning method can be used to integrate the transferable knowledge representation into the initialization process of the policy model. The initial policy model is compressed into a lightweight reasoning model through the knowledge distillation method. For example, the student-teacher network structure can be used for knowledge distillation to transfer the knowledge of a large and complex model to a small and lightweight model.

[0061] Deploy a lightweight inference model and start performance evaluation. Calculate the performance difference before and after optimization to get the optimization improvement. For example, compare the difference in comprehensive performance scores before and after optimization. When the improvement of multiple consecutive rounds of optimization is lower than the preset improvement threshold, save the current tuning state, get a tuning snapshot, and stop the optimization process. For example, set the preset improvement threshold to 0.05. If the improvement of three consecutive rounds of optimization is less than 0.05, stop the optimization.

[0062] The solution of this application can: Improve collection efficiency and quality: Through real-time monitoring and progressive tuning, the collection strategy can be adjusted dynamically to make the data collection process more efficient while ensuring data quality. Reduce resource consumption: Through parameter optimization and knowledge distillation, the computing overhead, storage overhead, and bandwidth overhead can be reduced, and the resource consumption of data collection can be reduced. Enhance system adaptability: Through continuous learning and knowledge transfer, the system can automatically adapt to different data environments and application scenarios, improving the robustness and generalization ability of the system.

[0063] In an optional implementation, the transferable knowledge representation is used to initialize the policy model parameters to obtain an initial policy model, and the initial policy model is compressed into a lightweight reasoning model by a knowledge distillation method; the lightweight reasoning model is deployed and performance evaluation is started, and the performance difference before and after optimization is calculated to obtain the optimization improvement. When the optimization improvement is lower than a preset improvement threshold for multiple consecutive rounds, the current tuning state is saved to obtain a tuning snapshot and the optimization process is stopped, including: The transferable knowledge representation is used to initialize the parameters of the corresponding network layer in the policy model to obtain an initial policy model; the initial policy model is used as a teacher model, the policy distribution output and value function prediction of the teacher model are extracted as soft label information, and a student model with fewer layers and a parameter sharing mechanism is constructed; Constructing a knowledge distillation loss function based on the soft label information and the output of the student model, wherein the knowledge distillation loss function is obtained by linearly combining the strategy distribution KL divergence, the value function prediction mean square error, and the task objective function through weight coefficients; optimizing the parameters of the student model using the knowledge distillation loss function to obtain a lightweight reasoning model, deploying the lightweight reasoning model and starting performance evaluation; During the performance evaluation process, the matching degree between the output action of the lightweight inference model and the optimal action is calculated to obtain the strategy accuracy, and the deviation between the predicted value function and the true value function is calculated to obtain the value function error; the strategy accuracy and the value function error are calculated according to the preset weights to obtain a comprehensive performance index, and the optimization improvement is calculated based on the comprehensive performance index of the current round and the previous round; A sliding time window is set. When the optimization improvement in multiple consecutive rounds within the sliding time window is lower than a preset improvement threshold, the network structure parameters, optimizer status and performance indicators of the lightweight inference model are saved as a tuning snapshot, the optimization process is stopped and the tuning snapshot is output.

[0064] First, obtain transferable knowledge representations. Assume that in a simulated robot navigation task, a large policy model has been trained to successfully navigate in a complex environment. The network layer parameters extracted from this large model (such as the weights and biases of the convolutional layer) can be used as transferable knowledge representations. Specifically, the parameters of the convolutional layer responsible for feature extraction in the model can be extracted, which encode the general understanding of the environment, such as the recognition of walls, obstacles, and targets. These extracted parameters will be used to initialize the corresponding network layers of the student model.

[0065] The policy model is then initialized using the transferable knowledge representation. The extracted convolutional layer parameters are directly loaded into the corresponding layers of the student model (a smaller convolutional neural network). Assuming that the convolutional layer structure of the student model is similar to the feature extraction layer structure of the large model, the parameter values ​​can be directly copied. This allows the student model to have a certain degree of environmental perception at the beginning of training.

[0066] Next, build a lightweight inference model. Use the initialized policy model as the teacher model, and extract its policy distribution output and value function prediction as soft label information. For example, in a robot navigation task, the policy distribution output can be expressed as the probability distribution of selecting different actions (forward, backward, left turn, right turn) at each time step, and the value function prediction can be expressed as the expected cumulative reward of the current state. Build a student model with fewer layers and a parameter sharing mechanism. For example, the student model can use smaller convolution kernels or fewer convolution layers, and share some convolution kernels between different layers to reduce the number of parameters.

[0067] After that, knowledge distillation training is performed. A knowledge distillation loss function is constructed based on the soft label information and the output of the student model. The loss function consists of three items: the difference in policy distribution, the difference in value function prediction, and the task objective function (for example, the reward for reaching the target point in the navigation task), and is obtained by linear combination of weight coefficients. For example, the difference in policy distribution, the difference in value function prediction, and the task objective function can be given weights of 0.5, 0.3, and 0.2, respectively. The loss function is used to optimize the parameters of the student model to obtain a lightweight inference model.

[0068] Then, deploy the lightweight inference model and start the performance evaluation. During the performance evaluation, calculate the degree of match between the output action of the lightweight inference model and the optimal action to obtain the policy accuracy, for example, in 100 test scenarios, the proportion of times the action selected by the model is consistent with the optimal action. At the same time, calculate the deviation between the predicted value function and the true value function to obtain the value function error, for example, the average difference between the predicted value and the true value. Calculate the policy accuracy and value function error according to the preset weights (for example, 0.7 and 0.3) to obtain the comprehensive performance index.

[0069] Finally, monitor the optimization improvement and save the tuning snapshot. Calculate the optimization improvement based on the comprehensive performance indicators of the current round and the previous round, for example, (current round comprehensive performance indicator - previous round comprehensive performance indicator) / previous round comprehensive performance indicator. Set a sliding time window (for example, 5 rounds). When the optimization improvement of multiple consecutive rounds within the sliding time window is lower than the preset improvement threshold (for example, 0.01), save the network structure parameters, optimizer status and performance indicators of the lightweight inference model as a tuning snapshot, stop the optimization process and output the tuning snapshot.

[0070] The solution of this application can: Reduce model complexity: Through knowledge distillation, large policy models are compressed into lightweight inference models, which significantly reduces the number of model parameters and computation, allowing the model to be deployed and run on resource-constrained devices. Improve inference speed: Lightweight models have faster inference speeds and can meet the needs of real-time applications. For example, in areas such as robot control and autonomous driving, fast response is crucial. Maintain model performance: The knowledge distillation process uses the soft label information of the teacher model to guide the learning of the student model, allowing the student model to inherit the knowledge and experience of the teacher model, thereby maintaining a high performance level while reducing the model size, for example, maintaining a high navigation success rate and a low path length in navigation tasks.

[0071] In an optional implementation, the parameter sampling set is prioritized according to the parameter exploration priority to obtain a parameter tuning sequence, and an estimated mean and an estimated standard deviation of each parameter in the parameter tuning sequence are calculated; a confidence boundary adjustment strategy is constructed based on the estimated mean and the estimated standard deviation, and an optimized parameter scheme is obtained by determining a parameter optimization direction by adaptively adjusting a learning rate and a confidence coefficient. The scheme includes: Prioritizing the parameter sampling set according to the parameter exploration priority to obtain a parameter tuning sequence, calculating the correlation between adjacent parameters in the parameter tuning sequence, and combining the parameters whose correlation is greater than a preset correlation threshold into a parameter group; Establishing a historical data sampling window of the parameter tuning sequence, applying a time decay weight to the data in the historical data sampling window to obtain a weighted historical sample set; in the weighted historical sample set, calculating an estimated mean and an estimated standard deviation of each parameter in the parameter tuning sequence, and calculating the joint distribution characteristics of the parameters in the parameter group, wherein the estimated mean and the estimated standard deviation reflect the distribution characteristics of the parameters; A confidence boundary adjustment strategy is constructed based on the estimated mean and the estimated standard deviation, and the upper and lower boundaries of the parameter confidence interval are determined in combination with the joint distribution characteristics, wherein the confidence boundary adjustment strategy includes a single parameter confidence interval and a parameter group confidence interval; the degree of deviation between the current parameter value and the parameter confidence interval is calculated, and the combined deviation between the current value of the parameter group and the parameter group confidence interval is calculated, and a reference direction for parameter adjustment is determined based on the degree of deviation and the combined deviation; Analyze the performance change trend during the historical parameter optimization process, calculate the performance improvement sequence of the most recent optimizations, adaptively adjust the learning rate according to the fluctuation characteristics of the performance improvement sequence, and use the ratio of the learning rate to the performance improvement amplitude as the confidence coefficient; extract several optimization directions with the most significant performance improvement in the historical parameter optimization path, perform weighted combination of the several optimization directions and the parameter adjustment reference direction, correct the combined direction according to the confidence coefficient to obtain the parameter optimization direction, and multiply the parameter optimization direction by the learning rate to obtain the optimization parameter solution.

[0072] First, determine the priority of parameter exploration, such as sorting parameters according to their impact on model performance or parameter sensitivity. Assuming there are five parameters A, B, C, D, and E, based on experience, their priority is A > B > C > D > E, thus obtaining the parameter tuning sequence {A, B, C, D, E}.

[0073] Next, calculate the correlation between adjacent parameters in the parameter tuning sequence. For example, the correlation between parameters A and B can be measured by calculating the Pearson correlation coefficient between their historical adjustment data. Assume that the correlation between A and B is 0.8, the correlation between B and C is 0.2, the correlation between C and D is 0.9, and the correlation between D and E is 0.1. Set the preset correlation threshold to 0.7, then combine the parameters with a correlation greater than 0.7 into a parameter group, and obtain the parameter groups {A, B} and {C, D}.

[0074] Then, establish the historical data sampling window of the parameter tuning sequence. For example, the model training data of the last 100 times can be taken as the historical data sampling window. Apply time decay weights to the data in the window, such as using exponential decay, so that the most recent data has a higher weight. Assume that the values ​​of parameter A in the last 10 times are 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, respectively. After time decay weighting, a weighted historical sample set is obtained.

[0075] In the weighted historical sample set, calculate the estimated mean and estimated standard deviation of parameter A. For example, assuming the weighted sample set is {0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 4.5, 5}, the estimated mean is 3 and the estimated standard deviation is 1.58. At the same time, calculate the joint distribution characteristics of the parameters in the parameter group {A, B}, for example, the joint probability distribution of A and B can be calculated. Assuming that the joint probability distribution of A and B is P(A=x, B=y), it can be estimated by counting the number of combinations of A and B values ​​in the historical data.

[0076] Construct a confidence bound adjustment strategy based on the estimated mean and estimated standard deviation. For parameter A, assuming the confidence level is 95%, the upper and lower bounds of the confidence interval can be calculated based on the normal distribution as the mean plus or minus two standard deviations, that is, 3 ± 2 *1.58 = [-0.16, 6.16]. For the parameter group {A, B}, the upper and lower bounds of the confidence interval can be determined based on the joint distribution characteristics. Assume that the joint confidence interval of A and B is [A_lower, A_upper] x [B_lower, B_upper].

[0077] Calculate the deviation between the current parameter value and the confidence interval. Assuming that the current value of parameter A is 4, the deviation between it and the confidence interval is |4 - 3| / (6.16 - (-0.16)) = 0.16. At the same time, calculate the combined deviation between the current value of the parameter group {A, B} and the confidence interval. Assuming that the current value of parameter B is 2 and the confidence interval of the parameter group {A, B} is [2,6] x [1, 3], the combined deviation can be defined as the weighted sum of the deviations of A and B from their respective confidence intervals.

[0078] Analyze the performance change trend during the historical parameter optimization process. For example, calculate the performance improvement sequence of the last 10 optimizations, assuming that the performance improvement sequence is {0.1, 0.2, 0.3, 0.2, 0.1, 0, -0.1, 0.1, 0.2, 0.3}. Adaptively adjust the learning rate according to the fluctuation characteristics of the performance improvement sequence. For example, if the performance improvement sequence fluctuates greatly, reduce the learning rate; if the performance improvement sequence is relatively stable, increase the learning rate. Assuming that the current learning rate is 0.1 and the performance improvement is 0.3, the confidence coefficient is 0.1 / 0.3 = 0.33.

[0079] Extract several optimization directions with the most significant performance improvement in the historical parameter optimization path. For example, you can extract the top three optimization directions with the greatest performance improvement. Perform a weighted combination of these three optimization directions and the parameter adjustment benchmark direction. Modify the combined direction according to the confidence coefficient to obtain the parameter optimization direction. Multiply the parameter optimization direction by the learning rate to obtain the optimization parameter solution. For example, assuming the optimization direction is [0.1, 0.2, 0.3, 0.4, 0.5] and the learning rate is 0.1, the optimization parameter solution is [0.01, 0.02, 0.03, 0.04, 0.05].

[0080] The solution of this application can: Improve parameter optimization efficiency: Through priority sorting and correlation analysis, you can focus on optimizing parameters and parameter groups that have a greater impact on model performance, thereby improving parameter optimization efficiency. Enhance parameter optimization stability: The confidence boundary adjustment strategy can effectively avoid excessive parameter adjustment and improve parameter optimization stability. Achieve parameter optimization adaptation: The adaptive learning rate and confidence coefficient can dynamically adjust the parameter optimization strategy according to the trend of model performance changes to achieve parameter optimization adaptation.

[0081] Figure 2 FIG. 1 is a schematic diagram of the structure of a database adaptive data stream acquisition optimization system based on reinforcement learning according to an embodiment of the present invention. Figure 2 As shown, the system comprises: The first unit is used to build a distributed data acquisition agent network, obtain data flow information of the database in real time, and generate an initial feature set; perform multimodal decomposition on the initial feature set, extract data flow fluctuation spectrum eigenvalues, fine-grained type distribution eigenvalues, multi-scale time series eigenvalues ​​and graph structure association eigenvalues, and form a multimodal feature matrix; based on the multimodal feature matrix, calculate the feature weight coefficient through an improved attention mechanism, adaptively fuse different modal features, and output a fused feature vector; use the fused feature vector, combined with a genetic algorithm to calculate the optimal initial acquisition parameter combination, including a concurrent acquisition threshold, a cache capacity threshold and a batch size threshold, and generate an initial state space vector; The second unit is used to construct a two-layer deep Q learning network structure based on the initial state space vector and the acquisition parameter combination, set the input of the high-level network to the fused feature vector, and output a global acquisition strategy; set the input of the low-level network to the initial state space vector, and output a specific parameter adjustment plan; design a hierarchical reward function, use the feature weight coefficient as a balancing factor of the reward function, and establish a multi-objective evaluation system including acquisition delay rate, data consistency, throughput efficiency and resource utilization; perform a hierarchical training process based on the multi-objective evaluation system, use the global acquisition strategy output by the high-level network as a constraint condition of the low-level network, iteratively update the network parameters through a priority experience replay mechanism, and generate a hierarchical acquisition strategy model; The third unit is used to deploy the hierarchical collection strategy model to the production environment, continuously collect data stream features, calculate the similarity between the real-time features and the fused feature vector, and evaluate the degree of feature drift; when the degree of feature drift exceeds the preset feature threshold, trigger the high-level network to re-plan the collection strategy based on the current fused feature vector, and input the updated global strategy into the low-level network for parameter fine-tuning; monitor the collection performance index in real time, and when the performance index is lower than the preset target threshold, start the progressive tuning mechanism, input the newly collected state transition sequence into the training cache pool, and perform incremental learning based on historical tuning experience to achieve continuous optimization of the hierarchical collection strategy model.

[0082] According to a third aspect of the embodiments of the present invention, An electronic device is provided, comprising: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0083] A fourth aspect of the embodiments of the present invention is: A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0084] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A database adaptive data stream acquisition optimization method based on reinforcement learning, characterized in that: include: Build a distributed data collection agent network to obtain data flow information from the database in real time and generate an initial feature set; Performing multimodal decomposition on the initial feature set, extracting data stream fluctuation spectrum eigenvalues, fine-grained type distribution eigenvalues, multi-scale time series eigenvalues ​​and graph structure association eigenvalues, and forming a multimodal feature matrix; Based on the multimodal feature matrix, feature weight coefficients are calculated through an improved attention mechanism, different modal features are adaptively fused, and a fused feature vector is output; Using the fused feature vector and combining it with a genetic algorithm, an optimal initial acquisition parameter combination is calculated, including a concurrent acquisition threshold, a cache capacity threshold, and a batch size threshold, to generate an initial state space vector; Based on the initial state space vector and the acquisition parameter combination, a two-layer deep Q learning network structure is constructed, the input of the high-level network is set to the fused feature vector, and a global acquisition strategy is output; the input of the low-level network is set to the initial state space vector, and a specific parameter adjustment plan is output; a hierarchical reward function is designed, the feature weight coefficient is used as a balancing factor of the reward function, and a multi-objective evaluation system including acquisition delay rate, data consistency, throughput efficiency and resource utilization is established; a hierarchical training process is performed based on the multi-objective evaluation system, the global acquisition strategy output by the high-level network is used as a constraint condition of the low-level network, the network parameters are iteratively updated through a priority experience replay mechanism, and a hierarchical acquisition strategy model is generated; Deploy the hierarchical collection strategy model to the production environment, continuously collect data stream features, calculate the similarity between the real-time features and the fused feature vector, and evaluate the degree of feature drift; when the degree of feature drift exceeds the preset feature threshold, trigger the high-level network to re-plan the collection strategy based on the current fused feature vector, and input the updated global strategy into the low-level network for parameter fine-tuning; The acquisition performance indicators are monitored in real time. When the performance indicators are lower than the preset target threshold, the progressive tuning mechanism is started, the newly acquired state transition sequence is input into the training cache pool, and incremental learning is performed based on historical tuning experience to achieve continuous optimization of the hierarchical acquisition strategy model.

2. The method according to claim 1, characterized in that Based on the multimodal feature matrix, feature weight coefficients are calculated through an improved attention mechanism, different modal features are adaptively fused, and a fused feature vector is output; The fused feature vector is used in combination with a genetic algorithm to calculate the optimal initial acquisition parameter combination, including a concurrent acquisition threshold, a cache capacity threshold, and a batch size threshold, to generate an initial state space vector including: The multimodal feature matrix includes fluctuation spectrum features reflecting dynamic changes in data streams, fine-grained type features reflecting data distribution laws, multi-scale sequence features reflecting temporal fluctuation laws, and graph association features reflecting topological structures. A data multi-dimensional representation feature matrix is ​​constructed based on the fluctuation spectrum features, the fine-grained type features, the multi-scale sequence features, and the graph association features. Calculate the statistical distribution characteristics of the multidimensional characterization feature matrix of the data, select the corresponding normalization strategy according to the statistical distribution characteristics, use the maximum and minimum value normalization for the fluctuation spectrum characteristics to maintain the peak and trough relationship, use softmax normalization for the fine-grained type characteristics to highlight the main types, use z-score normalization for the multi-scale sequence characteristics to eliminate the dimension effect, and use degree centrality normalization for the graph association characteristics to balance the importance of nodes, and generate a standardized feature matrix; Analyze the information entropy distribution of each modal feature in the standardized feature matrix, use the information entropy distribution as a modal importance indicator, construct a modal adaptive factor based on the information entropy, dynamically adjust the modal adaptive factor with the information entropy to adaptively capture high-information features, and integrate the modal adaptive factor into the improved attention mechanism to calculate the dynamic attention weight; A two-layer feature fusion network is constructed using the dynamic attention weights. In the first layer, a self-attention operation is performed within the modality to extract the internal correlation of the modality. In the second layer, a cross-attention operation between modalities is performed to model the complementary relationship of the modalities. The outputs of the two layers of attention are fused through a residual structure and a normalization layer to obtain a fused feature vector containing multi-modal complementary information. Component analysis is performed on the fused feature vector, and the concurrent acquisition threshold boundary is determined by using the period and peak characteristics of the fluctuation spectrum feature, the cache capacity threshold boundary is determined by using the entropy value and skewness characteristics of the fine-grained type feature, and the batch processing threshold boundary is determined by using the autocorrelation and stationary characteristics of the multi-scale sequence feature, and the three threshold boundaries constitute a parameter search space; Extracting feature importance based on the principal component analysis result of the fused feature vector, mapping the feature importance into a weight coefficient, constructing a multi-objective fitness function including system throughput, processing delay and resource utilization, and dynamically weighting the three performance indicators according to the weight coefficient; Initializing a genetic algorithm population in the parameter search space, using the multi-objective fitness function to evaluate the quality of individuals in the population, adaptively adjusting the crossover probability based on the population diversity level to balance local and global searches, adaptively adjusting the mutation probability based on the optimization convergence degree to prevent premature convergence, and obtaining the optimal parameter combination through iterative optimization; The optimal parameter combination and the fused feature vector are deeply embedded to generate a state space vector of coupling parameter information and feature representation.

3. The method according to claim 1, characterized in that Based on the initial state space vector and the acquisition parameter combination, a double-layer deep Q learning network structure is constructed, the input of the high-level network is set as the fused feature vector, and a global acquisition strategy is output; The input of the low-level network is set to the initial state space vector, and the specific parameter adjustment scheme is outputted including: Based on the initial state space vector and the acquisition parameter combination, a double-layer deep Q learning network structure is constructed, the fused feature vector is set as the input of the high-level network, and the initial state space vector is set as the input of the low-level network, and the high-level network and the low-level network adopt a double network structure in which the target network and the evaluation network are separated; A dual-stream feature extraction channel is constructed in the high-level network, wherein the first channel extracts static features in the fused feature vector through a multi-layer fully connected network to obtain a static feature representation, and the second channel extracts dynamic features in the fused feature vector through a bidirectional LSTM network to obtain a dynamic feature representation. The static feature representation and the dynamic feature representation are adaptively fused through a multi-head attention mechanism to output a global acquisition strategy, wherein the global acquisition strategy includes a parameter adjustment direction, an adjustment step, and an adjustment priority. The adjustment direction is used to indicate a parameter increase or decrease, the adjustment step is used to control a parameter change amplitude, and the adjustment priority is used to determine a multi-parameter adjustment order. In the low-level network, based on the initial state space vector, the Dueling architecture is used to decompose the Q value into a state value function and an advantage function. The state value function evaluates the parameter configuration performance through a three-layer feedforward neural network to obtain a performance evaluation value. The advantage function evaluates and adjusts the action advantage through a deep residual network to obtain an action advantage value. The performance evaluation value and the action advantage value are weightedly combined to output a specific parameter adjustment plan.

4. The method according to claim 1, characterized in that: Deploy the hierarchical collection strategy model to the production environment, continuously collect data stream features, calculate the similarity between the real-time features and the fused feature vector, and evaluate the degree of feature drift; when the degree of feature drift exceeds the preset feature threshold, trigger the high-level network to re-plan the collection strategy based on the current fused feature vector, and input the updated global strategy into the low-level network for parameter fine-tuning, including: Deploy the hierarchical acquisition strategy model to the production environment, establish a feature similarity hybrid measurement model, continuously collect data stream features to obtain real-time feature vectors, calculate the directional deviation between the real-time feature vector and the fused feature vector on continuous features by cosine similarity to obtain continuous feature similarity, and calculate the distribution change amplitude between the real-time feature vector and the fused feature vector on discrete features by KL divergence to obtain discrete feature similarity; The continuous feature similarity and the discrete feature similarity are normalized and weighted to obtain a feature drift comprehensive index, and a multi-scale feature drift detection mechanism is constructed based on the feature drift comprehensive index; within a first time window, the feature drift comprehensive index is smoothed by an exponentially weighted moving average method to obtain a feature trend, within a second time window, a mutation point detection algorithm is used to identify the jump position of the feature drift comprehensive index to obtain a feature change point, and within a third time window, a trend analysis method is used to capture the gradual change process of the feature drift comprehensive index to obtain a feature trend; Input the feature trend, the feature change point and the feature trend into a decision fusion module, compare the feature drift index at each time scale with the corresponding preset feature threshold, and generate a feature drift degree assessment result; When the feature drift degree evaluation result indicates that the feature drift index at any time scale exceeds the corresponding preset feature threshold, recalculating the feature importance weight matrix based on the current fused feature vector, and inputting the feature importance weight matrix into the high-level network; The high-level network replans the acquisition strategy according to the feature importance weight matrix, updates the feature extraction scheme to obtain the feature weight update strategy, adjusts the parameter change direction and change step size based on the feature weight update strategy to obtain the global strategy update scheme, and inputs the global strategy update scheme into the low-level network for parameter fine-tuning.

5. The method according to claim 1, characterized in that: Real-time monitoring of acquisition performance indicators. When the performance indicators are lower than the preset target threshold, the progressive tuning mechanism is started, the newly acquired state transition sequence is input into the training cache pool, and incremental learning is performed based on historical tuning experience to achieve continuous optimization of the hierarchical acquisition strategy model, including: Construct a multi-dimensional performance monitoring model, collect data throughput, data delay time, and resource utilization in real time to obtain efficiency index data, collect data integrity, data accuracy, and data timeliness to obtain quality index data, and collect computing overhead, storage overhead, and bandwidth overhead to obtain cost index data; calculate the respective index weights for the efficiency index data, the quality index data, and the cost index data, and perform a weighted combination of the index weights and the corresponding index data to obtain a comprehensive performance score; compare the comprehensive performance score with a preset target threshold, and trigger a parameter optimization process when the comprehensive performance score is lower than the preset target threshold; In the parameter optimization process, the parameter space is randomly sampled based on the Thompson sampling strategy to obtain a parameter sampling set, and the uncertainty of each parameter interval in the parameter sampling set is calculated to obtain the parameter exploration priority; the parameter sampling set is prioritized according to the parameter exploration priority to obtain a parameter tuning sequence, and the estimated mean and estimated standard deviation of each parameter in the parameter tuning sequence are calculated; a confidence boundary adjustment strategy is constructed based on the estimated mean and the estimated standard deviation, and the parameter optimization direction is determined by adaptively adjusting the learning rate and the confidence coefficient to obtain an optimized parameter solution; the state transition data generated during the optimization process is collected, and the state transition data is stored in the first cache layer, the second cache layer, and the third cache layer respectively according to the data timestamp to construct a hierarchical training cache pool; Calculate the timeliness weight and value weight of the experience data in the hierarchical training cache pool, multiply the timeliness weight and the value weight to obtain the experience sampling weight; sample the training data set from the hierarchical training cache pool according to the experience sampling weight, analyze the common features in the training data set to obtain the transferable knowledge representation; use the transferable knowledge representation to initialize the policy model parameters to obtain the initial policy model, and compress the initial policy model into a lightweight reasoning model through the knowledge distillation method; deploy the lightweight reasoning model and start performance evaluation, calculate the performance difference before and after optimization to obtain the optimization improvement, and when the optimization improvement is lower than the preset improvement threshold for multiple consecutive rounds, save the current tuning state to obtain a tuning snapshot and stop the optimization process.

6. The method according to claim 5, characterized in that Prioritizing the parameter sampling set according to the parameter exploration priority to obtain a parameter tuning sequence, and calculating an estimated mean and an estimated standard deviation of each parameter in the parameter tuning sequence; Constructing a confidence boundary adjustment strategy based on the estimated mean and the estimated standard deviation, and determining the parameter optimization direction by adaptively adjusting the learning rate and the confidence coefficient to obtain an optimization parameter solution includes: Prioritizing the parameter sampling set according to the parameter exploration priority to obtain a parameter tuning sequence, calculating the correlation between adjacent parameters in the parameter tuning sequence, and combining parameters with correlations greater than a preset correlation threshold into a parameter group; Establishing a historical data sampling window of the parameter tuning sequence, applying a time decay weight to the data in the historical data sampling window, and obtaining a weighted historical sample set; in the weighted historical sample set, calculating an estimated mean and an estimated standard deviation of each parameter in the parameter tuning sequence, and calculating the joint distribution characteristics of the parameters in the parameter group, wherein the estimated mean and the estimated standard deviation reflect the distribution characteristics of the parameters; A confidence boundary adjustment strategy is constructed based on the estimated mean and the estimated standard deviation, and the upper and lower boundaries of the parameter confidence interval are determined in combination with the joint distribution characteristics, wherein the confidence boundary adjustment strategy includes a single parameter confidence interval and a parameter group confidence interval; the degree of deviation between the current parameter value and the parameter confidence interval is calculated, and the combined deviation between the current value of the parameter group and the parameter group confidence interval is calculated, and a reference direction for parameter adjustment is determined based on the degree of deviation and the combined deviation; Analyze the performance change trend in the parameter history optimization process, calculate the performance improvement sequence of the most recent optimizations, adaptively adjust the learning rate according to the fluctuation characteristics of the performance improvement sequence, and use the ratio of the learning rate to the performance improvement as the confidence coefficient; extract several optimization directions with the most significant performance improvement in the parameter history optimization path, perform weighted combination of the several optimization directions and the parameter adjustment reference direction, correct the combined direction according to the confidence coefficient to obtain the parameter optimization direction, and multiply the parameter optimization direction by the learning rate to obtain the optimization parameter solution.

7. A database adaptive data stream acquisition optimization system based on reinforcement learning, used to implement the method described in any one of claims 1 to 6, characterized in that: include: The first unit is used to build a distributed data collection agent network, obtain the data flow information of the database in real time, and generate an initial feature set; Performing multimodal decomposition on the initial feature set, extracting data stream fluctuation spectrum eigenvalues, fine-grained type distribution eigenvalues, multi-scale time series eigenvalues ​​and graph structure association eigenvalues, and forming a multimodal feature matrix; Based on the multimodal feature matrix, feature weight coefficients are calculated through an improved attention mechanism, different modal features are adaptively fused, and a fused feature vector is output; Using the fused feature vector and combining it with a genetic algorithm, an optimal initial acquisition parameter combination is calculated, including a concurrent acquisition threshold, a cache capacity threshold, and a batch size threshold, to generate an initial state space vector; The second unit is used to construct a two-layer deep Q learning network structure based on the initial state space vector and the acquisition parameter combination, set the input of the high-level network to the fused feature vector, and output a global acquisition strategy; set the input of the low-level network to the initial state space vector, and output a specific parameter adjustment plan; design a hierarchical reward function, use the feature weight coefficient as a balancing factor of the reward function, and establish a multi-objective evaluation system including acquisition delay rate, data consistency, throughput efficiency and resource utilization; perform a hierarchical training process based on the multi-objective evaluation system, use the global acquisition strategy output by the high-level network as a constraint condition of the low-level network, iteratively update the network parameters through a priority experience replay mechanism, and generate a hierarchical acquisition strategy model; The third unit is used to deploy the hierarchical collection strategy model to the production environment, continuously collect data stream features, calculate the similarity between the real-time features and the fused feature vector, and evaluate the degree of feature drift; when the degree of feature drift exceeds a preset feature threshold, trigger the high-level network to re-plan the collection strategy based on the current fused feature vector, and input the updated global strategy into the low-level network for parameter fine-tuning; The acquisition performance indicators are monitored in real time. When the performance indicators are lower than the preset target threshold, the progressive tuning mechanism is started, the newly acquired state transition sequence is input into the training cache pool, and incremental learning is performed based on historical tuning experience to achieve continuous optimization of the hierarchical acquisition strategy model.

8. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images