A cloud mobile phone data synchronization scheduling method based on reinforcement learning

Abstract

The application discloses a cloud mobile phone data synchronization scheduling method based on reinforcement learning, comprising the following steps: S1, collecting multi-dimensional state preprocessing to obtain a space-time state feature matrix; S2, inputting an improved PA-DDPG algorithm, mapping output discrete tasks and continuous resource parameters, and solving mixed scheduling instructions; S3, simulating execution in an isolated environment, tracking data flow and version trajectory; S4, comparing logical time stamps, releasing without conflict, intercepting and downgrading to priority lock rules with conflict, and obtaining a final scheme; S5, tracking real execution, and weightedly calculating real composite reward values; S6, generating virtual experience along a strategy, and coexisting with real experience into a playback pool; S7, extracting mixed experience to calculate errors and backward propagation, respectively evaluating task strategic value and resource tactical efficiency through a double-flow network, and updating parameters to complete iteration. The application realizes accurate scheduling decision and absolute conflict interception, and guarantees data strong consistency.

Description

Technical Field

[0001] This invention relates to the intersection of resource scheduling and artificial intelligence, and in particular to a cloud mobile phone data synchronization scheduling method based on reinforcement learning. Background Technology

[0002] Reinforcement learning-based intelligent scheduling methods, with their self-learning and decision-making optimization capabilities in dynamic and ever-changing environments, have been widely applied in recent years in areas such as cloud phone data synchronization, distributed storage scheduling, and edge computing resource allocation, becoming an important development direction for intelligent data flow management. However, in practical applications, cloud phone data synchronization scenarios face many challenges, including complex data dependency topologies, variable physical operating environments, and stringent requirements for strong data consistency. The deployment effectiveness of intelligent scheduling methods is still constrained by various factors.

[0003] Most current data synchronization scheduling methods rely on static rule matching or single-dimensional heuristic allocation, making it difficult to fully integrate the dependencies between data shards with multi-dimensional dynamic states such as network bandwidth, latency, cloud computing power, and end-side power consumption. This results in a lack of comprehensive modeling of the spatiotemporal state of the synchronization process. Some systems only employ discrete task scheduling strategies, neglecting the joint optimization of task execution order, bandwidth allocation ratio, synchronization protocol parameters, and other continuous resource allocations, thus limiting the adaptive tuning capability of scheduling strategies in complex physical environments. Furthermore, the scheduling logic lacks pre-execution security checks and conflict prediction mechanisms, making it difficult to effectively intercept and mitigate data version conflicts before they occur, affecting the absolute consistency of synchronization results and system reliability.

[0004] Furthermore, existing intelligent scheduling methods mostly rely on single online interactions in real-world environments for experience acquisition in data synchronization. This fails to provide efficient forward-looking extrapolation while ensuring absolute data consistency, resulting in excessively high trial-and-error costs and low sample utilization. Their network evaluation and feedback mechanisms often employ single value stream evaluation, failing to decouple and dynamically correct based on the strategic value of task selection and the tactical effectiveness of resource allocation. This makes it difficult to adapt to the constantly changing multiple constraints in cloud phone synchronization scenarios, severely impacting the practical value and convergence stability of the scheduling model in real-world scenarios.

[0005] Therefore, how to provide a cloud mobile phone data synchronization and scheduling method based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose a cloud mobile phone data synchronization scheduling method based on reinforcement learning. This invention fully integrates key steps such as spatiotemporal state feature extraction, improved PA-DDPG algorithm decision-making, isolated environment simulation and consistency verification, constructing an intelligent scheduling and security protection process with multi-dimensional representation of data dependencies and physical environment, joint solution of hybrid scheduling actions, pre-execution conflict interception and degradation, and experience replay pool construction. This achieves precise control and absolute consistency guarantee of data flow in complex cloud mobile phone synchronization scenarios. By introducing a dual-stream commentator network based on parameterized action decoupling, this invention achieves independent evaluation and dynamic correction of the strategic value of task selection and the tactical effectiveness of resource allocation. Simultaneously, the construction of an experience replay pool integrates real and virtual simulation data, significantly improving sample utilization and policy convergence speed. This invention possesses advantages such as comprehensive state modeling, optimized action space decoupling, controllable trial-and-error risk, and efficient experience sampling, effectively solving problems in existing methods such as the difficulty of joint decision-making between discrete and continuous actions, high trial-and-error costs in real environments, and easy disruption of data consistency.

[0007] A cloud mobile phone data synchronization scheduling method based on reinforcement learning according to an embodiment of the present invention includes the following steps:

[0008] S1. Collect the dependencies, network bandwidth, latency, cloud container computing power, and terminal battery status of the data shards to be synchronized in the cloud phone instance, and preprocess them to obtain the spatiotemporal state feature matrix.

[0009] S2. Input the spatiotemporal state feature matrix into the improved PA-DDPG algorithm, introduce a dual-stream critic network based on parameterized action decoupling, perform joint mapping through the actor network, output discrete task selection probability distribution and continuous resource parameter adjustment vector, take the discrete task with the highest probability as the synchronization time slice allocation and task execution sorting strategy, and extract the corresponding bandwidth splitting ratio and synchronization protocol parameters to solve for the hybrid scheduling action instructions.

[0010] S3. Reproduce the current synchronous state in the cloud-isolated environment, accelerate the execution of mixed scheduling action commands in this isolated environment, track data flow and version changes, and obtain the simulated data flow status and version evolution trajectory.

[0011] S4. Based on the simulated data flow status, compare the logical timestamps of each data shard in the simulated version evolution trajectory. If there is no conflict, the consistency check is determined to be passed and a release instruction is output. Otherwise, the check is determined to be failed and an interception instruction is output. The system is downgraded to a regular synchronization rule based on priority locks to obtain the final scheduling execution scheme.

[0012] S5. Track the execution of the final scheduling plan in the real environment, count the synchronization time, traffic consumption and end-side lag rate, and assess whether the degradation interception is triggered. Calculate the weighted average of various business indicators and interception events to obtain the real composite reward value.

[0013] S6. Starting from the real environment state after the final scheduling execution plan is executed, the cloud isolation environment is called to continue the deduction along the current actor network strategy, collect the virtual state, virtual action and virtual reward of each step of the deduction, generate virtual experience data, and store the real experience data and virtual experience data into the experience replay pool.

[0014] S7. Extract mixed experience data from the experience replay pool, calculate the temporal difference error based on the real composite reward value and the virtual reward, and backpropagate it. Through the first-level scheduling value network in the dual-stream commentator network, input state and discrete actions to evaluate the strategic value of task selection, and the second-level execution effectiveness network, input state, discrete actions and continuous parameters to evaluate the tactical effectiveness of resource allocation, generate a comprehensive evaluation value and calculate the gradient, update the parameters of the dual-stream commentator network and the actor network, and complete the iterative update of the scheduling strategy.

[0015] Optionally, S1 specifically includes:

[0016] S11. Read the synchronization logs and resource monitoring interface of the cloud phone instance in real time, extract the dependency list between the data shards to be synchronized, and obtain the current network bandwidth value, network latency value, cloud container computing power utilization rate and terminal power percentage.

[0017] S12. Take the data fragments to be synchronized as nodes of the graph, take the dependencies between data fragments as edges of the graph, construct a data dependency topology graph and extract the corresponding adjacency matrix, and concatenate the network bandwidth value, network latency value, cloud container computing power utilization rate and end-side power percentage into a one-dimensional physical state vector in a set order.

[0018] S13. Concatenate the adjacency matrix of the data-dependent topology graph with the one-dimensional physical state vector to obtain the concatenated feature matrix. Perform principal component analysis on the concatenated feature matrix to reduce the dimensionality and remove feature dimensions with a variance contribution rate lower than a preset threshold to obtain the spatiotemporal state feature matrix.

[0019] Optionally, S2 specifically includes:

[0020] S21. Input the spatiotemporal state feature matrix into the actor network of the improved PA-DDPG algorithm, perform a linear transformation on the spatiotemporal state feature matrix and the first fully connected layer of the actor network and input it into the ReLU activation function, set the negative elements to zero and keep the positive elements to obtain the basic state feature vector.

[0021] S22. Input the basic state feature vector into the discrete action head of the actor network, read the discrete mapping weight matrix and discrete mapping bias vector randomly generated during the initialization stage of the discrete action head, perform matrix multiplication on the basic state feature vector and the discrete mapping weight matrix and superimpose the discrete mapping bias vector to obtain the discrete linear output.

[0022] S23. Perform an exponential operation with the natural constant e as the base for each value in the discrete linear output to obtain each exponential result. Divide each exponential result by the sum of all exponential results to obtain the probability value of each data slice to be synchronized. Compare all probability values ​​and select the data slice corresponding to the probability value with the largest value as the synchronization time slice allocation and task execution sorting strategy.

[0023] S24. Construct a zero vector with a dimension equal to the total number of data fragments to be synchronized. Set the element at the position of the data fragment corresponding to the synchronization time slice allocation and task execution sorting strategy in the zero vector to one, and keep the other elements at zero. Convert the synchronization time slice allocation and task execution sorting strategy into a discrete one-hot encoded vector. Multiply the basic state feature vector with the corresponding element in the discrete one-hot encoded vector one by one to obtain the condition constraint feature vector.

[0024] S25. Input the conditional constraint feature vector into the continuous action head of the actor network, read the continuous mapping weight matrix and continuous mapping bias vector randomly generated during the initialization phase of the continuous action head, perform matrix multiplication on the conditional constraint feature vector and the continuous mapping weight matrix, and superimpose the continuous mapping bias vector to obtain a continuous linear output.

[0025] S26. Input the continuous linear output into the Tanh activation function, compress each value in the continuous linear output to an open interval between negative one and one, and obtain the compressed continuous value. Multiply the first value in the compressed continuous value by the preset bandwidth upper limit, multiply the second value in the compressed continuous value by the preset protocol parameter upper limit, and reassemble the two product results in the original order to obtain the continuous resource parameter adjustment vector.

[0026] S27. Extract the first value from the continuous resource parameter adjustment vector as the bandwidth allocation ratio, extract the second value from the continuous resource parameter adjustment vector as the synchronization protocol parameter, and concatenate the synchronization time slice allocation and task execution sorting strategy, bandwidth allocation ratio and synchronization protocol parameter in sequence into a one-dimensional action vector to obtain the hybrid scheduling action instruction.

[0027] Optionally, S3 specifically includes:

[0028] S31. Read the memory snapshot and data shard storage status of the current cloud phone instance in the cloud isolation environment, load the memory snapshot and data shard storage status into the running memory of the isolation container, overwrite the initial state in the isolation container, and obtain the reproduced current synchronization state.

[0029] S32. Transmit the hybrid scheduling action command to the control interface of the isolation container, start the preset acceleration clock, and execute the synchronization time slice allocation and task execution sorting strategy, bandwidth splitting ratio and synchronization protocol parameters in the isolation container according to the time step of the acceleration clock.

[0030] S33. During each step of execution, monitor the read and write operations of data fragments in the isolation container, identify the data fragments that have read operations as source fragments, identify the data fragments that have written operations as target fragments, record the amount of data transmitted from the source fragment to the target fragment, and obtain a single-step data flow record.

[0031] S34. While monitoring read and write operations, extract the system logic clock value when each write operation is completed, and bind it with the data fragment identifier to form a data version tag. Arrange all data version tags in the order of operation execution to obtain a single-step version evolution record.

[0032] S35. Concatenate the single-step data flow records of each step of the accelerated clock in chronological order to obtain the simulated data flow state after the execution of the hybrid scheduling action. Concatenate the single-step version evolution records of each step in chronological order to obtain the version evolution trajectory after the execution of the hybrid scheduling action.

[0033] Optionally, S4 specifically includes:

[0034] S41. Extract the data fragment identifiers of each data fragment that has been written from the simulated data flow state after the execution of the hybrid scheduling action, and read the data version tags with time attributes bound to each data fragment identifier from the version evolution trajectory after the execution of the hybrid scheduling action. Parse the system logical clock value in each data version tag to obtain the logical timestamp of each data fragment.

[0035] S42. Logical timestamps with the same data fragment identifier are divided into the same comparison group. Within the same comparison group, the absolute value of the difference between any two logical timestamps is calculated. If the absolute value of the difference is equal to zero, it is determined that there is no conflict between the logical timestamps in the comparison group. If the absolute value of the difference is greater than zero, it is determined that there is a conflict between the logical timestamps in the comparison group.

[0036] S43. Iterate through the judgment results of all comparison groups. If all comparison groups are judged to have no logical timestamp conflict, the consistency check is passed and a release instruction is generated. If any comparison group is judged to have a logical timestamp conflict, the check is failed and an interception instruction is generated.

[0037] S44. When a release instruction is generated, the hybrid scheduling action instruction is directly used as the final scheduling execution scheme to ensure absolute data consistency. When an interception instruction is generated, the hybrid scheduling action instruction is abandoned, the preset priority of all data fragments to be synchronized is extracted, the highest priority data fragment is allocated a synchronization time slice and a mutex lock is applied, and the remaining data fragments are synchronized in descending order of preset priority after the mutex lock is released.

[0038] S45. Degrade the rules that apply mutex locks and are executed in order of preset priority to regular synchronization rules based on priority locks, and output the final scheduling execution scheme as the final scheduling execution scheme to ensure absolute data consistency.

[0039] Optionally, S5 specifically includes:

[0040] S51. Execute the final scheduling execution plan in a real environment, record the system time at the start of execution and the system time to complete the synchronization of all data shards, calculate the difference between the system time to complete the system time and the system time to start the system time, and obtain the synchronization time.

[0041] S52. During the execution of the final scheduling plan in a real environment, read the number of bytes of data packets sent and received by the network interface, calculate the difference between the number of bytes of data packets after execution and before execution, and obtain the traffic consumption.

[0042] S53. During the execution of the final scheduling plan in a real environment, the rendering frame rate on the client side is collected at fixed time intervals. The percentage of collections that are lower than the preset rendering frame rate threshold is calculated out of the total number of collections to obtain the client side stuttering rate.

[0043] S54. Read the generation record of the final scheduling execution plan. If the generation record contains a regular synchronization rule based on priority lock, it is determined that a downgrade interception has been triggered and the value is set to 1. If the generation record contains a mixed scheduling action instruction, it is determined that a downgrade interception has not been triggered and the value is set to 0. The interception event value is obtained.

[0044] S55. Multiply the synchronization time by the preset time weighting coefficient, multiply the traffic consumption by the preset traffic weighting coefficient, multiply the terminal lag rate by the preset lag weighting coefficient, multiply the interception event value by the preset interception penalty weighting coefficient, and sum the four product results to obtain the true composite reward value.

[0045] Optionally, S6 specifically includes:

[0046] S61. After executing the final scheduling execution plan, read the memory data and hard disk data of the cloud mobile phone instance in the real environment and send them to the cloud isolation environment. In the cloud isolation environment, overwrite the running memory and storage space of the isolation container to obtain the initial state of the virtual simulation.

[0047] S62. Perform feature splicing and dimensionality reduction on the initial state of the virtual simulation to obtain the virtual state feature matrix. Input the virtual state feature matrix into the current actor network, perform linear transformation, matrix multiplication and weighted splicing operations, output the virtual synchronization time slice allocation and task execution sorting strategy, the virtual bandwidth splitting ratio and the virtual synchronization protocol parameters, and splice them into a one-dimensional vector to obtain the virtual hybrid scheduling action instructions.

[0048] S63. Execute virtual hybrid scheduling action instructions in the cloud isolation environment, record the virtual time and virtual data transmission volume from the start to the end of the execution, read the memory data and disk data of the isolation container after the execution is completed to obtain the virtual next state, add the virtual time and virtual data transmission volume in a weighted manner to obtain the virtual reward, and concatenate the virtual state feature matrix, virtual hybrid scheduling action instructions, virtual reward and virtual next state in sequence into a quadruple and record it as a virtual experience data.

[0049] S64. The spatiotemporal state feature matrix input to the actor network, the hybrid scheduling action instruction output by the actor network, the real composite reward value, and the real environmental state execution features after executing the final scheduling execution plan are concatenated and dimensionality reduced to obtain the post-execution state feature matrix. The spatiotemporal state feature matrix, the hybrid scheduling action instruction, the real composite reward value, and the post-execution state feature matrix are concatenated in sequence into a quadruple and recorded as a real experience data.

[0050] S65. Assign a preset virtual sampling weight to the virtual experience data and a preset real sampling weight to the real experience data. Randomly extract data from the virtual experience data and the real experience data according to the preset virtual sampling weight and the preset real sampling weight. After merging the extracted data, write it into the pre-allocated memory space to obtain the experience playback pool.

[0051] S66. Repeat steps S62 to S65. Each time the process is repeated, increment the virtual simulation step count by one. Determine whether the virtual simulation step count is equal to the preset step count. If it is not equal to the preset step count, use the next virtual state as the new virtual simulation initial state and continue repeating. If it is equal to the preset step count, stop the simulation.

[0052] Optionally, S7 specifically includes:

[0053] S71. Randomly extract a preset number of mixed experience data from the experience replay pool, separate the current state feature matrix, mixed scheduling action instruction, current reward value and next state feature matrix from each mixed experience data, extract the synchronization time slice allocation and task execution sorting strategy as discrete actions from the mixed scheduling action instruction, and extract the bandwidth splitting ratio and synchronization protocol parameters as continuous actions.

[0054] S72. Input the next state feature matrix into the target actor network, perform matrix multiplication and activation function operations, output the next discrete action and the next continuous action, concatenate the next state feature matrix, the next discrete action and the next continuous action in sequence to form the target joint input vector, and input it into the target network of the dual-stream critic network. Perform matrix multiplication between the target joint input vector and the weight matrix of the fully connected layer in the target network and superimpose the bias vector. After calculation by the ReLU activation function, output the target evaluation value.

[0055] S73. Determine whether the currently extracted mixed experience data is real experience data or virtual experience data. If it is real experience data, extract the real composite reward value as the reward value of the current step. If it is virtual experience data, extract the virtual reward as the reward value of the current step. Add the preset discount factor to the reward value of the current step and multiply it by the target evaluation value to obtain the target time series difference target value. Subtract the current evaluation value output by the dual-stream commentator network from the target time series difference target value to obtain the time series difference error.

[0056] S74. Input the current state feature matrix and discrete actions into the first-order scheduling value network of the dual-stream critic network, perform matrix multiplication and nonlinear activation operations to map the state features and discrete actions into a single value, and obtain the strategic value assessment value of task selection.

[0057] S75. The current state feature matrix, discrete actions and continuous actions are concatenated into a joint input vector. The joint input vector is then input into the second-stream execution efficiency network of the two-stream commentator network. Matrix multiplication and nonlinear activation operations are performed to map the joint input vector into a single value, thereby obtaining the tactical effectiveness evaluation value of resource allocation.

[0058] S76. Multiply the strategic value assessment value of task selection by the preset strategic weight coefficient, multiply the tactical effectiveness assessment value of resource allocation by the preset tactical weight coefficient, add the two product results to obtain the comprehensive assessment value, calculate the product of the comprehensive assessment value and the temporal difference error to obtain the gradient value, backpropagate the gradient value back to the two-stream commentator network, and update the weight matrix and bias vector of each layer in the two-stream commentator network layer by layer.

[0059] S77. Calculate the policy gradient based on the updated weight matrix and bias vector of the dual-stream commentator network, backpropagate the policy gradient back to the actor network, and update the discrete mapping weight matrix, discrete mapping bias vector, continuous mapping weight matrix and continuous mapping bias vector in the actor network layer by layer to complete the iterative update of the scheduling policy.

[0060] The beneficial effects of this invention are:

[0061] This invention constructs a spatiotemporal state feature matrix by extracting data dependency topology and physical operating environment features. Addressing the multidimensional and coupled state issues in cloud phone synchronization scenarios, it employs an improved PA-DDPG algorithm and introduces a dual-stream commentator network based on parameterized action decoupling. This performs a joint mapping of discrete task selection and continuous resource allocation, outputting hybrid scheduling action instructions. Accelerated simulation execution is achieved by reproducing the synchronization scenario in a cloud-isolated environment. Consistency checks are performed by comparing logical timestamps, and conflicting scheduling instructions are intercepted and downgraded to regular synchronization rules based on priority locks, generating a final scheduling execution scheme that guarantees absolute consistency. During the strategy iteration phase, real-world execution metrics are statistically analyzed to obtain the true composite reward value. Starting from the real-world state, virtual experience data is generated by continuing to deduce along the strategy in the isolated environment, constructing an experience replay pool. The dual-stream commentator network is then used to evaluate the strategic value of task selection and the tactical effectiveness of resource allocation, respectively. A weighted comprehensive evaluation value and temporal difference error are calculated for parameter updates. Ultimately, it achieves precise decision-making for hybrid actions in the cloud phone data synchronization process, pre-interception of data conflicts, and strategy self-evolution driven by hybrid virtual and real experience, effectively improving the resource matching degree of synchronization scheduling, the absolute guarantee of data consistency, and the convergence stability of reinforcement learning models. Attached Figure Description

[0062] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0063] Figure 1 This is a flowchart of a cloud mobile phone data synchronization and scheduling method based on reinforcement learning proposed in this invention;

[0064] Figure 2 This is a flowchart of the hybrid scheduling instruction generation and secure release process based on isolated environment pre-simulation and consistency verification proposed in this invention.

[0065] Figure 3 This is a flowchart illustrating the iterative update process of the scheduling strategy based on hybrid virtual-real experience and a dual-stream commentator network proposed in this invention. Detailed Implementation

[0066] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0067] refer to Figures 1-3 A cloud mobile phone data synchronization scheduling method based on reinforcement learning includes the following steps:

[0068] S1. Real-time collection of the dependencies, network bandwidth, latency, cloud container computing power and terminal power status of the data shards to be synchronized in the cloud phone instance. Map the dependencies between data shards as nodes and edges of a graph and combine them with other state parameters for feature splicing and dimensionality reduction to obtain a spatiotemporal state feature matrix containing data dependency topology and physical operating environment.

[0069] S2. Input the spatiotemporal state feature matrix into the improved PA-DDPG algorithm, introduce a dual-stream critic network based on parameterized action decoupling, perform joint mapping through the actor network, synchronously output discrete task selection probability distribution and continuous resource parameter adjustment vector, take the discrete task with the highest probability as the synchronous time slice allocation and task execution sorting strategy, and extract the corresponding bandwidth splitting ratio and synchronization protocol parameters, and jointly solve to obtain the hybrid scheduling action instruction.

[0070] S3. Reproduce the current synchronous state in the cloud-isolated environment, accelerate the execution of hybrid scheduling action instructions in this isolated environment, track data flow and version changes, and obtain the simulated data flow status and version evolution trajectory after the execution of hybrid scheduling actions.

[0071] S4. Based on the simulated data flow status, compare the logical timestamps of each data shard in the simulated version evolution trajectory. If there is no conflict in the logical timestamps, the consistency check is determined to be passed and a release instruction is output. If there is a conflict in the logical timestamps, the check is determined to be failed and an interception instruction is output. The system is downgraded to a regular synchronization rule based on priority locks to obtain the final scheduling and execution scheme that ensures absolute data consistency.

[0072] S5. Track the execution of the final scheduling plan in the real environment, count the synchronization time, traffic consumption and end-side lag rate, and assess whether the degradation interception is triggered. Calculate the weighted average of various business indicators and interception events to obtain the real composite reward value.

[0073] S6. Starting from the real environment state after the final scheduling execution plan is executed, the cloud isolation environment is called to continue to deduce the preset number of steps along the current actor network strategy, collect the virtual state, virtual action and virtual reward of each step, generate virtual experience data, and store the real experience data and virtual experience data into the experience playback pool after assigning different preset sampling weights to them.

[0074] S7. Extract mixed experience data from the experience replay pool, calculate the temporal difference error based on the real composite reward value and the virtual reward, and backpropagate it. Through the first-level scheduling value network in the dual-stream commentator network, input state and discrete actions to evaluate the strategic value of task selection, and the second-level execution effectiveness network, input state, discrete actions and continuous parameters to evaluate the tactical effectiveness of resource allocation. The weighted sum is used to obtain the comprehensive evaluation value to calculate the gradient, update the parameters of the dual-stream commentator network and the actor network, and complete the iterative update of the scheduling strategy.

[0075] This invention significantly improves the strong consistency and scheduling efficiency of cloud phone data synchronization. By constructing a spatiotemporal state feature matrix, it achieves unified modeling of data sharding dependency topology and physical operating environment, enhancing the ability to accurately perceive complex synchronization scenarios. When generating scheduling instructions, a dual-stream commentator network based on parameterized action decoupling is introduced to jointly output discrete task ordering and continuous resource parameters, achieving accurate decision-making for hybrid scheduling actions. Relying on the accelerated simulation and deduction mechanism of the cloud-isolated environment and the strict comparison mechanism of logical timestamps, all potential data version conflicts are accurately intercepted and automatically downgraded before the instructions are issued, completely eliminating data corruption and ensuring absolute data consistency. At the same time, by using a hybrid virtual-real experience replay pool and differentiated sampling weights, the trial-and-error costs and risks in the real environment are greatly reduced; the dual-stream commentator network evaluates the strategic value of task selection and the tactical effectiveness of resource allocation respectively, effectively overcoming the problem of inaccurate evaluation in the hybrid action space. Through continuous optimization of decision-making strategies, this invention exhibits stronger robustness in extremely complex scenarios such as network jitter and high concurrency, significantly shortening synchronization time, improving bandwidth utilization, and significantly improving the intelligent guarantee level of cloud phone data flow and collaboration.

[0076] In this embodiment, S1 specifically includes:

[0077] S11. Read the synchronization logs and resource monitoring interface of the cloud phone instance in real time, extract the dependency list between the data shards to be synchronized, and obtain the current network bandwidth value, network latency value, cloud container computing power utilization rate and terminal power percentage.

[0078] S12. Take the data fragments to be synchronized as nodes of the graph, take the dependencies between data fragments as edges of the graph, construct a data dependency topology graph and extract the corresponding adjacency matrix, and concatenate the network bandwidth value, network latency value, cloud container computing power utilization rate and end-side power percentage into a one-dimensional physical state vector in a set order.

[0079] S13. The adjacency matrix of the data-dependent topology graph and the one-dimensional physical state vector are concatenated to obtain a concatenated feature matrix. Principal component analysis is performed on the concatenated feature matrix to reduce its dimensionality. Feature dimensions with a variance contribution rate lower than a preset threshold are removed to obtain a spatiotemporal state feature matrix containing the data-dependent topology and the physical operating environment. The preset threshold is 0.05.

[0080] In this embodiment, S2 specifically includes:

[0081] S21. Input the spatiotemporal state feature matrix into the actor network of the improved PA-DDPG algorithm, perform a linear transformation on the spatiotemporal state feature matrix and the first fully connected layer of the actor network and input it into the ReLU activation function, set the negative elements to zero and keep the positive elements to obtain the basic state feature vector.

[0082] S22. Input the basic state feature vector into the discrete action head of the actor network, read the discrete mapping weight matrix and discrete mapping bias vector randomly generated during the initialization stage of the discrete action head, perform matrix multiplication on the basic state feature vector and the discrete mapping weight matrix and superimpose the discrete mapping bias vector to obtain the discrete linear output.

[0083] S23. Perform an exponential operation with the natural constant e as the base for each value in the discrete linear output to obtain each exponential result. Divide each exponential result by the sum of all exponential results to obtain the probability value of each data slice to be synchronized. Compare all probability values ​​and select the data slice corresponding to the probability value with the largest value as the synchronization time slice allocation and task execution sorting strategy.

[0084] S24. Construct a zero vector with a dimension equal to the total number of data fragments to be synchronized. Set the element at the position of the data fragment corresponding to the synchronization time slice allocation and task execution sorting strategy in the zero vector to one, and keep the other elements at zero. Convert the synchronization time slice allocation and task execution sorting strategy into a discrete one-hot encoded vector. Multiply the basic state feature vector with the corresponding element in the discrete one-hot encoded vector one by one to obtain the condition constraint feature vector.

[0085] S25. Input the conditional constraint feature vector into the continuous action head of the actor network, read the continuous mapping weight matrix and continuous mapping bias vector randomly generated during the initialization phase of the continuous action head, perform matrix multiplication on the conditional constraint feature vector and the continuous mapping weight matrix, and superimpose the continuous mapping bias vector to obtain a continuous linear output.

[0086] S26. Input the continuous linear output into the Tanh activation function, compress each value in the continuous linear output to an open interval between -1 and 1 to obtain a compressed continuous value, multiply the first value in the compressed continuous value by a preset bandwidth upper limit, multiply the second value in the compressed continuous value by a preset protocol parameter upper limit, and reassemble the two product results in the original order to obtain a continuous resource parameter adjustment vector, wherein the preset bandwidth upper limit is 100Mbps and the preset protocol parameter upper limit is 500 milliseconds;

[0087] S27. Extract the first value from the continuous resource parameter adjustment vector as the bandwidth allocation ratio, extract the second value from the continuous resource parameter adjustment vector as the synchronization protocol parameter, and concatenate the synchronization time slice allocation and task execution sorting strategy, bandwidth allocation ratio and synchronization protocol parameter in sequence into a one-dimensional action vector to obtain the hybrid scheduling action instruction.

[0088] In this embodiment, S3 specifically includes:

[0089] S31. Read the memory snapshot and data shard storage status of the current cloud phone instance in the cloud isolation environment, load the memory snapshot and data shard storage status into the running memory of the isolation container, overwrite the initial state in the isolation container, and obtain the reproduced current synchronization state.

[0090] S32. Transmit the hybrid scheduling action command to the control interface of the isolation container, start the preset acceleration clock, and execute the synchronization time slice allocation and task execution sorting strategy, bandwidth splitting ratio and synchronization protocol parameters in the isolation container according to the time step of the acceleration clock.

[0091] S33. During each step of execution, monitor the read and write operations of data fragments in the isolation container, identify the data fragments that have read operations as source fragments, identify the data fragments that have written operations as target fragments, record the amount of data transmitted from the source fragment to the target fragment, and obtain a single-step data flow record.

[0092] S34. While monitoring read and write operations, extract the system logic clock value when each write operation is completed, and bind it with the data fragment identifier being written to form a data version tag with time attributes. Arrange all data version tags according to the order of operation execution to obtain a single-step version evolution record.

[0093] S35. Concatenate the single-step data flow records of each step of the accelerated clock in chronological order to obtain the simulated data flow state after the execution of the hybrid scheduling action. Concatenate the single-step version evolution records of each step in chronological order to obtain the version evolution trajectory after the execution of the hybrid scheduling action.

[0094] In this embodiment, S4 specifically includes:

[0095] S41. Extract the data fragment identifiers of each data fragment that has been written from the simulated data flow state after the execution of the hybrid scheduling action, and read the data version tags with time attributes bound to each data fragment identifier from the version evolution trajectory after the execution of the hybrid scheduling action. Parse the system logical clock value in each data version tag to obtain the logical timestamp of each data fragment.

[0096] S42. Logical timestamps with the same data fragment identifier are divided into the same comparison group. Within the same comparison group, the absolute value of the difference between any two logical timestamps is calculated. If the absolute value of the difference is equal to zero, it is determined that there is no conflict between the logical timestamps in the comparison group. If the absolute value of the difference is greater than zero, it is determined that there is a conflict between the logical timestamps in the comparison group.

[0097] S43. Iterate through the judgment results of all comparison groups. If all comparison groups are judged to have no logical timestamp conflict, the consistency check is passed and a release instruction is generated. If any comparison group is judged to have a logical timestamp conflict, the check is failed and an interception instruction is generated.

[0098] S44. When a release instruction is generated, the hybrid scheduling action instruction is directly used as the final scheduling execution scheme to ensure absolute data consistency. When an interception instruction is generated, the hybrid scheduling action instruction is abandoned, the preset priority of all data fragments to be synchronized is extracted, the highest priority data fragment is allocated a synchronization time slice and a mutex lock is applied, and the remaining data fragments are synchronized in descending order of preset priority after the mutex lock is released.

[0099] S45. Degrade the rules that apply mutex locks and are executed in order of preset priority to regular synchronization rules based on priority locks, and output the final scheduling execution scheme as the final scheduling execution scheme to ensure absolute data consistency.

[0100] In this embodiment, S5 specifically includes:

[0101] S51. Execute the final scheduling execution plan in a real environment, record the system time at the start of execution and the system time to complete the synchronization of all data shards, calculate the difference between the system time to complete the system time and the system time to start the system time, and obtain the synchronization time.

[0102] S52. During the execution of the final scheduling plan in a real environment, read the number of bytes of data packets sent and received by the network interface, calculate the difference between the number of bytes of data packets after execution and before execution, and obtain the traffic consumption.

[0103] S53. During the execution of the final scheduling execution plan in the real environment, the rendering frame rate on the terminal side is collected at fixed time intervals. The percentage of collections that are lower than the preset rendering frame rate threshold is calculated to determine the terminal side stuttering rate. The preset rendering frame rate threshold is 30 frames per second.

[0104] S54. Read the generation record of the final scheduling execution plan. If the generation record contains a regular synchronization rule based on priority lock, it is determined that a downgrade interception has been triggered and the value is set to 1. If the generation record contains a mixed scheduling action instruction, it is determined that a downgrade interception has not been triggered and the value is set to 0. The interception event value is obtained.

[0105] S55. Multiply the synchronization time by a preset time weighting coefficient, multiply the traffic consumption by a preset traffic weighting coefficient, multiply the end-side lag rate by a preset lag weighting coefficient, multiply the interception event value by a preset interception penalty weighting coefficient, and sum the four product results to obtain the true composite reward value. The preset time weighting coefficient is 0.2, the preset traffic weighting coefficient is 0.2, the preset lag weighting coefficient is 0.3, and the preset interception penalty weighting coefficient is 0.3.

[0106] In this embodiment, S6 specifically includes:

[0107] S61. After executing the final scheduling execution plan, read the memory data and hard disk data of the cloud mobile phone instance in the real environment and send them to the cloud isolation environment. In the cloud isolation environment, overwrite the running memory and storage space of the isolation container to obtain the initial state of the virtual simulation.

[0108] S62. Perform feature splicing and dimensionality reduction on the initial state of the virtual simulation to obtain the virtual state feature matrix. Input the virtual state feature matrix into the current actor network, perform linear transformation, matrix multiplication and weighted splicing operations, output the virtual synchronization time slice allocation and task execution sorting strategy, the virtual bandwidth splitting ratio and the virtual synchronization protocol parameters, and splice them into a one-dimensional vector to obtain the virtual hybrid scheduling action instructions.

[0109] S63. Execute virtual hybrid scheduling action instructions in the cloud isolation environment, record the virtual time and virtual data transmission volume from the start to the end of the execution, read the memory data and disk data of the isolation container after the execution is completed to obtain the virtual next state, add the virtual time and virtual data transmission volume in a weighted manner to obtain the virtual reward, and concatenate the virtual state feature matrix, virtual hybrid scheduling action instructions, virtual reward and virtual next state in sequence into a quadruple and record it as a virtual experience data.

[0110] S64. The spatiotemporal state feature matrix input to the actor network, the hybrid scheduling action instruction output by the actor network, the real composite reward value, and the real environmental state execution features after executing the final scheduling execution plan are concatenated and dimensionality reduced to obtain the post-execution state feature matrix. The spatiotemporal state feature matrix, the hybrid scheduling action instruction, the real composite reward value, and the post-execution state feature matrix are concatenated in sequence into a quadruple and recorded as a real experience data.

[0111] S65. Assign a preset virtual sampling weight to the virtual experience data and a preset real sampling weight to the real experience data. Randomly extract data from the virtual experience data and the real experience data according to the preset virtual sampling weight and the preset real sampling weight. Merge the extracted data and write it into a pre-allocated memory space to obtain the experience playback pool. The preset virtual sampling weight is 0.3 and the preset real sampling weight is 0.7.

[0112] S66. Repeat steps S62 to S65. Each time the process is repeated, increment the virtual simulation step count by one. Determine whether the virtual simulation step count is equal to the preset step count. If it is not equal to the preset step count, use the next virtual state as the new virtual simulation initial state and continue repeating. If it is equal to the preset step count, stop the simulation. The preset step count is 10 steps.

[0113] This implementation method introduces a cloud-isolated environment and a hybrid virtual-real experience generation mechanism as innovative technologies, which have significant differences and advantages compared to traditional single-real-environment interaction models and the standard DDPG algorithm. Traditional reinforcement learning relies solely on trial and error in the real environment to gain experience. In cloud mobile phone data synchronization, a decision error can easily lead to data version conflicts and system paralysis, resulting in extremely high trial and error costs. On the other hand, the single experience stream of the standard DDPG algorithm is prone to getting trapped in local optima, and its policy evaluation convergence is slow in high-concurrency dynamic scenarios, making it difficult to guarantee absolute data consistency.

[0114] This invention constructs a lossless sandbox that is completely synchronized with the real-world environment by overwriting the entire memory and hard disk of a cloud-isolated environment. This transfers high-risk simulation and trial-and-error processes to the virtual space, completely eliminating data security risks inherent in the real environment. In particular, the virtual simulation and hybrid experience pool mechanism, consisting of S62 to S66, proactively explores 10 steps along the current strategy in the isolated environment, generating a large amount of non-destructive virtual experience. By assigning differentiated sampling weights of 0.3 to virtual experience and 0.7 to real experience, it fully utilizes the state space expansion and exploration gains brought by virtual simulation while ensuring the dominant role of real core experience in strategy evolution. This mechanism effectively overcomes the distribution offset problem of pure virtual simulation, significantly reducing trial-and-error costs, accelerating experience accumulation, and significantly enhancing the evaluation accuracy and robustness of the model's scheduling strategy under complex disturbances.

[0115] In this embodiment, S7 specifically includes:

[0116] S71. Randomly extract a preset number of mixed experience data from the experience replay pool, separate the current state feature matrix, mixed scheduling action instruction, current reward value and next state feature matrix from each mixed experience data, extract the synchronization time slice allocation and task execution sorting strategy as discrete actions from the mixed scheduling action instruction, and extract the bandwidth splitting ratio and synchronization protocol parameters as continuous actions. The preset number is 64.

[0117] S72. Input the next state feature matrix into the target actor network, perform matrix multiplication and activation function operations, output the next discrete action and the next continuous action, concatenate the next state feature matrix, the next discrete action and the next continuous action in sequence to form the target joint input vector, and input it into the target network of the dual-stream critic network. Perform matrix multiplication between the target joint input vector and the weight matrix of the fully connected layer in the target network and superimpose the bias vector. After calculation by the ReLU activation function, output the target evaluation value.

[0118] S73. Determine whether the currently extracted mixed experience data is real experience data or virtual experience data. If it is real experience data, extract the real composite reward value as the current step reward value. If it is virtual experience data, extract the virtual reward as the current step reward value. Add a preset discount factor to the current step reward value and multiply it by the target evaluation value to obtain the target time series difference target value. Subtract the current evaluation value output by the dual-stream commentator network from the target time series difference target value to obtain the time series difference error. The preset discount factor is 0.95.

[0119] S74. Input the current state feature matrix and discrete actions into the first-order scheduling value network of the dual-stream critic network, perform matrix multiplication and nonlinear activation operations to map the state features and discrete actions into a single value, and obtain the strategic value assessment value of task selection.

[0120] S75. The current state feature matrix, discrete actions and continuous actions are concatenated into a joint input vector. The joint input vector is then input into the second-stream execution efficiency network of the two-stream commentator network. Matrix multiplication and nonlinear activation operations are performed to map the joint input vector into a single value, thereby obtaining the tactical effectiveness evaluation value of resource allocation.

[0121] S76. Multiply the strategic value assessment value of the task selection by a preset strategic weight coefficient, multiply the tactical effectiveness assessment value of resource allocation by a preset tactical weight coefficient, add the two products to obtain a comprehensive assessment value, calculate the product of the comprehensive assessment value and the temporal difference error to obtain a gradient value, backpropagate the gradient value back to the dual-stream commentator network, and update the weight matrix and bias vector of each layer in the dual-stream commentator network layer by layer. The preset strategic weight coefficient is 0.4, and the preset tactical weight coefficient is 0.6.

[0122] S77. Calculate the policy gradient based on the updated weight matrix and bias vector of the dual-stream commentator network, backpropagate the policy gradient back to the actor network, and update the discrete mapping weight matrix, discrete mapping bias vector, continuous mapping weight matrix and continuous mapping bias vector in the actor network layer by layer to complete the iterative update of the scheduling policy.

[0123] This invention achieves precise iteration and efficient optimization of cloud phone hybrid scheduling strategies by introducing a parameterized action-decoupled dual-stream commentator network and a differentiated reward evaluation mechanism. Hybrid experience data is separated into discrete tasks and continuous resource actions, and a target evaluation value is output through target network deduction. Real or virtual rewards are dynamically extracted based on the data source, and temporal difference errors are calculated using a 0.95 discount factor. Innovatively, states and actions are input into the dual-stream network separately: the first stream focuses on evaluating the strategic value of task selection, while the second stream deeply measures the tactical effectiveness of resource allocation. A comprehensive evaluation value is obtained by weighting the two streams with weights of 0.4 and 0.6, respectively, and the gradient is calculated using the temporal difference error and backpropagated, simultaneously updating the parameters of the dual-stream commentator and actor networks. This invention effectively overcomes the problem of inaccurate evaluation in hybrid action spaces, significantly improving strategy convergence speed and decision robustness under the constraints of strong data consistency and high network concurrency.

[0124] The improved PA-DDPG algorithm of this invention is similar to the original PA-DDPG algorithm in that both retain the core architecture of the parameterized action deep deterministic policy gradient network, that is, both adopt the actor-critic framework, output mixed actions (discrete task selection and continuous resource parameters) through the actor network, and use the experience replay pool and the target network to learn off-track policies. Both also use iterative updates of network parameters based on temporal difference errors.

[0125] The difference lies in that this invention breaks the limitation of the original PA-DDPG algorithm, where a single commentator network directly evaluates the overall value of mixed actions, and introduces a two-stream commentator network based on parameterized action decoupling. In the original algorithm, a single commentator simply concatenates the state, discrete actions, and continuous actions for unified evaluation, which is prone to evaluation bias and gradient direction ambiguity due to differences in the spatial dimensions of mixed actions. In steps S74 and S75, this invention decouples the single network into a two-stream structure: the first-stream scheduling value network only takes the current state and discrete actions as input, focusing on evaluating the strategic value of task selection; the second-stream execution efficiency network takes the state, discrete actions, and continuous actions as input, deeply measuring the tactical efficiency of resource allocation. Subsequently, in step S76, heterogeneous weight coefficients of 0.4 and 0.6 are introduced to weight and sum the two-stream evaluation values, and the gradient is accurately calculated by combining temporal difference errors and backpropagated to update the network.

[0126] Based on the aforementioned improvements, the beneficial effects of this invention are as follows: Through parameterized action decoupling and a dual-stream evaluation mechanism, the improved PA-DDPG algorithm can decouple and independently evaluate the different characteristics of discrete tasks and continuous resources in cloud phone scheduling. This breaks the limitation of the original single commentator sharing the same set of parameters to process mixed actions, effectively alleviating the evaluation bias and gradient interference caused by the interweaving of high- and low-dimensional action spaces. This design significantly enhances the algorithm's evaluation accuracy and value judgment ability for mixed scheduling strategies, enabling more precise guidance for actor networks to optimize action output. Combined with a differentiated weight fusion mechanism, while improving the strategy convergence speed and decision precision, it effectively optimizes the gradient update direction, enhancing the system's robustness and adaptive response capability in scenarios with strong data consistency and high concurrency complex networks.

[0127] Example 1: To verify the feasibility of this invention in the field of cloud phone data synchronization scheduling, this invention was deployed in the intelligent data flow management platform of a large cross-cloud collaborative office service provider in Guangzhou. This platform handles cloud migration services for multiple government and enterprise clients in South China, connecting over 12,500 cloud phone instances of various types. The underlying distributed storage cluster contains 4,200 data shards, carrying the real-time interaction of massive amounts of office documents, instant messaging records, and multimedia stream data. The average daily data synchronization task triggering volume exceeds 8.5 million times, with a peak concurrent synchronization stream of 15,000 messages / second. The physical operating environment and network conditions are extremely variable, and the requirements for strong data consistency are stringent.

[0128] This cloud phone platform needs to process over 2.4TB of multi-dimensional data daily, including cloud-side application layer logs, edge operation sequences, physical node bandwidth and latency detection, and distributed storage shard version tags. Traditional data synchronization scheduling systems mainly rely on static heuristic allocation rules and post-event conflict repair mechanisms. When facing large-scale high-concurrency scenarios, they suffer from severe response delays, extremely high synchronization conflict rates, and an inability to dynamically adapt to edge-cloud network jitter and computing power fluctuations. Especially in extreme scenarios such as peak evening business surges and cross-regional cluster migrations, serious problems such as data version corruption, synchronization link congestion, and even data loss often occur. Post-event repairs not only consume a large amount of computing resources but also seriously affect the business continuity of government and enterprise customers.

[0129] In practical deployment, the method of this invention transforms the aforementioned multi-source heterogeneous data into a spatiotemporal state feature matrix, constructing a dynamic state map containing multi-dimensional attributes such as network bandwidth latency, end-side power and computing power, data sharding dependency topology, and logical timestamp offset. The feature matrix of the current task to be synchronized is extracted in real time and input into the improved PA-DDPG algorithm. The discrete action head of the actor network outputs the task execution sorting strategy, while the continuous action head outputs the bandwidth splitting ratio and synchronization protocol parameters, achieving joint decision-making for hybrid scheduling actions. Simultaneously, accelerated simulation and deduction of hybrid instructions are performed in a cloud-isolated environment. By comparing logical timestamps, potential data version conflicts are intercepted, and intercepted instructions are downgraded to regular synchronization rules based on priority locks, thus ensuring absolute data consistency before execution. In the strategy iteration phase, a dual-stream commentator network based on parameterized action decoupling is introduced to evaluate the strategic value of task selection and the tactical effectiveness of resource allocation, respectively. Adaptive weight correction and strategy optimization are performed using a virtual-real hybrid experience replay pool. Table 1 below shows the comparison data between the method of this invention and existing traditional heuristic scheduling methods in cloud mobile phone data synchronization tasks over a three-month period:

[0130] Table 1. Comparison of performance and consistency assurance between the present invention and traditional methods.

[0131]

[0132] Based on the comparative data shown in Table 1, it can be seen that the scheduling method proposed in this invention, which is based on hybrid virtual and real experience and dual-stream commentator network, shows significant performance advantages over traditional methods in cloud mobile phone data synchronization scenarios. In particular, it has achieved comprehensive improvements in key indicators such as data consistency guarantee, synchronization response timeliness, bandwidth resource utilization, and adaptive optimization in extreme environments.

[0133] Regarding data consistency assurance, this invention achieves a 0% conflict rate in all four typical synchronization scenarios, completely eliminating the frequent consistency disasters in traditional methods. For example, in the "sudden network jitter" scenario, traditional methods rely on post-event conflict detection and rollback mechanisms. Due to network packet loss causing logical clock misalignment, the conflict rate is as high as 3.21%. In contrast, this invention, through accelerated simulation and rigorous comparison of logical timestamps in a cloud-isolated environment, accurately intercepts all potential conflicts and automatically degrades them before scheduling instructions are issued, truly preventing problems before they occur and achieving absolute data consistency.

[0134] Regarding synchronization response timeliness, this invention achieves joint optimization of hybrid scheduling actions through an improved PA-DDPG algorithm, significantly shortening synchronization time. In four scenarios, the average synchronization time is drastically reduced from over 646 milliseconds using traditional methods to approximately 247 milliseconds, a nearly three-fold speedup. Particularly in the "evening peak business" scenario, the 95th percentile time of traditional methods reaches as high as 1245 milliseconds, while this invention achieves only 462 milliseconds, greatly alleviating link congestion under high concurrency pressure.

[0135] In terms of bandwidth utilization and degradation blocking, this invention also demonstrates superior intelligent decision-making capabilities. Traditional methods experience a sudden drop in bandwidth utilization to 35.2% under unforeseen circumstances, while this invention dynamically adjusts protocol parameters and segmentation ratios to maintain a stable utilization rate above 71.4%. Furthermore, regarding potential conflicts identified during simulations, this invention achieves an average degradation blocking accuracy rate of over 98%, precisely mitigating the risk of data contamination.

[0136] Overall, this invention achieves precise decision-making for mixed scheduling actions and absolute interception of data conflicts in core scenarios such as "daily off-peak hours," "evening peak business hours," "cross-regional migration," and "sudden network jitter" by integrating isolation pre-simulation verification and dual-stream decoupling optimization mechanisms. Its consistency assurance and resource adaptability are particularly significant in extreme network jitter and cross-regional migration scenarios, filling the technical shortcomings of traditional synchronization methods in strong consistency control under dynamic and complex environments.

[0137] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A cloud mobile phone data synchronization scheduling method based on reinforcement learning, characterized in that, Includes the following steps: S1. Collect the dependencies, network bandwidth, latency, cloud container computing power, and terminal battery status of the data shards to be synchronized in the cloud phone instance, and preprocess them to obtain the spatiotemporal state feature matrix. S2. Input the spatiotemporal state feature matrix into the improved PA-DDPG algorithm, introduce a dual-stream critic network based on parameterized action decoupling, perform joint mapping through the actor network, output discrete task selection probability distribution and continuous resource parameter adjustment vector, take the discrete task with the highest probability as the synchronization time slice allocation and task execution sorting strategy, and extract the corresponding bandwidth splitting ratio and synchronization protocol parameters to solve for the hybrid scheduling action instructions. S3. Reproduce the current synchronous state in the cloud-isolated environment, accelerate the execution of mixed scheduling action commands in this isolated environment, track data flow and version changes, and obtain the simulated data flow status and version evolution trajectory. S4. Based on the simulated data flow status, compare the logical timestamps of each data shard in the simulated version evolution trajectory. If there is no conflict, the consistency check is determined to be passed and a release instruction is output. Otherwise, the check is determined to be failed and an interception instruction is output. The system is downgraded to a regular synchronization rule based on priority locks to obtain the final scheduling execution scheme. S5. Track the execution of the final scheduling plan in the real environment, count the synchronization time, traffic consumption and end-side lag rate, and assess whether the degradation interception is triggered. Calculate the weighted average of various business indicators and interception events to obtain the real composite reward value. S6. Starting from the real environment state after the final scheduling execution plan is executed, the cloud isolation environment is called to continue the deduction along the current actor network strategy, collect the virtual state, virtual action and virtual reward of each step of the deduction, generate virtual experience data, and store the real experience data and virtual experience data into the experience replay pool. S7. Extract mixed experience data from the experience replay pool, calculate the temporal difference error based on the real composite reward value and the virtual reward, and backpropagate it. Through the first-level scheduling value network in the dual-stream commentator network, input state and discrete actions to evaluate the strategic value of task selection, and the second-level execution effectiveness network, input state, discrete actions and continuous parameters to evaluate the tactical effectiveness of resource allocation, generate a comprehensive evaluation value and calculate the gradient, update the parameters of the dual-stream commentator network and the actor network, and complete the iterative update of the scheduling strategy.

2. The cloud mobile phone data synchronization scheduling method based on reinforcement learning according to claim 1, characterized in that, S1 specifically includes: S11. Read the synchronization logs and resource monitoring interface of the cloud phone instance in real time, extract the dependency list between the data shards to be synchronized, and obtain the current network bandwidth value, network latency value, cloud container computing power utilization rate and terminal power percentage. S12. Take the data fragments to be synchronized as nodes of the graph, take the dependencies between data fragments as edges of the graph, construct a data dependency topology graph and extract the corresponding adjacency matrix, and concatenate the network bandwidth value, network latency value, cloud container computing power utilization rate and end-side power percentage into a one-dimensional physical state vector in a set order. S13. Concatenate the adjacency matrix of the data-dependent topology graph with the one-dimensional physical state vector to obtain the concatenated feature matrix. Perform principal component analysis on the concatenated feature matrix to reduce the dimensionality and remove feature dimensions with a variance contribution rate lower than a preset threshold to obtain the spatiotemporal state feature matrix.

3. The cloud mobile phone data synchronization scheduling method based on reinforcement learning according to claim 1, characterized in that, S2 specifically includes: S21. Input the spatiotemporal state feature matrix into the actor network of the improved PA-DDPG algorithm, perform a linear transformation on the spatiotemporal state feature matrix and the first fully connected layer of the actor network and input it into the ReLU activation function, set the negative elements to zero and keep the positive elements to obtain the basic state feature vector. S22. Input the basic state feature vector into the discrete action head of the actor network, read the discrete mapping weight matrix and discrete mapping bias vector randomly generated during the initialization stage of the discrete action head, perform matrix multiplication on the basic state feature vector and the discrete mapping weight matrix and superimpose the discrete mapping bias vector to obtain the discrete linear output. S23. Perform an exponential operation with the natural constant e as the base for each value in the discrete linear output to obtain each exponential result. Divide each exponential result by the sum of all exponential results to obtain the probability value of each data slice to be synchronized. Compare all probability values ​​and select the data slice corresponding to the probability value with the largest value as the synchronization time slice allocation and task execution sorting strategy. S24. Construct a zero vector with a dimension equal to the total number of data fragments to be synchronized. Set the element at the position of the data fragment corresponding to the synchronization time slice allocation and task execution sorting strategy in the zero vector to one, and keep the other elements at zero. Convert the synchronization time slice allocation and task execution sorting strategy into a discrete one-hot encoded vector. Multiply the basic state feature vector with the corresponding element in the discrete one-hot encoded vector one by one to obtain the condition constraint feature vector. S25. Input the conditional constraint feature vector into the continuous action head of the actor network, read the continuous mapping weight matrix and continuous mapping bias vector randomly generated during the initialization phase of the continuous action head, perform matrix multiplication on the conditional constraint feature vector and the continuous mapping weight matrix, and superimpose the continuous mapping bias vector to obtain a continuous linear output. S26. Input the continuous linear output into the Tanh activation function, compress each value in the continuous linear output to an open interval between negative one and one, and obtain the compressed continuous value. Multiply the first value in the compressed continuous value by the preset bandwidth upper limit, multiply the second value in the compressed continuous value by the preset protocol parameter upper limit, and reassemble the two product results in the original order to obtain the continuous resource parameter adjustment vector. S27. Extract the first value from the continuous resource parameter adjustment vector as the bandwidth allocation ratio, extract the second value from the continuous resource parameter adjustment vector as the synchronization protocol parameter, and concatenate the synchronization time slice allocation and task execution sorting strategy, bandwidth allocation ratio and synchronization protocol parameter in sequence into a one-dimensional action vector to obtain the hybrid scheduling action instruction.

4. The cloud mobile phone data synchronization scheduling method based on reinforcement learning according to claim 1, characterized in that, S3 specifically includes: S31. Read the memory snapshot and data shard storage status of the current cloud phone instance in the cloud isolation environment, load the memory snapshot and data shard storage status into the running memory of the isolation container, overwrite the initial state in the isolation container, and obtain the reproduced current synchronization state. S32. Transmit the hybrid scheduling action command to the control interface of the isolation container, start the preset acceleration clock, and execute the synchronization time slice allocation and task execution sorting strategy, bandwidth splitting ratio and synchronization protocol parameters in the isolation container according to the time step of the acceleration clock. S33. During each step of execution, monitor the read and write operations of data fragments in the isolation container, identify the data fragments that have read operations as source fragments, identify the data fragments that have written operations as target fragments, record the amount of data transmitted from the source fragment to the target fragment, and obtain a single-step data flow record. S34. While monitoring read and write operations, extract the system logic clock value when each write operation is completed, and bind it with the data fragment identifier to form a data version tag. Arrange all data version tags in the order of operation execution to obtain a single-step version evolution record. S35. Concatenate the single-step data flow records of each step of the accelerated clock in chronological order to obtain the simulated data flow state after the execution of the hybrid scheduling action. Concatenate the single-step version evolution records of each step in chronological order to obtain the version evolution trajectory after the execution of the hybrid scheduling action.

5. The cloud mobile phone data synchronization scheduling method based on reinforcement learning according to claim 1, characterized in that, S4 specifically includes: S41. Extract the data fragment identifiers of each data fragment that has been written from the simulated data flow state after the execution of the hybrid scheduling action, and read the data version tags with time attributes bound to each data fragment identifier from the version evolution trajectory after the execution of the hybrid scheduling action. Parse the system logical clock value in each data version tag to obtain the logical timestamp of each data fragment. S42. Logical timestamps with the same data fragment identifier are divided into the same comparison group. Within the same comparison group, the absolute value of the difference between any two logical timestamps is calculated. If the absolute value of the difference is equal to zero, it is determined that there is no conflict between the logical timestamps in the comparison group. If the absolute value of the difference is greater than zero, it is determined that there is a conflict between the logical timestamps in the comparison group. S43. Iterate through the judgment results of all comparison groups. If all comparison groups are judged to have no logical timestamp conflict, the consistency check is passed and a release instruction is generated. If any comparison group is judged to have a logical timestamp conflict, the check is failed and an interception instruction is generated. S44. When a release instruction is generated, the hybrid scheduling action instruction is directly used as the final scheduling execution scheme to ensure absolute data consistency. When an interception instruction is generated, the hybrid scheduling action instruction is abandoned, the preset priority of all data fragments to be synchronized is extracted, the highest priority data fragment is allocated a synchronization time slice and a mutex lock is applied, and the remaining data fragments are synchronized in descending order of preset priority after the mutex lock is released. S45. Degrade the rules that apply mutex locks and are executed in order of preset priority to regular synchronization rules based on priority locks, and output the final scheduling execution scheme as the final scheduling execution scheme to ensure absolute data consistency.

6. The cloud mobile phone data synchronization scheduling method based on reinforcement learning according to claim 1, characterized in that, S5 specifically includes: S51. Execute the final scheduling execution plan in a real environment, record the system time at the start of execution and the system time to complete the synchronization of all data shards, calculate the difference between the system time to complete the system time and the system time to start the system time, and obtain the synchronization time. S52. During the execution of the final scheduling plan in a real environment, read the number of bytes of data packets sent and received by the network interface, calculate the difference between the number of bytes of data packets after execution and before execution, and obtain the traffic consumption. S53. During the execution of the final scheduling plan in a real environment, the rendering frame rate on the client side is collected at fixed time intervals. The percentage of collections that are lower than the preset rendering frame rate threshold is calculated out of the total number of collections to obtain the client side stuttering rate. S54. Read the generation record of the final scheduling execution plan. If the generation record contains a regular synchronization rule based on priority lock, it is determined that a downgrade interception has been triggered and the value is set to 1. If the generation record contains a mixed scheduling action instruction, it is determined that a downgrade interception has not been triggered and the value is set to 0. The interception event value is obtained. S55. Multiply the synchronization time by the preset time weighting coefficient, multiply the traffic consumption by the preset traffic weighting coefficient, multiply the terminal lag rate by the preset lag weighting coefficient, multiply the interception event value by the preset interception penalty weighting coefficient, and sum the four product results to obtain the true composite reward value.

7. The cloud mobile phone data synchronization scheduling method based on reinforcement learning according to claim 1, characterized in that, S6 specifically includes: S61. After executing the final scheduling execution plan, read the memory data and hard disk data of the cloud mobile phone instance in the real environment and send them to the cloud isolation environment. In the cloud isolation environment, overwrite the running memory and storage space of the isolation container to obtain the initial state of the virtual simulation. S62. Perform feature splicing and dimensionality reduction on the initial state of the virtual simulation to obtain the virtual state feature matrix. Input the virtual state feature matrix into the current actor network, perform linear transformation, matrix multiplication and weighted splicing operations, output the virtual synchronization time slice allocation and task execution sorting strategy, the virtual bandwidth splitting ratio and the virtual synchronization protocol parameters, and splice them into a one-dimensional vector to obtain the virtual hybrid scheduling action instructions. S63. Execute virtual hybrid scheduling action instructions in the cloud isolation environment, record the virtual time and virtual data transmission volume from the start to the end of the execution, read the memory data and disk data of the isolation container after the execution is completed to obtain the virtual next state, add the virtual time and virtual data transmission volume in a weighted manner to obtain the virtual reward, and concatenate the virtual state feature matrix, virtual hybrid scheduling action instructions, virtual reward and virtual next state in sequence into a quadruple and record it as a virtual experience data. S64. The spatiotemporal state feature matrix input to the actor network, the hybrid scheduling action instruction output by the actor network, the real composite reward value, and the real environmental state execution features after executing the final scheduling execution plan are concatenated and dimensionality reduced to obtain the post-execution state feature matrix. The spatiotemporal state feature matrix, the hybrid scheduling action instruction, the real composite reward value, and the post-execution state feature matrix are concatenated in sequence into a quadruple and recorded as a real experience data. S65. Assign a preset virtual sampling weight to the virtual experience data and a preset real sampling weight to the real experience data. Randomly extract data from the virtual experience data and the real experience data according to the preset virtual sampling weight and the preset real sampling weight. After merging the extracted data, write it into the pre-allocated memory space to obtain the experience playback pool. S66. Repeat steps S62 to S65. Each time the process is repeated, increment the virtual simulation step count by one. Determine whether the virtual simulation step count is equal to the preset step count. If it is not equal to the preset step count, use the next virtual state as the new virtual simulation initial state and continue repeating. If it is equal to the preset step count, stop the simulation.

8. The cloud mobile phone data synchronization scheduling method based on reinforcement learning according to claim 1, characterized in that, Specifically, S7 includes: S71. Randomly extract a preset number of mixed experience data from the experience replay pool, separate the current state feature matrix, mixed scheduling action instruction, current reward value and next state feature matrix from each mixed experience data, extract the synchronization time slice allocation and task execution sorting strategy as discrete actions from the mixed scheduling action instruction, and extract the bandwidth splitting ratio and synchronization protocol parameters as continuous actions. S72. Input the next state feature matrix into the target actor network, perform matrix multiplication and activation function operations, output the next discrete action and the next continuous action, concatenate the next state feature matrix, the next discrete action and the next continuous action in sequence to form the target joint input vector, and input it into the target network of the dual-stream critic network. Perform matrix multiplication between the target joint input vector and the weight matrix of the fully connected layer in the target network and superimpose the bias vector. After calculation by the ReLU activation function, output the target evaluation value. S73. Determine whether the currently extracted mixed experience data is real experience data or virtual experience data. If it is real experience data, extract the real composite reward value as the reward value of the current step. If it is virtual experience data, extract the virtual reward as the reward value of the current step. Add the preset discount factor to the reward value of the current step and multiply it by the target evaluation value to obtain the target time series difference target value. Subtract the current evaluation value output by the dual-stream commentator network from the target time series difference target value to obtain the time series difference error. S74. Input the current state feature matrix and discrete actions into the first-order scheduling value network of the dual-stream critic network, perform matrix multiplication and nonlinear activation operations to map the state features and discrete actions into a single value, and obtain the strategic value assessment value of task selection. S75. The current state feature matrix, discrete actions and continuous actions are concatenated into a joint input vector. The joint input vector is then input into the second-stream execution efficiency network of the two-stream commentator network. Matrix multiplication and nonlinear activation operations are performed to map the joint input vector into a single value, thereby obtaining the tactical effectiveness evaluation value of resource allocation. S76. Multiply the strategic value assessment value of task selection by the preset strategic weight coefficient, multiply the tactical effectiveness assessment value of resource allocation by the preset tactical weight coefficient, add the two product results to obtain the comprehensive assessment value, calculate the product of the comprehensive assessment value and the temporal difference error to obtain the gradient value, backpropagate the gradient value back to the two-stream commentator network, and update the weight matrix and bias vector of each layer in the two-stream commentator network layer by layer. S77. Calculate the policy gradient based on the updated weight matrix and bias vector of the dual-stream commentator network, backpropagate the policy gradient back to the actor network, and update the discrete mapping weight matrix, discrete mapping bias vector, continuous mapping weight matrix and continuous mapping bias vector in the actor network layer by layer to complete the iterative update of the scheduling policy.

Images