Remote copy mode control method, apparatus and device

Abstract

This application provides a method, apparatus, and device for remote replication mode control. The method includes: collecting multi-dimensional state information associated with a remote replication task and obtaining service context information corresponding to the remote replication task; dynamically fusing the multi-dimensional state information based on the service context information to obtain current state features; obtaining topology information and historical state information of the data transmission network used to execute the remote replication task, and predicting the state of the network links based on the topology information and historical state information to obtain future network state features; determining the replication mode control actions for the remote replication task using a reinforcement learning policy network based on the current state features and future network state features, and controlling the replication mode of the remote replication task based on the replication mode control actions. The method provided in this application can improve the accuracy of replication mode control.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a remote copy mode control method, apparatus and device. Background Technology

[0002] Remote replication is a method of copying data from a primary storage device to a remote storage device, enabling business recovery using data from the remote storage device in the event of a primary device failure. Remote replication typically includes synchronous replication mode and asynchronous replication mode. In synchronous replication mode, the write operation is confirmed after data is written on both the primary and remote devices. In asynchronous replication mode, the write operation is confirmed after data is written on the primary device, and the data is transferred to the remote device at a later time.

[0003] In related technologies, servers can use a fixed threshold strategy based on network metrics to control the replication mode of remote replication tasks. Specifically, the server can collect the link latency of the data transmission network, compare the link latency with a preset high latency threshold or low latency threshold, and based on the comparison results of multiple consecutive collection cycles, switch the remote replication task from synchronous replication mode to asynchronous replication mode, or vice versa. However, the accuracy of determining the replication mode of remote replication tasks using the above-mentioned method based on network metrics is low when network status, storage status, or service status changes. Summary of the Invention

[0004] This application provides a remote copy mode control method, apparatus, and device to solve the problem of low accuracy in determining the copy mode of a remote copy task.

[0005] In a first aspect, embodiments of this application provide a remote replication mode control method, including:

[0006] Collect multi-dimensional status information associated with remote replication tasks and obtain the business context information corresponding to remote replication tasks;

[0007] The multi-dimensional state information is dynamically fused based on business context information to obtain the current state characteristics.

[0008] The topology and historical status information of the data transmission network are acquired, and the status of the network links is predicted based on the topology and historical status information to obtain the future network status characteristics; the data transmission network is used to perform remote replication tasks.

[0009] Based on the current state characteristics and future network state characteristics, a reinforcement learning strategy network is used to determine the replication mode control actions for remote replication tasks, and the replication mode of remote replication tasks is controlled based on the replication mode control actions.

[0010] By dynamically fusing multi-dimensional state information associated with remote replication tasks by combining business context information, and predicting network link states based on the topology and historical state information of the data transmission network, the reinforcement learning policy network can determine replication mode control actions by comprehensively considering current state characteristics and future network state characteristics. This enables replication mode control to adapt to current operating conditions, business needs, and future network state changes, thereby improving the accuracy and timeliness of remote replication mode control.

[0011] In one possible implementation, multi-dimensional state information is dynamically fused based on business context information to obtain current state features, including:

[0012] Normalize each state index in the multidimensional state information to obtain the normalized state index corresponding to each state index.

[0013] Determine business context adjustment information based on business context information, and obtain the basic importance parameters and business context sensitive parameters corresponding to each normalized state indicator;

[0014] Based on the index values, basic importance parameters, business context adjustment information, and business context sensitive parameters of each normalized state index, a weighting function is used to calculate the dynamic weights corresponding to each normalized state index.

[0015] The current state characteristics are obtained by weighting and fusing the normalized state indicators according to the dynamic weights.

[0016] By normalizing different status indicators and dynamically calculating the weights of each normalized status indicator in conjunction with business context adjustment information, basic importance parameters, and business context sensitive parameters, the contribution of different status indicators to the current status characteristics can be adjusted with changes in status and business context, thereby improving the accuracy of the current status characteristics in representing the overall operational status of the remote replication task.

[0017] In one possible implementation, the state of network links is predicted based on topology information and historical state information to obtain future network state characteristics, including:

[0018] Based on the topology information, network devices in the data transmission network are taken as nodes and network links as edges. The state information of each network link at multiple historical moments is used as the temporal features of the corresponding edges to obtain graph structure time series data.

[0019] Input graph-structured temporal data into a spatiotemporal graph convolutional network, and aggregate the neighbor features of each edge based on the normalized adjacency matrix to obtain the spatial features corresponding to each network link;

[0020] One-dimensional temporal convolution is performed on the spatial features of each network link at multiple historical moments to obtain spatiotemporal features.

[0021] By performing fully connected processing on the spatiotemporal features, the future network state features can be obtained.

[0022] By constructing the data transmission network as a graph-structured time-series data containing network devices, network links, and the historical states of network links, the spatial correlation between network links and the temporal variation features between multiple historical moments are extracted sequentially. This allows the obtained future network state features to comprehensively characterize the network topology correlation and the evolution trend of network link states, thereby improving the accuracy of predicting the future state of network links.

[0023] In one possible implementation, based on current state characteristics and future network state characteristics, a reinforcement learning policy network is used to determine the replication mode control actions for the remote replication task, including:

[0024] Retrieve the replication mode control actions of the remote replication task at the previous moment;

[0025] Input the current state features, future network state features, and the copy mode control action from the previous time step into the reinforcement learning policy network to obtain the copy mode control action and the decision confidence corresponding to the copy mode control action.

[0026] The replication mode control action is to maintain the current replication mode, switch to synchronous replication mode, or switch to asynchronous replication mode.

[0027] By inputting the replication mode control action from the previous time step, along with the current state features and future network state features, into the reinforcement learning policy network, the determination of the replication mode control action can comprehensively consider the current operating state, future network state changes, and historical action information, and simultaneously obtain the corresponding decision confidence level. This improves the matching degree between the replication mode control action and the operating state of the remote replication task, and provides a confidence basis for subsequent replication mode control.

[0028] In one possible implementation, controlling the replication mode of a remote replication task based on replication mode control actions includes:

[0029] Based on the decision confidence level corresponding to the replication mode control action, security gating is applied to the replication mode control action;

[0030] When the security gating process allows the execution of replication mode control actions, the replication mode of the remote replication task is controlled by executing these actions.

[0031] By applying security gating to replication mode control actions based on decision confidence, and controlling the replication mode of remote replication tasks only when the security gating is allowed, replication mode control actions can be pre-executed to reduce the risk of low-confidence replication mode control actions directly affecting remote replication tasks, thereby improving the security and stability of replication mode control.

[0032] In one possible implementation, after determining the replication mode control action, the method further includes:

[0033] Based on the running status information after executing the replication mode control action, determine the recovery point target satisfaction, application performance indicators, and resource efficiency indicators, and determine the replication mode switching result based on the replication mode control action;

[0034] The reward value is obtained by weighting the satisfaction of the recovery point target, application performance indicators, resource efficiency indicators, and the results of replication mode switching.

[0035] Based on the current state characteristics, future network state characteristics, replication mode control actions, and reward values, the dominance function estimation algorithm is used to determine the dominance estimate.

[0036] Based on the advantage estimate, a proximal policy optimization algorithm with truncation operation is used to update the reinforcement learning policy network.

[0037] The reward value is determined based on the satisfaction of the recovery point objective, application performance indicators, resource efficiency indicators, and the result of the replication mode switching after the replication mode control action is executed. The reinforcement learning policy network is updated based on the advantage estimate using a proximal policy optimization algorithm that includes truncation operation. This allows the reinforcement learning policy network to continuously adjust its decision-making strategy in combination with the actual execution feedback of the replication mode control action, thereby improving the adaptability and update stability of the reinforcement learning policy network.

[0038] Secondly, embodiments of this application provide a remote replication mode control device, including:

[0039] The acquisition module is used to collect multi-dimensional status information associated with the remote replication task and obtain the corresponding business context information of the remote replication task.

[0040] The fusion module is used to dynamically fuse multi-dimensional state information based on business context information to obtain the current state characteristics.

[0041] The processing module is used to acquire the topology information and historical status information of the data transmission network, and predict the status of the network links based on the topology information and historical status information to obtain the future network status characteristics. The data transmission network is used to execute remote replication tasks.

[0042] The control module is used to determine the replication mode control actions of the remote replication task based on the current state characteristics and future network state characteristics using a reinforcement learning strategy network, and to control the replication mode of the remote replication task based on the replication mode control actions.

[0043] Thirdly, embodiments of this application provide a remote replication mode control device, comprising:

[0044] At least one processor; and

[0045] A memory that is communicatively connected to at least one processor; wherein,

[0046] The memory stores instructions that can be executed by at least one processor, such that the at least one processor is able to perform the method shown in any of the first aspects.

[0047] Fourthly, embodiments of this application provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the method shown in any of the first aspects.

[0048] Fifthly, embodiments of this application provide a computer program product, including a computer program that, when executed by a processor, implements the method shown in any of the first aspects.

[0049] The remote replication mode control method, apparatus, and device provided in this application, by collecting multi-dimensional state information associated with the remote replication task and dynamically fusing it with business context information, can form current state characteristics that match the current operating state and business requirements of the remote replication task. By combining the topology information and historical state information of the data transmission network to predict the network link state, it can obtain future network state characteristics that characterize the subsequent trend of network changes. Furthermore, by using the current state characteristics and future network state characteristics together as the decision-making basis for the reinforcement learning policy network, the replication mode control action can simultaneously adapt to the current operating state, business requirements, and future network state changes, improving the accuracy and timeliness of remote replication mode control. Thus, it solves the problem that related technologies determine the replication mode based only on a single current network indicator or a fixed strategy, resulting in low accuracy of replication mode control. Attached Figure Description

[0050] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0051] Figure 1 A schematic diagram illustrating the application scenarios provided in the embodiments of this application;

[0052] Figure 2A flowchart illustrating a remote replication mode control method provided in an embodiment of this application;

[0053] Figure 3 A schematic diagram of the method for updating the reinforcement learning policy network provided in the embodiments of this application;

[0054] Figure 4 This is a schematic diagram of the structure of the remote copy mode control device provided in the embodiments of this application;

[0055] Figure 5 This is a schematic diagram of the structure of the remote copy mode control device provided in the embodiments of this application.

[0056] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0057] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0058] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0059] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with relevant laws, regulations and standards, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0060] To facilitate understanding, the following will be combined with... Figure 1 The application scenarios applicable to the embodiments of this application will be described.

[0061] Figure 1 This is a schematic diagram illustrating an application scenario provided in an embodiment of this application. Please refer to [link / reference]. Figure 1 This application scenario can include a source storage system 101, a destination storage system 102, a data transmission network 103, and a remote replication mode control device 104. The source storage system 101 transmits replication data to the destination storage system 102 via the data transmission network 103, according to the replication mode currently used by the remote replication task. The remote replication mode control device 104 can communicate with the monitoring components corresponding to the source storage system 101, the destination storage system 102, and the data transmission network 103 to obtain network status information, storage status information, and other operational information associated with the remote replication task. It can also obtain business context information from the remote replication task configuration, storage policy data, or the management system. Based on the obtained information, the remote replication mode control device 104 determines the replication mode control action and controls the remote replication task to maintain the current replication mode, switch to synchronous replication mode, or switch to asynchronous replication mode.

[0062] The above application scenario can be a Storage Area Network (SAN) disaster recovery scenario. The source storage system 101 and the destination storage system 102 can be deployed in different data centers and establish a remote replication relationship through the data transmission network 103. When the source storage system 101 fails or the source service cannot operate normally, the replicated data stored in the destination storage system 102 can be used for service recovery. The remote replication mode control device 104 can be deployed independently outside of the source storage system 101 and the destination storage system 102, or it can be deployed as part of the remote replication management function in the source storage system 101, the destination storage system 102, or a management device used to manage remote replication tasks. This application embodiment does not limit the specific deployment location of the remote replication mode control device 104.

[0063] Remote replication tasks can employ either synchronous or asynchronous replication modes. In synchronous replication mode, after completing local data processing, the source storage system 101 waits for the destination storage system 102 to complete its corresponding data processing before confirming the data write request. Synchronous replication reduces data discrepancies between the source and destination storage systems 101 and 102, but the data write process is susceptible to disruptions caused by factors such as link latency in the data transmission network 103. In asynchronous replication mode, after completing local data processing, the source storage system 101 can confirm the data write request and subsequently transmit the data to be replicated to the destination storage system 102. Asynchronous replication reduces the impact of cross-datacenter data transfer on data write processing, but a certain time lag may exist between the source and destination storage systems 101 and 102.

[0064] The data protection and application performance requirements of the services associated with different remote replication tasks may vary. Furthermore, the link latency, network link jitter, network link packet loss rate, and available bandwidth of the data transmission network 103, as well as the read / write status, cache status, and replication queue status of the storage system, may all change over time. Therefore, the appropriate replication mode for the same remote replication task may differ depending on its operating state and service context.

[0065] In related technologies, the replication mode of remote replication tasks is typically controlled based on a comparison between the current network status indicators and preset thresholds. For example, the current link latency can be collected at preset intervals. If the current link latency is higher than a preset high latency threshold for several consecutive collection periods, the remote replication task is switched from synchronous replication mode to asynchronous replication mode; if the current link latency is lower than a preset low latency threshold for several consecutive collection periods, the remote replication task is switched from asynchronous replication mode to synchronous replication mode; otherwise, the current replication mode is maintained.

[0066] However, the operational status of remote replication tasks is influenced not only by the current network status but also by the storage's operational status and business requirements. The same network status may correspond to different replication mode control requirements under different storage statuses or different business operating backgrounds. Judging solely based on the current network status is insufficient to reflect subsequent changes in the network link's status, potentially causing replication mode control actions to lag behind actual changes in the data transmission network. Therefore, related technologies that determine replication mode control actions based solely on a single or limited network status information may result in a mismatch between replication mode control actions and the actual operational requirements of the remote replication task, leading to low accuracy in remote replication mode control.

[0067] To address the aforementioned issues, this application provides a remote replication mode control method. First, multi-dimensional state information associated with the remote replication task is collected, and corresponding business context information is obtained. The multi-dimensional state information may include network state information, storage state information, and business state information; the business context information may include business time period information and business type information. The multi-dimensional state information is dynamically fused based on the business context information to obtain current state characteristics. Further, the topology information and historical state information of the data transmission network used to execute the remote replication task are obtained. Based on the topological relationship between network devices and network links, and the state changes of each network link at multiple historical moments, the state of the network links is predicted to obtain future network state characteristics. The current state characteristics and future network state characteristics are used as decision inputs to a reinforcement learning policy network, which determines the replication mode control actions for the remote replication task. By combining business context information with multi-dimensional state information associated with remote replication tasks, and obtaining future network state characteristics based on the topology and historical state information of the data transmission network, the determination of replication mode control actions can comprehensively consider the current operating state, business requirements, and future network state changes. This makes the replication mode control actions more aligned with the actual operating requirements of remote replication tasks, thereby improving the accuracy of remote replication mode control.

[0068] The method described in this application will now be illustrated through specific embodiments. It should be noted that the following embodiments may exist independently or in combination with each other; identical or similar content will not be repeated in different embodiments.

[0069] Figure 2 This is a flowchart illustrating a remote replication mode control method provided in an embodiment of this application. Please refer to... Figure 2 The method may include:

[0070] S201: Collect multi-dimensional status information associated with the remote replication task and obtain the business context information corresponding to the remote replication task.

[0071] The execution entity in this application embodiment can be a remote replication mode control device or a remote replication mode control apparatus installed within the remote replication mode control device. The remote replication mode control apparatus can be implemented through software or a combination of software and hardware. The remote replication mode control device can be a server.

[0072] In this embodiment, a remote replication task refers to a processing task that copies data from a source end to a destination end according to a preset replication relationship. A remote replication task can correspond to one or more data objects to be replicated and has a task identifier used to distinguish different remote replication tasks. Multidimensional state information refers to a set of state data collected from multiple technical dimensions related to the operation of the remote replication task, used to characterize the operating environment and operating state of the remote replication task at the current moment or within the current observation period. Business context information refers to information used to characterize the operating background, business constraints, and business requirements of the business corresponding to the remote replication task.

[0073] When collecting multi-dimensional status information, the data transmission environment, storage execution environment, and business operation environment associated with the remote replication task can be determined first based on the task identifier of the remote replication task. Status data can then be obtained from the data collection interface of the corresponding environment. Status data can be obtained through periodic collection, event-triggered reporting, active querying, or a combination of these methods. For continuously changing status data, continuous collection can be performed according to a preset collection period; for data generated when the status changes, collection can be performed after a status change event is detected. By associating status data from different sources with the same remote replication task, it is possible to avoid using status data from other remote replication tasks for the replication mode control of the current remote replication task.

[0074] The collected status data can carry task identifiers and collection time information. Based on the task identifier, status data belonging to the same remote replication task are grouped into corresponding multi-dimensional status information. Based on the collection time information, status data from different sources are time-correlated, ensuring that each status data point in the multi-dimensional status information corresponds to the same or similar observation period. For multiple status data points obtained within the same observation period, the status data corresponding to the current processing time can be selected, or the status data corresponding to that observation period can be formed according to a preset data aggregation method. This ensures that the collected multi-dimensional status information consistently represents the current operational status of the remote replication task.

[0075] Business context information can be obtained from business configuration data, business management data, or task policy data associated with the remote replication task. When obtaining business context information, the business corresponding to the remote replication task is determined based on the association between the remote replication task and the corresponding business, and the currently applicable business context information for that business is read. For cases where the same remote replication task is associated with multiple businesses, the business context information corresponding to each business can be obtained separately and organized according to the association between the remote replication task and each business, so that the business context information can reflect the current business operation background and business requirements corresponding to the remote replication task.

[0076] S202: Dynamically fuse multi-dimensional state information based on business context information to obtain the current state characteristics.

[0077] In this step, dynamic fusion processing refers to adaptively adjusting the decision contribution of different state data in the multidimensional state information based on the business operation background and business requirements represented by the business context information, and combining the adjusted multidimensional state information into feature data that can uniformly represent the current operating state of the remote replication task. The current state feature is the processing result obtained from dynamic fusion processing, used to reflect the comprehensive relationship between the data transmission environment, storage operation environment, and business operation requirements of the remote replication task in the current observation period.

[0078] During dynamic fusion processing, business context information can be associated with multidimensional state information belonging to the same remote replication task based on the task identifier of the remote replication task. For multidimensional state information obtained according to different collection cycles or from different data sources, the state data corresponding to the current processing time can be determined based on the collection time information. This ensures that the multidimensional state information participating in dynamic fusion processing corresponds to the currently applicable business context information in time, reducing data deviations caused by state data and business context information not belonging to the same operational stage.

[0079] When processing multidimensional state information based on business context information, the current business requirements reflected in the business context information are identified, and the relative contributions of different state data in the fusion processing are adjusted accordingly. For state data that is highly correlated with the current business requirements, the influence of that state data on the current state characteristics can be increased; for state data that is less correlated with the current business requirements, the influence of that state data on the current state characteristics can be decreased. The relative contribution changes dynamically with changes in business context information and multidimensional state information, enabling the current state characteristics obtained at different operational stages to adapt to changes in the business requirements corresponding to the remote replication task.

[0080] After dynamically adjusting the various state data, the multi-dimensional state information is combined to obtain the current state features. These current state features can be organized according to a preset data arrangement into feature data that can be invoked by subsequent processing, while maintaining the correlation between each state data point and its corresponding technical dimension. The current state features are updated as the multi-dimensional state information or business context information changes, characterizing the comprehensive operational status of the remote replication task at the current moment. By dynamically fusing multi-dimensional state information using business context information, the current state features not only reflect the actual operational status of the remote replication task but also embody the current needs of the corresponding business, providing a state basis that matches the current operating environment and business requirements for subsequent replication mode control.

[0081] S203: Obtain the topology and historical status information of the data transmission network, and predict the status of the network links based on the topology and historical status information to obtain the future network status characteristics. The data transmission network is used to perform remote replication tasks.

[0082] In this step, the data transmission network refers to the network that carries the data transmission process of the remote replication task and provides a data transmission channel between the source and destination ends of the remote replication task. Topology information is used to characterize network devices, network links, and their connections within the data transmission network. Historical state information refers to the network operating state data continuously collected and saved for the data transmission network over multiple observation periods prior to the current processing time. Future network state characteristics refer to the characteristic data predicted based on topology information and historical state information, used to characterize the operating state of the data transmission network in future observation periods.

[0083] In practice, the data transmission network used by the remote replication task is determined based on the task identifier, and topology information is read from the configuration data, management data, or operational monitoring data of the data transmission network. Topology information may include network device information, network link information, and the connection relationships between network devices and network links within the data transmission network. When the same data transmission network carries multiple remote replication tasks, topology information related to that remote replication task can be obtained based on the source, destination, and data transmission range of that task.

[0084] The system retrieves network link status data prior to the current processing time from the network operation monitoring records or historical status data storage area. Based on the link identifier, the historical status information is associated with the corresponding network link in the topology information. Based on the time information, the historical status information of the same network link is organized chronologically to form predictive input data that simultaneously reflects network connectivity and network status changes.

[0085] By utilizing topology information to determine the relationships between different network links and combining this with the historical state changes of each network link, the state of network links in the data transmission network can be predicted during future observation periods. In the prediction process, the historical state changes of a single network link itself, as well as the state changes of other network links with which it has a topological relationship, can be comprehensively analyzed to obtain the predicted results of the future operating state of each network link.

[0086] Based on the prediction results, future network state characteristics are obtained. These future network state characteristics can be represented using a preset data arrangement so that they can be input together with the current state characteristics into the subsequent reinforcement learning policy network. It should be noted that as topology information or historical state information is updated, network link state prediction can be re-executed, and the future network state characteristics can be updated accordingly. By combining the topology information and historical state information of the data transmission network to predict the future state of network links, subsequent replication mode control can not only rely on the current operating state but also consider the development trend of the data transmission network, obtaining corresponding predictive basis before the actual network state changes, thus improving the timeliness and accuracy of replication mode control.

[0087] S204: Based on the current state characteristics and future network state characteristics, a reinforcement learning strategy network is used to determine the replication mode control actions for the remote replication task, and the replication mode of the remote replication task is controlled based on the replication mode control actions.

[0088] In this step, the reinforcement learning policy network refers to a network model that determines the replication mode control actions corresponding to the remote replication task based on the input state features and configured policy parameters. Replication mode control actions refer to the control information used to maintain or change the current replication mode of the remote replication task. The reinforcement learning policy network learns from the replication mode control results under different operating states, thus establishing a corresponding decision relationship between the input state features and the replication mode control actions.

[0089] When determining the replication mode control action, the current and future network state characteristics after organization are input into the reinforcement learning policy network. The reinforcement learning policy network processes the input characteristics based on the established policy parameters, analyzes the impact of the current operating state, business requirements, and future network state on the remote replication task, and outputs a replication mode control action that matches the current input characteristics. A correspondence is established between the replication mode control action and the remote replication task to ensure that the replication mode control action output by the reinforcement learning policy network applies to the corresponding remote replication task.

[0090] Subsequently, based on the control content indicated by the replication mode control action, the current replication mode used by the remote replication task is maintained, or the current replication mode is adjusted to the replication mode corresponding to the replication mode control action. After the replication mode control is completed, the remote replication task performs data replication between the source and destination ends according to the controlled replication mode. During the continuous operation of the remote replication task, the current state characteristics and future network state characteristics can be reacquired according to a preset decision cycle, and a reinforcement learning strategy network is used to determine the replication mode control action corresponding to the updated state characteristics, so that the replication mode of the remote replication task can be adjusted accordingly with changes in the running state and future network state.

[0091] By using both current state characteristics and future network state characteristics as the decision-making basis for the reinforcement learning policy network, the replication mode control process can reflect both the current operational needs of the remote replication task and the future changing trends of the data transmission network. This reduces decision-making bias caused by relying solely on the current single state for replication mode control, and improves the accuracy and timeliness of replication mode control.

[0092] This application provides a remote replication mode control method, including: collecting multi-dimensional state information associated with a remote replication task and obtaining business context information corresponding to the remote replication task; dynamically fusing the multi-dimensional state information according to the business context information to obtain current state features; obtaining topology information and historical state information of the data transmission network used to execute the remote replication task, and predicting the state of the network links according to the topology information and historical state information to obtain future network state features; determining the replication mode control action of the remote replication task using a reinforcement learning strategy network according to the current state features and future network state features, and controlling the replication mode of the remote replication task based on the replication mode control action. In this embodiment, by dynamically fusing the multi-dimensional state information associated with the remote replication task with business context information, current state features that reflect the current operating state and business requirements are obtained. Furthermore, by combining the topology information and historical state information of the data transmission network to predict the network link state, future network state features characterizing the subsequent trend of network changes are obtained. Then, replication mode control actions are determined based on the current state features and future network state features, enabling replication mode control to simultaneously consider the current operating state, business requirements, and future network state changes, thereby improving the accuracy and timeliness of remote replication mode control.

[0093] In one possible implementation, the multi-dimensional state information is dynamically fused based on business context information to obtain the current state features. This includes: normalizing each state indicator in the multi-dimensional state information to obtain a normalized state indicator corresponding to each state indicator; determining business context adjustment information based on the business context information and obtaining the basic importance parameters and business context sensitive parameters corresponding to each normalized state indicator; calculating the dynamic weight corresponding to each normalized state indicator using a weighting function based on the indicator value, basic importance parameters, business context adjustment information, and business context sensitive parameters; and performing weighted fusion of each normalized state indicator based on the dynamic weights to obtain the current state features.

[0094] The system comprises three components: business context adjustment information, basic importance parameters, and business context sensitivity parameters. Business context adjustment information characterizes the moderating effect of business context information on the importance of each normalized state indicator. Basic importance parameters characterize the basic importance of the corresponding normalized state indicator in remote replication mode control. Business context sensitivity parameters characterize the degree to which the corresponding normalized state indicator is affected by changes in business context information. Each state indicator may include network link latency, network link jitter, network link packet loss rate, available bandwidth ratio, number of I / O operations per second for read operations, number of I / O operations per second for write operations, cache hit rate, replication queue depth, business criticality information, and recovery point information corresponding to the current replication mode. Since different state indicators have different dimensions and numerical ranges, they can be normalized according to their corresponding reference numerical ranges, mapping each state indicator to a unified preset numerical range, enabling different state indicators to participate in dynamic weight calculation under a unified numerical scale. Basic importance parameters and business context sensitivity parameters can be pre-configured and associated with their corresponding normalized state indicators.

[0095] In practical processing, each status indicator is normalized, and business context adjustment information is determined based on the current business context information. For example, business context information may include business time period information and business type information. When the business corresponding to the remote replication task enters a critical business period, the business context adjustment information can enhance the role of business criticality information in dynamic fusion processing. Further, the indicator values ​​of each normalized status indicator, the corresponding basic importance parameters, the business context adjustment information, and the corresponding business context sensitivity parameters are input into a weighting function to calculate the dynamic weights corresponding to each normalized status indicator. In one implementation, the weighting function can be a weighted soft maximum function, which characterizes the current state and basic importance of the corresponding status indicator based on the indicator values ​​and basic importance parameters of the normalized status indicator, and adjusts the importance of the corresponding status indicator under the current business context information using the business context adjustment information and business context sensitivity parameters to obtain the dynamic weights corresponding to each normalized status indicator.

[0096] Subsequently, the corresponding normalized state indicators are weighted and fused according to their dynamic weights, and the weighted and fused data is organized according to the correspondence between the normalized state indicators to obtain the current state characteristics. When the business context information or the indicator values ​​of each state indicator change, the dynamic weights corresponding to each normalized state indicator can change accordingly, so that the current state characteristics can dynamically reflect the relative contribution of network state information, storage state information, and business state information in the current business operation context.

[0097] By adopting the above method, the state indicators of different dimensions are normalized, and the adjustment effect of business context information on the importance of each state indicator is introduced into the dynamic weight calculation. This allows the contribution of different state indicators to the current state characteristics to be dynamically adjusted according to the running status of the remote replication task and the business running background, thereby improving the accuracy of the current state characteristics in representing the comprehensive running status of the remote replication task.

[0098] Building upon the aforementioned embodiments, the state of network links is further predicted based on topology information and historical state information to obtain future network state features. This includes: using topology information, taking network devices in the data transmission network as nodes and network links as edges, and using the state information of each network link at multiple historical moments as the temporal features of the corresponding edges to obtain graph-structured temporal data; inputting the graph-structured temporal data into a spatiotemporal graph convolutional network, and aggregating the neighboring edge features of each edge based on a normalized adjacency matrix to obtain the spatial features corresponding to each network link; performing one-dimensional temporal convolution processing on the spatial features corresponding to each network link at multiple historical moments to obtain spatiotemporal features; and performing fully connected processing on the spatiotemporal features to obtain future network state features.

[0099] Specifically, the normalized adjacency matrix is ​​used to characterize the adjacency relationships between network links determined based on the topological information of the data transmission network. Specifically, the adjacent edges corresponding to each network link can be determined according to the connection relationship between network devices and network links, and the adjacency matrix used to characterize the adjacency relationships of network links is normalized. The spatiotemporal graph convolutional network can be composed of graph convolutional layers and temporal convolutional layers. The graph convolutional layers are used to extract the spatial associations between adjacent network links at the same time, and the temporal convolutional layers are used to extract the dependencies of network link states as they change over historical time periods.

[0100] In practical implementation, a network graph model is first established based on the topology information, and the state vector of each network link at multiple consecutive historical moments is associated with its corresponding edge, forming graph-structured time-series data containing node connection relationships and edge temporal information. Then, this graph-structured time-series data is input into a spatiotemporal graph convolutional network, where a normalized adjacency matrix is ​​used to weighted aggregate the edge features adjacent to the target network link, generating spatial features reflecting the coupling relationships of the local network. Next, the spatial features corresponding to the same network link at multiple historical moments are processed by one-dimensional temporal convolution, performing convolution operations along the time dimension to extract short-term and medium-term temporal patterns in the network link state evolution, obtaining spatiotemporal features. Finally, the spatiotemporal features are input into a fully connected layer for mapping, and the output is a prediction result serving as the future network state features.

[0101] The prediction results can simultaneously characterize the topological associations and historical evolution trends of network links, and serve as input for subsequent replication mode control. Since spatial features are obtained by aggregating adjacent network links and temporal features are obtained by convolution of continuous historical states, the output future network state features can reflect the future latency, congestion, or packet loss trends of network links, providing a predictive basis for mode selection in remote replication tasks.

[0102] By adopting the above method, future network state characteristics no longer rely solely on a single historical threshold, but are inferred by comprehensively considering topological relationships and network link states at multiple time points. Therefore, it has a stronger ability to characterize local network link fluctuations and changes in the correlation between adjacent network links. When conducting subsequent control based on this prediction result, the ability to proactively identify changes in network link state can be improved, and the stability and adaptability of remote replication mode control can be enhanced.

[0103] In one possible implementation, based on the current state characteristics and future network state characteristics, a reinforcement learning policy network is used to determine the replication mode control action of the remote replication task, including: obtaining the replication mode control action of the remote replication task at the previous time step; inputting the current state characteristics, future network state characteristics, and the replication mode control action at the previous time step into the reinforcement learning policy network to obtain the replication mode control action and the decision confidence corresponding to the replication mode control action; the replication mode control action is to maintain the current replication mode, switch to synchronous replication mode, or switch to asynchronous replication mode.

[0104] Specifically, the replication mode control action from the previous time step is used to characterize the historical action information of the remote replication task at the previous decision time step. This information is input into the reinforcement learning policy network in the form of action identifiers or action vectors, enabling the current decision to be associated with the replication mode control action from the previous decision time step. The decision confidence is used to characterize the degree of certainty of the reinforcement learning policy network regarding the determined replication mode control action, and can be obtained based on the action probability corresponding to the determined replication mode control action.

[0105] In practical implementation, the current state features, future network state features, and the copy mode control action from the previous time step can be vectorized and combined in a preset order before being input into the reinforcement learning policy network. The copy mode control action from the previous time step can be converted into a corresponding action identifier or action vector depending on whether the current copy mode is maintained, a synchronous copy mode is switched to, or an asynchronous copy mode is switched to. The reinforcement learning policy network performs joint processing based on the above inputs, outputting the action probability corresponding to each candidate copy mode control action, and determines the copy mode control action and the corresponding decision confidence based on each action probability. For example, the candidate copy mode control action with the highest action probability can be selected as the copy mode control action, and the action probability corresponding to this copy mode control action can be used as the decision confidence. For the case implemented using deep reinforcement learning, the reinforcement learning policy network can be trained based on historical interaction samples to learn the correspondence between different state combinations and copy mode control actions.

[0106] By using the above method, the replication mode control action of the previous time step, the current state features, and the future network state features are all used as inputs to the reinforcement learning policy network. This allows the determination of the replication mode control action to comprehensively consider historical action information, the current running state, and the future network state change trend, and output the corresponding decision confidence level. This improves the matching degree between the replication mode control action and the running state of the remote replication task, and provides a confidence basis for subsequent control execution.

[0107] Based on the foregoing embodiments, further, the replication mode of the remote replication task is controlled based on the replication mode control action, including: performing security gating processing on the replication mode control action according to the decision confidence level corresponding to the replication mode control action; and controlling the replication mode of the remote replication task by executing the replication mode control action when the security gating processing allows the execution of the replication mode control action.

[0108] The decision confidence level characterizes the credibility of the replication mode control action in the current state, and the security gating process verifies the decision confidence level before executing the replication mode control action to determine whether to allow the execution of the replication mode control action. In one embodiment, the security gating process can compare the decision confidence level with a preset confidence threshold and determine whether to allow the execution of the replication mode control action based on the comparison result.

[0109] In its implementation, the security gating process acquires the replication mode control action and its corresponding decision confidence level, and generates a gating result. When the gating result indicates that execution is permitted, the replication mode control action is executed, allowing the remote replication task to maintain its current replication mode, switch to synchronous replication mode, or switch to asynchronous replication mode. When the gating result indicates that execution is not permitted, the current replication mode of the remote replication task is maintained, and the replication mode control action and its corresponding decision confidence level are recorded. The decision confidence level can be obtained from the output of the reinforcement learning policy network or from the action probability corresponding to the replication mode control action output by the reinforcement learning policy network.

[0110] In its working principle, the security gating process first constrains the replication mode control actions based on decision confidence, and then determines whether to execute the replication mode control actions, so that the replication mode of the remote replication task is not directly adjusted based on the output of a single policy. Therefore, the replication mode of the remote replication task is controlled only when the replication mode control actions meet the execution requirements of the security gating process.

[0111] In this way, the replication mode control actions are screened by security gating before execution, so that the replication mode control is based on decision confidence verification, and the replication mode of the remote replication task is actually controlled only when the security gating process allows execution, thereby improving the stability and security of remote replication mode control.

[0112] Figure 3 This is a schematic flowchart illustrating the method for updating the reinforcement learning policy network provided in an embodiment of this application. Please refer to... Figure 3 After determining the replication mode control actions, the following are also included:

[0113] S301, based on the running status information after executing the replication mode control action, determine the target satisfaction of the recovery point, application performance indicators and resource efficiency indicators, and determine the replication mode switching result based on the replication mode control action;

[0114] S302, the reward value is obtained by weighting the recovery point target satisfaction, application performance indicators, resource efficiency indicators and replication mode switching results;

[0115] S303, Based on the current state characteristics, future network state characteristics, replication mode control actions and reward values, the dominance function estimation algorithm is used to determine the dominance estimate;

[0116] S304, based on the advantage estimate, uses a proximal policy optimization algorithm with truncation operation to update the reinforcement learning policy network.

[0117] Specifically, the recovery point target refers to the historical point in time that the system or data needs to be restored to after a disaster, measuring the tolerance for data loss. Operational status information characterizes the actual operation of the remote replication task after the replication mode control action is executed. Recovery point target satisfaction characterizes the degree to which the recovery point target is achieved. Application performance indicators characterize changes in business response, throughput, and latency. Resource efficiency indicators characterize the utilization of bandwidth, storage, and computing resources. Replication mode switching results characterize whether a mode switch occurred and the type of mode after the switch. The reward value is obtained by weighted summation of the above indicators, assigning corresponding weights, and the switching cost can be introduced as a correction term to form a feedback signal consistent with the disaster recovery target. It should be noted that the weights are the reward item weights corresponding to the recovery point target satisfaction, application performance indicators, resource efficiency indicators, and replication mode switching results, respectively, and are different parameters from the dynamic weights corresponding to the status indicators in the dynamic fusion processing.

[0118] In the specific implementation, after executing the replication mode control action, operational status information is collected based on the source storage system, destination storage system, data transmission network, and related operational monitoring components. The recovery point target satisfaction is then calculated according to a preset evaluation model. If synchronous replication mode is used, the recovery point corresponding to the current replication mode can be determined based on the data consistency status or data synchronization time difference between the source and destination storage systems. If asynchronous replication mode is used, the recovery point corresponding to the current replication mode can be determined based on replication lag, the time range of data to be replicated in the replication queue, or the data time difference between the source and destination storage systems. The recovery point corresponding to the current replication mode is then compared with the target recovery point to obtain the recovery point target satisfaction. Application performance indicators can be characterized by average response time of business requests, throughput, or queue waiting time. Resource efficiency indicators can be characterized by link utilization, cache occupancy, and storage write efficiency. The replication mode switching result can be determined based on the replication mode control action and the actual effective replication mode to identify whether the replication mode control action has actually taken effect, avoiding misjudging an ineffective replication mode switch as a completed switch.

[0119] The dominance function estimation algorithm is used to determine the difference in payoff between the current replication mode control action and the baseline payoff under the current decision state. The baseline payoff can be determined based on the value estimation result corresponding to the current decision state. In one implementation, a generalized dominance estimation method can be used, combining the reward values ​​in continuous decision samples and the value estimation results corresponding to adjacent decision states to determine the dominance estimate. After dominance function estimation based on current state features, future network state features, replication mode control action, and reward value, a generalized dominance estimation method can be used to jointly calculate short-term rewards and long-term returns to obtain the dominance estimate. Subsequently, a proximal policy optimization algorithm including a truncation operation updates the reinforcement learning policy network based on this dominance estimate, and by limiting the range of change in the policy update ratio, the adjustment magnitude of the policy parameters in each iteration is controlled.

[0120] Through the above methods, the reward value can simultaneously reflect the recovery point objective, application performance, and resource utilization; the advantage estimate can reflect the quality of actions and long-term benefits; and the proximal policy optimization algorithm enables the reinforcement learning policy network to continuously adapt to changes in network and business states while maintaining stable updates. Therefore, the reinforcement learning policy network can generate decision outputs that are more aligned with disaster recovery objectives in subsequent replication mode control, and improve the stability and adaptability of replication mode switching.

[0121] In one possible implementation, the multidimensional state information includes network state information, storage state information, and service state information. The process involves collecting multidimensional state information related to the remote replication task and obtaining service context information associated with the remote replication task, including: collecting transmission quality state information and bandwidth state information of the data transmission network to obtain network state information; collecting storage read / write state information, cache state information, and replication queue state information related to the remote replication task to obtain storage state information; obtaining service-critical information associated with the remote replication task and recovery point information corresponding to the current replication mode to obtain service state information; and obtaining service time period information and service type information associated with the remote replication task to obtain service context information.

[0122] Among them, transmission quality status information is used to characterize the network link transmission stability, latency fluctuations, and packet loss changes of the data transmission network at the current moment; bandwidth status information is used to characterize the network link bandwidth occupancy and available margin. Storage read / write status information is used to characterize the read / write load of the volume or array related to remote replication; cache status information is used to characterize cache hit status, cache occupancy rate, or dirty data backlog; replication queue status information is used to characterize the queue length and backlog of data to be replicated. Business criticality information is used to characterize the level, disaster recovery priority, or continuity requirements of the business corresponding to the remote replication task; recovery point information corresponding to the current replication mode is used to characterize the level of data recovery points that can be retained in the current synchronous or asynchronous mode.

[0123] In practical implementation, the network monitoring interface can periodically read link latency, jitter, packet loss rate, throughput, and bandwidth utilization, and normalize them to form network status information. The storage monitoring interface can collect storage queue depth, read / write response time, cache hit rate, cache occupancy rate, and replication queue length, forming storage status information. The service management interface can read service classification identifiers, recovery point target configurations, replication mode identifiers, service calendars, and service type tags, forming service status information and service context information. These interfaces can be deployed in storage controllers, replication management nodes, or operation and maintenance management platforms.

[0124] By unifying network, storage, and service attributes into the status acquisition scope of remote replication tasks, subsequent mode control can make decisions based on network link quality, storage load, and service requirements simultaneously, and use service time period and service type as context input, providing a more complete status basis for replication mode switching. This enhances the ability to identify network link fluctuations, storage pressure, and service differences.

[0125] In one possible implementation, the method further includes: when security gating does not allow the execution of replication mode control actions, running a reinforcement learning policy network in shadow mode, recording the replication mode control actions output by the reinforcement learning policy network, and not controlling the replication mode of the remote replication task based on the recorded replication mode control actions; obtaining expert feedback information on the recorded replication mode control actions; updating the reinforcement learning policy network based on the expert feedback information, and using the updated reinforcement learning policy network to redetermine the replication mode control actions and the corresponding decision confidence levels; and performing security gating processing on the redetermined replication mode control actions based on the decision confidence levels corresponding to the redetermined replication mode control actions.

[0126] Specifically, shadow mode refers to the operation mode in which the reinforcement learning policy network performs inference and outputs control suggestions without changing the current replication mode of the remote replication task. The actions output by the reinforcement learning policy network are only used for recording and accumulating learning samples and do not directly affect the switching control between synchronous and asynchronous replication modes. Decision confidence can be obtained from the output probability of the reinforcement learning policy network for each candidate replication mode control action or a normalized confidence score, and is used to characterize the degree of certainty of the reinforcement learning policy network regarding the control action for the current replication mode. Expert feedback information can be generated by storage operation and maintenance experts based on business-critical information, replication queue status, link status, and recovery point target constraints. The content may include action rationality evaluation, action correction suggestions, or security markings.

[0127] In practical implementation, the reinforcement learning policy network can maintain consistency with the input interfaces of the aforementioned current state features and future network state features, enabling it to reason about the same state in shadow mode and record the replication mode control actions corresponding to the current business scenario. Expert feedback information can be written to the training sample cache and, together with the recorded replication mode control actions, state features, and expert feedback tags, constitute updated data for network parameter correction. The updated reinforcement learning policy network can re-output the replication mode control actions and their decision confidence based on the corrected parameters. This decision confidence is then subjected to security gating and judged together with preset thresholds, blacklist / whitelist rules, or business time constraints. Subsequent execution is only allowed when the judgment result meets the security conditions.

[0128] By recording actions in a shadow mode and introducing expert feedback to update the network when security gating fails, the solution can continuously accumulate experience samples without affecting the stable operation of the current remote replication task. This allows the reinforcement learning strategy network to gradually adapt to business constraints and human experience in enterprise-level disaster recovery scenarios. Since the redefined actions must be processed through security gating again, this mechanism creates a closed-loop constraint between learning capability and execution security, reducing the impact of non-compliant replication mode switching on business continuity and the consistency of recovery goals.

[0129] Building upon the aforementioned embodiments, the state of network links is further predicted based on topology information and historical state information to obtain future network state characteristics, including:

[0130] Monitor node and link changes in the data transmission network, and update the topology information based on these changes; construct graph-structured time-series data based on the updated topology information and historical status information.

[0131] The graph-structured temporal data is input into the spatiotemporal graph convolutional network. Based on the graph attention mechanism, the aggregation weights of the adjacent network links corresponding to each network link are determined. The state features of the adjacent network links are aggregated according to the aggregation weights, and the time features of the aggregated state features are extracted to obtain the first network state prediction result.

[0132] Based on historical state information, statistical prediction of network link status is performed to obtain a second network state prediction result; the first network state prediction result and the second network state prediction result are weighted and fused to obtain the future network state characteristics.

[0133] Specifically, node change information is used to characterize the addition, removal, or state switching of network devices in the data transmission network; link change information is used to characterize the establishment, disconnection, bandwidth adjustment, or quality fluctuation of network links; and updated topology information is used to reflect the current data transmission network connection relationships and network link attributes. Graph structure time-series data is used to map network devices as graph nodes and network links as graph edges, and organize the state information of each edge at consecutive historical moments into time-series features. Graph attention mechanism is used to calculate the influence weights of adjacent network links based on the correlation strength between network links, and time feature extraction is used to perform time-series convolution modeling on the aggregated feature sequences. Network link state statistical prediction is used to generate statistical prediction results based on the mean, trend, variance, or distribution parameters in historical state information, and weighted fusion is used to combine the confidence levels of the two types of prediction results to output a unified future network state feature.

[0134] In practical implementation, link probe messages, routing announcements, and interface status information can be periodically collected to identify node and link changes. These changes are then written to the topology database to complete topology updates. Based on the updated topology, latency, packet loss rate, jitter, or bandwidth utilization of each network link at multiple historical moments can be used as edge features. This data is then used to construct a graph-structured time-series data set and input into a spatiotemporal graph convolutional network. The graph attention mechanism scores the correlation between the target network link and its neighboring network links using learnable parameters, aggregates the state features of neighboring network links with normalized weights, and extracts short-term dependencies through one-dimensional temporal convolution to obtain the first network state prediction result. Simultaneously, sliding window statistical analysis can be performed on historical state information. Mean extrapolation, trend regression, or distribution parameter estimation can be used to obtain the second network state prediction result. The fusion weights are determined based on the error evaluation results of the two predictions, resulting in a weighted fusion output.

[0135] By employing the above methods, the future network state characteristics simultaneously incorporate information on the evolution of network link relationships and historical statistical patterns. This allows for a more complete network state representation upon which subsequent replication mode control is based, and provides better adaptability to network topology changes and network link fluctuations.

[0136] In one specific embodiment, the remote replication mode control method is applied to a storage area network disaster recovery scenario across data centers.

[0137] At the current decision time t, multi-dimensional state information associated with the remote replication task is collected. This includes network state information such as network link latency l(t), network link jitter j(t), network link packet loss rate p(t), and available bandwidth ratio b(t); and storage state information such as the number of input / output operations per second corresponding to local read operations. The number of input / output operations per second corresponding to local write operations The cache hit rate c(t) and replication queue depth q(t) are used as parameters. Business status information includes business criticality information k and the theoretical recovery point corresponding to the current replication mode. The value range of the business-critical information k is [0,1], which can be obtained by mapping according to the storage strategy corresponding to the business associated with the remote replication task. Simultaneously, business context information such as the business time period and business type is obtained, and the business context function is determined based on this information. .

[0138] The minimum-maximum normalization method is used to normalize each status indicator. The indicator values ​​are mapped to a unified numerical range based on the reference range corresponding to each indicator. The normalized link latency, network link jitter, network link packet loss rate, available bandwidth ratio, number of I / O operations per second for read operations, number of I / O operations per second for write operations, cache hit rate, replication queue depth, business-critical information, and the theoretical recovery point corresponding to the current replication mode are used as inputs for dynamic weight calculation.

[0139] The weights are not fixed, but rather a function that varies with time t and the business context (such as time period and business type). For the i-th normalized state indicator, its basic importance parameter α is obtained. i and business context sensitive parameter β i Based on the index value s of the i-th normalized state index. i (t), basic importance parameter α i Business context functions and business context sensitive parameter β i The dynamic weights are calculated using a weighted soft maximum function:

[0140]

[0141] Generate weight vector W(t)=[w l w j ,...,w k Where n is the number of state indicators participating in dynamic fusion, and w i (t) represents the dynamic weight of the i-th normalized state index at the current decision time. Business context function. Based on the current business operation context, corresponding adjustment information is generated. For example, when the remote replication task-related business enters a critical business period, a positive bias can be injected into the business criticality information k through the business context function, thereby increasing the dynamic weight corresponding to the business criticality information.

[0142] The corresponding normalized state indicators are weighted according to each dynamic weight to form the current state feature S. tTaking link latency, network link jitter, and service-critical information as examples, the current state characteristics can be represented as follows:

[0143]

[0144] in, This indicates that the corresponding state indicators have undergone normalization. Other normalized state indicators participating in dynamic fusion are written into the current state features according to their corresponding dynamic weights.

[0145] Simultaneously, using network devices such as routers in the data transmission network as nodes and network links connecting different network devices as edges, the data transmission network is modeled as a graph G=(V,E), where V represents the set of nodes and E represents the set of edges. For any network link e, the link delay and link utilization status corresponding to the historical time t are used as edge features to obtain the edge feature vector: .

[0146] For a network link e, its first-order neighbor set N(e) is determined, and the neighbor information is aggregated in the spatial graph convolution process. The network link features of layer l+1 are obtained as follows:

[0147]

[0148] in, This represents the characteristics of network link e at layer l. This represents the trainable feature transformation matrix of the l-th layer. The degree information after adding self-connects is represented, σ represents the activation function, and the normalized adjacency relationship is reflected by the neighbor set and the degree normalization coefficient. Through the above spatial graph convolution processing, the neighbor state information with topological association with the target network link is aggregated into the spatial features corresponding to the target network link.

[0149] In the edge feature sequence Then, one-dimensional convolution is used to capture temporal dependencies. The result of temporal convolution can be represented as:

[0150]

[0151] Where K represents the convolution range corresponding to the temporal convolution processing. This represents the convolution parameters corresponding to the historical position. This represents the spatial characteristics of network link e at the corresponding historical moment.

[0152] After processing through multiple spatiotemporal convolutional blocks, the obtained spatiotemporal features are input into the fully connected layer, which outputs the edge feature prediction results for each network link at the next H time steps: .

[0153] Predicting future delays of critical links As a characteristic of future network state Input into the DRL decision engine.

[0154] The current state feature S t Characteristics of future network states The copy mode control action a from the previous moment t-1 By combining these elements, the state input of the reinforcement learning policy network at the current decision moment is formed: .

[0155] Copy mode control action a t The motion space is set as follows: The reinforcement learning policy network is represented by parameter θ, and its policy is... Input the status as s t The input is a reinforcement learning policy network, which outputs the action probability distribution corresponding to each candidate copy mode control action. Based on the action probability distribution, the control action 'a' for the current copy mode is determined. t And the corresponding decision confidence level.

[0156] After obtaining the replication mode control action and decision confidence level, both are input into the security gating module. The security gating module compares the decision confidence level with a preset confidence threshold. When the decision confidence level meets the execution requirements corresponding to the security gating process, the replication mode control action is allowed to be executed. When the replication mode control action is action 0, the remote replication task continues to use the current replication mode; when the replication mode control action is action 1, an asynchronous replication mode switching command is issued to the corresponding storage system; when the replication mode control action is action 2, a synchronous replication mode switching command is issued.

[0157] When security gating disallows replication mode control actions, the replication mode of the remote replication task is not adjusted based on the current replication mode control action, and the reinforcement learning policy network continues to run in shadow mode. In shadow mode, the reinforcement learning policy network uses the same state inputs as the actual decision-making process for inference, records the output replication mode control actions and their decision confidence levels, but does not distribute the recorded replication mode control actions to the storage system. For the recorded replication mode control actions, expert feedback information from storage operations experts can be obtained, and the reinforcement learning policy network is updated based on this feedback. The updated reinforcement learning policy network reprocesses the corresponding state inputs, determines new replication mode control actions and decision confidence levels, and the new replication mode control actions re-enter security gating.

[0158] For replication mode control actions permitted by security gating, the runtime status information of the remote replication task is re-acquired after the replication mode control action is executed. The corresponding target recovery point is determined based on the business criticality information k. And obtain the theoretical recovery point corresponding to the current replication mode. The reward corresponding to the target recovery point is determined based on the relationship between the target recovery point and the theoretical recovery point corresponding to the current replication mode. :

[0159]

[0160] Wherein, λ1 is the adjustment hyperparameter. When the theoretical recovery point corresponding to the current replication mode meets the target recovery point corresponding to the business-critical information, a positive reward is given; otherwise, the degree of penalty is determined based on the deviation between the two.

[0161] The application performance bonus item R is determined based on the application performance after the execution of the replication mode control action. perf In synchronous replication mode, network link latency can affect the write confirmation process of the source application. Therefore, the application performance bonus is represented as follows:

[0162]

[0163] Where λ2 is the adjustment hyperparameter. This is an indicator function. When the remote replication task is in synchronous replication mode, the corresponding performance penalty is determined based on the network link latency.

[0164] Determine the replication mode switching reward item R based on the replication mode control action. switch :

[0165]

[0166] Wherein, λ3 and λ4 are the adjustment hyperparameters. When the replication mode control action indicates a replication mode switch, a corresponding switching cost is introduced; when the remote replication task is detected to frequently switch replication modes within a preset time range, the switching penalty is further increased.

[0167] Resource efficiency bonus item R resource Represented as:

[0168]

[0169] Wherein, λ5 is the adjustment hyperparameter. When the remote replication task adopts asynchronous replication mode and the reward item corresponding to the recovery point target is not less than zero, a positive reward is given for resource usage.

[0170] The reward value at the current decision moment is obtained by combining the reward items corresponding to the fulfillment of the recovery point objective, application performance, replication mode switching, and resource efficiency:

[0171]

[0172] Input the current state as s t Copy mode control action a t Reward value R t The new state formed after the execution of the replication mode control action is written into the experience data. After the replication mode control action is executed, network state information, storage state information, and service state information are re-collected and dynamically weighted to obtain the current state feature S for the next time step. t+1 Simultaneously, the new network link states are added to the historical state information for subsequent spatiotemporal graph convolutional network prediction.

[0173] The advantage estimate is determined based on the current state characteristics, future network state characteristics, replication mode control actions, and reward values. The advantage estimate is used to characterize the superiority or inferiority of the replica mode control action relative to the average decision level under the current decision state. Batch data is periodically sampled from the stored empirical data, and the reinforcement learning policy network is updated using the Proximal Policy Optimization (PPO) algorithm. The policy update process corresponding to PPO is represented as follows:

[0174]

[0175] in, This indicates the policy parameters before the update. This indicates the updated policy parameters. The parameter represents the truncation range, and `clip` represents the truncation operation. The policy change is determined based on the action probabilities output by the reinforcement learning policy network for the copy mode control action before and after the update, and the truncation operation limits the range of policy change during a single update.

[0176] After updating the policy parameters, the updated reinforcement learning policy network is used to process the current state features, future network state features, and the replication mode control action from the previous time step at the next decision moment, determining the next replication mode control action and its decision confidence. The above processes of data acquisition, dynamic fusion, network state prediction, replication mode control action determination, security gating, action execution, reward calculation, and reinforcement learning policy network update are repeated according to the decision cycle.

[0177] Figure 4 This is a schematic diagram of the remote replication mode control device provided in an embodiment of this application. Please refer to... Figure 4The remote replication mode control device may include:

[0178] The acquisition module 41 is used to acquire multi-dimensional status information associated with the remote replication task and obtain the business context information corresponding to the remote replication task.

[0179] The fusion module 42 is used to dynamically fuse multi-dimensional state information based on business context information to obtain the current state features;

[0180] Processing module 42 is used to acquire the topology information and historical status information of the data transmission network, and predict the status of the network links based on the topology information and historical status information to obtain the future network status characteristics. The data transmission network is used to perform remote replication tasks.

[0181] The control module 44 is used to determine the replication mode control action of the remote replication task by adopting a reinforcement learning strategy network based on the current state characteristics and future network state characteristics, and to control the replication mode of the remote replication task based on the replication mode control action.

[0182] In one possible implementation, the fusion module 42 is specifically used for:

[0183] Normalize each state index in the multidimensional state information to obtain the normalized state index corresponding to each state index.

[0184] Determine business context adjustment information based on business context information, and obtain the basic importance parameters and business context sensitive parameters corresponding to each normalized state indicator;

[0185] Based on the index values, basic importance parameters, business context adjustment information, and business context sensitive parameters of each normalized state index, a weighting function is used to calculate the dynamic weights corresponding to each normalized state index.

[0186] The current state characteristics are obtained by weighting and fusing the normalized state indicators according to the dynamic weights.

[0187] In one possible implementation, the processing module 42 is specifically used for:

[0188] Based on the topology information, network devices in the data transmission network are taken as nodes and network links as edges. The state information of each network link at multiple historical moments is used as the temporal features of the corresponding edges to obtain graph structure time series data.

[0189] Input graph-structured temporal data into a spatiotemporal graph convolutional network, and aggregate the neighbor features of each edge based on the normalized adjacency matrix to obtain the spatial features corresponding to each network link;

[0190] One-dimensional temporal convolution is performed on the spatial features of each network link at multiple historical moments to obtain spatiotemporal features.

[0191] By performing fully connected processing on the spatiotemporal features, the future network state features can be obtained.

[0192] In one possible implementation, the control module 44 is specifically used for:

[0193] Retrieve the replication mode control actions of the remote replication task at the previous moment;

[0194] Input the current state features, future network state features, and the copy mode control action from the previous time step into the reinforcement learning policy network to obtain the copy mode control action and the decision confidence corresponding to the copy mode control action.

[0195] The replication mode control action is to maintain the current replication mode, switch to synchronous replication mode, or switch to asynchronous replication mode.

[0196] In one possible implementation, the control module 44 is further specifically used for:

[0197] Based on the decision confidence level corresponding to the replication mode control action, security gating is applied to the replication mode control action;

[0198] When the security gating process allows the execution of replication mode control actions, the replication mode of the remote replication task is controlled by executing these actions.

[0199] In one possible implementation, the device further includes an update module, which, after determining the replication mode control action, is used to:

[0200] Based on the running status information after executing the replication mode control action, determine the recovery point target satisfaction, application performance indicators, and resource efficiency indicators, and determine the replication mode switching result based on the replication mode control action;

[0201] The reward value is obtained by weighting the satisfaction of the recovery point target, application performance indicators, resource efficiency indicators, and the results of replication mode switching.

[0202] Based on the current state characteristics, future network state characteristics, replication mode control actions, and reward values, the dominance function estimation algorithm is used to determine the dominance estimate.

[0203] Based on the advantage estimate, a proximal policy optimization algorithm with truncation operation is used to update the reinforcement learning policy network.

[0204] The remote replication mode control device provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0205] Figure 5 This is a schematic diagram of the remote replication mode control device provided in an embodiment of this application. Please refer to... Figure 5 The remote copy mode control device 50 may include a memory 51 and a processor 52. Exemplarily, the memory 51 and the processor 52 are interconnected via a bus 53.

[0206] Memory 51 is used to store program instructions;

[0207] The processor 52 is used to execute the program instructions stored in the memory, so that the remote copy mode control device 50 performs the method shown in the above method embodiment.

[0208] The remote replication mode control device 50 provided in this application embodiment can execute the technical solution shown in the above method embodiment. Its implementation principle and beneficial effects are similar, and will not be described again here.

[0209] This application provides a computer-readable storage medium storing computer-executable instructions, which are used to implement the above-described method when executed by a processor.

[0210] This application embodiment may also provide a computer program product, including a computer program that, when executed by a processor, can implement the above-described method.

[0211] All or part of the steps in the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above-described method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), random access memory (RAM), flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disc, and any combination thereof.

[0212] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0213] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0214] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0215] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of the claims of this application and their equivalents, this application also intends to include these modifications and variations.

[0216] In this application, the term "comprising" and its variations can refer to non-limiting inclusion; the term "or" and its variations can refer to "and / or". The terms "first", "second", etc., in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

Claims

1. A remote replication mode control method, characterized in that, include: Collect multi-dimensional status information associated with the remote replication task, and obtain the business context information corresponding to the remote replication task; The multi-dimensional state information is dynamically fused based on the business context information to obtain the current state features; The topology and historical status information of the data transmission network are obtained, and the status of the network links is predicted based on the topology and historical status information to obtain the future network status characteristics. The data transmission network is used to perform remote replication tasks; Based on the current state characteristics and the future network state characteristics, a reinforcement learning policy network is used to determine the replication mode control action of the remote replication task, and the replication mode of the remote replication task is controlled based on the replication mode control action.

2. The method according to claim 1, characterized in that, The step of dynamically fusing the multi-dimensional state information based on the business context information to obtain the current state features includes: The state indicators in the multidimensional state information are normalized to obtain the normalized state indicators corresponding to each state indicator. Based on the business context information, determine the business context adjustment information, and obtain the basic importance parameters and business context sensitive parameters corresponding to each of the normalized state indicators; Based on the index values ​​of each normalized state index, the basic importance parameter, the business context adjustment information, and the business context sensitivity parameter, a weighting function is used to calculate the dynamic weight corresponding to each normalized state index. The normalized state indicators are weighted and fused according to the dynamic weights to obtain the current state features.

3. The method according to claim 1, characterized in that, The step of predicting the state of network links based on the topology information and the historical state information to obtain future network state characteristics includes: Based on the topology information, network devices in the data transmission network are used as nodes and network links are used as edges. The state information of each network link at multiple historical moments is used as the temporal features of the corresponding edges to obtain graph structure temporal data. The graph structure temporal data is input into a spatiotemporal graph convolutional network, and the neighbor features of each edge are aggregated based on the normalized adjacency matrix to obtain the spatial features corresponding to each network link. One-dimensional temporal convolution is performed on the spatial features of each network link at multiple historical moments to obtain spatiotemporal features. The spatiotemporal features are processed by a fully connected layer to obtain the future network state features.

4. The method according to claim 1, characterized in that, The step of determining the replication mode control action of the remote replication task using a reinforcement learning policy network based on the current state characteristics and the future network state characteristics includes: Obtain the replication mode control action of the remote replication task at the previous moment; The current state features, the future network state features, and the copy mode control action from the previous time step are input into the reinforcement learning policy network to obtain the copy mode control action and the decision confidence corresponding to the copy mode control action. The replication mode control action is to maintain the current replication mode, switch to synchronous replication mode, or switch to asynchronous replication mode.

5. The method according to claim 4, characterized in that, The control of the replication mode of the remote replication task based on the replication mode control action includes: Based on the decision confidence level corresponding to the replication mode control action, the replication mode control action is subjected to security gating processing; When the security gating process allows the execution of the replication mode control action, the replication mode of the remote replication task is controlled by executing the replication mode control action.

6. The method according to claim 1 or 4, characterized in that, After determining the replication mode control action, the method further includes: Based on the running status information after executing the replication mode control action, determine the recovery point target satisfaction, application performance indicators, and resource efficiency indicators, and determine the replication mode switching result based on the replication mode control action; The reward value is obtained by weighting the recovery point target satisfaction, the application performance index, the resource efficiency index, and the replication mode switching result. Based on the current state characteristics, the future network state characteristics, the replication mode control action, and the reward value, an advantage function estimation algorithm is used to determine the advantage estimate. Based on the aforementioned advantage estimate, the reinforcement learning policy network is updated using a proximal policy optimization algorithm that includes a truncation operation.

7. A remote copy mode control device, characterized in that, The device includes: The acquisition module is used to acquire multi-dimensional status information associated with the remote replication task and obtain the business context information corresponding to the remote replication task. The fusion module is used to dynamically fuse the multi-dimensional state information based on the business context information to obtain the current state features; The processing module is used to acquire the topology information and historical state information of the data transmission network when the remote replication task is executed, and to perform network link prediction based on the topology information and the historical state information to obtain future network state characteristics. The control module is used to determine the replication mode control action of the remote replication task by using a reinforcement learning policy network based on the current state characteristics and the future network state characteristics, and to control the replication mode of the remote replication task based on the replication mode control action.

8. A remote replication mode control device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, in, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images