Tracking data in complex replication systems

Abstract

One exemplary operation may include one or more of: receiving a message from a proxy installed at a data replication server, the message including a check and validated status identifier of a data replication operation, identifying a latency value associated with the data replication server, and determining whether a data loss has occurred based on the checksum verified state identifier and the latency value, and in response to determining that the data loss has occurred, sending a notification of the data loss to a computing system associated with the data replication server.

Description

Background Art

[0001] In a complex replication environment, multiple replication sites (nodes) may be connected to each other in a distributed network topology. Within the network, each replication site may include a replication server (e.g., a canonical log server, etc.) that runs a workflow and replicates the workflow to one or more other sites in the network. For example, as part of a remote backup storage operation, data stored at one server may be replicated (transferred) to other servers within a distributed network, etc. In many cases, data may be transferred using SQL replication, etc.

[0002] In some cases, data may be lost during a replication operation. For example, when a table reaches its destination in a replication operation, a small portion of the data rows within the table may be lost. Lost data can be very difficult to detect, especially in complex environments where servers must be continuously and in real time available. In addition, when data is lost, there is often no error reporting. Therefore, a better mechanism is needed for tracking data during a data replication operation and notifying when data is lost. Summary of the invention

[0003] An example embodiment provides an apparatus comprising: a network interface configured to: receive a message from an agent installed at a data replication server, wherein the message may include a status identifier of a checksum verification of a data replication operation; and a processor configured to: identify a wait time value associated with the data replication server, determine whether data loss has occurred based on the status identifier of the checksum verification and the wait time value, and in response to determining that data loss has occurred, send a notification of data loss to a computing system associated with the data replication server via the network interface.

[0004] Another example embodiment provides a method, comprising: receiving a message from an agent installed at a data replication server, the message including a status identifier of a checksum verification of a data replication operation, identifying a wait time value associated with the data replication server, determining whether data loss has occurred based on the status identifier of the checksum verification and the wait time value, and in response to determining that data loss has occurred, sending a notification of data loss to a computing system associated with the data replication server.

[0005] Yet another example embodiment provides a computer-readable medium comprising instructions which, when read by a processor, cause the processor to perform one or more of the following operations: receive a message from an agent installed at a data replication server, the message comprising a status identifier for a checksum verification of a data replication operation; identify a wait time value associated with the data replication server; determine whether data loss has occurred based on the status identifier for the checksum verification and the wait time value; and in response to determining that data loss has occurred, send a notification of data loss to a computing system associated with the data replication server. BRIEF DESCRIPTION OF THE DRAWINGS

[0006] The patent or application file contains at least one drawing drawn in color. Copies of this patent or patent application publication with color drawing(s) will be provided by the Office upon request and payment of the necessary fee.

[0007] Figure 1 is a diagram illustrating a cloud computing environment interacting with various devices according to an example embodiment.

[0008] Figure 2A is a diagram illustrating abstract model layers of a cloud computing environment according to an example embodiment.

[0009] Figure 2B is a diagram illustrating a system for identifying data loss in a complex replication system according to an example embodiment.

[0010] Figures 3A-3C is a diagram illustrating an example of a permission network according to an example embodiment.

[0011] Figure 3D is a diagram illustrating a machine learning process via a cloud computing platform according to an example embodiment.

[0012] Figure 3E is a diagram illustrating a quantum computing environment associated with a cloud computing platform according to an example embodiment.

[0013] Figure 4A is a diagram illustrating a process of transmitting replication messages between data replication sites according to an example embodiment.

[0014] Figure 4B is a diagram illustrating a process of verifying a status of a data copy operation based on checksum verification according to an example embodiment.

[0015] Figure 4C is a diagram illustrating a process of determining whether data loss has occurred according to an example embodiment.

[0016] Figure 4D is a diagram illustrating a process of training a latency model according to an example embodiment.

[0017] Figure 5 is a diagram illustrating a method of determining whether data loss has occurred according to an example embodiment.

[0018] Figure 6 is a diagram illustrating an example of a computing system that supports one or more example embodiments. DETAILED DESCRIPTION

[0019] It should be understood that although the present disclosure includes a detailed description of cloud computing, the implementation of the teachings set forth herein is not limited to a cloud computing environment. Instead, embodiments of the present invention can be implemented in conjunction with any other type of computing environment now known or later developed.

[0020] Currently, there are no products that can adequately monitor data replication systems and detect data loss in replication operations in an efficient manner. Instead, the traditional strategy is to add diagnostic code to one or more components, rebuild the product, stop the program, and redeploy the product to the customer's production system. The diagnostic code can capture detailed information that can be analyzed for data loss. Some disadvantages of this process include bringing the server down (and potentially causing downtime for one or more customers). In addition, code changes may result in unintentional changes to the operation and maintenance of the product, and further, problems may not be easily reproduced because some problems (e.g., random problems, etc.) are unpredictable. Likewise, the computing resources required to debug the code are also unpredictable, which may cause additional problems for the host.

[0021] Example embodiments are directed to a tracking system for monitoring and tracking data loss within a complex data replication network. The network may include a distributed network of replication sites. Each replication site may include one or more servers having replication software installed therein. The replication software may be configured to receive data and transmit the data to other servers in the network and synchronize these servers. For example, a source server may wish to back up a copy of its data, applications, etc. Here, the source server may select a destination server for storing the backup. The destination server may be multiple hops (sites) away from the source server. The replication network may form a path of servers / sites therein for transmitting the data to be backed up from the source server to the destination server. In addition, the replication software may ensure and verify that a synchronized copy of the data from the source server is stored at the destination server.

[0022] The tracking system may include an agent installed at each server (site) and a central coordinator configured to exchange messages with the agent and determine whether data has been lost during the replication operation. As an example, the agent can be a device, key, box, etc. that is physically plugged into the server (e.g., via USB, LAN cable, etc.). As another example, the agent can be a software program installed within the server without the need for a physical device to be plugged in. A pluggable device can be beneficial because it is easy to maintain and the user can simply unplug it when it is no longer needed instead of uninstalling it. Moreover, since the hardware is included, it is easy to plan dedicated resources (e.g., processor, memory, etc.) for the agent.

[0023] The coordinator may be installed or otherwise operated on a host platform such as a cloud platform, an on-site server, etc. The coordinator may be referred to as a suggestion system or an intermediary here. The coordinator communicates with the agent using a status check message that provides the coordinator with insight into the data replication process. For example, the source site may transmit / backup data to the target site (destination site) via one or more intermediate sites. In this example, the source site may create a checksum or multiple checksums based on the data being transmitted (e.g., the checksum of one or more replicated transactions within the data being transmitted, etc.), and add the checksum to the data replication message including the data being transmitted. The intermediate server may receive the data replication message, verify the (one or more) checksums, and report the verification results to the coordinator. In response, the coordinator may determine whether data loss has occurred and whether additional actions should be taken. If necessary, the coordinator may also request to resend the transaction content between the servers.

[0024] The messages exchanged between the agent and the coordinator include a status indicator (checksum status indicator) that provides the result of the checksum verification process performed by the agent. For example, the message may include an identifier indicating that the checksum verification is a match (successful) or that the checksum verification did not result in a match (suspicious). The message may also include an identifier of the agent sending the message, an identifier of one or more transactions for one or more transactions that respectively created the one or more checksums, and the verification status of the one or more checksums. The exchanged messages may also include control messages for management or maintenance, such as heartbeat messages, reconfirmation requests, etc.

[0025] According to various embodiments, the messages exchanged between the agents themselves and between the agents and the coordinator are different from the messages exchanged by the replication servers because they include checksum verification and an identifier of the transaction used to create the checksum. At the same time, the messages between the replication servers typically only include application / user data without any verification data for data loss. Therefore, the agents and coordinators can provide benefits to traditional data replication networks by incorporating data loss detection capabilities and real-time notification capabilities into the data replication network, which can reduce downtime, save resources, and ensure efficient operation of the data replication process.

[0026] Cloud computing is a service delivery model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services) that can be quickly provisioned and released with minimal management effort or interaction with the provider of the service. The cloud model may include at least five characteristics, at least three service models, and at least four deployment models.

[0027] Examples of cloud computing features that may be associated with example embodiments include the following.

[0028] On-demand self-service: Cloud consumers can unilaterally and automatically provision computing capabilities, such as server time and network storage, as needed without requiring manual interaction with the service provider.

[0029] Broad Network Access: Capabilities are available over the network and accessed through standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs).

[0030] Resource pooling: A provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically allocated and reallocated based on demand. There is location independence in the sense that consumers typically do not control or know the exact location of the resources provided, but are able to specify the location at a higher level of abstraction (e.g., country, state, or data center).

[0031] Rapid elasticity: In some cases, the ability to scale out quickly and in quickly can be provisioned quickly and elastically. To the consumer, the capacity available for provisioning often appears to be unlimited and can be purchased in any quantity at any time.

[0032] Measurable services: Cloud systems automatically control and optimize resource usage by leveraging metering capabilities at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency to both providers and consumers of the utilized services.

[0033] Examples of service models that may be associated with example embodiments include the following:

[0034] Software as a Service (SaaS): The capability provided to the consumer is to use the provider's applications running on the cloud infrastructure. The applications are accessible from a variety of client devices through a thin client interface such as a web browser (e.g., web-based email). The consumer does not manage or control the underlying cloud infrastructure including the network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.

[0035] Platform as a Service (PaaS): The capability provided to consumers is to deploy consumer-created or acquired applications onto cloud infrastructure, where the applications are created using programming languages ​​and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but has control over the deployed applications and possible configuration of the application hosting environment.

[0036] Infrastructure as a Service (IaaS): The capabilities provided to consumers are processing, storage, networking, and other basic computing resources on which consumers can deploy and run arbitrary software, which may include operating systems and applications. Consumers do not manage or control the underlying cloud infrastructure, but have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0037] Examples of deployment models that may be associated with example embodiments include the following:

[0038] Private cloud: The cloud infrastructure is operated only for the organization. It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0039] Community cloud: The cloud infrastructure is shared by several organizations and supports a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). It can be managed by the organization or a third party and can exist on-premises or off-premises.

[0040] Public cloud: Cloud infrastructure is available to the general public or large industrial groups and is owned by an organization that sells cloud services.

[0041] Hybrid cloud: A cloud infrastructure is a combination of two or more clouds (private, community or public) that remain a unique entity but are bound together by standardized or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds).

[0042] The cloud computing environment is service-oriented, with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the core of cloud computing is an infrastructure consisting of a network of interconnected nodes.

[0043] Reference now Figure 1 , depicts an illustrative cloud computing environment 50. As shown, the cloud computing environment 50 includes one or more cloud computing nodes 10 with which a local computing device used by a cloud consumer can communicate, such as a personal digital assistant (PDA) or cellular phone 54A, a desktop computer 54B, a laptop computer 54C, and / or an automobile computer system 54N. The cloud computing nodes 10 can communicate with each other. They can be physically or virtually grouped (not shown) in one or more networks, such as a private cloud, community cloud, public cloud, or hybrid cloud, or a combination thereof, as described above. This allows the cloud computing environment 50 to provide infrastructure, platform, and / or software as a service for which the cloud consumer does not need to maintain resources on a local computing device. It should be understood that Figure 1 The types of computing devices 54A-N shown in are intended for illustration only, and cloud computing node 10 and cloud computing environment 50 may communicate with any type of computing device over any type of network and / or network addressable connection (eg, using a web browser).

[0044] Reference now Figure 2A , showing that Figure 1 A set of functional abstraction layers provided by the cloud computing environment 50. It should be understood in advance that Figure 2A The components, layers, and functions shown in are intended to be illustrative only, and embodiments of the present invention are not limited thereto. As depicted, the following layers and corresponding functions are provided: Hardware and software layer 60 includes hardware and software components. Examples of hardware components include: host 61; server 62 based on RISC (Reduced Instruction Set Computer) architecture; server 63; blade server 64; storage device 65; and network and network components 66. In some embodiments, software components include network application server software 67 and database software 68. Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual server 71; virtual storage 72; virtual network 73, including virtual private network; virtual application and operating system 74; and virtual client 75. In one example, management layer 80 can provide the functions described below.

[0045] Resource provisioning 81 provides dynamic procurement of computing resources and other resources for performing tasks within a cloud computing environment. Metering and pricing 82 provides cost tracking when utilizing resources in a cloud computing environment, as well as bills or invoices for consuming these resources. In one example, these resources may include application software licenses. Security provides authentication for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management so that the required service levels are met. Service level agreement (SLA) planning and fulfillment 85 provides pre-arrangement and procurement of cloud computing resources, where future demand is anticipated based on the SLA.

[0046] The workload layer 90 provides examples of functions that can take advantage of the cloud computing environment. Examples of workloads and functions that can be provided from this layer include: mapping and navigation 91; software development and lifecycle management 92; virtual classroom education delivery 93; data analysis processing 94; transaction processing 95; and replication data loss detection 96.

[0047] Figure 2B The corresponding Figure 2A 2. An example of a process 200 for detecting replication data loss 96 in a replication network. The process 200 may be performed by a system including a plurality of agents 212, 222, 232, and 242 installed at a plurality of sites 210, 220, 230, and 240, respectively, in a replication network. Here, each of the sites 210-240 may include replication instances 214, 224, 234, and 244 that process replication processes including data transfer between different sites 210-240. As an example, the agents 212, 222, 232, and 242 may include software programs installed within a data server at the corresponding site. As another example, the agents 212, 222, 232, and 242 may be hardware devices such as appliances, boxes, and the like that may be physically inserted into a data server. Each replication instance may include attributes of a replication operation, including the number of rows to be submitted each time, an identifier of data to be written, an identifier of data to be deleted, an identifier of data to be updated, and the like.

[0048] Figure 2BThe system shown in also includes a coordinator 250, which in this example is installed in the cloud platform. The cloud platform is located separately from the sites 210, 220, 230, and 240, although this is not required. It should be understood that the coordinator 250 can be incorporated into a server within the premises, a data server at a replication site, etc. The coordinator 250 includes a latency model 252. The latency model 252 may include one or more machine models that are trained to analyze the logs of the replication history of the data server and determine the corresponding latency values ​​of each agent 212, 222, 232, and 242 installed at each site 210, 220, 230, and 240. The latency value represents the duration of each agent (e.g., 300 seconds, 1000 seconds, etc.).

[0049] If data is lost in the transmission between the two servers, but the waiting time value of the agent installed at the sending server has not been exceeded, the coordinator 250 can request the sending server to resend the lost data. As another example, even if the waiting time value of the corresponding agent is exceeded, the coordinator 250 can also request resending.

[0050] Figures 3A-3E Various examples of additional features that can be used in association with the cloud computing environment described herein are provided. These examples should be considered additional extensions or additional examples of the embodiments described herein.

[0051] Figure 3A An example of a permissioned blockchain network 300 is shown, which features a distributed, decentralized peer-to-peer architecture. The blockchain network can interact with a cloud computing environment 50, allowing additional functionality such as peer authentication of data written to a distributed ledger. In this example, a blockchain user 302 can initiate a transaction to a permissioned blockchain 304. In this example, the transaction can be a deployment, call, or query, and can be issued through a client-side application utilizing an SDK, directly through an API, etc. The network can provide access to a regulator 306 (such as an auditor). The blockchain network operator 308 manages member permissions, such as registering a regulator 306 as an "auditor" and registering a blockchain user 302 as a "client." Auditors can be restricted to querying only the ledger, while clients can be authorized to deploy, call, and query certain types of chaincodes.

[0052] Blockchain developers 310 can write chaincodes and client applications. Blockchain developers 310 can deploy chaincodes directly to the network through an interface. In order to include credentials from traditional data sources 312 in the chaincode, developers 310 can use out-of-band connections to access the data. In this example, blockchain users 302 connect to the permissioned blockchain 304 through peer nodes 314. Before any transaction is made, peer nodes 314 retrieve the user's registration and transaction certificates from the certificate authority 316 that manages user roles and permissions. In some cases, blockchain users must have these digital certificates in order to trade on the permissioned blockchain 304. At the same time, users who attempt to utilize chaincodes may need to verify their credentials on traditional data sources 312. In order to confirm the user's authorization, the chaincode can use an out-of-band connection to the data through a traditional processing platform 318.

[0053] Figure 3B Another example of a permissioned blockchain network 320 is shown, featuring a distributed, decentralized peer-to-peer architecture. In this example, a blockchain user 322 may submit transactions to a permissioned blockchain 324. In this example, a transaction may be a deployment, call, or query, and may be issued through a client-side application utilizing an SDK, directly through an API, or the like. The network may provide access to regulators 326, such as auditors. A blockchain network operator 328 manages member permissions, such as registering regulators 326 as “auditors” and blockchain users 322 as “clients.” Auditors may be restricted to querying only the ledger, while clients may be authorized to deploy, call, and query certain types of chaincodes.

[0054] Blockchain developers 330 write chaincodes and client applications. Blockchain developers 330 can deploy chaincodes directly to the network through interfaces. In order to include credentials from traditional data sources 332 in chaincodes, developers 330 can use out-of-band connections to access data. In this example, blockchain users 322 connect to the network through peer nodes 334. Before conducting any transactions, peer nodes 334 retrieve the user's registration and transaction certificates from certificate authorities 336. In some cases, blockchain users must have these digital certificates in order to trade on the permissioned blockchain 324. At the same time, users trying to use chaincodes may need to verify their certificates on traditional data sources 332. To confirm the user's authorization, the chaincode can use an out-of-band connection to the data through a traditional processing platform 338.

[0055] In some embodiments, the blockchain herein may be a permissionless blockchain. In contrast to a permissioned blockchain that requires permission to join, anyone can join a permissionless blockchain. For example, to join a permissionless blockchain, a user can create a personal address and begin interacting with the network by submitting transactions and thereby adding entries to the ledger. Additionally, all parties may choose to run nodes on the system and employ a mining protocol to help validate transactions.

[0056] Figure 3C A process 350 of a transaction processed by a permissionless blockchain 352 including a plurality of nodes 354 is shown. A sender 356 desires to send a payment or some other form of value (e.g., a deed, medical record, contract, goods, services, or any other asset that can be encapsulated in a digital record) to a receiver 358 via the permissionless blockchain 352. In one embodiment, each of the sender device 356 and the receiver device 358 may have a digital wallet (associated with the blockchain 352) that provides user interface controls and display of transaction parameters. In response, the transaction is broadcast to the nodes 354 throughout the blockchain 352. Depending on the network parameters of the blockchain 352, the nodes verify 360 the transaction based on rules established by the creator of the permissionless blockchain 352 (which may be predefined or dynamically assigned). For example, this may include verifying the identities of the parties involved, etc. The transaction may be verified immediately, or it may be placed in a queue with other transactions, and the nodes 354 determine whether the transaction is valid based on a set of network rules.

[0057] In structure 362, valid transactions are formed into blocks and sealed with locks (hashes). This process can be performed by a mining node in node 354. The mining node can utilize additional software dedicated to mining and creating blocks of the permissionless blockchain 352. Each block can be identified by a hash (e.g., a 256-bit number, etc.) created using an algorithm agreed upon by the network. Each block can include a header, a pointer or reference to the hash of the previous block header in the chain, and a set of valid transactions. The reference to the previous block hash is associated with the creation of a secure independent block chain.

[0058] Before blocks can be added to the blockchain, they must be verified. Verification of the permissionless blockchain 352 may include proof of work (PoW) as a solution to a puzzle derived from the header of the block. Figure 3C Not shown in the example, but another process for validating blocks is Proof of Stake. Unlike Proof of Work, where an algorithm rewards miners for solving mathematical problems, with Proof of Stake, the creators of new blocks are selected in a deterministic manner based on their wealth, also defined as “stake”. Similar proofs are then performed by the selected / chosen nodes.

[0059] Through mining 364, nodes attempt to solve the block by incrementally changing a variable until the solution meets a network-wide goal. This creates a PoW, thereby ensuring a correct answer. In other words, a potential solution must prove that computing resources are exhausted in solving the problem. In some types of permissionless blockchains, miners can be rewarded with value (e.g., coins, etc.) for correctly mining a block.

[0060] Here, the PoW process, together with the block chain, makes modification of the block chain extremely difficult, as an attacker must modify all subsequent blocks in order for a modification of a block to be accepted. In addition, as new blocks are mined, the difficulty of modifying blocks increases, and the number of subsequent blocks increases. Through distribution, successfully verified blocks are distributed through the permissionless blockchain 352, and all nodes 354 add the block to the majority chain, which is an auditable ledger of the permissionless blockchain 352. In addition, the value in the transaction submitted by the sender 356 is deposited or otherwise transferred to the digital wallet of the recipient device 358.

[0061] Figure 3D and 3E Additional examples of use cases for cloud computing that may be incorporated and used herein are shown. Figure 3D An example 370 of a cloud computing environment 50 storing machine learning (artificial intelligence) data is shown. Machine learning relies on large amounts of historical data (or training data) to build predictive models for making accurate predictions about new data. Machine learning software (e.g., neural networks, etc.) can often sift through millions of records to find non-intuitive patterns.

[0062] exist Figure 3D In the example of , host platform 376 builds and deploys a machine learning model for predictive monitoring of asset 378. Here, host platform 366 may be a cloud platform, an industrial server, a web server, a personal computer, a user device, etc. Asset 378 may be any type of asset (e.g., a machine or equipment, etc.), such as an airplane, a locomotive, a turbine, medical machinery and equipment, oil and gas equipment, a ship, a vessel, a vehicle, etc. As another example, asset 378 may be a non-tangible asset, such as a stock, currency, a digital currency, insurance, etc.

[0063] The cloud computing environment 50 can be used to significantly improve both the training process 372 of the machine learning model and the prediction process 374 based on the trained machine learning model. For example, in 372, historical data can be stored on the cloud computing environment 50 by the asset 378 itself (or through a medium, not shown), rather than requiring a data scientist / engineer or another user to collect the data, which can significantly reduce the collection time required by the host platform 376 when performing predictive model training. For example, data can be directly and reliably transferred directly from its original location to the cloud computing environment 50. By using the cloud computing environment 50 to ensure the security and ownership of the collected data, smart contracts can send data directly from the asset to the individual who uses the data to build a machine learning model. This allows data to be shared between assets 378.

[0064] In addition, training the machine learning model on the collected data can be performed by the host platform 376 for several rounds of refinement and testing. Each round can be based on additional data or data that was not previously considered to help expand the knowledge of the machine learning model. In 372, the different training and testing steps (and the data associated therewith) can be stored by the host platform 376 on the cloud computing environment 50. Each refinement of the machine learning model (e.g., changes in variables, weights, etc.) can be stored in the cloud computing environment 50 to provide verifiable proof of how the model was trained and what data was used to train the model. For example, the machine learning model can be stored on a blockchain to provide verifiable evidence. In addition, when the host platform 376 has implemented the trained model, the resulting model can be stored on the cloud computing environment 50.

[0065] After the model has been trained, it can be deployed to a field environment where predictions / decisions can be made based on the execution of the final trained machine learning model. For example, in 374, the machine learning model can be used for condition-based maintenance (CBM) of assets such as aircraft, wind turbines, healthcare machines, etc. In this example, data fed back from asset 378 can be input into the machine learning model and used to make event predictions, such as failure events, error codes, etc. The determination made by executing the machine learning model at the host platform 376 can be stored on the cloud computing environment 50 to provide auditable / verifiable proof. As a non-limiting example, the machine learning model can predict a future failure / failure of a portion of the asset 378 and create an alert or notification to replace the portion. The data behind the decision can be stored by the host platform 376 and / or stored on the cloud computing environment 50, and in one embodiment, the features and / or actions described and / or depicted herein can occur on or with respect to the cloud computing environment 50.

[0066] Figure 3EAn example 380 of a quantum-safe cloud computing environment 382 is shown that implements quantum key distribution (QKD) to prevent quantum computing attacks. In this example, cloud computing users can verify each other's identities using QKD. This uses quantum particles such as photons to send information, which an eavesdropper cannot copy without destroying the quantum particles. In this way, the sender and receiver can be confident of each other's identities through the cloud computing environment.

[0067] exist Figure 3E In the example of , there are four users 384, 386, 388, and 390. Each pair of users can share a key 392 between themselves (i.e., QKD). Since there are four nodes in this example, there are six pairs of nodes, so six different keys 392 are used, including QKD AB , QKD AC , QKD AD , QKD BC , QKD BD and QKD CD Each pair can create a QKD by sending information using quantum particles such as photons, which an eavesdropper cannot copy without destroying the QKD. In this way, a pair of users can be confident of each other's identity.

[0068] The operation of the cloud computing environment 382 is based on two processes: (i) the creation of transactions, and (ii) the construction of blocks that aggregate new transactions. New transactions can be created similarly to traditional networks such as blockchain networks. Each transaction can contain information about the sender, the receiver, the creation time, the amount (or value) to be transferred, a list of reference transactions that prove that the sender has funds for the operation, etc. The transaction record is then sent to all other nodes, where it is entered into a pool of unconfirmed transactions. Here, two parties (i.e., a pair of users in 384-390) authenticate the transaction by providing their shared secret key 392 (QKD). The quantum signature can be attached to each transaction, making it very difficult to tamper with. Each node checks its entry on the local copy of the cloud computing environment 382 to verify that each transaction has sufficient funds.

[0069] Figures 4A-4D is a diagram illustrating a data error detection process within a data replication system such as a distributed data replication system. In this example, a source site (e.g., a source server, etc.) is attempting to transfer / synchronize its content to a backup copy of a source server (e.g., a target server, etc.) at a target location, and both the source and the target may include agents installed therein according to various embodiments. In addition, intermediate servers along the transmission path between the source server and the target server may also include agents. Each agent may communicate with a coordinator hosted by a cloud platform, one of the sites, etc.

[0070] Figure 4A A process 400 for transmitting replication messages between data replication sites according to an example embodiment is shown. Figure 4A , a source site 410 is backing up data to a destination site (not shown). The transmission path for transmitting data from the source site 410 to the destination site includes an intermediate site 420. It may also include multiple intermediate sites (not shown). In this example, according to various embodiments, the source site includes an agent 412 and a replication instance 414 of a workload prepared for replication. The workload may include operations to be performed, such as new data values ​​to be added, existing data values ​​to be modified, existing data values ​​to be deleted, etc. Here, the replication instance 414 can generate a replication message 402 with replication data 404 for transmission to the destination site.

[0071] In this example, agent 412 (e.g., inserted or otherwise installed on a server at source site 410) can generate one or more checksums and inject them into replication message 402. For example, agent 412 can generate a checksum (e.g., a hash value, etc.) of a first transaction (e.g., transaction A) and add it to a transaction message of a second transaction (e.g., transaction B) and a transaction message of a third transaction (e.g., transaction C). The combination of the checksum of transaction A, transaction message B with checksum A added, and transaction message C with checksum A added can be sent to the next agent / server (i.e., agent 422 at intermediate site 420). The replication operation may include dozens or even hundreds of transactions (e.g., database transactions, etc.) to be performed in order to synchronize the target site with source site 410. Here, agent 412 can randomly select transactions and then generate checksums of the randomly selected transactions. As another example, agent 412 can select predetermined transactions and use them for checksums. Then, replication message 402 is forwarded to intermediate site 420, which also includes agent 422 installed therein. The agent 422 installed at the next server may generate a checksum of transaction A and compare it with the checksum value of A stored in transaction message B or transaction message C, the selection of which transaction message may be random? The coordinator may request the corresponding agent to perform a checksum re-evaluation before / after reaching the waiting time threshold, so that it can reduce false alarms caused by waiting time. The coordinator may not require the replication server to resend the data replication message, but it will notify the user to take action on the data loss.

[0072] That is, agents can be inserted into the source server, the target destination server, and the intermediate server. The agent at the source server can inject an additional checksum value into the replicated transaction data message. In addition, the agent at the intermediate server can verify the checksum and report to the coordinator ( Figure 4C) to send a suspicion message. The agent can also be installed in the target destination server and perform the same role as the agent inserted in the intermediate server.

[0073] Reference again Figure 4A In the example in FIG. 4 , an agent 412 installed at a source site 410 (source server) can collect replication transaction messages before they are sent by replication instance 414, and add a checksum value to the message before the message is sent. The implementation of replication transaction messages may be different for different replication product providers. As an example, the message may be in XML format. In this case, the checksum may be stored anywhere in XML format with the correct schema. In other words, it can be anywhere in the message as long as the message can be correctly interpreted by the next server and the existing replication server.

[0074] Figure 4B A process 440 of verifying the status of a data copy operation based on checksum verification according to an example embodiment is shown. Figure 4B , replication message 402 is received by intermediate site 420. In response, replication instance 424 may forward replication message 402 to agent 422, which is installed locally on intermediate site 420 (server). Agent 422 may identify which transactions in replication transaction data 404 are used to create checksum values ​​406 and 408 based on the transaction ID in data replication message 402, and recreate the checksum value based on replication transaction data 404.

[0075] Next, the agent 422 may compare the recreated checksum value with the checksum values ​​406 and 408 in the replication message 402 to determine if they match. If the two match, the agent 422 determines that there is no data loss. If they do not match, there is a chance that data may be lost. The results of the determination may be transmitted from the agent 422 to the coordinator 430 hosted on the central platform. Here, the agent 422 may specify that the checksum matches (success) or the checksum does not match (suspicious).

[0076] Figure 4C FIG. 4 shows a process 450 for determining whether data loss has occurred according to an example embodiment. The process 450 may be performed by Figure 4B The coordinator 430 shown in FIG. Figure 4C , at 451, the coordinator may receive a request from a proxy (e.g., Figure 4B m, etc.) receives the checksum verification message, which may include an agent identifier (e.g., an agent ID, etc.) that is unique to an agent in the agent network. In addition, the verification message may include a status identifier of the checksum verification (e.g., pass / fail, etc.), and in addition, the verification message may include an identifier of the transaction used to generate the checksum.

[0077] At 452, the coordinator may add an entry to a tracking table (eg, Figure 4D , etc.), each agent verification may include its own entry in tracking table 470. For example, if a replication message passes through three intermediate sites (and agents installed therein), coordinator 430 will receive three verifications of the replication data included in the replication message from the three agents. Each verification message and the attributes therein may be added to an entry in tracking table 470. At 453, the coordinator may determine whether the message received at 451 indicates a successful verification (i.e., the checksum created by the agent matches the checksum in the message) or an unsuccessful verification (i.e., the checksum does not match). If the checksum verification is successful, then at 454, the coordinator may send a notification of compliance to one or more administrators of the replication operation.

[0078] However, if the checksum verification is unsuccessful, then in 455, the coordinator may determine whether a wait time value has been exceeded. The wait time value may be a time range that can ensure that the unsuccessful confirmation is not due to a false alarm (i.e., network latency). Here, the coordinator may identify how much time has passed since the replication message was sent based on the timestamp of the replication message. The coordinator may compare the amount of time to the time range in the wait time value. If the amount of time is within the time range created by the wait time value, the coordinator may send a request to the source site 410 to resend one or more transactions that were not correctly received to the intermediate site 420.

[0079] Reference again Figure 4D The wait time value can be a time range that represents an acceptable amount of wait time (e.g., 300 seconds, 1000 seconds, etc.). The wait time value can be determined by Figure 4D 456. The coordinator may determine whether or not data has been lost by the coordinator 450 ...

[0080] Figure 5A method 500 for simulating a workflow based on a simulated workflow environment according to an example embodiment is shown. For example, the method 500 may be performed by a computer system such as a cloud platform, a web server, a personal computer or other user device. Figure 5 In 510, the method may include receiving a message from an agent installed at the data replication server. Here, the message may include a status identifier of the verification and validation of the data replication operation.

[0081] At 520, the method may include identifying a wait time value associated with the data replication server. For example, the wait time value may be determined by analyzing a log / summary of historical messages from an agent installed on the data replication server. Here, the host may detect an acceptable (not erroneous) amount of wait time in a particular case under particular conditions, and use the wait time value to determine whether the checksum verification may be incomplete because all data has not yet arrived. At 530, the method may include determining whether data loss has occurred based on a status identifier and the wait time value for the checksum verification. At 540, in response to determining that data loss has occurred, the method may include sending a notification of the data loss to a computing system associated with the data replication server.

[0082] In some embodiments, the message may also include an agent identifier of an agent installed at the data replication server and an identifier of a transaction within the data replication operation. In some embodiments, the method may further include receiving a plurality of messages from an agent installed on the data replication server during the data replication operation, and generating a tracking table having identifiers of a plurality of transactions included in the data replication operation and status identifiers of checks and verifications of the plurality of transactions respectively included in the plurality of messages. In some embodiments, the method may also include executing a machine learning model on the generated tracking table to predict a wait time value. In some embodiments, the method may also include receiving a request for a status of the data replication operation, and in response to the request, displaying content from the tracking table via a user interface.

[0083] In some embodiments, the method may further include determining to request revalidation of transactions included in the data replication operation based on the number of occurrences of requests for revalidation that have been performed. In some embodiments, the method may further include receiving status identifiers of verification and confirmation of the data replication operation from a plurality of agents installed on a plurality of data replication servers included in the data replication operation, respectively. In some embodiments, the method may further include identifying transactions that were not successfully received based on the determination, and transmitting a control signal to the source data replication server to resend the transaction.

[0084] The above embodiments may be implemented in hardware, a computer program executed by a processor, firmware, or a combination thereof. The computer program may be contained on a computer-readable medium, such as a storage medium. For example, the computer program may reside in a random access memory ("RAM"), a flash memory, a read-only memory ("ROM"), an erasable programmable read-only memory ("EPROM"), an electrically erasable programmable read-only memory ("EEPROM"), a register, a hard disk, a removable disk, a compact disk read-only memory ("CD-ROM"), or any other form of storage medium known in the art.

[0085] An exemplary storage medium may be coupled to a processor such that the processor can read information from the storage medium and write information to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and the storage medium may reside in an application specific integrated circuit ("ASIC"). In an alternative, the processor and the storage medium may reside as discrete components. For example, Figure 6 An example computer system architecture 600 is shown, which may be representative of or integrated in any of the above components, among others.

[0086] Figure 6 An example system 600 is shown that supports one or more of the example embodiments described and / or depicted herein. System 600 includes a computer system / server 602 that can operate with many other general or special computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with computer system / server 602 include, but are not limited to, personal computer systems, server computer systems, thin clients, fat clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems or devices, etc.

[0087] Computer system / server 602 may be described in the general context of computer system executable instructions, such as program modules, executed by the computer system. In general, program modules may include routines, programs, objects, components, logic, data structures, etc. that perform specific tasks or implement specific abstract data types. Computer system / server 602 may be practiced in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules may be located in local and remote computer system storage media, including memory storage devices.

[0088] like Figure 6As shown, the computer system / server 602 in the cloud computing node 600 is shown in the form of a general-purpose computing device. The components of the computer system / server 602 may include, but are not limited to, one or more processors or processing units 604, a system memory 606, and a bus that couples various system components including the system memory 606 to the processor 604.

[0089] Bus refers to one or more of any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, and a processor or local bus using any of a variety of bus architectures. By way of example and not limitation, these architectures include an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0090] Computer system / server 602 generally includes various computer system readable media. Such media can be any available media that can be accessed by computer system / server 602, and it includes volatile and non-volatile media, removable and non-removable media. In one embodiment, system memory 606 implements the flow charts of other figures. System memory 606 can include computer system readable media in the form of volatile memory, such as random access memory (RAM) 610 and / or cache memory 612. Computer system / server 602 can also include other removable / non-removable, volatile / non-volatile computer system storage media. As an example only, storage system 614 can be provided for reading from and writing to non-removable, non-volatile magnetic media (not shown, and generally referred to as "hard drive"). Although not shown, a disk drive for reading from and writing to a removable, non-volatile disk (e.g., "floppy disk") can be provided, as well as an optical disk drive for reading from or writing to a removable, non-volatile optical disk such as a CD-ROM, DVD-ROM or other optical media. In this case, each can be connected to the bus through one or more data medium interfaces. As will be further depicted and described below, memory 606 may include at least one program product having a set (eg, at least one) of program modules configured to perform the functions of various embodiments of the present application.

[0091] A program / utility 616 having a set (at least one) of program modules 618, as well as an operating system, one or more application programs, other program modules, and program data may be stored in memory 606, by way of example and not limitation. Each of the operating system, one or more application programs, other program modules, and program data, or some combination thereof, may include an implementation of a networking environment. Program modules 618 generally perform the functions and / or methods of various embodiments of the application as described herein.

[0092] As will be appreciated by those skilled in the art, various aspects of the present application may be implemented as a system, method or computer program product. Therefore, various aspects of the present application may take the form of a complete hardware embodiment, a complete software embodiment (including firmware, resident software, microcode, etc.) or a combination of software and hardware embodiments, which may be collectively referred to herein as a "circuit", "module" or "system". In addition, various aspects of the present application may take the form of a computer program product, which includes one or more computer-readable media having a computer-readable program code embodied therewith.

[0093] The computer system / server 602 may also communicate with one or more external devices 620, such as a keyboard, a pointing device, a display 622, etc.; one or more devices that enable a user to interact with the computer system / server 602; and / or any device that enables the computer system / server 602 to communicate with one or more other computing devices (e.g., a network card, a modem, etc.). Such communication may occur via an I / O interface 624. In addition, the computer system / server 602 may communicate with one or more networks, such as a local area network (LAN), a general wide area network (WAN), and / or a public network (e.g., the Internet), via a network adapter 626. As depicted, the network adapter 626 communicates with other components of the computer system / server 602 via a bus. It should be understood that, although not shown, other hardware and / or software components may be used in conjunction with the computer system / server 602. Examples include, but are not limited to, microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data archival storage systems, etc.

[0094] Although exemplary embodiments of at least one of the systems, methods, and non-transient computer-readable media are shown in the drawings and described in the foregoing detailed description, it will be understood that the present application is not limited to the disclosed embodiments, but is capable of numerous rearrangements, modifications, and substitutions as set forth and defined in the appended claims. For example, the capabilities of the systems of the drawings may be performed by one or more of the modules or components described herein or in a distributed architecture, and may include a transmitter, a receiver, or a pairing of both. For example, all or part of the functions performed by each module may be performed by one or more of these modules. In addition, the functions described herein may be performed at various times and may be related to various events inside or outside the module or component. Moreover, the information sent between the various modules may be sent between the modules via at least one of the following: a data network, the Internet, a voice network, an Internet protocol network, a wireless device, a wired device, and / or via multiple protocols. Moreover, messages sent or received by any module may be sent or received directly and / or via one or more other modules.

[0095] Those skilled in the art will appreciate that a "system" may be implemented as a personal computer, server, console, personal digital assistant (PDA), cellular phone, tablet computing device, smart phone, or any other suitable computing device, or combination of devices. Presenting the above functions as being performed by a "system" is not intended to limit the scope of the present application in any way, but is intended to provide an example of many embodiments. In fact, the methods, systems, and devices disclosed herein may be implemented in localized and distributed forms consistent with computing technology.

[0096] It should be noted that some of the system features described in this specification have been presented as modules in order to more specifically emphasize their implementation independence. For example, a module can be implemented as a hardware circuit including a custom very large scale integration (VLSI) circuit or gate array, an off-the-shelf semiconductor such as a logic chip, a transistor, or other discrete components. A module can also be implemented in a programmable hardware device such as a field programmable gate array, a programmable array logic, a programmable logic device, a graphics processing unit, etc.

[0097] Modules can also be implemented at least in part with software executed by various types of processors. The identification unit of the executable code can, for example, include one or more physical or logical blocks of computer instructions, which can, for example, be organized as objects, processes, or functions. However, the executable code of the identification module does not need to be physically located together, but can include different instructions stored in different locations, which, when logically combined together, include the module and achieve the purpose specified by the module. In addition, the module can be stored on a computer-readable medium, which can be, for example, a hard drive, a flash memory device, a random access memory (RAM), a magnetic tape, or any other such medium for storing data.

[0098] In practice, a module of executable code may be a single instruction or multiple instructions, and may even be distributed over several different code segments, between different programs, and across several memory devices. Similarly, operational data may be identified and shown within a module, and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set, or may be distributed over different locations including different storage devices, and may exist, at least in part, only as electronic signals on a system or network.

[0099] It is easy to understand that the components of the present application, as generally described and shown in the drawings herein, can be arranged and designed in a variety of different configurations. Therefore, the detailed description of the embodiments is not intended to limit the scope of the claimed application, but is only a representative of the selected embodiments of the present application.

[0100] Those skilled in the art will readily appreciate that the above content may be practiced with steps in different orders and / or with hardware elements having configurations different from those disclosed. Therefore, although the present application has been described based on these preferred embodiments, some modifications, variations, and alternative configurations will be apparent to those skilled in the art.

[0101] Although preferred embodiments of the present application have been described, it should be understood that the described embodiments are merely illustrative and that the scope of the present application is limited solely by the appended claims when considering its full range of equivalents and modifications (e.g., protocols, hardware devices, software platforms, etc.).

Claims

1. A device comprising: a network interface configured to receive a message from an agent installed at a data replication server, the message including a status identifier of a checksum validation of a data replication operation; as well as The processor is configured as: identifying a latency value associated with the data replication server; determining whether data loss has occurred based on the checksum verified status identifier and the wait time value; as well as In response to determining that the data loss has occurred, a notification of the data loss is sent via the network interface to a computing system associated with the data replication server.

2. The device according to claim 1, wherein: The message further includes an agent identifier of the agent installed at the data replication server and an identifier of a transaction within the data replication operation.

3. The device according to claim 1, wherein: The network interface receives a plurality of messages from the agent installed on the data replication server during the data replication operation, and the processor is configured to generate a tracking table having identifiers of a plurality of transactions included in the data replication operation and status identifiers of checks and verifications of the plurality of transactions respectively included in the plurality of messages.

4. The device according to claim 3, wherein: The processor is further configured to execute a machine learning model on the generated tracking table to predict the latency value.

5. The device according to claim 3, wherein: The processor is further configured to receive a request for a status of the data copy operation and, in response to the request, display content from the tracking table via a user interface.

6. The device according to claim 1, wherein: The processor is further configured to determine to request revalidation of transactions included in the data copy operation based on a number of occurrences of requests for revalidation that have been performed.

7. The device according to claim 1, wherein: The processor is further configured to receive status identifiers of verification and validation of the data replication operation from a plurality of agents respectively installed on a plurality of data replication servers included in the data replication operation.

8. The device according to claim 1, wherein: The processor is further configured to identify transactions that were not successfully received based on the determination, and to send a control signal to a source data replication server to resend the transactions.

9. A method comprising: receiving a message from an agent installed at a data replication server, the message including a status identifier of a checksum validation of a data replication operation; identifying a latency value associated with the data replication server; determining whether data loss has occurred based on the checksum verified status identifier and the wait time value; as well as In response to determining that the data loss has occurred, a notification of the data loss is sent to a computing system associated with the data replication server.

10. The method according to claim 9, wherein: The message further includes an agent identifier of the agent installed at the data replication server and an identifier of a transaction within the data replication operation.

11. The method according to claim 9, wherein: The method further includes receiving a plurality of messages from the agent installed on the data replication server during the data replication operation, and generating a tracking table having identifiers of a plurality of transactions included in the data replication operation and status identifiers of checks and verifications of the plurality of transactions respectively included in the plurality of messages.

12. The method according to claim 11, wherein: The method further includes executing a machine learning model on the generated tracking table to predict the latency value.

13. The method according to claim 11, wherein: The method further includes receiving a request for a status of the data copy operation, and in response to the request, displaying contents from the tracking table via a user interface.

14. The method according to claim 9, wherein: The method further includes determining to request revalidation of transactions included in the data copy operation based on a number of occurrences of requests for revalidation that have been performed.

15. The method according to claim 9, wherein: The method further includes receiving status identifiers of verification and validation of the data replication operation from a plurality of agents respectively installed on a plurality of data replication servers included in the data replication operation.

16. The method according to claim 9, wherein: The method further includes identifying transactions that were not successfully received based on the determination, and sending a control signal to the source data replication server to resend the transactions.

17. A computer-readable storage medium comprising instructions which, when read by a processor, cause the processor to perform a method comprising: receiving a message from an agent installed at a data replication server, the message including a status identifier of a checksum validation of a data replication operation; identifying a latency value associated with the data replication server; determining whether data loss has occurred based on the checksum verified status identifier and the wait time value; as well as In response to determining that the data loss has occurred, a notification of the data loss is sent to a computing system associated with the data replication server.

18. The computer-readable storage medium of claim 17, wherein: The message further includes an agent identifier of the agent installed at the data replication server and an identifier of a transaction within the data replication operation.

19. The computer-readable storage medium of claim 17, wherein: The method further includes receiving a plurality of messages from the agent installed on the data replication server during the data replication operation, and generating a tracking table having identifiers of a plurality of transactions included in the data replication operation and status identifiers of checks and verifications of the plurality of transactions respectively included in the plurality of messages.

20. The computer-readable storage medium of claim 19, wherein: The method further includes executing a machine learning model on the generated tracking table to predict the latency value.

Images