System and method for parallel transmission of data between computer systems

Abstract

Embodiments of the present disclosure include techniques for moving data. In one embodiment, a computer system imports and / or exports data. The computer system may be an application server coupled to a content management system accessible through a plurality of tenant front ends. Data movement may move content packages between systems using multiple jobs executing in parallel. In one embodiment, before a job is executed, other jobs are checked to ensure that any overlap does not cause data conflicts.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is related to and filed concurrently with the following: U.S. patent application serial number (unassigned; agent docket number 000005-105500US), filed on December 6, 2023, with inventor Sahana DurgamUdaya, entitled “SYSTEMS AND METHODS FOR AUTHORIZED MOVEMENT OF INFORMATION BETWEEN COMPUTER SYSTEMS,” the disclosure of which is incorporated herein by reference.

[0003] This application is related to and filed concurrently with the following: U.S. patent application serial number (unassigned; agent docket number 000005-105600US), filed on December 6, 2023, with inventors SahanaDurgam Udaya and Pranav Kumar, entitled “SYSTEMS AND METHODS FOR SCHEDULING PACKAGESTO SYNCHRONIZE CONTENT ACROSS COMPUTER SYSTEMS,” the disclosure of which is incorporated herein by reference.

[0004] This application is related to and filed concurrently with the following: U.S. patent application serial number (unassigned; agent docket number 000005-106300US), filed on December 6, 2023, entitled “SYSTEMS AND METHODS FOR COPYING DATA BETWEEN COMPUTER SYSTEMS,” with inventors SahanaDurgam Udaya, Soumya Basavaraju, Abhishek Nagendra, Ashokkumar KandasamyNarayanan and Mickey Wong, the disclosure of which is incorporated herein by reference.

[0005] This application is related to and filed concurrently with the following: U.S. patent application serial number (unassigned; agent docket number 000005-106400US), filed on December 6, 2023, with inventor SahanaDurgam Udaya, entitled “SYSTEMS AND METHODS FOR STORING AND RETRIEVING PUBLIC DATA,” the disclosure of which is incorporated herein by reference. Technical Field

[0006] The present disclosure relates generally to software systems and, more particularly, to systems and methods for transferring data in parallel between computer systems. Background Art

[0007] Computer systems require data to produce useful and meaningful results. Data preparation and analysis can involve complex, time-consuming relationship preparation, visualization, and data compilation. When data is prepared in this manner, it can be beneficial to share data across multiple computer systems. However, moving complex data structures across different systems can be challenging. One particular challenge is ensuring that the latest version of the data is transmitted. In some cases, the data being transmitted may be an older version. Therefore, when newer data is available, the new data can be overwritten with the old data and / or the old data can be retrieved from the system.

[0008] The present disclosure addresses these and other challenges and is directed to techniques for moving data between computer systems. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 A system for moving data according to an embodiment is shown.

[0010] Figure 2 A method for moving data according to an embodiment is shown.

[0011] Figure 3A An example system according to another embodiment is shown.

[0012] Figure 3B An example package according to an embodiment is shown.

[0013] Figure 3C Example data objects are shown according to an embodiment.

[0014] Figure 3D A transmission process according to an embodiment is shown.

[0015] Figure 4A An example flow chart for creating a data packet according to an embodiment is shown.

[0016] Figure 4B An example flow chart for triggering an export according to an embodiment is shown.

[0017] Figure 4C An example flow chart for importing data into a server according to an embodiment is shown.

[0018] Figure 4D An example flow chart for moving data according to an embodiment is shown.

[0019] Figure 4E An example flow chart for moving data according to an embodiment is shown.

[0020] Figure 5 The hardware of a special purpose computing system configured according to the above disclosure is shown. DETAILED DESCRIPTION

[0021] This article describes technologies for backing up data. In the following description, for the purpose of explanation, many examples and specific details are set forth in order to provide a thorough understanding of some embodiments. Various embodiments defined by the claims may include some or all of the features in these examples alone or in combination with other features described below, and may further include modifications and equivalents of the features and concepts described herein.

[0022] Figure 1 A system for moving data according to an embodiment is shown. Features and advantages of the present invention include techniques for moving data between computer systems using an automated process. For example, the process can prevent incorrect versions of data from being moved, advantageously ensuring that data moved from one computer system to another is the correct version and / or preventing overwriting. A user can use a front end 101 to interface with a computer system. The front end 101 may include a user interface (UI) 101a that allows a user to access other computer systems, such as computer system 102 and / or computer system 103. In some embodiments, computer system 103 may be an application server (also referred to as a back end) that stores application data in a database 104. An example application server is a server that hosts an application or software that delivers a business application through a communication protocol. In addition, in some embodiments, computer system 102 may be a content management system. An example content management system (CMS) is an application for managing content that allows multiple contributors to create, edit, and publish. Content in a CMS is typically stored in a database and displayed in a presentation layer based on a set of templates (such as a website), which can be accessed, for example, through the front end 101. Content management system 102 may include a database 113 and an object data store 112. Object data store 112 may store objects, eg, an object is a data structure that holds multiple attributes of data and metadata.

[0023] In various embodiments, users can use the techniques described herein to move data from content management system 102 to server 103 (e.g., data is imported to server) or to move data from server 103 to content management system 102 (e.g., data is exported from server). For example, content management system 102 can similarly import data from server or export data to server. In some embodiments, users of multiple different application servers 103 can store data generated locally on server 103 in content management system 102 to share data with other users on other application servers (not shown). Therefore, it is desirable to import and export data between content management system 102 and server 103. In some embodiments, data can include data packets. Data packets transferred between computer systems can include multiple data objects, metadata describing multiple data objects and dependencies between data objects, and information summarizing data packets. Example objects in object data stores that can be shared include objects corresponding to complex analytical content (e.g., models, stories, visualizations, dimensions, connections, etc. of data), which can be used to discover unseen data patterns to improve, for example, business productivity.

[0024] To facilitate the movement of data between systems, the content management system 102 includes an automated movement process 110. In various embodiments, the movement of data between systems may include starting an automated process 110 to move data between a first computer system and a second computer system. Advantageously, data may be moved in parallel. For example, the automated process 110 may instantiate multiple jobs 111a-n to move data units between a first computer system and a second computer system. In some embodiments, the data unit is a data packet, which includes multiple data objects, metadata describing the dependencies between multiple data objects and data objects, and information summarizing the data packet. For example, each job 111a-n may assemble and move a data packet. However, because the data is shared, in various scenarios, for example, it may be desirable to ensure that the data being moved is the latest version and / or that other users do not modify the data during the process of being moved. Therefore, before executing each job 111a-n, the system may determine whether there is overlap between the data units of the job. For example, if one job is moving data out of the content management system and another job is moving a modified version of the same data out of the content management system, overlap may be detected. For example, when there is no overlap between data units moved by any of the currently executing jobs 111a-n and the particular job to be executed, the job may be executed.Other examples and advantages of the present technology are further described below.

[0025] Figure 2A method for moving data according to an embodiment is shown. At 201, the system initiates an automated process to move data between a first computer system and a second computer system. At 202, the system instantiates multiple jobs to move data units between the first computer system and the second computer system. At 203, before executing each job, the system determines whether there is overlap between the data units of the jobs. At 204, when there is no overlap between the data units moved by the currently executed job and each specific job, the system executes each specific job.

[0026] Figure 3A An example database system according to another embodiment is shown. In this example, a user accesses a content management system 302 and a server 303 through a tenant front end 301. The content management system 302 includes a content manager service 310, a content manager database (DB) server 311, an object data store 312, and a database 313.

[0027] For example, the present disclosure can be used in the context of an analytical content network in a cloud computer system that combines business intelligence (BI) and planning and forecasting capabilities. In any business intelligence application, analytical content (models, stories, visualizations, etc.) plays a central role in discovering unseen patterns to enhance business productivity. Therefore, sharing analytical content across users is very helpful for better collaboration. In addition, standard content templates can be reused by all users by inserting their corresponding data. This infrastructure for sharing analytical content is sometimes referred to as an "analytic content network" (ACN). The content entity containing the content to be shared is referred to as a "package."

[0028] ACN can be arranged as a central component through landscape, which is all connected. Application landscape is a coherent collection of interconnected applications usually within an enterprise, business or organization, which is usually associated with, for example, different geographical regions. Logically, ACN is "a global content network" that can provide or share any content with servers and users across landscapes. ACN can support the following end-user workflows. Content creators create content in the form of stories, models, dimensions, connections, value-driven trees (VDTs), etc. If authorized, the user can then export the content from a tenant (a part of the system resources securely assigned to a specific group) to ACN by creating a "content package", which can contain any number of these content items and, for example, share the content item with multiple other tenants. Another SAC content user can view all available content packages in their list and import those packages related to their analysis workflow. This includes public content (templates or demo content) and private content (privately shared with them). In order to achieve cross-tenant sharing, content is bundled in so-called "packages". For example, a package may contain details of each object present in the package, dependency information between these objects, and an overview summarizing the content details. For example, data objects are stored in data store 312, and data describing data objects and packages may be stored in database 313. Figure 3B An example package is shown in FIG. 4 , which includes package metadata 320 and data objects 321 a - n , wherein data objects 321 a - n include object metadata, for example. Figure 3C An example data object 321x is shown in FIG.

[0029] Embodiments of the present disclosure may include three steps for transferring data content and objects across tenants, such as Figure 3D At 330, the source creates a package, at 331, the source shares the package to the destination, and at 332, the destination imports the package.

[0030] Figure 4AAn example flow chart for creating a data package according to an embodiment is shown. At 410, a user logs in to a tenant front end 401 and clicks on the "New Export" option to create a new package. Then, at 411, the user selects an object, such as a story, model, dimension, currency, role, folder, etc. For example, the user can provide information such as a package name, description, details, and can optionally be shared with a specific destination (e.g., landscape, server, other users). The user can view the package summary and export the package to create a new package. At 412, the content manager service 403 obtains the selected object, metadata, and object content from the server 402. At 413, the content is sent from the server 402 to the service 403. At 414, the package metadata is saved by the service 403 in the content manager DB server 404. At 415, the package metadata and package content are saved in the object store 405, thereby creating a content package to be transferred. At 416, a confirmation is sent to the tenant front end. For example, the content management system can now receive package requests from other tenants.

[0031] Figure 4B An example flow chart for triggering export according to an embodiment is shown. At 420, a user of the tenant front end 401 selects a package to be edited. At 421, the tenant 401 obtains package details from the content manager service 403. The user can edit the content on the front end 401 and then trigger the export at 422. The content manager 403 then initiates the export, as further described below. Sharing and exporting content may create several conflicting scenarios. For example, package A may be created by user A, which includes story 1, model 1, model 2, dimension 1, connection 1. However, multiple users can modify package A at the same or different time periods. For example, another user C modifies package A by adding another object data action 1 (in addition to story 1, model 1, model 2, dimension 1, connection 1) and re-exports package A. User D can modify package A and re-export package A by deleting model 2 (including story 1, model 1, dimension 1, connection 1). Here, user C and user D are unaware of each other's actions. If the changes made by user C and user D are at different times, no data is lost. Package A contains story 1, model 1, dimension 1, connection 1, data action 1. However, if user C and user D data movements are triggered at overlapping times, there is a chance of overwriting, for example resulting in the loss of one of the user updates.

[0032] Figure 4CAn example flow chart for importing data into a server according to an embodiment is shown. The process for importing a package into a backend server (SAC XS) 402 is shown below. At 430, a user logs in to a tenant 401 and clicks on a package to be imported. Check the grouping summary and update the import preferences and any object locations as needed. Trigger the import at 431. Here, the analytical content owned by each tenant is processed and stored in the backend server (SAC XS) 402. The content management system (CMS) receives an import package request from any tenant of the server 402. The content manager service 403 obtains package metadata from the content manager DB server 404 at 432. The package metadata can be used to obtain package content from the object storage 405 at 433. The service 403 receives the package and sends the package to the server 402 at 434. For example, a content structure such as an object is created in the server 402 using the information in the package. The content is now imported into the server 402. At 435, confirmation of the package import is received by the tenant 401.

[0033] One issue with significant packages from the CMS to the server is that packages can contain public objects.

[0034] If tenant 1 has package:

[0035] Package A - Story 1, Model 1, Model 2, Dimension 1, Connection 1; and

[0036] Package B - Story 1, Model 1, Currency 3

[0037] Then Story 1 and Model 1 are common objects.

[0038] Therefore, if more than one user triggers the import of a package containing the same set of objects at the same time, there is a chance that data overwrites may occur. For example, the following scenario may occur. User C creates (exports) Story 1, adds a table view, and creates Package A containing Story 1 and its dependencies, and if User C imports Package A in the following import order (for example, at 4 pm and takes one hour to complete): Dimension 1, Connection 1, Model 1, Model 2, Story 1. User D modifies Story 1, adds a pie chart, a line chart (in addition to the table view added above), and creates (exports) Package B containing Story 1 and its dependencies, where User D imports Package B at 4:30 pm with the import order: Currency 3, Model 1, Story 1. For the above scenario, if Package A and Package B are executed at non-overlapping times, there is no conflict. However, if Package A import is in progress when Package B import is initiated, there is a chance that Story 1, Model 1 objects will be overwritten. Package B imports Story 1 (Table View, Pie Chart, and Line Chart). For example, Package A imports Story 1 (Table View) and may overwrite the changes made by User D. Note that usually the exported object structure is maintained so it can be recreated in another system. If new elements are added / removed from a package (story), the package in the CMS will not be affected.

[0039] Features and advantages of the present disclosure allow for parallel post-transfer analysis of already executed jobs (packets and / or content) to move data faster between systems without overlap or conflict to avoid data loss as described above. Figure 4D The process for exporting a job is shown. A user triggers a new export job at 440 using tenant 401. At 441, the tenant creates an export job on CMS 408. At 442, the job is selected for execution. At 443, the CMS checks whether another export for the same package has been executed on tenant 401. At 444, CMS 408 verifies the other exports for tenant 401. During verification, a query is sent to the database to check whether there are any jobs running in the tenant. If so, the system determines, for example, whether there are conflicting objects across the running job and the current job. CMS 408 can poll pending jobs and check database (DB) 409 to execute export jobs running on the CMS for the tenant. If any pending jobs are being executed for the tenant, the system checks whether the currently executed job is for the same package on the same tenant. If so, the current job cannot be selected for execution, and the job is returned to the job queue. Otherwise, the current job can be selected for execution. In various embodiments, the system checks whether another job for the tenant is already executing, and if so, other jobs are waiting in the queue. Additionally, for example, the system can check whether exports for the same package are being requested multiple times. At 445, the job is executed.

[0040] Figure 4EThe process of importing a job is shown. A user triggers a new import job at 450. At 451, tenant 401 sends a message to CMS408 to create a new import job. At 452, CMS 408 selects a job to execute. CMS 408 can poll pending jobs. For example, CMS 408 can check database 409 for executing import jobs for tenant 401. If there are any jobs, the system can check at 453 whether the currently executed job and the new job share the same set of objects on the same tenant. The system can verify other imports for tenant 401 at 454, for example, to determine whether the currently executed job and the new job share the same set of objects on the same tenant. If another import with the same objects has been executed on tenant 401, the job cannot be selected and the job is returned to the queue. Otherwise, the job can be selected to execute. The job is executed at 455.

[0041] Using the techniques described in the above examples, ACN-supported package transfers can be performed in parallel to reduce the time spent exporting or importing multiple packages, improve the use of resources (e.g., number of DB connection pools, instances, and threads), and reduce the time spent during tenant migrations, which may involve batch package transfers.

[0042] Figure 5 The hardware of a special purpose computing system 500 configured according to the above disclosure is shown. The following hardware description is only an example. It will be understood that various computer topologies can be used to implement the above described technology. Figure 5 501 for processing information. The computer system 510 includes a bus 505 or other communication mechanism for transmitting information, and one or more processors 501 coupled to the bus 505 for processing information. The computer system 510 also includes a memory 502 coupled to the bus 505 for storing information and instructions to be executed by the processor 501, including information and instructions for performing some of the techniques described above, for example. The memory 502 can also be used to store programs executed by the processor (or multiple) 501. Possible implementations of the memory 502 can be, but are not limited to, random access memory (RAM), read-only memory (ROM), or both. A storage device 503 is also provided for storing information and instructions. Common forms of storage devices include, for example, hard drives, magnetic disks, optical disks, CD-ROMs, DVDs, solid-state disks, flash memory or other non-volatile memories, USB memory cards, or any other electronic storage media from which a computer can read. For example, the storage device 503 may include source code, binary code, or software files for performing the above-mentioned techniques. The storage device 503 and the memory 502 are both examples of non-temporary computer-readable storage media (also referred to as storage media).

[0043] In some systems, computer system 510 may be coupled to a display 512 via bus 505 for displaying information to a computer user. Input devices 511, such as a keyboard, touch screen, and / or mouse, are coupled to bus 505 for communicating information and command selections from the user to processor 501. The combination of these components allows the user to communicate with the system. In some systems, bus 505 represents, for example, multiple dedicated buses for coupling various components of a computer together.

[0044] The computer system 510 also includes a network interface 504 coupled to the bus 505. The network interface 504 can provide two-way data communication between the computer system 510 and the local network 520. The network 520 can represent one or more networking technologies, such as Ethernet, a local wireless network (e.g., WiFi), or a cellular network. For example, the network interface 504 can be a wireless or wired connection. For example, the computer system 510 can send and receive information across a wired or wireless local area network, an intranet, or a cellular network to the Internet 530 through the network interface 504. In some embodiments, the front end (e.g., a browser) can, for example, access data and features on a back-end software system, where the back-end software system can reside on multiple different hardware servers on prem5 31 or reside on servers 532-534 across a network 530 (e.g., an extranet or the Internet). One or more of the servers 532-534 can also reside in, for example, a cloud computing environment.

[0045] Further examples

[0046] Each of the following non-limiting features in the following examples may exist independently, or may be combined with one or more of the other features in the following examples in various permutations or combinations. In various embodiments, the present disclosure may be implemented as a system, method, or computer-readable medium.

[0047] Embodiments of the present disclosure may include a system, method, or computer-readable medium. In one embodiment, the present disclosure includes a computer system comprising: at least one processor and at least one non-transitory computer-readable medium (e.g., memory) storing computer-executable instructions, wherein, when executed by at least one processor, the computer-executable instructions cause the computer system to perform a method as described herein and in the following examples. In another embodiment, the present disclosure includes a non-transitory computer-readable medium storing computer-executable instructions, wherein the computer-executable instructions, when executed by at least one processor, perform a method as described herein and in the following examples.

[0048] In some embodiments, a method of moving data includes: initiating an automated process to move data between a first computer system and a second computer system; instantiating multiple jobs to move data units between the first computer system and the second computer system; before executing each job, determining whether there is overlap between the data units of the jobs; and executing each specific job when there is no overlap between the data units moved by the currently executed job and each specific job.

[0049] In some embodiments, the first computer system is an application server and the second computer system is a content management system.

[0050] In some embodiments, the content management system instantiates a plurality of jobs, and wherein the data unit is a data package including a plurality of data objects, metadata describing the plurality of data objects and dependencies between the data objects, and information summarizing the data package.

[0051] In some embodiments, the automated process for moving data is a data import operation.

[0052] In some embodiments, determining whether there is overlap between data units of the jobs includes determining whether the currently executing job and the new job share the same set of objects, wherein the new job is not executed when the currently executing job and the new job share the same set of objects.

[0053] In some embodiments, the automated process of moving data is a data export operation.

[0054] In some embodiments, determining whether there is overlap between data units of the jobs includes determining whether the currently executing job and the new job operate on the same data packet, wherein when the currently executing job and the new job operate on the same data packet, the new job is not executed.

[0055] The above description shows various embodiments and examples of how aspects of some embodiments may be implemented. The above examples and embodiments should not be considered to be the only embodiments, and are presented to illustrate the flexibility and advantages of some embodiments defined by the appended claims. Based on the above disclosure and the following claims, other arrangements, embodiments, implementations and equivalents may be adopted without departing from the scope of the invention defined by the claims.

Claims

1. A method for moving data, comprising: initiating an automated process to move data between the first computer system and the second computer system; instantiating a plurality of jobs to move data units between a first computer system and a second computer system; Before executing each job, determining whether there is overlap between data units of the job; as well as When there is no overlap between data units moved by the currently executed job and each specific job, each specific job is executed.

2. The method according to claim 1, wherein: The first computer system is an application server and the second computer system is a content management system.

3. The method according to claim 2, wherein: The content management system instantiates a plurality of jobs, and wherein the data unit is a data package, the data package including a plurality of data objects, metadata describing the plurality of data objects and dependencies between the data objects, and information summarizing the data package.

4. The method according to claim 3, wherein: The automated process of moving data is the data import operation.

5. The method according to claim 4, wherein: Determining whether there is overlap between data units of the jobs includes determining whether the currently executing job and the new job share the same set of objects, wherein the new job is not executed when the currently executing job and the new job share the same set of objects.

6. The method according to claim 3, wherein: The automated process of moving data is the data export operation.

7. The method according to claim 6, wherein: Determining whether there is overlap between data units of the jobs includes determining whether the currently executing job and the new job operate on the same data packet, wherein when the currently executing job and the new job operate on the same data packet, the new job is not executed.

8. A computer system comprising: at least one processor; At least one non-transitory computer-readable medium storing computer-executable instructions, which when executed by at least one processor cause a computer system to perform a method of moving data, the method comprising: initiating an automated process to move data between the first computer system and the second computer system; instantiating a plurality of jobs to move data units between a first computer system and a second computer system; Before executing each job, determining whether there is overlap between data units of the job; and When there is no overlap between data units moved by the currently executed job and each specific job, each specific job is executed.

9. The computer system according to claim 8, wherein: The first computer system is an application server and the second computer system is a content management system.

10. The computer system according to claim 9, wherein: The content management system instantiates a plurality of jobs, and wherein the data unit is a data package, the data package including a plurality of data objects, metadata describing the plurality of data objects and dependencies between the data objects, and information summarizing the data package.

11. The computer system according to claim 10, wherein: The automated process of moving data is the data import operation.

12. The computer system according to claim 11, wherein: Determining whether there is overlap between data units of the jobs includes determining whether the currently executing job and the new job share the same set of objects, wherein the new job is not executed when the currently executing job and the new job share the same set of objects.

13. The computer system according to claim 10, wherein: The automated process of moving data is the data export operation.

14. The computer system of claim 13, wherein: Determining whether there is overlap between data units of the jobs includes determining whether the currently executing job and the new job operate on the same data packet, wherein when the currently executing job and the new job operate on the same data packet, the new job is not executed.

15. A non-transitory computer readable medium storing computer executable instructions, the computer executable instructions when executed by at least one processor performing a method of moving data, the method comprising: initiating an automated process to move data between the first computer system and the second computer system; instantiating a plurality of jobs to move data units between a first computer system and a second computer system; Before executing each job, determining whether there is overlap between data units of the job; as well as When there is no overlap between data units moved by the currently executed job and each specific job, each specific job is executed.

16. The non-transitory computer readable medium of claim 15, wherein: The first computer system is an application server and the second computer system is a content management system.

17. The non-transitory computer readable medium of claim 16, wherein: The content management system instantiates a plurality of jobs, and wherein the data unit is a data package, the data package including a plurality of data objects, metadata describing the plurality of data objects and dependencies between the data objects, and information summarizing the data package.

18. The non-transitory computer readable medium of claim 17, wherein: The automated process of moving data is the data import operation.

19. The non-transitory computer readable medium of claim 18, wherein: Determining whether there is overlap between data units of the jobs includes determining whether the currently executing job and the new job share the same set of objects, wherein the new job is not executed when the currently executing job and the new job share the same set of objects.

20. The non-transitory computer readable medium of claim 17, wherein: The automated process of moving data is a data export operation, and wherein determining whether there is overlap between data units of jobs includes determining whether a currently executing job and a new job operate on the same data packet, wherein when the currently executing job and the new job operate on the same data packet, the new job is not executed.

Images