Data coordination system
The data coordination controller generates combined dimension members and refreshes the data set using execution delimiter values, solving the costly and time-consuming problem of data coordination in the prior art, and achieving fast and accurate data set integrity checks.
Patent Information
- Application Number
- CN202380022221.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-01
- Filing Date
- 2023-03-28
- Publication Date
- 2025-09-02
- Estimated Expiration
- 2043-03-28
AI Technical Summary
Existing data coordination technologies are expensive, time-consuming and error-prone, making it difficult to effectively check the integrity and consistency of data during data migration or after system updates.
Through the data coordination controller, dimensional members and bridge members are used to generate combined dimension members, refresh the data set with execution delimiter values, generate data coordination reports, and provide a graphical user interface to display coordination results.
Reduces data coordination time, provides richer and faster visible information trajectories, allowing users to quickly identify and resolve problems, ensuring the integrity and consistency of the data set.
Smart Images

Figure CN118805165B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to data reconciliation. Background Art
[0002] Data reconciliation is a process used to validate data, for example, during or after a data migration. That is, the target data can be compared to the original data to ensure that it was correctly migrated, or in the case of system updates or changes, to ensure that data that should remain static has not changed. In this context, the data reconciliation process checks for missing records or values, incorrect records or values, duplicate values or records, and various formatting issues. Conventional data reconciliation techniques are expensive, time-consuming, error-prone, or some combination of these. Summary of the Invention
[0003] One aspect of the present disclosure provides a system or computer-implemented method that, when executed on data processing hardware, causes the data processing hardware to perform operations for data reconciliation. The operations include receiving a data reconciliation request from a user device in communication with the data processing hardware, the data reconciliation request requesting data reconciliation for a first data set and a second data set. The operations also include obtaining a first data set from a first data source. Here, the first data set includes one or more dimensions, each dimension having multiple dimension members. The operations also include obtaining a second data set from a second data source. Here, the second data set includes one or more dimensions, each dimension having multiple dimension members. For each corresponding dimension of the first data set, the operations also include obtaining a corresponding bridge member that associates the corresponding dimension of the first data set with the corresponding dimension of the second data set. The operations also include generating a first set of combined dimension members using each pair of dimension members of the one or more dimensions of the first data set and the corresponding bridge member. The operations also include generating a second set of combined dimension members using each pair of dimension members of the one or more dimensions of the second data set and the corresponding bridge member. The operations also include generating a third set of combined dimension members using the first set of combined dimension members and the second set of combined dimension members, and generating a data reconciliation report based on the third set of combined dimension members.
[0004] Another aspect of the present disclosure provides a computer-implemented method that, when executed on data processing hardware, causes the data processing hardware to perform operations for data reconciliation. The operations include receiving a data reconciliation request from a user device in communication with the data processing hardware, the data reconciliation request requesting data reconciliation for a first data set and a second data set. The operations also include obtaining a first data set from a first data source. Here, the first data set includes one or more dimensions, each dimension having a plurality of dimension members. The operations also include obtaining a second data set from a second data source. Here, the second data set includes one or more dimensions, each dimension having a plurality of dimension members. For each corresponding dimension of the first data set, the operations also include obtaining a corresponding bridge member that associates the corresponding dimension of the first data set with the corresponding dimension of the second data set. The operations also include generating a first set of combined dimension members using each pair of dimension members of the one or more dimensions of the first data set and the corresponding bridge member. The operations also include generating a second set of combined dimension members using each pair of dimension members of the one or more dimensions of the second data set and the corresponding bridge member. The operations also include refreshing the first set of combined dimension members and the second set of combined dimension members based on an execution delimiter value corresponding to the number of dimension members to be refreshed simultaneously. The operation further includes generating a third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members, and generating a data reconciliation report based on the third set of combined dimension members.
[0005] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, refreshing the first set of combined dimension members and the second set of combined dimension members based on an execution delimiter value further includes refreshing the first set of combined dimension members based on a first execution delimiter value, and refreshing the second set of combined dimension members based on a second execution delimiter value. Refreshing and generating the first set of combined dimension members and the second set of combined dimension members based on an execution delimiter value may also include obtaining updated dimension members from a plurality of dimension members of a first data set from a first data source, and obtaining updated dimension members from a plurality of dimension members of a second data set from a second data source.
[0006] In some examples, generating a third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes suppressing a portion of the combined dimension members. Here, the portion of the combined dimension members may include combined dimension members associated with zero or no data. In these examples, suppressing the portion of the combined dimension members includes zeroing the second portion of the combined dimension members based on a variance threshold.
[0007] In some implementations, generating a third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes inverting values associated with the refreshed second set of combined dimension members. In other implementations, generating the third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes combining the refreshed first set of combined dimension members and the refreshed second set of dimension members into a linear format. The operations may also include sending a data reconciliation report to a user device. Here, receiving the data reconciliation report causes the user device to display the data reconciliation report via a graphical user interface.
[0008] Another aspect of the present disclosure provides a system comprising data processing hardware and memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations. The operations include receiving a data reconciliation request from a user device in communication with the data processing hardware, the data reconciliation request requesting data reconciliation for a first data set and a second data set. The operations also include obtaining the first data set from a first data source. Here, the first data set includes one or more dimensions, each dimension having multiple dimension members. The operations also include obtaining a second data set from a second data source. Here, the second data set includes one or more dimensions, each dimension having multiple dimension members. For each corresponding dimension of the first data set, the operations also include obtaining a corresponding bridge member that associates the corresponding dimension of the first data set with the corresponding dimension of the second data set. The operations also include generating a first set of combined dimension members using each pair of dimension members of the one or more dimensions of the first data set and the corresponding bridge member. The operations also include generating a second set of combined dimension members using each pair of dimension members of the one or more dimensions of the second data set and the corresponding bridge member. The operations also include refreshing the first set of combined dimension members and the second set of combined dimension members based on an execution delimiter value corresponding to the number of dimension members to be refreshed simultaneously. The operation further includes: generating a third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members, and generating a data reconciliation report based on the third set of combined dimension members.
[0009] Implementations of the present disclosure may include one or more of the following optional features. In some implementations, refreshing the first set of combined dimension members and the second set of combined dimension members based on an execution delimiter value further includes refreshing the first set of combined dimension members based on a first execution delimiter value, and refreshing the second set of combined dimension members based on a second execution delimiter value. Refreshing and generating the first set of combined dimension members and the second set of combined dimension members based on an execution delimiter value may also include obtaining updated dimension members from a plurality of dimension members of a first data set from a first data source, and obtaining updated dimension members from a plurality of dimension members of a second data set from a second data source.
[0010] In some examples, generating a third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes suppressing a portion of the combined dimension members. Here, the portion of the combined dimension members may include combined dimension members associated with zero or no data. In these examples, suppressing the portion of the combined dimension members includes zeroing the second portion of the combined dimension members based on a variance threshold.
[0011] In some implementations, generating a third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes inverting values associated with the refreshed second set of combined dimension members. In other implementations, generating the third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes combining the refreshed first set of combined dimension members and the refreshed second set of dimension members into a linear format. The operations may also include sending a data reconciliation report to a user device. Here, receiving the data reconciliation report causes the user device to display the data reconciliation report via a graphical user interface.
[0012] The details of one or more implementations of the present disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS
[0013] Figure 1 is a diagram of an example system for coordinating data.
[0014] Figure 2 A flow chart illustrating an exemplary operational arrangement of a data coordination controller.
[0015] Figure 3A A flow chart illustrating an exemplary operational arrangement of a coordination tag module is shown.
[0016] Figure 3B An example graphical user interface (GUI) view of a coordination tag module is shown.
[0017] Figure 4A A flow chart illustrating an exemplary operational arrangement of a dimension linking module.
[0018] Figure 4B-4D An example graphical user interface (GUI) view of a dimension linking module is shown.
[0019] Figure 5A A flow chart illustrating an exemplary operational arrangement of an import module is shown.
[0020] Figure 5B An example graphical user interface (GUI) view of an import module is shown.
[0021] Figure 6A A flow chart illustrating an exemplary operational arrangement of a bridge module is shown.
[0022] Figure 6B and Figure 6C An example graphical user interface (GUI) view of a bridge module is shown.
[0023] Figure 7A A flow diagram illustrating an exemplary operational arrangement of a bridge execution module is shown.
[0024] Figure 7B An example graphical user interface (GUI) view of a bridge execution module is shown.
[0025] Figure 8A A flow chart illustrating an exemplary arrangement of operations for transforming an identifier.
[0026] Figure 8B An example graphical user interface (GUI) view of a transformation identifier is shown.
[0027] Figure 9A A flow diagram illustrating an exemplary operational arrangement of a transform execution module.
[0028] Figure 9B An example graphical user interface (GUI) view of a transformation execution module is shown.
[0029] Figure 10A A flow chart illustrating an exemplary operational arrangement of a report execution module.
[0030] Figure 10B An example graphical user interface (GUI) view of a report execution module is shown.
[0031] Figure 11 is a flow diagram of an example arrangement of operations for a method of performing data reconciliation.
[0032] Figure 12is a schematic diagram of an example computing device that can be used to implement the systems and methods described herein.
[0033] Like reference numbers in the various drawings indicate like elements. DETAILED DESCRIPTION
[0034] Data reconciliation is a process used to validate data, such as during or after a data migration. For example, target data is compared to original data to ensure it was correctly migrated, or in the case of system updates or changes, to ensure that data that should remain static has not changed. In this context, the data reconciliation process checks for missing records or values, incorrect records or values, duplicate values or records, and various formatting issues. Conventional data reconciliation techniques are expensive, time-consuming, error-prone, or some combination of these factors.
[0035] Implementations herein are directed to a data reconciliation controller that not only reduces the time required to initiate and perform data reconciliation (e.g., of financial data or other values), but also provides a richer and more valuable information trail that is more quickly visible, thereby allowing users to focus on solving problems rather than finding them. The data reconciliation controller reveals errors in the approach to problem solving. For example, when a problem is not properly resolved, the data reconciliation controller allows users to find the problem and revert to an appropriate solution in a shorter time. The approach taken by the data reconciliation controller ensures that users focus not only on the top and bottom layers, but also on every layer in between, thereby focusing on important matters (e.g., matters that often cause delays) early in the process. The data reconciliation tool provides audit tools that allow users to enter and filter information in a practical manner, enabling faster conclusions to be reached while creating and maintaining an audit trail.
[0036] Now refer to Figure 1 In some implementations, the example system 100 includes a processing system 140 that communicates with one or more user devices 110 via a network 120. The processing system 140 can be a single computer, multiple computers, user devices, or a distributed system (e.g., a cloud computing environment) with fixed or scalable / elastic computing resources 142 (e.g., data processing hardware) and / or storage resources 144 (e.g., memory hardware). The data source 135 can be overlaid on the storage resource 144 by one or more users (e.g., user devices 110) or computing resources 142. The data source 135 is configured to store one or more data sets 130.
[0037] The processing system 140 is configured to receive a coordination request (e.g., a data coordination request) 102 from a user device 110 associated with a respective user 10. The user device 110 can correspond to any computing device, such as a desktop workstation, a laptop workstation, or a mobile device (i.e., a smartphone). The user device 110 includes computing resources 112 (e.g., data processing hardware), storage resources 114 (e.g., memory hardware), and / or a display 116 (e.g., a graphical user interface (GUI)). In some examples, the coordination request 102 includes the storage location of the first data set 130, 130a stored at a first data source 135, 135a, and the storage location of the second data set 130, 130b stored at a second data source 135, 135b. Thus, the coordination request requests data coordination for the first data set 130a and the second data set 130b. In other examples, the user 10 directly provides the first data set 130a and the second data set 130b as part of the coordination request 102 to the data coordination controller 200 via the user device 110.
[0038] The processing system 140 executes a data reconciliation controller 200 configured to reconcile the first data set 130a and the second data set 130b based on (i.e., in response to) receiving the reconciliation request 102. Each data source 135 can be a relevant application (i.e., "app"), table, database, or any other source of data sets 130 for reconciliation. Therefore, the data sources 135 may also be interchangeably referred to herein as "applications 135 or apps 135." In some examples, the data sets 130 represent financial information (e.g., financial account balances, debts, credits, etc.) or other numerical data (e.g., inventory quantities). In other examples, the data sets 130 include string values (e.g., employee names, positions, descriptions, etc.). In some examples, each data set 130 includes several dimensions 132. Each dimension 132 can correspond to a data column of the data set 130. For example, a dimension 132 can correspond to a data column representing "Accounts" or "Year" data of the data set 130. In some implementations, the data reconciliation controller 200 obtains or receives dimension alignment data. Dimensionally aligned data aligns the dimensions 132 of each data set 130, for example, via a bridge member 136. For example, a first dimension 132 labeled "Account" for a first data set 130 may be aligned with a second dimension 132 labeled "Acct" for a second data set 130, as discussed in more detail below.
[0039] Furthermore, each dimension 132 includes one or more members 134 (e.g., dimension members) of the dimension 132. For example, the dimension "Account" may include several members (i.e., dimension members) 134 representing different individuals or entities with accounts associated with a dataset 130, a data source 135, or a server associated with an application. Here, each dimension member 134 represents a corresponding data value for the corresponding dimension 132. Along with the dimensions 132 of each dataset 130, the data reconciliation controller 200 obtains each dimension member 134 of the dataset 130 for reconciliation. Each dimension member 134 may correspond to a base member and / or a parent member. In some examples, one or more dimension members 134 include or are associated with one or more annotations, comments, and / or descriptions. For example, a user may provide an annotation that provides a brief description of one or more dimension members 134. For example, annotations may include audit notes, cube or system-defined details to annotate responsible parties, merge different codes to assist in filtering data in different ways, member exceptions to one or more general rules, and the like.
[0040] Figure 2-Figure 1 0 shows a flowchart of various exemplary operation arrangements and various GUI views of the data coordination controller 200 that performs data coordination. It should be understood that Figure 2-Figure 1 The flowcharts shown in FIG. 0 are exemplary only and are not intended to limit the present disclosure in any way. For example, the flowcharts may repeat any operation any number of times, include more or fewer operations, and / or perform any operations in a different order without departing from the scope of the present disclosure. Furthermore, various GUI views may be displayed on the GUI 116 of the user device 110, thereby enabling the user to provide user input to modify the GUI views and / or data.
[0041] Now refer to Figure 2 The data reconciliation controller 200 includes a user login module 210, a reconciliation tag (i.e., naming) module 300, a dimension linking module 400, an import module 500, a bridging module 600, a bridging execution module 700, a transformation identifier 800, a transformation execution module 900, and a report execution module 1000. The data reconciliation controller 200 is configured to reconcile one or more datasets 130 in response to receiving a reconciliation request 102. Each dataset 130 is stored at a corresponding data source 135, and thus, reconciliation of a dataset 130 may also be referred to as reconciliation of a data source 135 (e.g., datasets 130 stored at corresponding data sources 135).
[0042] In some implementations, the coordination request 102 may be initiated by the user 10 via the user device 110 ( Figure 1) manually. In other implementations, the coordination controller 200 generates a coordination request based on the detection of a triggering event, thereby initiating the coordination process. For example, the triggering event can be a timer, causing coordination to occur at fixed intervals, or it can be a data modification event, indicating an update of data in one or more data sets 130. The user can configure the triggering event as any trigger, causing the data coordination controller 200 to perform coordination in response to the detection of the triggering event without any direct user interaction.
[0043] Thus, the data coordination controller 200 obtains the data set 130 (e.g., from the corresponding data source 135) in response to receiving the coordination request 102. For example, the coordination request 102 may provide a username, password, or any other credentials required to access the data set 130 from the corresponding data source 135. In some examples, the data sources 135 are applications, and the data coordination controller 200 obtains a username, password, and connection name for each application (e.g., from a user of the processing system 140 or a remote device in communication with the processing system 140). Here, the data coordination controller 200 authenticates the obtained username and password for the corresponding application 135 before retrieving the data set 130 from the data source 135. The user 10 may access the data set 130 via the GUI 116 ( Figure 1 ) manually provide some or all of the data set 130 to the data coordination controller 200, the GUI 116 being another interface executed on the user device 110 in communication with the processing system 140 or executed on the processing system 140 itself.
[0044] A user may have read access or read / write access to the dataset 130. For example, a read-access user may only read the dataset 130 and perform reconciliation, while a user with read / write access (e.g., an administrator) may update data included in the dataset 130. Thus, the data reconciliation controller 200 may grant or deny a data reconciliation request 102 based on whether the corresponding user has read access, read / write access, or no access (i.e., neither read access nor read / write access) to the requested dataset for reconciliation.
[0045] Now refer to Figure 3A and Figure 3B , the coordination tag module 300 is configured to select the data source 135 for coordination. That is, the example coordination tag module 300 ( Figure 3A) first determines whether the first data source 135a or the second data source 135b corresponds to a new data source based on the coordination request 102 at operation 310. Based on determining that neither the first data source 135a or the second data source 135b corresponds to a new data source, the coordination tag module 300 selects a data source (e.g., application) 135 to be coordinated and selects a version of the selected data source 135 for coordination at operations 320 and 330, respectively. For example, the coordination tag module 300 may select a version of the selected data source 135 based on the coordination request 102 ( Figure 1 ) to select the first data source 135a and the second data source 135b (and the corresponding data source 135 versions).
[0046] On the other hand, based on determining that at least one (or both) of the first data source 135a or the second data source 135b corresponds to a new data source, the coordination tag module 300 creates a data source tag and a coordination execution tag at operations 340 and 350, respectively. Here, the user 10 may include the data source tag and the coordination execution tag as the coordination request 102 ( Figure 1 ) is included in the data reconciliation controller 200. For example, the user 10 may specify a new data source label as "app1" and a reconciliation execution label as "abc2." After selecting the data source 135 for reconciliation, the reconciliation label module 300 provides the reconciliation execution label including the data source 135 and its corresponding version to the dimension linking module 400.
[0047] Figure 3B An example GUI view 301 is shown depicting a graphical representation of the reconciliation tag module 300. In the example shown, the example GUI view 301 depicts a reconciliation execution label (e.g., run name) 362 for each row in a table. Each reconciliation execution label 362 includes a description 364 (e.g., demonstrating a reconciliation run) and a corresponding status 366. The status 366 can indicate which step of the reconciliation process the corresponding reconciliation is currently in. For example, the status 366 can indicate a reporting, dimension update, or bridging state. In addition, the example GUI view 301 can display metadata including, but not limited to, creation data 368 indicating a creating user (e.g., administrator or user) and a creation timestamp, modification data indicating a modifying user (e.g., administrator or user) and a modification timestamp, and an execution timestamp.
[0048] Now refer to Figures 4A-4C , the dimension linking module 400 is configured to align dimensions 132 from data sources 135. Example dimension linking module 400 ( Figure 4A) selects a reconciliation run for execution at operation 410. That is, the dimension linking module 400 selects the reconciliation run tag 362 created by the reconciliation tag module 300 ( FIG. 3 ). At operation 420, the dimension linking module 400 sets a variance threshold 422. For example, the data reconciliation controller 200 may receive one or more variance thresholds 422 as part of the reconciliation request 102. The variance thresholds 422 determine the margin available when performing data reconciliation.
[0049] For example, when the variance threshold 422 is 0.05, the data coordination controller 200 may determine that values that are within 0.05 of each other are the same, or treat values that are within 0.05 of zero as zero (i.e., variance values below the variance threshold are converted to zero and do not show any variance within the coordination). In short, the data coordination controller 200 determines that when the difference between the values from the first data set 130a and the second data set 130b is within the variance threshold 422, the values do not need to be reconciled. The user can configure the variance threshold 422 to any value, and unless otherwise configured, the data coordination controller 200 can implement a default variance threshold. In some examples, the data coordination controller 200 determines the variance threshold 422 based on the data source 135, the data set 130, and / or other contextual information.
[0050] At operation 430, the dimension linking module 430 selects an execution delimiter 432 for the first data source 135a and the second data source 135b. The execution delimiter 432 can be a semicolon, a colon, a pound sign, or any other character. The execution delimiters 432 for the first data source 135a and the second data source 135b can be the same execution delimiter or different execution delimiters (e.g., a first execution delimiter value and a second execution delimiter value). In some examples, the data coordination controller 200 receives or obtains the execution delimiter 432 from the data coordination request 102. In other examples, the data coordination controller 200 generates the execution delimiter 432 based on the data source 135.
[0051] The execution delimiter (i.e., the "line run rate") determines the maximum number of combinations (e.g., rows or lines of a table) that are processed or refreshed simultaneously. In other words, the dimension linking module 400 refreshes the dimension members 134 of the first dataset 130a (e.g., the first set of combined dimension members) and the dimension members 134 (e.g., the second set of combined dimension members) based on the execution delimiter value. Here, the dimension linking module 400 refreshes the dimension members by obtaining updated dimension members (e.g., updated data values) from the plurality of dimension members 134 of the first dataset 130a from the first data source 135a and obtaining updated dimension members from the plurality of dimension members 134 of the second dataset 130b from the second data source 135b. Although, in some scenarios, no data updates occur in the first dataset 130a and the second dataset 130b, and the dimension linking module 400 does perform a refresh because there is no new data to update, the use of the term "dimension member 134" (or a set of combined dimension members) herein can refer to either refreshed dimension members or unrefreshed dimension members 134. In some examples, the dimension linking module 400 refreshes a first set of combined dimension members based on a first execution delimiter value and refreshes a second set of combined dimension members based on a second execution delimiter value that is different from the first execution delimiter value. In some scenarios, processing or refreshing all combinations (e.g., thousands or millions of combinations or rows) simultaneously can result in failures (e.g., crashes, freezes, etc.) or other suboptimal behavior.
[0052] The execution delimiter 432 may include a corresponding position in the data set 130 that "splits" the processing or refreshing of the combination or row at the execution delimiter position into "batches" to ensure stable execution. The user can configure the execution delimiter 432 to have an arbitrary position so that the user can control the batch size for execution. Unless otherwise configured, the data coordination controller 200 can implement a default execution delimiter, for example, with a default batch size. In some implementations, the data coordination controller 200 determines the execution delimiter 432 based on the data source 135, the data set 130, or other contextual information (e.g., available computing resources). Reference Figure 4B-4D In more detail, the dimension linking module 400 aligns the dimensions 132 of the first and second data sources 135 at operation 440 and generates (i.e., creates) additional dimensions at operation 450. Thereafter, the dimension linking module 400 saves the aligned first and second data sources 135a, 135b at operation 460 and outputs them to the import module 500.
[0053] Figure 4B-4DVarious example GUI views 401 are shown, each depicting a graphical representation of the dimension linking module 400. As shown in the example GUI views 401, 401a, a user can select the reconcile execution tab 362 for alignment and select a dataset 130 import type. For example, the first dataset 130a and the second dataset 130b can be imported from their respective data sources 135 via file upload, direct connection, user input from a reconcile request, or some combination thereof. Continuing with this example, the user can also select a currency delimiter 432, a currency symbol to be removed 434, and / or a currency delimiter 436 that the dimension linking module 400 removes from the data source 135 for each data source 135 (e.g., app1 and app2). In some implementations, the example GUI view 401a also allows the user to set a variance threshold 422 and indicate whether each data source 135 has a header 438 or no header.
[0054] Now refer to Figure 4C and Figure 4D In some implementations, the dimension linking module 400 displays (e.g., via the GUI 116) alignments between the data sources 135. For example, the example GUI views 401 and 401b display alignment data for a first data source (e.g., app1) 135a, and the example GUI views 401 and 401c display alignment data for a second data source (e.g., app2) 135b. The GUI views 401b and 401c display the alignment data separately, but it will be appreciated that the alignment data can be displayed adjacent to each other. That is, the GUI views 401b and 401c can be displayed side by side to facilitate easy comparison between the data sources 135.
[0055] The dimension linking module 400 displays aligned data including a data source 135 (e.g., app1 or app2), a dimension 132, and an indicator 442 (e.g., yes or no) indicating whether the corresponding dimension 132 is included in the corresponding data source 135. In the example shown, the dimension 132 corresponding to "Year" is not included in the first data source 135a and the second data source 135b, and therefore, the top member 446 of the "Year" dimension is displayed as "2022." Therefore, in this example, the dimension members 134 of the dimension 132 for "Year" only include the value "2022." Furthermore, the dimension 132 corresponding to "Amount" is included in both the first data source 135a and the second data source 135b, and therefore, the displayed field 444 is '4'. Field 444 can configure the display position of the corresponding dimension 132 for coordination. That is, field 444 can align the "Amount" dimension 132 between the data sources 135. In some implementations, a user manually aligns and populates the alignment data or imports additional dimensions 132. In other implementations, the dimension linking module 400 aligns and populates the alignment data based on the data source 135 without any user input.
[0056] Figure 5A and Figure 5B An import module 500 is shown configured to bridge alignment data from a data source 135. In particular, the example import module 500 ( Figure 5A ) receives the data source 135 (e.g., aligned data source) from the dimension linking module 400 and generates a corresponding bridge member 136 for each dimension 132 of the corresponding data set 130 at operation 510. The bridge member 136 links the data sets 130 together for coordination purposes. In some implementations, the data coordination controller 200 generates a new data structure or table ( Figure 5B ), which includes a first set of combined dimension members of the first data set 130a and a second set of combined dimension members of the second data set 130b.
[0057] That is, for each data set 130, the data coordination controller 200 generates each combination (i.e., pair) of input dimension members 134. For example, when the data set 130 includes five dimensions 132, where the first dimension 132 has 40 dimension members 134, the second dimension 132 has 35 dimension members 134, the third dimension 132 has 3 dimension members 134, the fourth dimension 132 has 10 dimension members 134, and the fifth dimension 132 has 32 dimension members 134, the data coordination controller 200 generates 1,344,000 combinations for the data set 130 (i.e., 40*35*3*10*32=1,344,000). Thus, the data coordination controller 200 generates a first set of combined dimension members (e.g., dimension members) 134 using each pair of dimension members 134 of one or more dimensions 132 of the first data set 130a, and generates a second set of combined dimension members (e.g., dimension members) 134 using each pair of dimension members 134 of one or more dimensions of the second data set 130b. Thereafter, the data coordination controller 200 adds a bridge member 136 associated with each combined dimension member 134 of the new data structure. That is, the data coordination controller 200 may place a corresponding bridge member 136 in association with each corresponding combined dimension member 134 of the new data structure.
[0058] After generating the composite dimension members and generating the data structure including the composite and bridge members, the data coordination controller 200 refreshes or updates each set in the composite at a rate controlled by the execution delimiter. The data coordination controller 200 repeats this process for each data set 130 from each data source 135 (i.e., each application), and continues to split the refresh for the larger data set 130 (based on the execution delimiter). After the data has been refreshed at least once, the data coordination controller 200 can determine whether any data in the data set 130 has changed before refreshing or updating the data again. That is, in some examples, the data coordination controller 200 only updates or refreshes the data when a data change (i.e., a change in the data set 130) is detected.
[0059] In some implementations, the data coordination controller 200 generates a suppression value data structure (e.g., a table) that includes two or more tables that include all data but no suppressed data, such as cells (i.e., column and row combinations) that are zero and / or have no data (i.e., unadjusted suppression values). In some examples, cells for which a variance threshold is zero are removed. As described above, the data coordination controller 200 can generate the suppression value data structure only when a data change is detected since the last execution of the data coordination controller 200.
[0060] In some implementations, after generating the suppression value data structure, the data coordination controller 200 combines the two tables of the suppression value data structure into a linear format. Next, the data coordination controller 200 may combine the adjusted data into a linear format. Next, the data coordination controller 200 or the tool may combine some or all of the previously generated tables (i.e., the tables of combined and bridged members and the adjusted data table) into a full data table (i.e., in a linear format). The data coordination controller 200 may reverse each quantity from the combination in the second data set 200b when generating the full data table (e.g., a value of 100 becomes -100, and a value of 14 becomes -14, etc.).
[0061] In some examples, the bridge member links the dimensions 132 together in a one-to-one manner. For example, when the first data source 135a has a dimension 132 titled "Accounts" and the second data source 135b has a dimension 132 titled "Accts" that maps to the dimension member 134 (e.g., data) in the Accounts dimension 132 of the first data source 135a, the bridge member between Accounts and Accts is a one-to-one relationship. In other examples, the bridge member links the dimensions 132 together to form a many-to-one, one-to-many, and / or many-to-many relationship with respect to the data sources 135. For example, when the first data source 135a has a dimension titled "Accounts," and the second data source 135b has two dimensions titled "Accts1" and "Accts2," thereby splitting the dimension member 134 into two separate dimensions 132, a bridge member can bridge "Accounts" to both "Accts1" and "Accts2," thereby forming a one-to-many relationship between the first data source 135a and the second data source 135b. Thus, the dimension 132 can be partitioned in any number of ways between the data sources 135, and a bridge member can be used to correctly map the dimension members 134 from the first data source 135a to the second data source 135b.
[0062] In some scenarios, the import module 500 has previously generated a bridge member 136 for the corresponding data source 135, so the import module 500 imports the previously generated bridge member 136 at operation 520 rather than generating the bridge member again. At operation 530, the user can manually edit any bridge member 136 generated by the import module 500. In addition, the user can add annotations to the dimension 132 or bridge member 136 at operation 540, such as exceptions to one or more general rules.
[0063] Figure 5BAn example GUI view 501 is shown depicting a graphical representation displayed by the import module 500. As shown, the graphical representation depicts a plurality of bridge members 136, 136a-e generated by the import module 500 for a first data source 135a and a second data source 135b. Here, each bridge member 136 is associated with a first composite dimension member 134 and / or a second composite dimension member 134. In particular, the example shows a first bridge member 136a between the first application 135a and the second application 135b, and a second bridge member 136b and a third bridge member 136c between the "Year" and "Period" dimensions 132 of the data source 135. In addition, the example shows a fourth bridge member 136d between the "Account" dimension 132 of the first data source 135a and the corresponding "Acct" dimension of the second data source 135b, and a fifth bridge member 136e between the "Entity" dimension 132 of the first data source 135a and the corresponding "Entity" dimension 132 of the second data source 135b. Below each bridge member 136 is a composite dimension member 134 corresponding to the bridge member 136.
[0064] Now refer to Figures 6A-6C In some implementations, the bridge module 600 is configured to synchronize the source member and the bridge member 136. Example bridge module 600 ( Figure 6A ) generates a concatenation 612 of a bridge member 136 and a dimension member 134 (e.g., one of the combined dimension members) for each corresponding dimension 132 at operation 610. Thus, the bridge module 600 can import the bridge member 136 generated by the import module 500 ( FIG. 5 ) for each data source 135 and generate the concatenation 612 using the bridge member 136 and the corresponding dimension member 134. In short, the bridge member 136 links the dimensions between the data sources 135, and the concatenation 612 associates the dimension members 134 (e.g., data values) of the dimension 132 of the bridge member 136. After generating the concatenation, the user can edit / configure any generated concatenation. Using the final concatenation 612, the bridge module 600 generates a synchronization map 622 at operation 620 that maps the source member (e.g., dimension member 134) to the bridge member 136, and submits the synchronization map 622 at operation 630.
[0065] Figure 6B and Figure 6CThe bridge module 600 is shown displaying (e.g., via the GUI 116) a representation corresponding to a synchronization map 622. For example, the example GUI views 601, 601a display an example synchronization map for a first data source (e.g., app1) 135a, and the example GUI views 601, 601b display an example synchronization map for a second data source (e.g., app2) 135b. The synchronization map includes a dimension 132, a dimension member 134, an inverse operation indicator 602, and a bridge member 136. Here, each row corresponds to a cascade 612 generated by the bridge module 600. For example, the synchronization map 622 for the first data source 135a ( Figure 6B ) shows that the "Period" dimension 132 has a dimension member 134 (e.g., data value) "600" and a bridge member 136 "EXP," whose reverse operation indicator 602 indicates "NO." Therefore, the data coordination controller 200 will not perform a reverse operation on the "Period" dimension based on the reverse operation indicator 602. Alternatively, in a scenario where the reverse operation indicator 602 indicates "YES," the data coordination controller 200 performs a reverse operation on the "Period" dimension.
[0066] Figure 7A and Figure 7B 1 and 2. The example bridging execution module 700 is shown as being configured to generate a source file table 722 that displays the dimensions 132 of the first data source 135a and the second data source 135b as headings with corresponding values below them. Figure 7A At operation 710, the bridge execution module 700 imports a first data set 130a and a second data set 130b from a first data source 135a and a second data source 135b, respectively. Each data set 130 includes one or more dimensions 132, and each dimension 132 includes a plurality of dimension members 134 (e.g., data values). At operation 720, the bridge execution module 700 generates a source file table 722 using the first and second data sets 130a, 130b and the data synchronization map 622. Thereafter, the bridge execution module 700 exports the source file table 722 at operation 730.
[0067] When the data coordination controller 200 imports the data set 130, the data coordination controller 200 generates a bridge member 136 for each dimension member (e.g., for each dimension member in a dimension member combination). However, if the bridge execution module 700 detects that one or more dimension members 134 do not include a bridge member 136, a bridge kick occurs, indicating that a bridge member is missing. In response to the bridge kick, the data coordination controller 200 and / or the user can update the bridge member (e.g., via the bridge module 600).
[0068] Figure 7B An example GUI view 701 is shown depicting a graphical representation of a source file table 722 generated by the bridge execution module 700. In the example shown, the source file table 722 includes the data sources 135 (e.g., app1 and app2), a "Year" dimension 132 next to a bridge member 136 for the "Year" dimension 132, and an "Account" or "Acct" dimension next to a bridge member for the account dimension 132.
[0069] Now refer to Figure 8A and Figure 8B In some examples, the transformation identifier 800 is configured to identify the transformation required for the source file table 722 (e.g., generated by the bridge execution module 700) prior to coordinated execution. In some implementations, one or more data sources 135 (e.g., the second data source 135b) has dimensions 132 that do not directly map (i.e., via bridge members 136) to any corresponding dimensions 132 of another data source 135 (e.g., the first data source 135a). For example, in some scenarios, the second data source 135b performs additional processing on dimension members 134 of the dimension 132 from the first data source 135a, thereby generating a new dimension 132 that does not exist in the first data source 135a. In such a scenario, because the first data source 135a does not have a corresponding dimension 132, the new dimension 132 of the second data source 135b does not have a direct mapping (e.g., no bridge member 136). Thus, where a dimension 132 does not exist in one data source 135 but exists in another data source 135 , the coordination controller 200 may generate additional tables to concatenate any number of columns to perform additional multi-dimensional mapping.
[0070] In the illustrated example, at operation 810, the transformation identifier 800 executes the source file table 722 to generate a bridge table. The bridge table identifies any transformations and displays the executed source file table at operation 820. Here, the identified transformations may indicate that one or more of the dimensions 132 and / or bridge members 136 are unmapped. At operation 830, the transformation identifier 800 determines whether the source file table has any unmapped dimensions 132 or missing dimensions. Based on the determination that there are no unmapped dimensions 132 (or bridge members 136), the transformation identifier 800 submits and exports the table at operations 840 and 850, respectively. Here, the data coordination controller 200 may enter the field number from the source file and enable the status to active.
[0071] On the other hand, based on determining that there are one or more unmapped dimensions 132 or missing dimensions 132, the transformation identifier 800 updates the one or more unmapped dimensions at operation 860. For example, the data coordination controller 200 can prompt the user to manually update the unmapped dimensions. Alternatively, the data coordination controller 200 can directly update the unmapped dimensions or missing dimensions, for example, by entering a placeholder value for the missing dimension. In addition, if there are missing source members, the data coordination controller can enter the top member value for the missing value (e.g., 2022 for the missing dimension 132 of "year"). After correcting the unmapped dimensions, the transformation identifier 800 submits and exports the bridge table at operations 840 and 850.
[0072] Figure 8B An example GUI view 801 is shown depicting a graphical representation of an executed source file table generated by a transformation identifier 800. Here, the executed source file table displays dimensions 132 and cascades 612 for each of a first data source 135a and a second data source 135b. In the example shown, each dimension 132 of a data source 135 is mapped to a corresponding dimension in the other data source 135. That is, there are no unmapped dimensions 132 in this example. However, if there were any unmapped dimensions 132, the executed source file table would display the unmapped dimensions 132 for correction. In particular, the data coordination controller 200 can be re-executed using updated data to map the unmapped dimensions 132, or a user can manually map the unmapped dimensions 132.
[0073] Figure 9A and Figure 9B 1 shows a transformation execution module 900 configured to generate a bridge synchronization table 922 that displays the dimensions 132 of the first data source 135a and the second data source 135b as headings with corresponding values below them. In particular, the example transformation execution module 900 ( Figure 9A ) receives a bridge table generated by the transformation identifier 800 at operation 910. Using the bridge table, the transformation execution module 900 generates a bridge synchronization table 922 at operation 920 and exports the bridge synchronization table at operation 930. Notably, operation 920 is similar to operation 720 ( FIG. 7 ), the only difference being that, at operation 920, the transformation execution module 900 uses the bridge table with the transformation. In short, operation 720 ( FIG. 7 ) is performed before any transformation (e.g., unmapped dimensions) are corrected, and operation 920 is performed using the bridge table, including any updates, to correct the unmapped dimensions.
[0074] Figure 9BAn example GUI view 901 is shown depicting a graphical representation of a bridge synchronization table 922 generated by the bridge execution module transformation execution module 900. In the example shown, the bridge synchronization table 922 includes the data sources 135 (e.g., app1 and app2), the "Year" dimension 132 next to the bridge member 136 for the "Year" dimension 132, and the "Account" or "Acct" dimension next to the bridge member 136 for the account dimension 132. It is worth noting that in this example, there are no transformations (e.g., from Figure 8B ), and therefore, the bridge synchronization table 922 and Figure 7B The source file table 722 shown in FIG. 7 is similar to (or the same as) that in FIG.
[0075] Now refer to Figure 10A and Figure 10B In some implementations, the report execution module 1000 is configured to reconcile reports 160. That is, using the full data table (e.g., the bridge synchronization table 922), the data reconciliation controller 200 generates one or more reconciliation reports 160, such as a reconciliation bridge report. The report provides the reconciliation results to, for example, a user in a visual manner. For example, the report highlights or annotates the data differences between the first data set 130a from the first data source 135a and the second data set 130b from the second data set 130b. In some implementations, the data reconciliation controller 200 generates additional dimension transformations (i.e., the data reconciliation controller 200 generates another opportunity to change or adjust the bridge account data based on the concatenation of existing dimensions to allow reconciliation in the event that a single dimension in one application is split into multiple dimensions in another application).
[0076] In particular, the report execution module 1000 generates the reconciliation report 160 at operation 1010. Optionally, the user may edit and / or submit a commit for the reconciliation report 160 at operations 1020 and 1030, respectively. Thereafter, the report execution module 1000 exports the reconciliation report 160 at operation 1040. For example, the report execution module 1000 may export the reconciliation report 160 to the user device 110 associated with the reconciliation request 102 that initiated the reconciliation of the dataset 130. In response to receiving the reconciliation report 160, the user device 110 may display (e.g., via the GUI 116) the reconciliation report 160 to the user.
[0077] Figure 10BAn example GUI view 1001 is shown, depicting a graphical representation of a reconciliation report 106 generated by the report execution module 1000. In the illustrated example, the reconciliation report 106 includes one or more bridge members 136, annotations 1004, and reconciliation values 1002. Specifically, the reconciliation report displays a first bridge member 136 between the "Account" and "Acct" dimensions 132 of a first data source 135a and a second data source 135b, as well as a second bridge member 136 between the "Entity" and "Entity" dimensions 132 of the first data source 135a and the second data source 135b. Any annotations 1004 entered by the user during the reconciliation process are displayed with the reconciliation report to inform the user of any special circumstances surrounding a particular reconciliation. Reconciliation values 1002 graphically indicate to the user whether any reconciliation is required between the datasets 130. In short, reconciliation values 1002 indicate whether any data differs, is missing, or otherwise differs between the datasets 130 from the data sources 135.
[0078] In some implementations, after generating the suppression value data structure, the data coordination controller 200 combines the two tables of the suppression value data structure into a linear format. Next, the data coordination controller 200 can combine the adjusted data into a linear format. Next, the data coordination controller 200 can combine some or all of the previously generated tables (i.e., the tables of combined and bridged members and the adjusted data table) into a full data table (i.e., in a linear format). When generating the full data table, the data coordination controller 200 can invert each quantity of the combination from the second data set 200b (e.g., a value of 100 becomes -100, and a value of 14 becomes -14, etc.).
[0079] Figure 111 is a flow chart of an exemplary operational arrangement for performing a method 1100 of performing data reconciliation. The method 1100 may be performed, for example, by a data reconciliation controller 200 operating at a processing system 140. At operation 1102, the method 1100 includes obtaining a first data set 130a from a first data source 135a. The first data set 130a includes one or more dimensions 132, each dimension 132 having a plurality of dimension members 134 (e.g., data values). At operation 1104, the method 1100 includes obtaining a second data set 130b from a second data source 135b. Similarly, the second data set 130b also includes one or more dimensions 132, each dimension 132 having a plurality of dimension members 134. Obtaining the first data set 130a and the second data set 130b may be in response to receiving a data reconciliation request 102 from a user device 110 in communication with data processing hardware 142 (e.g., via a network 120), the data reconciliation request 102 requesting data reconciliation for the first data set 130a and the second data set 130b. The data reconciliation request 102 may be sent by a user 10 associated with the user device 110 or in response to a triggering event.
[0080] For each corresponding dimension 132 of the first data set 130a, the method 1100 includes obtaining and / or generating a corresponding bridge member 136 at operation 1106. The corresponding bridge member 136 associates the corresponding dimension 132 of the first data set 130a with the corresponding dimension 132 of the second data set 130b. For example, the corresponding bridge member 136 can associate the dimension 132 of "Entity" of the first data set 130a with the dimension 132 of the corresponding "Entity" of the second data set. The bridge member 136 between the two dimensions 132 instructs the data reconciliation controller 200 to perform reconciliation between the data values (e.g., dimension members 134) of the two dimensions 132. Therefore, each dimension member 134 included in the two dimensions 132 can be associated with a corresponding bridge member 136.
[0081] At operation 1108, the method 1100 includes generating a first set of combined dimension members 134 using each pair of dimension members 134 and corresponding bridge members 136 for one or more dimensions 132 of the first data set 130a. Similarly, at operation 1110, the method 1100 includes generating a second set of combined dimension members 134 using each pair of dimension members 134 and corresponding bridge members 136 for one or more dimensions 132 of the second data set 130b. For example, to generate a set of combined dimension members 134 for a data set 130 including five dimensions 132, where a first dimension has 40 members, a second dimension has 35 members, a third dimension has 3 members, a fourth dimension has 10 members, and a fifth dimension has 32 members, the data coordination controller 200 generates 1,344,000 combinations (e.g., rows) for the data set 130 (i.e., 40*35*3*10*32=1,344,000). In simple terms, the set of combined dimension members 134 represents every combination of dimension members 134 from the dimensions 132 of the data set 130 .
[0082] Optionally, method 1100 may further include refreshing the first and second sets of combined dimension members based on an execution delimiter value corresponding to the number of dimension members being refreshed simultaneously. Advantageously, the execution delimiter value splits rows into "batches," thereby ensuring stable execution. Therefore, when method 1100 refreshes the first and second combined dimension members, the method uses the refreshed first and second combined dimension members. On the other hand, when no refresh occurs, the method only uses the first and second combined dimension members.
[0083] At operation 1112, method 1100 includes generating a third set of combined dimension members using the first set of combined dimension members 134 and the second set of combined dimension members 134. Here, the third set of combined dimension members associates dimension members 134 between the first data set 130a and the second data set 130b. For example, the third set of combined dimension members may correspond to a full data table (e.g., source file table 722 (FIG. 7) or bridge synchronization table 922 (FIG. 9)). Thereafter, at operation 1114, method 1100 includes generating a data reconciliation report 160 from the third set of combined dimension members (e.g., source file table 722 (FIG. 7) or bridge synchronization table 922 (FIG. 9)). For example, when no transformation of data set 130 is required, data reconciliation controller 200 generates data reconciliation report 160 using source file table 722. Otherwise, when transformation of data set 130 is required, data reconciliation controller 200 generates reconciliation report using bridge synchronization table 922. It is worth noting that the reconciliation report reconciles data between the first data set 130a and the second data set 130b so that the data reconciliation controller 200 detects whether any data between the first data set 130a and the second data set 130b has been modified, added, or deleted.
[0084] Figure 12 1 is a schematic diagram of an example computing device 1200 that can be used to implement the systems and methods described in this document. Computing device 1200 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The components shown here, their connections and relationships, and their functions are exemplary only and are not intended to limit implementations of the inventions described and / or claimed in this document.
[0085] Computing device 1200 includes a processor 1210, a memory 1220, a storage device 1230, a high-speed interface / controller 1240 connected to memory 1220 and a high-speed expansion port 1250, and a low-speed interface / controller 1260 connected to a low-speed bus 1270 and storage device 1230. Each of components 1210, 1220, 1230, 1240, 1250, and 1260 is interconnected using various buses and can be mounted on a common motherboard or in other suitable ways. Processor 1210 can process instructions for execution within computing device 1200, including instructions stored in memory 1220 or on storage device 1230, to display graphical information for a graphical user interface (GUI) on an external input / output device (e.g., a display 1280 coupled to high-speed interface 1240). In other implementations, multiple processors and / or multiple buses, as well as multiple memories and multiple types of memories, can be used as needed. Furthermore, multiple computing devices 1200 may be connected, with each device providing portions of the necessary operations (eg, as a server bank, a group of blade servers, or a multi-processor system).
[0086] Memory 1220 stores information non-temporarily within computing device 1200. Memory 1220 may be a computer-readable medium, a volatile memory unit, or a non-volatile memory unit. Non-temporary memory 1220 may be a physical device used to temporarily or permanently store programs (e.g., sequences of instructions) or data (e.g., program state information) for use by computing device 1200. Examples of non-volatile memory include, but are not limited to, flash memory and read-only memory (ROM) / programmable read-only memory (PROM) / erasable programmable read-only memory (EPROM) / electronically erasable programmable read-only memory (EEPROM) (e.g., typically used for firmware, such as bootloaders). Examples of volatile memory include, but are not limited to, random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), phase change memory (PCM), and disk or tape.
[0087] Storage device 1230 can provide mass storage for computing device 1200. In some implementations, storage device 1230 is a computer-readable medium. In various implementations, storage device 1230 can be a floppy disk device, a hard disk device, an optical disk device, or a magnetic tape device, a flash memory or other similar solid-state memory device, or an array of devices, including devices in a storage area network or other configuration. In additional implementations, a computer program product is tangibly embodied in an information carrier. The computer program product includes instructions that, when executed, perform one or more methods, such as the methods described above. The information carrier is a computer or machine-readable medium, such as memory 1220, storage device 1230, or memory on processor 1210.
[0088] The high-speed controller 1240 manages bandwidth-intensive operations for the computing device 1200, while the low-speed controller 1260 manages less bandwidth-intensive operations. This division of responsibilities is exemplary only. In some implementations, the high-speed controller 1240 is coupled to the memory 1220, the display 1280 (e.g., via a graphics processor or accelerator), and the high-speed expansion ports 1250, which can accept various expansion cards (not shown). In some implementations, the low-speed controller 1260 is coupled to the storage device 1230 and the low-speed expansion ports 1290. The low-speed expansion ports 1290 can include various communication ports (e.g., USB, Bluetooth, Ethernet, wireless Ethernet) and can be coupled to one or more input / output devices (e.g., keyboards, pointing devices, scanners), or network devices (e.g., switches or routers), for example, via a network adapter.
[0089] The computing device 1200 can be implemented in a variety of different forms, as shown. For example, it can be implemented as a standard server 1200a or multiple times in a group of such servers 1200a, as a laptop computer 1200b, or as part of a rack server system 1200c.
[0090] Various implementations of the systems and techniques described herein can be implemented in digital electronic and / or optical circuitry, integrated circuitry, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementations in one or more computer programs that are executable and / or interpretable on a programmable system that includes at least one programmable processor, which can be special-purpose or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to send data and instructions to the storage system, at least one input device, and at least one output device.
[0091] A software application (i.e., a software resource) may refer to computer software that enables a computing device to perform tasks. In some examples, a software application may be referred to as an "application," "app," or "program." Example applications include, but are not limited to, system diagnostic applications, system management applications, system maintenance applications, word processing applications, spreadsheet applications, messaging applications, media streaming applications, social networking applications, and gaming applications.
[0092] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and may be implemented in high-level procedural and / or object-oriented programming languages and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, non-transitory computer-readable medium, apparatus, and / or device (e.g., a disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including machine-readable media that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.
[0093] The processes and logic flows described in this specification can be performed by one or more programmable processors (also referred to as data processing hardware) to execute one or more computer programs to perform functions by operating on input data and generating outputs. The processes and logic flows can also be performed by dedicated logic circuit systems, such as FPGAs (field programmable gate arrays) or ASICs (application-specific integrated circuits). By way of example, processors suitable for executing computer programs include both general-purpose and special-purpose microprocessors, as well as any one or more processors of any type of digital computer. Typically, a processor will receive instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data, or be operably coupled to receive data from one or more mass storage devices (e.g., magnetic disks, magneto-optical disks, or optical disks) for storing data or to transmit data to them or both. However, a computer need not have such devices. Computer-readable media suitable for storing computer program instructions and data include all forms of nonvolatile memory, media, and storage devices, including, by way of example, semiconductor memory devices such as EPROM, EEPROM, and flash memory devices; magnetic disks, such as internal hard disks or removable disks; magneto-optical disks; and CD ROM and DVD-ROM disks. The processor and memory can be supplemented by, or incorporated in, special purpose logic circuitry.
[0094] To provide for interaction with a user, one or more aspects of the present disclosure may be implemented on a computer having a display device (e.g., a CRT (cathode ray tube), an LCD (liquid crystal display) monitor, or a touch screen) for displaying information to the user, and optionally having a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices may also be used to provide for interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback, such as visual feedback, auditory feedback, or tactile feedback; and input from the user may be received in any form, including acoustic, voice, or tactile input. Additionally, a computer may interact with a user by sending documents to and receiving documents from a device used by the user; for example, by sending a web page to a web browser in response to a request received from a web browser on the user's client device.
[0095] Many implementations have been described. Nevertheless, it should be understood that various modifications can be made without departing from the spirit and scope of this disclosure. Therefore, other implementations are within the scope of the appended claims.
Claims
1. A computer-implemented method which, when executed by data processing hardware, causes the data processing hardware to perform operations comprising: receiving a data coordination request from a user device in communication with the data processing hardware, the data coordination request requesting data coordination for a first data set and a second data set; Obtaining the first data set from a first data source, the first data set comprising one or more dimensions, each dimension of the one or more dimensions of the first data set comprising a first plurality of dimensional members; obtaining the second data set from a second data source, the second data set comprising one or more dimensions, each dimension of the one or more dimensions of the second data set comprising a second plurality of dimensional members; For each corresponding dimension of the first data set, obtaining a corresponding bridge member, the corresponding bridge member associating the corresponding dimension of the first data set with the corresponding dimension of the second data set; generating a first set of combined dimension members using each pair of dimension members in the one or more dimensions of the first data set and the corresponding bridge member; generating a second set of combined dimension members using each pair of dimension members in the one or more dimensions of the second data set and the corresponding bridge member; refreshing the first set of combined dimension members and the second set of combined dimension members based on an execution delimiter value corresponding to the number of dimension members refreshed simultaneously; Generate a third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members; and A data reconciliation report is generated based on the third set of combined dimension members.
2. The method according to claim 1, wherein Refreshing the first set of combined dimension members and the second set of combined dimension members based on the execution delimiter value further comprises: refreshing the first set of composite dimension members based on a first execution delimiter value; and The second set of combined dimension members is refreshed based on a second execution delimiter value.
3. The method according to claim 1, wherein Refreshing the first set of combined dimension members and the second set of combined dimension members based on the execution delimiter value further comprises: Obtaining updated dimension members from the first plurality of dimension members of the first data set from the first data source; and Updated dimension members are obtained from the second plurality of dimension members of the second data set from the second data source.
4. The method according to claim 1, wherein Generating the third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes suppressing a portion of the combined dimension members.
5. The method according to claim 4, wherein The portion of the combined dimension members includes combined dimension members associated with zero or no data.
6. The method according to claim 4, wherein: Suppressing the portion of the combined dimension members includes zeroing a second portion of the combined dimension members based on a variance threshold.
7. The method according to claim 1, wherein Generating the third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes inverting values associated with the refreshed second set of combined dimension members.
8. The method according to claim 1, wherein Generating the third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes combining the refreshed first set of combined dimension members and the refreshed second set of combined dimension members into a linear format.
9. The method according to claim 1, wherein The first data set and the second data set each include numerical values.
10. The method according to claim 1, wherein The operations further include sending the data reconciliation report to the user device, wherein receiving the data reconciliation report causes the user device to display the data reconciliation report via a graphical user interface.
11. A system comprising: Data processing hardware; as well as Memory hardware in communication with the data processing hardware, the memory hardware storing instructions that, when executed on the data processing hardware, cause the data processing hardware to perform operations comprising: receiving a data coordination request from a user device in communication with the data processing hardware, the data coordination request requesting data coordination for a first data set and a second data set; Obtaining the first data set from a first data source, the first data set comprising one or more dimensions, each dimension of the one or more dimensions of the first data set comprising a first plurality of dimensional members; obtaining the second data set from a second data source, the second data set comprising one or more dimensions, each dimension of the one or more dimensions of the second data set comprising a second plurality of dimensional members; For each corresponding dimension of the first data set, obtaining a corresponding bridge member, wherein the corresponding bridge member associates the corresponding dimension of the first data set with the corresponding dimension of the second data set; generating a first set of combined dimension members using each pair of dimension members in the one or more dimensions of the first data set and the corresponding bridge member; generating a second set of combined dimension members using each pair of dimension members in the one or more dimensions of the second data set and the corresponding bridge member; refreshing the first set of combined dimension members and the second set of combined dimension members based on an execution delimiter value corresponding to the number of dimension members to be refreshed simultaneously; generating a third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members; and A data reconciliation report is generated based on the third set of combined dimension members.
12. The system according to claim 11, wherein Refreshing the first set of combined dimension members and the second set of combined dimension members based on the execution delimiter value further comprises: refreshing the first set of composite dimension members based on a first execution delimiter value; and The second set of combined dimension members is refreshed based on a second execution delimiter value.
13. The system according to claim 11, wherein: Refreshing the first set of combined dimension members and the second set of combined dimension members based on the execution delimiter value further comprises: Obtaining updated dimension members from the first plurality of dimension members of the first data set from the first data source; and Updated dimension members are obtained from the second plurality of dimension members of the second data set from the second data source.
14. The system according to claim 11, wherein: Generating the third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes suppressing a portion of the combined dimension members.
15. The system according to claim 14, wherein: The portion of the combined dimension members includes combined dimension members associated with zero or no data.
16. The system of claim 14, wherein: Suppressing the portion of the combined dimension members includes zeroing a second portion of the combined dimension members based on a variance threshold.
17. The system according to claim 11, wherein: Generating the third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes inverting values associated with the refreshed second set of combined dimension members.
18. The system according to claim 11, wherein: Generating the third set of combined dimension members using the refreshed first set of combined dimension members and the refreshed second set of combined dimension members includes combining the refreshed first set of combined dimension members and the refreshed second set of combined dimension members into a linear format.
19. The system according to claim 11, wherein: The first data set and the second data set each include numerical values.
20. The system of claim 11, wherein: The operations further include sending the data reconciliation report to the user device, wherein receiving the data reconciliation report causes the user device to display the data reconciliation report via a graphical user interface.
Citation Information
Patent Citations
Method and system for checking data
CN102207940A
Generating multidimensional database queries
CN110928958A