A COMPUTER SYSTEM AND METHOD THAT AUTOMATICALLY ENSURES DATA EQUALITY.

Abstract

This invention relates to an information system (1) and method (1000) that automatically maintains data parity after large data transfers to geographically redundant analytical systems (2). The invention proposes an information system (1) and method (1000) that automatically maintains data parity in both analytical systems (2) in a resource and time-optimal way after a large data transfer to both analytical systems (2) that are geographically redundant and both are actively running.

Description

1 TARIFF A SYSTEM THAT AUTOMATICALLY ENSURES DATA EQUALITY IS MAINTAINED. INFORMATION SYSTEMS AND METHODS Technical Area This invention enables big data integration into geographically redundant analytics systems. a system that automatically ensures data parity is maintained after data transfers It is related to information systems and methods. State of the Art Analytical systems, which provide services to many services in information systems, are related to those services. After processing records related to associated products, 10 in the reporting and visualization areas. They provide various services. If the information system requires critical services and the necessary components for these services... If it contains important data, it can provide uninterrupted service in the event of natural disasters or catastrophic failures. In order to provide this service, analytical systems generally require both systems to be active. They are configured to be geographically redundant, in the way they operate. In this way In structured analytical systems, the records of data processed live in both systems are 15. For synchronization, the system needs to receive input from another analytical system or external source. Even when there is a large data transfer from the source, the data in these two analytical systems is quickly displayed. They need to be equalized in some way. In information systems where service continuity is critical, reporting and visualization are essential. Analytics 20 that perform their functions and are configured with geographical redundancy. The reports and visual graphics produced by the systems must be consistent across both systems. This is very important. Therefore, the information in the databases of both analytical systems... Synchronization is achieved through various methods. This includes the live data flowing into these analytical systems. The order may be larger than that of another analytical system or an external source for various reasons. A data transfer of a certain size may be required. In such cases, system resources may be limited to 25. transferring data without changing it and without significantly affecting system performance data in two analytical systems completed and then configured with geographical redundancy. The aim is to maintain equality. In this way, it is configured with geographical redundancy and the databases are synchronized. Maintaining data equality when transferring large amounts of data to working analytical systems is crucial. 30 2 One of the current methods followed is to examine the relationship between the two systems before data transfer. Transferring data to the first analytical system after turning off data synchronization. to perform the first analytical analysis of all data transferred to the system after data transfer. The aim is to ensure that the data is sent from the first system to the second analytical system. This method transfers the data. Although suitable for situations where the data is not very large, 5 from another analytical system In situations where a large amount of data is transferred to the existing analytical system, the existing analytics the system's resources are insufficient and the units responsible for data synchronization are both This causes them to experience problems in terms of both processing power and memory. In order to eliminate the resource problems experienced by the relevant units in the method, More processing and memory resources are being allocated to the units, which also improves analytics 10. systems consume more resources and are therefore more costly. This causes them to become like that. Another method currently being followed involves analyzing data between two analytical systems. After turning off synchronization, transfer data to the first analytical system and After data transfer, back up the first analytical system and restore it to the second analytical system. 15 This method involves backing up and restoring, thus adding to analytical systems. Since no funding is required, there is no increase in costs. However, The second system will be out of service until the backup from the first system is uploaded to the second system. Because of this, there is a general risk of service interruption. The analytical system... In large systems with high data volumes, the restore time is both long and short of backup time. Because the service will take a long time due to both data transfer and restoration processes. Due to the risk of interruption, this method is not frequently used by IT system administrators. It is not preferred. Today, both systems operate actively in a geographically redundant manner. After transferring large amounts of data to structured analytical systems, both 25 automatic optimization of data equality in the system in terms of resources and time Structures that enable this to be done are needed. Patent application number TR2021 / 010157, which is included in the prior art. The document describes the geographic segmentation system and method. However, the relevant The application document states that both are configured with geographical redundancy and have 30 active slots. After transferring large amounts of data to the analytical systems it works with, both optimizing data equality in the analytical system in terms of source and time. No structure that enables this to be done automatically is described. 3 In conclusion, solutions that address the needs described above are relevant to the subject. Due to its shortcomings, it has become necessary to make improvements in the relevant technical field. Brief Description of the Invention The invention was created by drawing inspiration from existing situations and overcoming the aforementioned drawbacks. It aims to solve. 5 The aim of this invention is to provide solutions for analytical systems configured with geographical redundancy. Automatically maintain data parity after large data transfers It is the development of an information system and method that provides this. The invention describes a system where both devices are configured with geographical redundancy and both are actively operating. After transferring large amounts of data to the analytical systems, 10 in both analytical systems Data equality is automatically optimized in terms of source and time. It proposes an information system and method that enables this to be done. The structural and characteristic features and all the advantages of the invention are given in the figures below. This becomes clearer thanks to the detailed explanation written with references to these figures. This will be understood as such, and therefore the evaluation will also be based on these forms and detailed explanations. 15 This should be done taking that into consideration. Figures that will help understand the invention. Figure 1 is a schematic representation of the system that is the subject of the invention. Figure 2 shows the flowchart of the method described in the invention. Description of Part References 20 1. Information system 2. Analytical system 3. Database 4. Data transfer unit 5. Data transfer synchronization unit 25 1000. Information method 4 A. External sources Detailed Description of the Invention In this detailed explanation, the information system (1) and method (1000) that are the subject of the invention are preferred. the structures that have been established are solely for the purpose of better understanding the subject. It is explained. 5 This invention represents a major contribution to the development of geographically redundant analytical systems (2). a system that automatically ensures data parity is maintained after data transfers It is related to information system (1) and method (1000). The invention describes a system where both devices are configured with geographical redundancy and both are actively operating. After transferring large amounts of data to analytical systems (2), both analytical systems 10 (2) data equality in the system is automatically optimized in terms of source and time. It proposes an information system (1) and method (1000) that enables this to be done. The subject of the invention is the information system (1), the schematic representation of which is given in Figure 1;  After processing the large amount of data transmitted to it, it records it and generates the necessary reports. Multiple analytical systems that generate visual graphics (2), 15  multiple that hold all data and configuration information in the analytical system (2) number of databases (3),  to the analytical system (2) from another analytical system (2) or from an external source (A) multiple data transfer units (4) that enable data transfer and  Data to one of the geographically redundant structured analytical systems (2) 20 Data transferred via the transfer unit (4), indexes in the database (3) by dividing them into predetermined time intervals and transferring them to the other, thus combining both analytical methods. one of the (2) that automatically maintains data equality in the system large number of data transfer synchronization units (5) It includes. 25 Information system (1) is active in order to solve problems in existing applications. data equality of analytical systems configured with active geographical redundancy (2) to automatically resume after big data transfer to systems (2) transferring data without interfering with the live data flow and the analytical system (2) It proposes a structure that automatically synchronizes without affecting performance. 30 In the information system (1), it is configured with geographical redundancy and is actively working. data equality between the two analytical systems (2) to one of the analytical systems (2) transferring data from another analytical system (2) or from an external source (A) without disrupting it This is done so that the performance of the live data stream coming to the analytical system (2) is not affected. Data transfer is generally carried out during periods when live data flow is low. 5 Care is taken to ensure this is done. The following part of the explanation will involve another analytical approach. data transferred from the system (2) or from an external source (A) to the first analytical system (2) loaded and subsequently from the first analytic system (2) to the second analytic system (2) Data equality between the two analytical systems (2) is ensured by transferring the data. It was assumed. However, the second analytic system (2) also has 10 that the first analytic system (2) has. It has all the units that are. For this reason, the structure and needs of the information system (1) Taking into consideration, the external data to be transferred will first be sent to the second analytical system (2) loading and then from the second analytic system (2) to the first analytic system (2) Ensuring data equality through data transfer can also be a preferred solution. In the information system (1), geographic 15 will work actively in two different geographic regions. from one of the redundant analytical systems (2) to another analytical system (2) or a data transfer operation is initiated from an external source (A). Data transfer unit (4) Data transfer synchronization is complete when the data transfer process carried out via the intermediary is finished. Unit (5) is activated. Data transfer synchronization unit (5) is connected to the first analytical system (2) During the transmission of the transferred data to the second analytical system (2), the analytical system (2) 20 To use resources effectively, the data to be transferred should be categorized into data ranges. It divides it into parts. The data to be transferred is located in (3) different indexes in the database. In these cases, this splitting process is performed according to the size of the data in the index. In indexes where the data to be transferred is small, transferring all the data is both time-consuming. and it is more efficient in terms of resource utilization, especially when the data is very large. The indexes divide the data into chunks and then appropriately feed these chunks into a second analytical system. (2) It is more accurate to transfer. Data ranges can be set separately for each data index in the analytical system (2). For example, in a case where the transferred data is 5 years old, this would apply to large-scale indexes. 30 The data is divided into 1-week intervals and sent to the second analytical system (2) As can be configured, data for smaller indexes can be grouped into 1-year intervals. The division can be configured. This configuration includes the sizes of the indices and the indices themselves. Analytical system (2) by taking into account many parameters such as the structure of the data found can be determined by the manager. The indexes (3) in the database (2) of the analytical system 6 for all indexes when they have similar size and data structure It is also possible to create a joint configuration. Data transfer synchronization unit (5), after determining the appropriate intervals for each data index then using the start and end times of the first interval, both analytic systems (2) The transferred data is used to synchronize data in databases. First Analytics 5 It transmits from the system (2) to the second analytical system (2). Data transfer synchronization unit (5), apart from the unit that enables the synchronization of live data in analytical systems (2) Because it's working, it doesn't cause any problems in synchronizing live data. After the first interval is successfully completed, the data synchronization process will proceed to the second and... Continuing for the subsequent intervals, all data ranges for the relevant index are processed and 10 It continues until it is transferred to the second analytical system (2). Similarly, the transferred data data synchronization process of the analytical system (2) all indexes in the database (3) The process is completed by repeating the steps. Data transfer synchronization is for data synchronization. live data because it uses the available resources in the analytical system (2). Live data is slower during periods of high synchronization. 15 In cases where synchronization is light, data intervals can be accessed more quickly. by processing, it enables more efficient use of resources and an optimal solution. It contributes to the creation of parts of the synchronization process of the transferred data. In this way, in the event of a possible error, intervention can be made in a shorter time and If necessary, the equalization process can be repeated for a smaller range, and the associated 20 This allows for more efficient use of time. Thus, it is established with geographical redundancy to operate in an active-active manner and in terms of data. large amounts of data to analytical systems (2) that provide synchronous service Maintaining data synchronization of the analytical system (2) after transfers This is done automatically by using the system (2) resources efficiently. 25 During this data synchronization process, there were no issues as in existing applications. Since no backup and restore operation is required, the second analytical system (2) is out of service. and therefore the service continuity of analytical systems in general (2) This prevents a situation that could lead to putting someone at risk. Furthermore... During the transfer of the transferred data to the second analytical system (2), the data to be transferred is 30 Since the process is carried out by dividing it into parts, both the resources of the analytical system (2) can be increased. This eliminates the need to stop the synchronization of live data. because of the live data, there is a mismatch between the two analytical systems (2). This also prevents its occurrence. 7 Data transfer synchronization unit (5);  Geographically redundant, operating actively in two different geographical regions. from one of the established analytical systems (2) to another analytical system (2) when the data transfer process from an external source (A) is completed, this information (4) to be received from the data transfer unit, 5  data by reading the configuration file on the analytical system (2) Determining the defined data ranges for the (3) indices in the base,  Data in the (3) indexes in the database, data defined for each index dividing into parts according to their intervals,  The data block belonging to each data range for the relevant index is the first analytical system (2) 10 Transfer of the database (3) to the database (2) of the second analytical system (3),  between the data transfer unit (4), the data transferred between the two analytical systems (2) Notification that data synchronization has been successfully completed,  Automatic data synchronization after data transfer completion and thus the analytical systems (2) synchronous 15 ensuring they continue working It is configured to perform its operations. The subject of the invention, the information method (1000), is presented in the flowchart in Figure 2;  Geographically redundant, operating actively in two different geographical regions. from one of the established analytical systems (2) to another analytical system (2) or 20 initiating the data transfer process from an external source (A) (1001),  data transfer unit (4) data transfer synchronization unit (5) first analytical (1002) notifying the system that the data transfer process (2) has been completed,  Configuration of the data transfer synchronization unit (5) on the analytical system (2) By reading the file, the data defined for the (3) indexes in the database 25 determining the intervals (1003),  data transfer synchronization unit (5) data in the indexes (3) in the database each dividing an index into parts according to defined data ranges (1004),  Synchronization of data transfer of data blocks belonging to each data range for the relevant index unit (5) from the database (2) of the first analytical system (3) second analytical 30 (2) transfer of the system to the database (3) (1005), 8  data transfer synchronization unit (5) to data transfer unit (4), transferred data In terms of successful data synchronization between two analytical systems (2) reporting that it is completed (1006) and  After data transfer, automatically via the data transfer synchronization unit (5) as a result of completing data synchronization, analytics 5 (2) ensuring that the systems continue to operate synchronously (1007) It includes the steps involved in the process. The flow in the information method (1000) will work actively in two different geographical regions. One of the analytical systems (2) established with geographical redundancy, another analytical 10 by initiating the data transfer process from the system (2) or from an external source (A) is triggered (1001). Then the data transfer unit (4) is sent to the data transfer synchronization unit (5) In the first analytical system (2), it reports that the data transfer process is complete (1002). In this flow, the initial data transfer operation is done to the first analytical system (2). It is assumed that the second analytic system (2) is just like the first analytic system (2) and both 15 It has both a data transfer unit (4) and a data transfer synchronization unit (5). For this reason data in accordance with the structure of the information system (1) and the preference of the information system (1) manager The transfer process can also be set to start with the second analytical system (2). In the first analytical system (2), the data receives information that the data transfer process is complete. Transfer synchronization unit (5), configuration file on analytical system (2) 20 by reading, it identifies the data ranges defined for the (3) indexes in the database. (1003). Then the data transfer synchronization unit (5), in the indexes (3) in the database divides the data into chunks according to the data ranges defined for each index (1004). The next step involves data transfer for each data range's data block for the relevant index. synchronization unit (5) from the database (2) of the first analytical system (3) second 25 The data is transferred to the database (3) of the analytical system (2) (1005). The data is transferred to the specified data ranges. According to this, the data is divided into parts and blocks and sent to the database of the second analytic system (2) (3). The transfer is carried out by the synchronization unit (5) and is located in the database (3). The process repeats in a loop for all indexes. After the data transfer process, which is carried out within a loop, is completed, the data will be 30. The data transfer synchronization unit (5) has two analytical aspects in terms of the data transferred to the data transfer unit (4). It reports that data synchronization between systems (2) has been successfully completed (1006). This notification to the data transfer unit (4) is one of the other notifications that can be made to the analytical system (2) 9 before data transfer, the data transfer synchronization unit (5) becomes available This is done for the purpose of providing information. Data transfer synchronization after data transfer. Data synchronization is completed automatically via unit (5) and analytics systems (2) continue to work synchronously (1007). In this way both data in the analytical system (2) and data from another analytical system (2) 5 or transferred from an external source (A) and streamed live into the analytical system (2) They become synchronized in terms of data. Thus, it is set up with geographical redundancy to operate in active-active mode, and data analytical systems that work synchronously with each other in terms of (2), as proposed by the invention thanks to the system (1) and method (1000), from another analytical system (2) or an external 10 Data is also transferred automatically in cases where data needs to be transferred from source (A). The data in their bases (3) continue to be served in a synchronized manner. The data transferred to the analytical system (2) in the system (1) and method (1000) are second Since they are transferred to the analytic system (2) by being divided into parts, the analytic system (2) There is no need to increase resources. Also, analytical systems (2) 15 because there is no requirement to stop the synchronization of live data between them A possible mismatch between two analytical systems (2) in terms of live data or This also prevents delays from occurring. In addition, the system (1) and method (1000) in question are used in existing applications. Since no backup and restore operations are required, Analytics 20 by taking one of the systems (2) out of service for a certain period of time, geographical redundant analytics This also prevents situations that would jeopardize the service continuity of the systems (2).

Claims

REQUESTS 1. It processes and records the large amount of data transmitted to it, and then prepares the necessary reports. multiple analytical systems that create visual graphics (2), analytical multiple of the system holding all data and configuration information (2) to the database (3), to the analytical system (2) from another analytical system (2) or a 5 Multiple data transfers that enable data transfer from an external source (A) (2) to the geographically redundant analytical systems that have (4) units. Automated data parity maintenance after large data transfers It is an information system (1) which provides the following features:  Data to one of the geographically redundant structured analytical systems (2) 10 data transferred via the transfer unit (4) in the database (3) by dividing the indexes into defined time intervals and transferring them to each other automatic maintenance of data equality in two analytical systems (2) Multiple data transfer synchronization units (5) It includes. 15 2. The information system mentioned in accordance with claim 1 is (1), and its feature is;  Geographically redundant, operating actively in two different geographical regions. from one of the established analytical systems (2) to another analytical system (2) or when the data transfer process from an external source (A) is completed This information is received from the data transfer unit (4), 20  data by reading the configuration file on the analytical system (2) Determining the defined data ranges for the (3) indices in the base,  The data in the (3) indexes in the database are defined for each index splitting into parts according to data ranges,  The first analytical 25 of the data block for each data range for the relevant index from the system's (2) database (3) to the second analytical system's (2) database (3) transfer,  data transfer unit (4), two analytical systems (2) in terms of transferred data notification that data synchronization has been successfully completed,  Automatic data synchronization after data transfer 30 completion and thus analytical systems (2) synchronous ensuring they continue working 11 Data transfer synchronization unit (5) configured to perform its operations It includes.

3. Big data to geographically redundant structured analytical systems (2) a system that automatically ensures data parity is maintained after data transfers The information method is (1000), and its feature is; 5  Geographically redundant, operating actively in two different geographical regions. from one of the established analytical systems (2) to another analytical system (2) or initiating a data transfer operation from an external source (A) (1001),  data transfer unit (4) data transfer synchronization unit (5) first analytical (1002), 10  data transfer synchronization unit (5) on the analytical system (2) by reading the configuration file for the (3) indexes in the database Detecting defined data ranges (1003),  data transfer synchronization unit (5) data in the indexes (3) in the database 15 (1004),  Synchronization of data transfer of data blocks belonging to each data range for the relevant index unit (5) from the database (2) of the first analytical system (3) second Transfer of the analytical system (2) to the database (3) (1005),  data transfer synchronization unit (5) to data transfer unit (4), transferred data 20 In terms of successful data synchronization between two analytical systems (2) reporting that it was completed in this way (1006) and  After data transfer, via the data transfer synchronization unit (5) automatic completion of data synchronization and thus 25 provision (1007) It includes the steps of the process.